I'm wondering if there's some proper way (internet protocol, browser API, etc.), for a web app in a modern browser, to get a stream of large data from a Web Server (in my case, nodejs, but the question is focused on client side), and to save it to one big file in the client's file system. The assumption is that the amount of data is too large to be entirely buffered in the App's assigned memory, so I guess somehow the process should append chunks of data from the stream directly to a file.

The data is basically tens of thousands of db records, that should be written to a format such as csv.

Hope I'm making any sense here.

I'm wondering if there's some proper way (internet protocol, browser API, etc.), for a web app in a modern browser, to get a stream of large data from a Web Server (in my case, nodejs, but the question is focused on client side), and to save it to one big file in the client's file system. The assumption is that the amount of data is too large to be entirely buffered in the App's assigned memory, so I guess somehow the process should append chunks of data from the stream directly to a file.

The data is basically tens of thousands of db records, that should be written to a format such as csv.

Hope I'm making any sense here.

• javascript
• html
• node.js

Share Improve this question Follow edited Nov 14, 2019 at 16:46 Luke Girvin 13.5k1010 gold badges6767 silver badges8585 bronze badges asked Mar 22, 2017 at 14:29 IsraelIsrael 1,22422 gold badges1414 silver badges2626 bronze badges 4

• You mean like a normal file download? Aka, generating the file on the server and then sending that file to the client. – George Commented Mar 22, 2017 at 14:31
• 1 @George no, the server should stream data from the db and the client should process and build the file – Israel Commented Mar 22, 2017 at 14:34
• You will want to use pipe(). nodejs/dist/latest-v6.x/docs/api/… – Diego ZoracKy Commented Mar 22, 2017 at 15:04
• 1 @DiegoZoracKy thanks, but this is an approach for the server side, while I'm asking specifically about the client side – Israel Commented Mar 22, 2017 at 15:43

Add a ment  | 

1 Answer 1

Sorted by: Reset to default 4

You asked if there was a proper way. No, I'm afraid the technology that does it is badly supported and brand new. However, if you are brave enough to try anyway this is what you have.

1. Getting a stream of data from Web Server

If you don't have to support IE then you can use the Fetch API. See this question. If you do have to support IE then you could use a websocket to push down little bits at a time.

2. Writing a stream of data to the disk

StreamSaver.js Is a project that tries to acplish this using the ReadableStream API, however it currently only works in Opera and Chrome.