I have to handle large amount of data retrieved from server using ajax and store it in JavaScript. I am currently using an array to store all the data. this is how I get data and store it in the javascript:

var buffer=new Array();
//when each ajax returns 
buffer=buffer.concat(JSON.parse(ajaxReq.responseText));

there are two questions in my mind:

1. Is using an array to store all the data efficient, is there an overflow issue with 1 million rows of data??
2. How is the concat performance? how can it be optimized here?

thanks for any input.

I have to handle large amount of data retrieved from server using ajax and store it in JavaScript. I am currently using an array to store all the data. this is how I get data and store it in the javascript:

var buffer=new Array();
//when each ajax returns 
buffer=buffer.concat(JSON.parse(ajaxReq.responseText));

there are two questions in my mind:

1. Is using an array to store all the data efficient, is there an overflow issue with 1 million rows of data??
2. How is the concat performance? how can it be optimized here?

thanks for any input.

• javascript
• arrays
• performance

Share Follow asked Dec 18, 2011 at 18:47 bingjie2680bingjie2680 7,77388 gold badges5050 silver badges7272 bronze badges 2

• 1 This question might be useful: stackoverflow./questions/4833480/… – Matt Ball Commented Dec 18, 2011 at 18:49
• I would avoid it, since you don't have any idea what the various client machines will do or can handle. I can't think of a valid reason to have that much data in the browser. I'd setup a method to offload the processing to the server. – Jared Farrish Commented Dec 18, 2011 at 18:54

Add a ment  | 

2 Answers 2

Sorted by: Reset to default 10

I know, that I risk a downvote, but: The only valid answer to "How do I store 1 million of rows in JavaScript" is "Don't."

It looks like you are dumping your whole database on the client at every pageload, so that at the particular page load he may or may not have to wait for quick ajax responses. This is honestly by far the most insane thing I have ever heard of, and hopefully you can see it as such as well.

Even if your users wouldn't have to wait for small ajax calls that don't really take long (look at google autosuggest), having to have them wait for 1 million database rows to transfer and load in memory on every pageload is even worse user experience.

What you are doing is only feasible if your data fits in localStorage (5MB), which it probably doesn't.