I have a situation where a user can upload a csv file. This CSV file contains a lot of data, but I am only interested in 2 columns (ID and Date). At the moment, I am parsing the CSV using Papaparse

Papa.parse(ev.data, {
    delimiter: "",
    newline: "",
    quoteChar: '"',
    header: true,
    error: function(err, file, inputElem, reason) { },
    plete: function (results) {
        this.parsed_csv = results.data;

    }
});

When this is run this.parsed_csv represents objects of data keyed by the field name. So if I JSON.stringify the output is something like this

[
  {
    "ID": 123456,
    "Date": "2012-01-01",
    "Irrelevant_Column_1": 123,
    "Irrelevant_Column_2": 234,
    "Irrelevant_Column_3": 345,
    "Irrelevant_Column_4": 456
  },
  ...
]

So my main question is how can I get rid of the columns I dont need, and just produce a new csv containing the columns ID and Date?

Thanks

One thing I realised, is there a way to add dynamic variables. For instance I am letting users select the columns I want to map. Now I need to do something like this

let ID = this.selectedIdCol;
this.parsed_csv = results.data.map(element => ({ID: element.ID, Date: element.Date}));

It is saying that ID is unused however. Thanks

I have a situation where a user can upload a csv file. This CSV file contains a lot of data, but I am only interested in 2 columns (ID and Date). At the moment, I am parsing the CSV using Papaparse

Papa.parse(ev.data, {
    delimiter: "",
    newline: "",
    quoteChar: '"',
    header: true,
    error: function(err, file, inputElem, reason) { },
    plete: function (results) {
        this.parsed_csv = results.data;

    }
});

When this is run this.parsed_csv represents objects of data keyed by the field name. So if I JSON.stringify the output is something like this

[
  {
    "ID": 123456,
    "Date": "2012-01-01",
    "Irrelevant_Column_1": 123,
    "Irrelevant_Column_2": 234,
    "Irrelevant_Column_3": 345,
    "Irrelevant_Column_4": 456
  },
  ...
]

So my main question is how can I get rid of the columns I dont need, and just produce a new csv containing the columns ID and Date?

Thanks

One thing I realised, is there a way to add dynamic variables. For instance I am letting users select the columns I want to map. Now I need to do something like this

let ID = this.selectedIdCol;
this.parsed_csv = results.data.map(element => ({ID: element.ID, Date: element.Date}));

It is saying that ID is unused however. Thanks

• javascript
• csv
• papaparse

Share Improve this question Follow edited Jan 27, 2019 at 12:42 katie hudson asked Jan 27, 2019 at 12:19 katie hudsonkatie hudson 2,8931313 gold badges5454 silver badges101101 bronze badges

Add a ment  | 

3 Answers 3

Sorted by: Reset to default 2

let data = [
  {
    "ID": 123456,
    "Date": "2012-01-01",
    "Irrelevant_Column_1": 123,
    "Irrelevant_Column_2": 234,
    "Irrelevant_Column_3": 345,
    "Irrelevant_Column_4": 456
  },
  ...
]

just produce results by using the following code:

data = data.map(element => ({ID: element.ID, Date: element.Date}))

Now you have desired column, please generate a new CSV on these columns

As Serrurier pointed out above, You should use the step/chunk function to alter the data rather than after parse map as in memory data is already available.

PapaParse.parse(file, { skipEmptyLines: true, header: true, step: (results, parser) => {
                  results.data = _.pick(results.data , [ 'column1' 'column2']);
                  return results;
           }});

Note that if you are loading a huge file, you will have the whole file in memory right after the parsing. Moreover it may freeze the browser due to the heavy workload. You can avoid that by reading and discarding columns :

• row by row
• chunk by chunk.

You should read Papaparse's FAQ before implementing that. To sum up, you will store required columns by extracting them from the step or chunk callbacks.