I have a file with name "file.csv", this file have data below:

ID          Full name
1           Steve
2           John
3           nam
4           Hạnh
5           Thủy

I use segment code below to parse this file to json file. But my results is not utf8

Code:

var fastCsv         = require("fast-csv");
var fs              = require("fs");
var iconv           = require('iconv-lite');
var fileStream      = fs.createReadStream("file.csv");

fastCsv
    .fromStream(fileStream, {headers : ["id", "full_name"]})
    .on("data", function(data){
        console.log("------------------------");
        console.log("data: ", data);
    })
    .on("end", function(){
        console.log("done");
});

Results:

data:  { id: '��I\u0000D\u0000', full_name: '\u0000F\u0000u\u0000l\u0000l\u0000 \u0000n\u0000a\u0000m\u0000e\u0000' }

data:  { id: '\u00001\u0000',full_name: '\u0000S\u0000t\u0000e\u0000v\u0000e\u0000' }

data:  { id: '\u00002\u0000',full_name: '\u0000J\u0000o\u0000h\u0000n\u0000' }

data:  { id: '\u00003\u0000',full_name: '\u0000n\u0000a\u0000m\u0000' }

data:  { id: '\u00004\u0000', full_name: '\u0000H\u0000�\u001en\u0000h\u0000' }

data:  { id: '\u00005\u0000',full_name: '\u0000T\u0000h\u0000�\u001ey\u0000' }

data:  { id: '\u0000', full_name: '' }

How to convert my result to utf8?

I have a file with name "file.csv", this file have data below:

ID          Full name
1           Steve
2           John
3           nam
4           Hạnh
5           Thủy

I use segment code below to parse this file to json file. But my results is not utf8

Code:

var fastCsv         = require("fast-csv");
var fs              = require("fs");
var iconv           = require('iconv-lite');
var fileStream      = fs.createReadStream("file.csv");

fastCsv
    .fromStream(fileStream, {headers : ["id", "full_name"]})
    .on("data", function(data){
        console.log("------------------------");
        console.log("data: ", data);
    })
    .on("end", function(){
        console.log("done");
});

Results:

data:  { id: '��I\u0000D\u0000', full_name: '\u0000F\u0000u\u0000l\u0000l\u0000 \u0000n\u0000a\u0000m\u0000e\u0000' }

data:  { id: '\u00001\u0000',full_name: '\u0000S\u0000t\u0000e\u0000v\u0000e\u0000' }

data:  { id: '\u00002\u0000',full_name: '\u0000J\u0000o\u0000h\u0000n\u0000' }

data:  { id: '\u00003\u0000',full_name: '\u0000n\u0000a\u0000m\u0000' }

data:  { id: '\u00004\u0000', full_name: '\u0000H\u0000�\u001en\u0000h\u0000' }

data:  { id: '\u00005\u0000',full_name: '\u0000T\u0000h\u0000�\u001ey\u0000' }

data:  { id: '\u0000', full_name: '' }

How to convert my result to utf8?

• javascript
• node.js
• unicode

Share Improve this question Follow edited Oct 19, 2015 at 11:04 DinhNguyen asked Oct 19, 2015 at 10:10 DinhNguyenDinhNguyen 35633 silver badges1515 bronze badges 1

• One small suggestion. Post the output as text, not an image so others can use it to test. – Evan Plaice Commented Oct 19, 2015 at 10:33

Add a ment  | 

2 Answers 2

Sorted by: Reset to default 4

Your input file is encoded in UTF-16LE, but it has been read as if it were UTF-8.

Try opening the file with fs.createReadStream('file.csv', {encoding: 'utf-16le'}).

Take a look at Javascript Has a Unicode Problem

In your case you need to decode the escaped unicode chars. A library included with node called punycode can handle this.

Import punycode via:

var punycode = require("punycode");

Change:

console.log("firstName: ", data);

To:

console.log("firstName: ", punycode.ucs2.decode(data));

You might have to break down the data object further to decode it's properties but I can't tell from your answer what their structure is.