jillix / a-csv Goto Github PK

Sometimes, I must deal with CSV having header row (name to each column). It may be very interesting to have an option to allow skipping first line.

What do you think about that?

The following does not parse as expected;

"Now is The time For all Great men",second col first col,second col
multiline test.txt

[email protected] can't be used because of this line.

$ node index.js 

/home/ionicabizau/Documents/csv-to-json/node_modules/a-csv/index.js:214
        if (arrMatches[2]) {�
                            ^
SyntaxError: Unexpected token ILLEGAL
    at exports.runInThisContext (vm.js:69:16)
    at Module._compile (module.js:432:25)
    at Object.Module._extensions..js (module.js:467:10)
    at Module.load (module.js:349:32)
    at Function.Module._load (module.js:305:12)
    at Function.Module.runMain (module.js:490:10)
    at startup (node.js:123:16)
    at node.js:1128:3

In the following snippet:

Firma,Vorname,Nachname,Zusatz,Kdnr.,Region,Filiale,GF,SOW,Besuchsgebiet,HP,Mitglied seit
,Anton,Studer,,10088,RML,OEN,KTM-GFG,5F,,0,Wed Mar 21 2012 00:00:00 GMT+0000 (UTC)
,Anton,Jenal,Arina-Garage,20193,ROS,CHU,KTM-GFG,4F,,42749,Mon Apr 04 2011 00:00:00 GMT+0000 (UTC)
,Ruedi,Jost,Topcar,33241,RML,TH,KTM-GFG,5F,,47037,Wed Mar 07 2012 00:00:00 GMT+0000 (UTC)
Garage P. Müller,Peter,Müller,Garage,54,RZH,WT,KTM-GFG,5F,,490,Wed Mar 02 2011 00:00:00 GMT+0000 (UTC)
,Fritz,Brändle,Garage,16132,RZH,BD,KTM-GFG,5F,,0,Tue Jun 18 2013 00:00:00 GMT+0000 (UTC)
,R.,Brändle,Garage,2080,RZH,DI,KTM-GFG,4F,,40256,Tue Mar 08 2011 00:00:00 GMT+0000 (UTC)
Hans Blöchliger,Hans,Blöchliger,Seegarage-Carrosserie,14009,RZH,JO,KTM-GFG,4F,,291,Fri Sep 13 2013 00:00:00 GMT+0000 (UTC)
,Claudius,Kupschina,,6609,RZH,DI,KTM-GFG,4F,,0,Sun Mar 27 2011 00:00:00 GMT+0000 (UTC)
Auto Regli GmbH,Emanuel,Regli,Garage,728,RML,LU,KTM-GFG,3M,,40678,Tue Mar 01 2011 00:00:00 GMT+0000 (UTC)
"Garage Fisch AG, Aadorf",,,,42,RZH,WT,KTM-GFG,5M,,0,Thu Dec 15 2011 00:00:00 GMT+0000 (UTC)

several lines start with the separator. That means the first column is missing.

The parsing will produce wrong output by displacing columns in the respective line with one (ignoring the first column)

Hi,
how is the buffer mechanism intended? Is it intended in terms of chunks that are buffered and the reading is continued, of is is actually the maximum file size that can be read?

I noticed two things:
a) the reading stopped in-between reading a larger file (>30.000 lines), with-out any hint that EOF was not reached. Using "a-csv" via npm module "read-csv-json" --> ... --> a-csv", (on Windows 7)

b) the CLI tool "a-csv" seems to ignore the "-l" parameter. changing it does not make a difference.

c) after increasing the "default buffer" size to 256+1024 I was able to read to my large file, but of cause on larger files that might fail again. Therefore the question, how the "buffer" usage is indented.

Thank you very much for you great tool!
Best Regards
Michael

Just found this repo searching for npm CSV modules. How does it compare with @wdavidw's https://github.com/wdavidw/node-csv-parser ?

$ node -v
v0.11.11

It never calls the callback function passed as last argument. I made some debugging.
I suppose that the line that causes the crash is this one:

 67                     // fill buffer with emptiness
>68                     buffer.fill("");

Can you explain what is the functionality of buffer.fill("")? The code running stops here.

test.csv

a,b

c,d
e,f

(data, empty_line, data, data, empty_line)

code:

var csv = require('a-csv')
csv.parse('test.csv', {delimiter: ','}, (err, row, next) => {
  console.log(row)
  return next()
})

Output using [email protected]:

[ 'a', 'b' ]
[]
[ 'c', 'd' ]
[ 'e', 'f' ]
[]
null

Output using [email protected]

[ 'a', 'b' ]
null
[ 'c', 'd' ]
[ 'e', 'f' ]
null
null

The older behavior lets us differentiate between empty lines and end of file in the callback function. In v2.0.0 even empty rows are returned as null. Is there any other way to detect if we've reached the end of the file inside the callback function?