cyrusofeden / csv.js Goto Github PK
View Code? Open in Web Editor NEWA simple, blazing-fast CSV parser and encoder. Full RFC 4180 compliance.
License: MIT License
A simple, blazing-fast CSV parser and encoder. Full RFC 4180 compliance.
License: MIT License
In Interned Explorer 11, I get an Invalid Calling Object error when executing the following code:
var op_val_objects = new CSV(op_val_string, {lineDelimiter: "\n", header: true}).parse();
Where op_val_string is the following string:
"option,value
Cap,1
Trim,2
Post Caps,4
Top Trim,8
Overlap,16
Good Neighbor,32"
Here's the error:
SCRIPT65535: Invalid calling object
File: csv.js, Line: 13, Column: 5
Here's the referenced line in context:
function isArray(object) {
return toString.call(object) == '[object Array]'; // This is line 13
}
I suspect that this might come from Internet Explorer not supporting "call" on many types of object that the other browsers commonly do. It seems I've seen reports of this kind of trouble with DOM objects; but I'm not sure if the same conditions apply here.
See http://tools.ietf.org/html/rfc4180 for an example of quoted fields, newlines in quotes, and escaped quotes.
Hello,
There is no trim() function when code parse fields. It's possible to add it, or to add it via options ?
4180 says in section 2 claim 6 that if there's a quote or a newline in the value, the field needs to be quoted
6. Fields containing line breaks (CRLF), double quotes, and commas
should be enclosed in double-quotes. For example:
"aaa","b CRLF
bb","ccc" CRLF
Here are four failed cases: embedded \n
, embedded \r
, embedded \r\n
, and embedded "
Johns-MacBook-Pro:scratch john$ node
> var csv = require('comma-separated-values');
undefined
> csv.encode([ [1,2,3,4], ["A\rB","C\nD","E\r\nF","G\"H"] ]);
'1,2,3,4\r\nA\rB,C\nD,E\r\nF,G"H'
I'm trying to parse the following csv but i get an extra quote at the end of the first field
Date Download,Image ID,Filename,Original Filename,License,Earnings,Tax
"2016-11-15 05:01:51","62944391","160700127","","240","24.0800","0.00"
"2016-11-15 05:01:57","62944391","160700127","","1080","108.3600","0.00"
I'm using and i get the date with an extra quote at the end
var csv = new Csv(data, { header: true });
i'm doing something wrong?
An example is Twitter's snowflake IDs. They are 64 bit integers, I believe, which causes truncation in Excel and Calc when loaded in as numbers.
An example CSV:
"Network","ID"
"twitter",479386071891505153
You'll notice that the number loses the precision and becomes 479386071891505000
or similar.
A fix is to quote the numbers so they become strings and then pull them in as strings in Excel/Calc.
Is there any way that we can get an option for this when encoding?
If my string only contains newlines and no carriage returns: "ab,cd\nef,gh" i will get {ab,c},{ef,g}
If the file contains empty cell in any row with heading, then that cell content is converted to number 0
require(['csv'], function (csv) { /* csv = undefined */ })
Its really strange, has CSV.js been tried with require.js?
Thanks!
Hi,
Currently, the parser reads empty lines and sets default values as follows:
If the first column is a number then it gets a 0
, otherwise if it is a string then it gets ""
. For the other columns numbers get NaN
and strings get undefined
.
var data = '\
year,age,status,sex,population\r\n\
1850,20,0,1,1017281\r\n
\r\n\
1850,20,0,2,1003841\r\n\
';
new CSV(data, { header: true }).parse();
/*
Returns:
[
{ year: 1850, age: 20, status: 0, sex: 1, population: 1017281 },
{ year: 0, age: NaN, status: NaN, sex: NaN, population: NaN },
{ year: 1850, age: 20, status: 0, sex: 2, population: 1003841 }
]
var data2 = '\
name,age\r\n\
bob,20\r\n
\r\n\
Chris,20\r\n\
';
new CSV(data2, { header: true }).parse();
/*
Returns:
[
{ name: "bob", age: 20 },
{ name: "", age: NaN },
{ name: "Chris", age: 20 }
]
*/
My initial assumption/hope was for the empty lines to simply be ignored.
I would love to hear your thoughts on that.
Cheers,
Salim
Is there a demo page like http://papaparse.com/demo ?
I added some auto detections for line endings and value delimiters in a chrome app I built. Something worth adding to the library? These functions can fail, but should work most of the time. Maybe behind boolean options like autoDetectLine
and autoDetectDelimiter
?
/**
* Detects and returns used line ending
* @param data The raw data string
* @return '\r\n', '\r' or '\n'
*/
function detectLineEnding(data) {
if (Utils.strCount('\r\n', data) > 0) {
return '\r\n'; // Windows
} else if (Utils.strCount('\r', data) > 0) {
return '\r'; // Old mac
} else {
return '\n'; // Unix/Mac
}
}
/**
* Detects and returns used value delimiter
* @param data The raw data string
* @param line The line ending character
* @return ';', ',' or '\t'
*/
function detectValueDelimiter(data, line) {
var arr = data.split(line);
var chars = [';', '\t'];
for(var i = 0; i < chars.length; i++) {
if(testChar(chars[i]))
return chars[i];
}
return ',';
function testChar(c) {
var first = Utils.strCount(c, arr[0]);
if (first === 0)
return false;
for (var i = 1; i < arr.length; i++) {
var count = Utils.strCount(c, arr[i]);
if (count !== first)
return false;
}
return true;
}
}
/**
* Sample implementation of counting substring occurrences.
*/
Utils.strCount = function (needle, haystack) {
return (haystack.match(new RegExp(needle, 'g')) || []).length
}
Maybe the heading has to be escaped.
Any help?
node_modules/comma-separated-values/csv.js:58
cc = function(text, i) { return text.charCodeAt(i); };
has no method 'charCodeAt'
at cc (/vagrant/node_modules/comma-separated-values/csv.js:58:46)
at new CSV (/vagrant/node_modules/comma-separated-values/csv.js:59:36)
at ReadStream. (/vagrant/util/tula_importer.coffee:13:3, :16:14)
at ReadStream.EventEmitter.emit (events.js:95:17)
at ReadStream. (stream_readable.js:745:14)
at ReadStream.EventEmitter.emit (events.js:92:17)
at emitReadable (_stream_readable.js:407:10)
at emitReadable (_stream_readable.js:403:5)
at readableAddChunk (_stream_readable.js:165:9)
at ReadStream.Readable.push (_stream_readable.js:127:10)
at onread (fs.js:1563:12)
at Object.wrapper as oncomplete
Thanks!
In file csv.js line 260, you should also check for double quotes. Better to do it in a single replace with regexp. See this answer for code example http://stackoverflow.com/a/24922761/139361
Hy, In the unminified version (at least) I have:
return data instanceof Array ? this.encode(stream) : this.parse(stream);
but I think it should be
return this.data instanceof Array ? this.encode(stream) : this.parse(stream);
If a cell has two lines, and the first line tail with two spaces, for example
1,"test
abc"
2,"test
abc"
3,"test
abc"
There are two space after test
, What expect to get is
[["1", "test \nabc"], ["2", "test \nabc"], ["3", "test \nabc"]]
It work fine in version 3.6.1 csv.js.
But when using csv.min.js, I got
[["1,"test", "", ""], ["abc"", undefined, undefined], ["2,"test", "", ""], ["abc"", undefined, undefined], ["3,"test", "", ""], ["abc"", undefined, undefined]]
It doesn't make sense. And I use uglify-js create a min.js myself, it didn't have this bug.
Hi
I think that it is not acceptable to remove tags in the repository, because there are projects which might fail to deploy/run one day with the following error:
bower comma-separated-values#~3.2.0 ENORESTARGET No tag found that was able to satisfy ~3.2.0
Additional error details:
Available versions: 3.3.0, 2.2.0, 1.2.3
rake aborted!
Command failed with status (1): [cd /home/avsej/prj/foobar && bower install...]
Please, don't do that
So you can do fs.createReadStream().pipe(new Csv).pipe(outStream)
FYI, I get the error:
Object doesn't support property or method 'call'
at this line:
return Array.isArray?Array.isArray(t):"[object Array]"==toString.call(t)
Maybe put somewhere in the README a minimum browser support.
Cheers
new CSV([{a:1, b:2}, {a:1, b:2, c:3}], {
header: ['a','b','c'],
}).encode();
will return:
"a","b","c"
1,2
1,2
value "c" will not render
same apply if first value is null:
new CSV([{a:1, b:2, c: null}, {a:1, b:2, c:3}], {
header: ['a','b','c'],
}).encode();
To reproduce, run the following code example from README.md
var data = [[1850, 20, 0, 1, 1017281], [1850, 20, 0, 2, 1003841]];
new CSV(data).encode();
generates the string:
"1850,20,,1,1017281
1850,20,,2,1003841"
Rather than the expected:
"1850,20,0,1,1017281
1850,20,0,2,1003841"
For my own, limited context, I added:
if (value == 0) return "0";
to the top of the stringify function. That fixed it for me. I'm not particularly familiar with this code base, so I doubt that's the right thing to do.
I don't know why this commit is a bug should be fix, so v3.6.4 is a breaking change and break my code, and let me busy this morning 😢
If ignore empty lines, we can’t one-to-one align the row index between parsing code and csv editor. For example, I generate a csv let use to edit, and use a empty line separate each section for for pretty look.
"title"
"desc"
"command"
"" <- as a separator
"User ID"
"integer"
"uid"
"1234"
When parse back the csv, if ignore the empty lines, I need to shift the row index, it is hard to debug because the rows
which return by parse()
, the array index is different from csv editor.
So, I request a option the let me setting this behavior.
I'm getting the following error:
/app/node_modules/comma-separated-values/csv.js:50
return string.replace(/"/g,'\\"');
^
TypeError: Cannot read property 'replace' of undefined
at sanitizeString (/app/node_modules/comma-separated-values/csv.js:50:18)
When executing the following code. Everything works fine without the configuration object.
new CSV(fileData, {
header: [
'monthNum',
'projectedLossRateAtMonth',
'actualLossRateAtMonth',
'adjustedProjectedLossRate',
'stressedLossRate',
'breakEvenLossRate',
'pim',
'rating'
]
}).parse();
Hi, i found a case that when the last column does not have a header, it can not be parsed.
The codes look like this
const data = "user_id,name\r\n1234,xxx,extra\r\n";
console.log(new CSV(data).parse());
// =>
[
[user_id, name],
[1234, xxx],
]
Actually i want to get a result like this
[
[user_id, name, ''],
[1234,xxx,extra],
]
Besides, i found that if there is a comma at the end of the header, the last column can be parsed, but there seem to be some other problems
const data = "user_id,name,\r\n1234,xxx,extra\r\n";
console.log(new CSV(data).parse());
// =>
[
[user_id, name,0],
[1234, xxx,NaN],
]
https://codesandbox.io/s/determined-bell-4bb6r?file=/src/App.js
Is there any way to solve it?
‘var Phonebook = new CSV(listStr,
{
header : true
}
).forEach(function (options)
{
Arraytmp.push(options["name"]);
console.log("~~~~~~~~" + options["name"]);
Arraytmp.push(options["office_number"]);
console.log("%%%%%%%%" + options["office_number"]);
}
);’例如原数组为:da,34\r\n
23,rf\r\n
rf不会被解析
I find this little lib very useful, but I've stumbled upon a strange failure, happening sometimes (not always) when null values are involved.
A sample reproduction on Chrome DevTools console:
> a=data[12]
["2015-07-15 00:00:00.000", 711, "ihstelekom", null, null]
> b=data[13]
[null, null, "sifcepsa", 724, "2015-07-16 12:00:00.000"]
> new CSV([a]).encode()
""2015-07-15 00:00:00.000",711,"ihstelekom",,"
> new CSV([b]).encode()
",,"sifcepsa",724,"2015-07-16 12:00:00.000""
> new CSV([a,b]).encode()
""2015-07-15 00:00:00.000",711,"ihstelekom",,<new-line>"null",,"sifcepsa",,"
As it can be seen, encoding a or b separately works ok. But encoding both of them fails (there's a spurious "null" value in b's encoded value, instead of the expected "nothing".
I did take a quick glance but didn't find the culprit. Could you help, please?
I have need for reading CSV files from multiple sources, some of which have multi-row headers (only one of which has actual column headers). It would be nice to be able to tell the parser to ignore the first X rows of a file and/or look for headers at a set row number.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.