ryu1kn / csv-writer Goto Github PK
View Code? Open in Web Editor NEWConvert objects/arrays into a CSV string or write them into a CSV file
Home Page: https://www.npmjs.com/package/csv-writer
License: MIT License
Convert objects/arrays into a CSV string or write them into a CSV file
Home Page: https://www.npmjs.com/package/csv-writer
License: MIT License
Hi. I'm using csv writer to integrate with a warehouse and they require that all values, including null values, in the csv be double quoted. I set the alwaysQuote option to true, but it looks like null values still get output as
,,,,,
rather than
,"","","","",
Would it be possible to add a new option to double quote null values or to update the existing option so that it works that way?
TSV is also a popular data format and especially useful as tab characters appear less frequent compared to comma characters used in CSV.
Compared to CSV, TSV is even less strictly defined and I haven't found a document which defines how the tab characters in fields should be escaped.
So when I implement this, probably I'll follow the below document, i.e. do not allow tab characters appear in field values.
const date = new Date().toISOString().split('T')[0];
const csvWriter = createCsvWriter({
path: `${require('path').join(require('os').homedir(), 'Desktop')}/${city}-${date}.csv`,
header: [
{id: 'name', title: 'name'},
{id: 'status', title: 'status'},
{id: 'registration', title: 'registration'},
{id: 'city', title: 'city'},
{id: 'prov', title: 'prov'},
{id: 'phone', title: 'phone'},
{id: 'distance', title: 'distance'},
{id: 'clinic', title: 'clinic'},
]
});
csvWriter
.writeRecords(data)
.then(()=> {
console.log(`The ${city}-${date}.csv file was written successfully`)
win.webContents.send('ping', `done ${city}-${date}.csv file was written successfully`);
}
).catch(err=> {
console.log(err)
win.webContents.send('ping', `Couldn't save file, please restart process`);
})
i am using abpove code to write a file to desktop , it works for macOS but doesn't works on windows 10, getting following error on windows
[Error: ENOENT: no such file or directory, open 'C:\Users\Dell\Desktop\vancouver-2019-12-27.csv'] {
errno: -4058,
code: 'ENOENT',
syscall: 'open',
path: 'C:\\Users\\Dell\\Desktop\\vancouver-2019-12-27.csv'
}
I need to pass the parameter from function and write the parameter to the csv file
Looking for a way to specify custom delimiters, between fields and between records.
These variables could be passed with the other constructor variables and (if set) update the abstract.js FIELD_DELIMITER and RECORD_DELIMITER variables.
For example:
createCsvWriter({ path: 'path/to/file.csv' , header: ['one', 'two'] , fieldDelimiter: '|' , recordDelimiter: '\n\n' });
If I try to pass structured data into this during a continuous operation, it fails to enter the record and only empty contents are passed to the csv.
eg
async function(){
for(const record of records) {
await csvWriter.writeRecords(record);
}
}
I know its better to just pass the entire the record into it, but this is for illustration like an ongoing http request process that keeps recurring.
I am downloading data from google firestore and saving it to a file. Names withs ÆÅØ and other letters are mis-interpretation because of missing UTF-8 encoding.
How v´can I fix that? my code are running in nodejs 8
Hi,
After the completion of the write operation (using writeRecords function) how to close the writer?
I tried using writer.end() or writer.close(). Both gave errors that end or close is not a function.
Regards,
Suparna
Hi, Really nice library, and its cool that it supports arrays and objects, and mapping of headers 👍
Seen a few similar previous issues regarding the Delimiter, but I was curious about having enclosures e.g. "value"
I can see you already have a
needsQuote()
for having enclosures for fields that would require it, but it be pretty nice to have the option for all fields to have this.
Although CSV isn't a defined spec, it is mentioned on RFC 4180,
Each field may or may not be enclosed in double quotes
and a other languages library(ies) have supported these options:
[1] https://github.com/thephpleague/csv
My thoughts would be having the enclosure be optional, and if undefined and a field needs quoting it gets quoted as currently works.
But that if a enclosure of "
is passed through it could wrap all fields.
More information here:
https://csv.thephpleague.com/9.0/connections/controls/
Hello Guys,
by simply parsing the example code :
``
const createCsvWriter = require('csv-writer').createObjectCsvWriter;
console.log(createCsvWriter);
const csvWriter = createCsvWriter({
path: 'PATH',
header: [
{id: 'name', title: 'NAME'},
{id: 'lang', title: 'LANGUAGE'}
]
});
const records = [
{name: 'Bob', lang: 'French'},
{name: 'Mary', lang: 'English'}
];
csvWriter.writeRecords(records) // returns a promise
.then(() => {
console.log('...Done');
`});``
I get the Error mentioned in the Title.
I can't find anything helpful here..
Best Regards
Please include MIT LICENSE on version (1.5.0) repository
I'm facing difficulty converting nested arrays/objects to separate columns in CSV. Can't there be any mode in which the library accepts the response and automatically picks up nested objects/arrays in separate columns with headers being the key values?
Is there a way to write to a separate tab or sheet in the csv?
Thanks!
Is it possible to change the delimiter to ;
for example ?
Excel doesn't parse a CSV with comma as delimiter
Hi, I've stumbled upon what seems to be an issue when trying to save on a mounted usb drive (or key) on linux.
The library works well and I've used it many times on a few projects, but the project I'm currently working on requires data to be saved on an external storage.
The media is properly mounted and accessible, and I'm even writing and reading JSON files on it using node's FS module.
As an exemple, the accessible mounted path for my storage is : "/mnt/usbKey" .
Whenever I try to write at this path I get the following error:
{ [Error: EINVAL: invalid argument, open '/mnt/usbKey/logs-2020-06-25T23:46:26+00:00.csv']
errno: -22,
code: 'EINVAL',
syscall: 'open',
path: '/mnt/usbKey/logs-2020-06-25T23:46:26+00:00.csv' }
At first I thought I was facing a path issue so I've tried many things like passing the absolute path, resolving it with node's path module, etc... but no cigars. I always end up stuck here.
My code works perfectly if I try to save it on the box's disk, but as soon as I try the usb storage, I get this error.
I'm still doubting this is a path issue, but I usually get around those pretty quickly. Unfortunately, this one.... I can't seem to get it right.
And as I mentioned earlier, I can save other files on the storage, using the same path, with no issues.
Any idea ?
Help would be greatly appreciated.
Thanks a mil for the lib, it's awesome.... just not today. :)
strange error.
in the console :
TypeError: records.map is not a function
at ObjectCsvStringifier.stringifyRecords (C:\Users\my\path\to\node_modules\csv-writer\lib\csv-stringifiers\abstract.js:20:14)
at CsvWriter.writeRecords (C:\Users\my\path\to\node_modules\csv-writer\lib\csv-writer.js:19:52)
at C:\Users\my\path\to\dev\app.js:33:10
at Layer.handle [as handle_request] (C:\Users\my\path\to\node_modules\express\lib\router\layer.js:95:5)
at next (C:\Users\my\path\to\node_modules\express\lib\router\route.js:137:13)
at Route.dispatch (C:\Users\my\path\to\node_modules\express\lib\router\route.js:112:3)
at Layer.handle [as handle_request] (C:\Users\my\path\to\node_modules\express\lib\router\layer.js:95:5)
at C:\Users\my\path\to\node_modules\express\lib\router\index.js:281:22
at Function.process_params (C:\Users\my\path\to\node_modules\express\lib\router\index.js:335:12)
at next (C:\Users\my\path\to\node_modules\express\lib\router\index.js:275:10)
here the dependancies of my app :
"dependencies": {
"body-parser": "^1.18.3",
"browser-sync": "^2.24.6",
"csv-express": "^1.2.2",
"csv-writer": "^1.0.1",
"my-app": "file:dev/private/my-app",
"express": "^4.16.3",
"fs": "0.0.1-security",
"gulp": "^3.9.1",
"gulp-concat": "^2.6.1",
"gulp-jshint": "^2.1.0",
"gulp-live-server": "0.0.31",
"gulp-rename": "^1.4.0",
"gulp-sass": "^4.0.1",
"gulp-shell": "^0.6.5",
"gulp-uglify": "^3.0.1",
"http": "0.0.0",
"jquery": "^3.3.1",
"jsdom": "^11.12.0",
"jshint": "^2.9.6",
"json2csv": "^4.2.1",
"npm": "^6.4.0"
},
"devDependencies": {
"gulp-node-inspector": "^0.2.1",
"gulp-nodemon": "^2.2.1"
}
If you run the example file and replace the records with numerical values - it does not seem to return the expected results:
const csvWriter = createCsvWriter({
path: '/www/pages/node/socket.io/examples/IR/public/configurations/export/test.csv',
header: [
{id: 'name', title: 'NAME'},
{id: 'lang', title: 'LANGUAGE'}
]
});
const records = [
{name: 'Bob', lang: '400,300,200,100,150,250,400,450'},
{name: 'Mary', lang: 'English'}
];
csvWriter.writeRecords(records)
.then(() => {
console.log('...Done');
});
This will output:
NAME | LANGUAGE |
---|---|
Bob | 400,300,200,100,150,000,000,000 |
Mary | English |
Everything after the 5th index is converted to 000. In this example, the 250,400,450 will be converted to 000, 000, 000.
A workaround is to add a string into the data, like this: {name: 'Bob', lang: '400,300,200,100,150,250,400,450, string'}. Is there a way to write an array of numbers to a csv with this without adding in a string to the data?
const job = new CronJob('*/30 * * * * *', async function () {
var data = [{"place": "Puducherry","state":"PY"},{"place":"Chennai","state":"TN"}]
await csv.csvWriter.writeRecords(data)
}
By executing this first time csv looklike
place,state
puducherry,PY
chennai,TN
By executing this second time csv looklike
place,state
puducherry,PY
chennai,TN
puducherry,PY
chennai,TN
By executing this third time csv looklike
place,state
puducherry,PY
chennai,TN
puducherry,PY
chennai,TN
puducherry,PY
chennai,TN
Similarly the value keep on increasing
Hi I use your createObjectCsvWriter demo code and this not work I get this inside my CSV file
`const createCsvWriter = require('csv-writer').createObjectCsvWriter;
const csvWriter = createCsvWriter({
path: 'path/to/file.csv',
header: [
{id: 'name', title: 'NAME'},
{id: 'lang', title: 'LANGUAGE'}
]
});
const records = [
{name: 'Bob', lang: 'French, English'},
{name: 'Mary', lang: 'English'}
];
csvWriter.writeRecords(records) // returns a promise
.then(() => {
console.log('...Done');
});
`
Allowing this library to accept field delimiters other than comma (,
) and semi-colon (;
) is sometimes requested, and I'm thinking to support this by introducing a new writer class called DsvWriter
.
The problem is that I haven't been able to find an authoritative specification for DSV (Delimiter-Separated Values), whereas we have RFC4180 for CSV (Comma-separated values).
This feature would be aligned with something described in this Wikipedia page; so:
,
/ |
/ ~
/
/ ...) except double-quote ("
), plus tab character.HI there! I'm getting this error:
Uncaught (in promise) TypeError: _this2._fs.writeFile is not a function
at csv-writer.js:46
Any ideas about what's going on?
Thanks!
If alwaysQuote
is enabled, writing a record which contains the empty string doesn't quote the empty string.
Writing record ['hello' , '']
Expected: "hello",""
but get "hello",
Hi, I would like to use csv-writer in a project, in order to reference this library correctly I need the copyright for this library.
Hello,
When I try to reproduce the example below, I get a Unhandled Rejection (TypeError): Cannot read property 'apply' of undefined error at the csvWriter.writeRecords(records) line.
const createCsvWriter = require('csv-writer').createObjectCsvWriter;
const csvWriter = createCsvWriter({
path: 'path/to/file.csv',
header: [
{id: 'name', title: 'NAME'},
{id: 'lang', title: 'LANGUAGE'}
]
});
const records = [
{name: 'Bob', lang: 'French, English'},
{name: 'Mary', lang: 'English'}
];
csvWriter.writeRecords(records) // returns a promise
.then(() => {
console.log('...Done');
});
promise.js:28 Uncaught (in promise) TypeError: Cannot read property 'apply' of undefined
at promise.js:28
at new Promise ()
at promise.js:23
at FileWriter. (file-writer.js:175)
at step (file-writer.js:128)
at Object.next (file-writer.js:59)
at file-writer.js:31
at new Promise ()
at push../node_modules/csv-writer/dist/lib/file-writer.js.__awaiter (file-writer.js:10)
at FileWriter.write (file-writer.js:169)
at CsvWriter. (csv-writer.js:175)
at step (csv-writer.js:128)
at Object.next (csv-writer.js:59)
at csv-writer.js:31
at new Promise ()
at push../node_modules/csv-writer/dist/lib/csv-writer.js.__awaiter (csv-writer.js:10)
at CsvWriter.writeRecords (csv-writer.js:166)
at App.handleClick (App.js:24)
at HTMLUnknownElement.callCallback (react-dom.development.js:363)
at Object.invokeGuardedCallbackDev (react-dom.development.js:412)
at invokeGuardedCallback (react-dom.development.js:466)
at invokeGuardedCallbackAndCatchFirstError (react-dom.development.js:481)
at executeDispatch (react-dom.development.js:614)
at executeDispatchesInOrder (react-dom.development.js:639)
at executeDispatchesAndRelease (react-dom.development.js:744)
at executeDispatchesAndReleaseTopLevel (react-dom.development.js:753)
at forEachAccumulated (react-dom.development.js:725)
at runEventsInBatch (react-dom.development.js:770)
at runExtractedPluginEventsInBatch (react-dom.development.js:916)
at handleTopLevel (react-dom.development.js:6171)
at batchedEventUpdates (react-dom.development.js:2422)
at dispatchEventForPluginEventSystem (react-dom.development.js:6271)
at dispatchEvent (react-dom.development.js:6301)
at unstable_runWithPriority (scheduler.development.js:674)
at runWithPriority$2 (react-dom.development.js:11834)
at discreteUpdates$1 (react-dom.development.js:22935)
at discreteUpdates (react-dom.development.js:2440)
at dispatchDiscreteEvent (react-dom.development.js:6254)
function writeCsv() {
const csvWriter = createCsvWriter({
path: argv.file.replace('.json', '.csv'),
header: [
{ id: "title", title: "TITLE" },
{ id: "phone", title: "PHONE" },
{ id: "link", title: "LINK" },
],
});
csvWriter.writeRecords(all) // returns a promise
.then(() => {
console.log("...Done");
});
}
I just get an empty csv file
This will overwrite the existing file data
const createCsvWriter = require('csv-writer').createObjectCsvWriter;
const csvWriter = createCsvWriter({
path: 'out.csv',
header: [
{id: 'name', title: 'Name'},
{id: 'surname', title: 'Surname'},
{id: 'age', title: 'Age'},
{id: 'gender', title: 'Gender'},
]
});
const data = [
{
name: 'John',
surname: 'Snow',
age: 26,
gender: 'M'
}, {
name: 'Clair',
surname: 'White',
age: 33,
gender: 'F',
}, {
name: 'Fancy',
surname: 'Brown',
age: 78,
gender: 'F'
}
];
csvWriter
.writeRecords(data)
.then(()=> console.log('The CSV file was written successfully'));
Documentation:
createObjectCsvStringifier(params)
Parameters:
params <Object>
header <Array<{id, title}|string>>
Array of objects (id and title properties) or strings (field IDs)
^^^^^^^^^^^^^^^^^
fieldDelimiter <string> (optional)
Default: ,. Only either comma , or semicolon ; is allowed.
Returns:
<ObjectCsvStringifier>
But code shows this:
https://github.com/ryu1kn/csv-writer/blob/master/lib/csv-stringifiers/object.js#L17
Hi there,
nice project! I just found it on npm and noticed some unusual behavior I thought worth reporting.
In the example you write:
`console.log(csvStringifier.getHeaderString());
// => 'NAME,LANGUAGE\n'
console.log(csvStringifier.stringifyRecords(records));
// => 'Bob,"French, English"\nMary,English\n'`
but the output then is:
`
NAME,LANGUAGE
Bob,"French, English"
Mary,English
`
Note the empty newlines after header and after output. I'm not quite sure if this really is correct output.
As often you need to get data from database and then export it in CSV, it would be very handy if we could put nested IDs in header object.
Example:
object = {name: Test, surname: Test, nested: {time: test}}
And our header object would look like:
header: [{id: 'name', title: 'First name' }, {id: 'surname', title: 'Last name'}, {id: 'nested.time', title:'My time'}]
Does this make sense?
Hello,
a library shouldn't be forcing users what is allowed and what isn't allowed,
you could have console.error and
say, hey csv standards are only ; and ,
You shouldn't be throwing an error. if a user wants to have a character to be the delimater.
If I want my csv to simply output a column with a stringified object, it will end up with double quotes
Desired
{"can_debit_product_attributes":{"experience_type":"ecommerce","is_comp":false}}
Actual output
“{”“can_debit_product_attributes”“:{“”experience_type”“:”“ecommerce”“,”“is_comp”“:false}}”
If I don't stringify my objects then it's output as
[object Object]
Anyway around this?
I did see we check if it has a quote already and then replace it with a double quote here: https://github.com/ryu1kn/csv-writer/blob/master/src/lib/field-stringifier.ts#L27
This is a known misbehavior in the cli-table module. See Automattic/cli-table#65 for the details.
I'm calling csvwriter as so:
return new Promise((resolve, reject) => {
csvwriter(data, {
table: true, lineNumbers: true
}, (err, csv) => {
if (err) reject(err);
else resolve(csv);
});
});
In this case the data object is:
System {
_id: 44,
_module_quan: 504,
_total_kw: 128.52,
_meter_id: 'TBD',
_longitude: 0,
_latitude: 0,
_projects_id: 23,
_devices: undefined }
Notice that one field has an undefined value.
The cli-table module crashes saying it cannot call toString on undefined.
[ 1, 44, 504, 128.52, 'TBD', 0, 0, 23, undefined ]
1
44
504
128.52
'TBD'
0
0
23
undefined
TypeError: Cannot read property 'toString' of undefined
at /Users/david/bin/node_modules/cli-table/lib/index.js:186:26
at Array.forEach (native)
at generateRow (/Users/david/bin/node_modules/cli-table/lib/index.js:184:11)
at /Users/david/bin/node_modules/cli-table/lib/index.js:279:16
at Table.forEach (native)
at Table.toString (/Users/david/bin/node_modules/cli-table/lib/index.js:262:10)
at createCLITable (/Users/david/bin/node_modules/csvwriter/lib/csvwriter.js:164:18)
at csvwriter (/Users/david/bin/node_modules/csvwriter/lib/csvwriter.js:108:9)
at Promise (/Users/david/bin/node_modules/xlsetup/cli.js:700:13)
at filterOutput (/Users/david/bin/node_modules/xlsetup/cli.js:698:16)
In the issue on cli-table, I notice two things ....
how is the project licensed? If it is public, can you add a license file?
thanks!
I am trying to write to a CSV-file from my electron renderer.js process. Unfortunately, most of the time the writing isn't working, and the .then
addition isn't executed. Strangely, there is no kind of error message or anything that would tell me my mistake. Some few times it has worked though, the file was written but the confirmation from .then
wasn't displayed in the console. I don't have any clue as to what went differently these times.
When reloading the application with ctrl+r
after the failed attempt the saving process is run again (somehow the onclick
attribute of a button, containing a function call for the function all this stuff here ↓ is in) and that always works (including the .then
call).
My code:
var settings = [
["devtools", false, devtools, ""],
["language", "en", language, ""]
]
var csvWriter = window.createCsvWriter({
header: ['ABBREVIATION', 'DEFAULT', 'CUSTOM', ''],
path: 'saves/settings.csv'
});
csvWriter.writeRecords(settings)
.then(() => {
console.log('Saved succesfully!');
});
window.createCsvWriter
is a preloaded script, devtools
and language
are script wide variables that are updated shortly before this.
I have no idea where the error is coming from, as it can't be in the path or something like that as it has run successfully multiple times. I even tried to follow the debugging process line by line, but all I think I have found out is that the settings array is fully dealt with, the script ends somewhere in a jungle of loops and if-clauses concerning the path or something.
I also know that a normal CSV-file wouldn't have a comma on the end of the rows, my code importing the settings later just can't deal with that, which I will fix later. If you need any more information just ask.
EDIT:
I just followed the code again line by line and notices that it stops after the return __awaiter()
in CsvWriter.prptotype.writeRecords = function (records) {...}
. records
is an array with the correct data for the CSV. Maybe that is useful information.
Hi,
How can I make this return a buffer instead of csv file. I would like to generate the buffer that I can then upload to a remote directory via sftp.
Regards.
I noticed in package.json
that the types
field is pointing to the TypeScript source files instead of the .d.ts
files. In practice this means that consumers are basically re-compiling csv-writer
, which is more brittle than it needs to be.
Hi Ryuichi,
Currently the record delimiter is '\n' (LF). However, as per RFC4180, the line break is expected to be '\r\n' (CRLF).
Each record is located on a separate line, delimited by a line break (CRLF).
I do understand that under Linux environment, the extra '\r' (CR) may cause a lot of problems, therefore I'm wondering if it is possible to at least have a choice for people to choose either to use '\n' (LF) or '\r\n' (CRLF) as the line delimiter.
Just to contribute more information, the reason why I'm sending this request through is many applications based on Windows platform will not be able to read proper line breaks without seeing '\r\n' (CRLF), which will result in a fail in reading the CSV file.
Thanks,
Ming
Currentlyonly , and ; are allowed. It would be wonderful to just open this field up to be anything (including \t for example).
I have an existing csv file without any header titles. appending new row deletes previous records.
Working on a personal project for maintaining an album-of-the-year list, and using csv-writer in NodeJS to maintain a csv that holds all the values (repo here). The first time I do a POST call, it overwrites the CSV correctly, but if I call it more than once, it instead appends the consecutive post calls to the existing CSV instead of overwriting as intended.
Sometimes (in a webserver scenario) one does not need to write a file. Instead the CSV could be streamed or written to a string.
Possible to add this here?
How can I add default values to my csv?
is encoding to ANSI working on createArrayCsvWriter(params)?
If auto-quote is in effect, records containing line feeds (\n) are correctly auto-quoted.
Also, records containing carriage return/line feed pairs (\r\n) are correctly auto-quoted.
But records containing only carriage returns (\r) are not auto-quoted.
hi developer team
i cannot set encoding: 'SJIS' for createCsvWriter function, please support me !
I have these numbers:
const score = [
Math.round(Math.random()) + '',
Math.round(Math.random()) + '',
Math.round(Math.random()) + '',
Math.round(Math.random()) + '',
Math.round(Math.random()) + '',
Math.round(Math.random()) + '',
Math.round(Math.random()) + '',
Math.round(Math.random()) + '',
Math.round(Math.random()) + '',
Math.round(Math.random()) + ''
]
In my use case is for data analysis with Weka, and the numbers should be represented as string. But, when I passed it into csv-writer with finalCsv.writeRecords(finalData).then(result => console.log(result))
gives me:
...
1,1,0,1,1,1,0,0,0,0,26,f,White-European,no,no,United States,no,0,18 and more,Self,NO
...
How can I force it to not convert the numbers?
webpack
Hash: 0ffcdcf02595080aba95
Version: webpack 1.15.0
Time: 16348ms
Asset Size Chunks Chunk Names
./public/builds/r2c-admin-build.js 11.5 MB 0 [emitted] main
+ 1270 hidden modules
ERROR in ./public/builds/r2c-admin-build.js from UglifyJs
SyntaxError: Unexpected token punc «}», expected punc «:» [./~/csv-writer/index.js:6,47]
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.