csv-parser vs json2csv vs csv-writer
CSV Handling Libraries Comparison
1 Year
csv-parserjson2csvcsv-writerSimilar Packages:
What's CSV Handling Libraries?

CSV handling libraries are essential tools in web development for parsing and writing CSV (Comma-Separated Values) files, which are commonly used for data exchange. These libraries simplify the process of reading data from CSV files into JavaScript objects and writing JavaScript objects back into CSV format. They help developers manage data efficiently, especially when dealing with large datasets or integrating with external systems that utilize CSV for data interchange. Each library has its unique features and use cases, making it important to choose the right one based on project requirements.

Package Weekly Downloads Trend
Github Stars Ranking
Stat Detail
Package
Downloads
Stars
Size
Issues
Publish
License
csv-parser1,383,0191,46229.5 kB584 months agoMIT
json2csv1,076,1862,72651.2 kB172 years agoMIT
csv-writer926,425253-325 years agoMIT
Feature Comparison: csv-parser vs json2csv vs csv-writer

Parsing Efficiency

  • csv-parser:

    csv-parser is designed for high performance and efficiency, utilizing a streaming approach to parse CSV files. This allows it to handle large files without loading the entire dataset into memory, which is crucial for performance in data-intensive applications.

  • json2csv:

    json2csv is primarily focused on converting JSON to CSV. Its efficiency is tied to the size of the JSON object being converted. For large JSON datasets, performance may vary based on the complexity of the transformation and the output format.

  • csv-writer:

    csv-writer focuses on writing CSV files rather than parsing. While it doesn't handle large datasets as efficiently as csv-parser, it provides a simple API for writing data, which is sufficient for most use cases involving smaller datasets.

Customization Options

  • csv-parser:

    csv-parser offers limited customization options as it primarily focuses on parsing CSV data into JavaScript objects. It does allow for some configuration, such as specifying delimiters and headers, but is not as flexible as the other libraries in terms of output formatting.

  • json2csv:

    json2csv provides extensive customization options for transforming JSON data into CSV format. Users can select specific fields, rename headers, and apply transformations to data, making it a powerful tool for exporting data in a desired format.

  • csv-writer:

    csv-writer excels in customization, allowing developers to define headers, specify field order, and format data before writing to CSV. This makes it highly suitable for creating structured CSV files tailored to specific requirements.

Ease of Use

  • csv-parser:

    csv-parser has a straightforward API that is easy to use for parsing CSV files. However, it may require additional handling for complex CSV structures, which could introduce a slight learning curve for beginners.

  • json2csv:

    json2csv is also user-friendly, especially for developers familiar with JSON. Its API is intuitive, allowing for quick conversions from JSON to CSV, but may require some understanding of the options available for customization.

  • csv-writer:

    csv-writer is designed with simplicity in mind, making it very user-friendly for writing CSV files. Its clear API and documentation help developers quickly implement CSV writing functionality without much overhead.

Streaming Support

  • csv-parser:

    csv-parser supports streaming, which is a significant advantage when dealing with large CSV files. It processes data in chunks, allowing for efficient memory usage and faster parsing times, making it ideal for real-time data processing.

  • json2csv:

    json2csv does not support streaming either, as it is designed for converting JSON objects to CSV format. It processes the entire JSON object before generating the CSV output, which may not be optimal for very large datasets.

  • csv-writer:

    csv-writer does not support streaming natively, as it is primarily focused on writing complete datasets to CSV files. For large datasets, this may require loading all data into memory before writing, which could be a limitation.

Community and Support

  • csv-parser:

    csv-parser has a growing community and is actively maintained, providing a decent level of support through documentation and community contributions. However, it may not have as extensive a user base as some other libraries.

  • json2csv:

    json2csv enjoys a robust community and is widely used, which translates to extensive documentation and numerous examples available online. This makes it easier for developers to find help and resources when needed.

  • csv-writer:

    csv-writer has a solid community and good documentation, making it easy to find examples and support. Its popularity among developers ensures that common issues are often addressed in community forums.

How to Choose: csv-parser vs json2csv vs csv-writer
  • csv-parser:

    Choose csv-parser if you need a lightweight and efficient solution for parsing large CSV files. It streams data, making it suitable for handling big datasets without consuming excessive memory.

  • json2csv:

    Select json2csv if you need to convert JSON data to CSV format. It offers flexibility in customizing the output, including field selection and transformation, making it perfect for data export scenarios.

  • csv-writer:

    Opt for csv-writer if your primary requirement is to create and write CSV files easily. It provides a straightforward API for defining headers and formatting data, making it ideal for generating reports or exporting data.

README for csv-parser

csv-parser

tests cover size

Streaming CSV parser that aims for maximum speed as well as compatibility with the csv-spectrum CSV acid test suite.

csv-parser can convert CSV into JSON at at rate of around 90,000 rows per second. Performance varies with the data used; try bin/bench.js <your file> to benchmark your data.

csv-parser can be used in the browser with browserify.

neat-csv can be used if a Promise based interface to csv-parser is needed.

Note: This module requires Node v8.16.0 or higher.

Benchmarks

⚡️ csv-parser is greased-lightning fast

→ npm run bench

  Filename                 Rows Parsed  Duration
  backtick.csv                       2     3.5ms
  bad-data.csv                       3    0.55ms
  basic.csv                          1    0.26ms
  comma-in-quote.csv                 1    0.29ms
  comment.csv                        2    0.40ms
  empty-columns.csv                  1    0.40ms
  escape-quotes.csv                  3    0.38ms
  geojson.csv                        3    0.46ms
  large-dataset.csv               7268      73ms
  newlines.csv                       3    0.35ms
  no-headers.csv                     3    0.26ms
  option-comment.csv                 2    0.24ms
  option-escape.csv                  3    0.25ms
  option-maxRowBytes.csv          4577      39ms
  option-newline.csv                 0    0.47ms
  option-quote-escape.csv            3    0.33ms
  option-quote-many.csv              3    0.38ms
  option-quote.csv                   2    0.22ms
  quotes+newlines.csv                3    0.20ms
  strict.csv                         3    0.22ms
  latin.csv                          2    0.38ms
  mac-newlines.csv                   2    0.28ms
  utf16-big.csv                      2    0.33ms
  utf16.csv                          2    0.26ms
  utf8.csv                           2    0.24ms

Install

Using npm:

$ npm install csv-parser

Using yarn:

$ yarn add csv-parser

Usage

To use the module, create a readable stream to a desired CSV file, instantiate csv, and pipe the stream to csv.

Suppose you have a CSV file data.csv which contains the data:

NAME,AGE
Daffy Duck,24
Bugs Bunny,22

It could then be parsed, and results shown like so:

const csv = require('csv-parser')
const fs = require('fs')
const results = [];

fs.createReadStream('data.csv')
  .pipe(csv())
  .on('data', (data) => results.push(data))
  .on('end', () => {
    console.log(results);
    // [
    //   { NAME: 'Daffy Duck', AGE: '24' },
    //   { NAME: 'Bugs Bunny', AGE: '22' }
    // ]
  });

To specify options for csv, pass an object argument to the function. For example:

csv({ separator: '\t' });

API

csv([options | headers])

Returns: Array[Object]

options

Type: Object

As an alternative to passing an options object, you may pass an Array[String] which specifies the headers to use. For example:

csv(['Name', 'Age']);

If you need to specify options and headers, please use the the object notation with the headers property as shown below.

escape

Type: String
Default: "

A single-character string used to specify the character used to escape strings in a CSV row.

headers

Type: Array[String] | Boolean

Specifies the headers to use. Headers define the property key for each value in a CSV row. If no headers option is provided, csv-parser will use the first line in a CSV file as the header specification.

If false, specifies that the first row in a data file does not contain headers, and instructs the parser to use the column index as the key for each column. Using headers: false with the same data.csv example from above would yield:

[
  { '0': 'Daffy Duck', '1': 24 },
  { '0': 'Bugs Bunny', '1': 22 }
]

Note: If using the headers for an operation on a file which contains headers on the first line, specify skipLines: 1 to skip over the row, or the headers row will appear as normal row data. Alternatively, use the mapHeaders option to manipulate existing headers in that scenario.

mapHeaders

Type: Function

A function that can be used to modify the values of each header. Return a String to modify the header. Return null to remove the header, and it's column, from the results.

csv({
  mapHeaders: ({ header, index }) => header.toLowerCase()
})
Parameters

header String The current column header.
index Number The current column index.

mapValues

Type: Function

A function that can be used to modify the content of each column. The return value will replace the current column content.

csv({
  mapValues: ({ header, index, value }) => value.toLowerCase()
})
Parameters

header String The current column header.
index Number The current column index.
value String The current column value (or content).

newline

Type: String
Default: \n

Specifies a single-character string to denote the end of a line in a CSV file.

quote

Type: String
Default: "

Specifies a single-character string to denote a quoted string.

raw

Type: Boolean

If true, instructs the parser not to decode UTF-8 strings.

separator

Type: String
Default: ,

Specifies a single-character string to use as the column separator for each row.

skipComments

Type: Boolean | String
Default: false

Instructs the parser to ignore lines which represent comments in a CSV file. Since there is no specification that dictates what a CSV comment looks like, comments should be considered non-standard. The "most common" character used to signify a comment in a CSV file is "#". If this option is set to true, lines which begin with # will be skipped. If a custom character is needed to denote a commented line, this option may be set to a string which represents the leading character(s) signifying a comment line.

skipLines

Type: Number
Default: 0

Specifies the number of lines at the beginning of a data file that the parser should skip over, prior to parsing headers.

maxRowBytes

Type: Number
Default: Number.MAX_SAFE_INTEGER

Maximum number of bytes per row. An error is thrown if a line exeeds this value. The default value is on 8 peta byte.

strict

Type: Boolean
Default: false

If true, instructs the parser that the number of columns in each row must match the number of headers specified or throws an exception. if false: the headers are mapped to the column index less columns: any missing column in the middle will result in a wrong property mapping! more columns: the aditional columns will create a "_"+index properties - eg. "_10":"value"

outputByteOffset

Type: Boolean
Default: false

If true, instructs the parser to emit each row with a byteOffset property. The byteOffset represents the offset in bytes of the beginning of the parsed row in the original stream. Will change the output format of stream to be { byteOffset, row }.

Events

The following events are emitted during parsing:

data

Emitted for each row of data parsed with the notable exception of the header row. Please see Usage for an example.

headers

Emitted after the header row is parsed. The first parameter of the event callback is an Array[String] containing the header names.

fs.createReadStream('data.csv')
  .pipe(csv())
  .on('headers', (headers) => {
    console.log(`First header: ${headers[0]}`)
  })

Readable Stream Events

Events available on Node built-in Readable Streams are also emitted. The end event should be used to detect the end of parsing.

CLI

This module also provides a CLI which will convert CSV to newline-delimited JSON. The following CLI flags can be used to control how input is parsed:

Usage: csv-parser [filename?] [options]

  --escape,-e         Set the escape character (defaults to quote value)
  --headers,-h        Explicitly specify csv headers as a comma separated list
  --help              Show this help
  --output,-o         Set output file. Defaults to stdout
  --quote,-q          Set the quote character ('"' by default)
  --remove            Remove columns from output by header name
  --separator,-s      Set the separator character ("," by default)
  --skipComments,-c   Skip CSV comments that begin with '#'. Set a value to change the comment character.
  --skipLines,-l      Set the number of lines to skip to before parsing headers
  --strict            Require column length match headers length
  --version,-v        Print out the installed version

For example; to parse a TSV file:

cat data.tsv | csv-parser -s $'\t'

Encoding

Users may encounter issues with the encoding of a CSV file. Transcoding the source stream can be done neatly with a modules such as:

Or native iconv if part of a pipeline.

Byte Order Marks

Some CSV files may be generated with, or contain a leading Byte Order Mark. This may cause issues parsing headers and/or data from your file. From Wikipedia:

The Unicode Standard permits the BOM in UTF-8, but does not require nor recommend its use. Byte order has no meaning in UTF-8.

To use this module with a file containing a BOM, please use a module like strip-bom-stream in your pipeline:

const fs = require('fs');

const csv = require('csv-parser');
const stripBom = require('strip-bom-stream');

fs.createReadStream('data.csv')
  .pipe(stripBom())
  .pipe(csv())
  ...

When using the CLI, the BOM can be removed by first running:

$ sed $'s/\xEF\xBB\xBF//g' data.csv

Meta

CONTRIBUTING

LICENSE (MIT)