fs-extra vs graceful-fs vs memfs vs fs
File System Libraries in Node.js Comparison
1 Year
fs-extragraceful-fsmemfsfsSimilar Packages:
What's File System Libraries in Node.js?

File system libraries in Node.js provide APIs for interacting with the file system, enabling developers to read, write, and manipulate files and directories. These libraries are essential for tasks such as file uploads, data storage, and configuration management in server-side applications. They offer varying levels of functionality, performance, and ease of use, catering to different development needs and scenarios.

NPM Package Downloads Trend
Github Stars Ranking
Stat Detail
Package
Downloads
Stars
Size
Issues
Publish
License
fs-extra54,910,8929,49954.9 kB16a year agoMIT
graceful-fs34,681,4951,27632.5 kB492 years agoISC
memfs8,765,9051,785683 kB625 days agoApache-2.0
fs1,199,359158-48 years agoISC
Feature Comparison: fs-extra vs graceful-fs vs memfs vs fs

Functionality

  • fs-extra:

    'fs-extra' builds upon 'fs' by adding methods like 'copy', 'remove', and 'mkdirp', which simplify common tasks. It also supports promises and async/await, making it easier to work with asynchronous code.

  • graceful-fs:

    'graceful-fs' enhances the standard 'fs' module by implementing a queue system to handle file operations more efficiently. It reduces the likelihood of hitting system limits on file descriptors (EMFILE errors) during high concurrency operations.

  • memfs:

    'memfs' provides a complete in-memory file system that mimics the behavior of a real file system. It allows developers to create, read, and manipulate files and directories entirely in memory, facilitating testing and temporary data storage.

  • fs:

    The 'fs' module provides basic file system functionalities such as reading, writing, and deleting files, as well as working with directories. It supports both synchronous and asynchronous operations, allowing for flexibility in handling file I/O.

Performance

  • fs-extra:

    'fs-extra' maintains performance similar to 'fs' while adding extra functionalities. It is optimized for common file operations, making it efficient for tasks that require additional features.

  • graceful-fs:

    'graceful-fs' is designed to improve performance by managing file descriptor limits more effectively. It queues file operations to prevent errors under heavy load, ensuring smoother performance in high-demand scenarios.

  • memfs:

    'memfs' operates entirely in memory, providing extremely fast read and write operations. However, it is not suitable for persistent storage, as data is lost when the process ends.

  • fs:

    The performance of 'fs' is generally good for standard file operations, but it can struggle under high concurrency due to system limits on file descriptors, leading to potential errors.

Error Handling

  • fs-extra:

    'fs-extra' inherits error handling from 'fs' but provides additional convenience methods that can simplify error management. It also supports promises, allowing for easier error handling with try/catch blocks.

  • graceful-fs:

    'graceful-fs' improves error handling by preventing common issues such as EMFILE errors. It automatically retries failed operations, making it more resilient in high-load situations.

  • memfs:

    'memfs' allows for controlled error handling in a testing environment. Since it simulates a file system, developers can easily manage errors without affecting the real file system.

  • fs:

    The native 'fs' module can throw errors directly, which requires developers to implement their own error handling strategies. This can lead to unhandled exceptions if not managed properly.

Use Cases

  • fs-extra:

    Use 'fs-extra' for applications that require advanced file manipulation features, such as copying files or creating directories recursively. It is ideal for build scripts and file management tasks.

  • graceful-fs:

    Use 'graceful-fs' in applications that perform many concurrent file operations, such as web servers or data processing applications, to avoid file descriptor limits and improve reliability.

  • memfs:

    Use 'memfs' in testing scenarios where you need a temporary file system. It is perfect for unit tests that require file operations without affecting the actual file system.

  • fs:

    Use 'fs' for simple file operations in applications where performance and additional features are not critical. It is suitable for basic file reading and writing tasks.

Learning Curve

  • fs-extra:

    'fs-extra' is also easy to learn, especially for those already familiar with 'fs'. Its additional methods are intuitive and enhance the learning experience for file manipulation.

  • graceful-fs:

    'graceful-fs' requires minimal learning, as it extends 'fs' without introducing significant complexity. Developers can use it as a drop-in replacement for improved performance.

  • memfs:

    'memfs' has a simple API that mimics the 'fs' module, making it easy to learn for those familiar with file operations in Node.js. Its in-memory nature simplifies the learning process for testing scenarios.

  • fs:

    The 'fs' module has a straightforward API, making it easy to learn for those familiar with Node.js. However, handling asynchronous operations may require additional understanding of callbacks or promises.

How to Choose: fs-extra vs graceful-fs vs memfs vs fs
  • fs-extra:

    Choose 'fs-extra' if you require additional methods for file manipulation, such as recursive directory creation and file copying. It extends the native 'fs' module with extra functionality, making it suitable for more complex file operations.

  • graceful-fs:

    Choose 'graceful-fs' if you need improved performance and reliability when dealing with file system operations, particularly in high-load scenarios. It addresses issues with the native 'fs' module by providing a more resilient implementation that prevents EMFILE errors.

  • memfs:

    Choose 'memfs' if you need an in-memory file system for testing or temporary file storage. It allows you to simulate file operations without affecting the actual file system, making it perfect for unit tests and scenarios where persistence is not required.

  • fs:

    Choose 'fs' for basic file system operations when you need a lightweight solution without additional features. It's built into Node.js, so no installation is required, making it ideal for simple file manipulations.

README for fs-extra

Node.js: fs-extra

fs-extra adds file system methods that aren't included in the native fs module and adds promise support to the fs methods. It also uses graceful-fs to prevent EMFILE errors. It should be a drop in replacement for fs.

npm Package License build status downloads per month JavaScript Style Guide

Why?

I got tired of including mkdirp, rimraf, and ncp in most of my projects.

Installation

npm install fs-extra

Usage

CommonJS

fs-extra is a drop in replacement for native fs. All methods in fs are attached to fs-extra. All fs methods return promises if the callback isn't passed.

You don't ever need to include the original fs module again:

const fs = require('fs') // this is no longer necessary

you can now do this:

const fs = require('fs-extra')

or if you prefer to make it clear that you're using fs-extra and not fs, you may want to name your fs variable fse like so:

const fse = require('fs-extra')

you can also keep both, but it's redundant:

const fs = require('fs')
const fse = require('fs-extra')

ESM

There is also an fs-extra/esm import, that supports both default and named exports. However, note that fs methods are not included in fs-extra/esm; you still need to import fs and/or fs/promises seperately:

import { readFileSync } from 'fs'
import { readFile } from 'fs/promises'
import { outputFile, outputFileSync } from 'fs-extra/esm'

Default exports are supported:

import fs from 'fs'
import fse from 'fs-extra/esm'
// fse.readFileSync is not a function; must use fs.readFileSync

but you probably want to just use regular fs-extra instead of fs-extra/esm for default exports:

import fs from 'fs-extra'
// both fs and fs-extra methods are defined

Sync vs Async vs Async/Await

Most methods are async by default. All async methods will return a promise if the callback isn't passed.

Sync methods on the other hand will throw if an error occurs.

Also Async/Await will throw an error if one occurs.

Example:

const fs = require('fs-extra')

// Async with promises:
fs.copy('/tmp/myfile', '/tmp/mynewfile')
  .then(() => console.log('success!'))
  .catch(err => console.error(err))

// Async with callbacks:
fs.copy('/tmp/myfile', '/tmp/mynewfile', err => {
  if (err) return console.error(err)
  console.log('success!')
})

// Sync:
try {
  fs.copySync('/tmp/myfile', '/tmp/mynewfile')
  console.log('success!')
} catch (err) {
  console.error(err)
}

// Async/Await:
async function copyFiles () {
  try {
    await fs.copy('/tmp/myfile', '/tmp/mynewfile')
    console.log('success!')
  } catch (err) {
    console.error(err)
  }
}

copyFiles()

Methods

Async

Sync

NOTE: You can still use the native Node.js methods. They are promisified and copied over to fs-extra. See notes on fs.read(), fs.write(), & fs.writev()

What happened to walk() and walkSync()?

They were removed from fs-extra in v2.0.0. If you need the functionality, walk and walkSync are available as separate packages, klaw and klaw-sync.

Third Party

CLI

fse-cli allows you to run fs-extra from a console or from npm scripts.

TypeScript

If you like TypeScript, you can use fs-extra with it: https://github.com/DefinitelyTyped/DefinitelyTyped/tree/master/types/fs-extra

File / Directory Watching

If you want to watch for changes to files or directories, then you should use chokidar.

Obtain Filesystem (Devices, Partitions) Information

fs-filesystem allows you to read the state of the filesystem of the host on which it is run. It returns information about both the devices and the partitions (volumes) of the system.

Misc.

Hacking on fs-extra

Wanna hack on fs-extra? Great! Your help is needed! fs-extra is one of the most depended upon Node.js packages. This project uses JavaScript Standard Style - if the name or style choices bother you, you're gonna have to get over it :) If standard is good enough for npm, it's good enough for fs-extra.

js-standard-style

What's needed?

  • First, take a look at existing issues. Those are probably going to be where the priority lies.
  • More tests for edge cases. Specifically on different platforms. There can never be enough tests.
  • Improve test coverage.

Note: If you make any big changes, you should definitely file an issue for discussion first.

Running the Test Suite

fs-extra contains hundreds of tests.

  • npm run lint: runs the linter (standard)
  • npm run unit: runs the unit tests
  • npm run unit-esm: runs tests for fs-extra/esm exports
  • npm test: runs the linter and all tests

When running unit tests, set the environment variable CROSS_DEVICE_PATH to the absolute path of an empty directory on another device (like a thumb drive) to enable cross-device move tests.

Windows

If you run the tests on the Windows and receive a lot of symbolic link EPERM permission errors, it's because on Windows you need elevated privilege to create symbolic links. You can add this to your Windows's account by following the instructions here: http://superuser.com/questions/104845/permission-to-make-symbolic-links-in-windows-7 However, I didn't have much luck doing this.

Since I develop on Mac OS X, I use VMWare Fusion for Windows testing. I create a shared folder that I map to a drive on Windows. I open the Node.js command prompt and run as Administrator. I then map the network drive running the following command:

net use z: "\\vmware-host\Shared Folders"

I can then navigate to my fs-extra directory and run the tests.

Naming

I put a lot of thought into the naming of these functions. Inspired by @coolaj86's request. So he deserves much of the credit for raising the issue. See discussion(s) here:

  • https://github.com/jprichardson/node-fs-extra/issues/2
  • https://github.com/flatiron/utile/issues/11
  • https://github.com/ryanmcgrath/wrench-js/issues/29
  • https://github.com/substack/node-mkdirp/issues/17

First, I believe that in as many cases as possible, the Node.js naming schemes should be chosen. However, there are problems with the Node.js own naming schemes.

For example, fs.readFile() and fs.readdir(): the F is capitalized in File and the d is not capitalized in dir. Perhaps a bit pedantic, but they should still be consistent. Also, Node.js has chosen a lot of POSIX naming schemes, which I believe is great. See: fs.mkdir(), fs.rmdir(), fs.chown(), etc.

We have a dilemma though. How do you consistently name methods that perform the following POSIX commands: cp, cp -r, mkdir -p, and rm -rf?

My perspective: when in doubt, err on the side of simplicity. A directory is just a hierarchical grouping of directories and files. Consider that for a moment. So when you want to copy it or remove it, in most cases you'll want to copy or remove all of its contents. When you want to create a directory, if the directory that it's suppose to be contained in does not exist, then in most cases you'll want to create that too.

So, if you want to remove a file or a directory regardless of whether it has contents, just call fs.remove(path). If you want to copy a file or a directory whether it has contents, just call fs.copy(source, destination). If you want to create a directory regardless of whether its parent directories exist, just call fs.mkdirs(path) or fs.mkdirp(path).

Credit

fs-extra wouldn't be possible without using the modules from the following authors:

License

Licensed under MIT

Copyright (c) 2011-2017 JP Richardson