p-map vs p-queue vs p-all vs p-series
Concurrency Control in JavaScript Comparison
1 Year
p-mapp-queuep-allp-seriesSimilar Packages:
What's Concurrency Control in JavaScript?

These npm packages provide utilities for managing asynchronous operations in JavaScript, allowing developers to handle multiple promises concurrently while controlling their execution flow. They are particularly useful in scenarios where you need to optimize performance by running tasks in parallel or sequentially, manage resource limits, and improve code readability. Each package offers unique features that cater to different concurrency needs, making them essential tools for modern JavaScript development.

NPM Package Downloads Trend
Github Stars Ranking
Stat Detail
Package
Downloads
Stars
Size
Issues
Publish
License
p-map22,585,7331,37821.2 kB12a month agoMIT
p-queue3,788,6623,51436 kB45a year agoMIT
p-all821,5073215.42 kB12 years agoMIT
p-series16,17566-03 years agoMIT
Feature Comparison: p-map vs p-queue vs p-all vs p-series

Execution Model

  • p-map:

    p-map runs promises concurrently but allows you to specify a concurrency limit. This means you can control how many promises are executed at the same time, which is useful for managing resources and preventing overload.

  • p-queue:

    p-queue manages a queue of tasks, executing them in the order they were added. It allows you to control the maximum number of concurrent executions, ensuring that tasks are processed in a controlled manner without overwhelming the system.

  • p-all:

    p-all executes all provided promises concurrently and resolves when all promises have settled. It does not impose any limit on the number of concurrent executions, making it suitable for scenarios where you want to maximize throughput.

  • p-series:

    p-series runs promises sequentially, ensuring that each promise is completed before the next one starts. This is ideal for tasks that need to be executed in a specific order, such as when one task depends on the result of another.

Error Handling

  • p-map:

    p-map allows for individual error handling within the mapped function, enabling you to handle errors for each promise separately while still maintaining control over the overall execution flow.

  • p-queue:

    p-queue allows you to handle errors as tasks are processed, giving you the flexibility to retry or skip tasks based on their success or failure, which is useful for long-running queues.

  • p-all:

    p-all rejects as soon as any of the promises reject, providing a straightforward way to handle errors in a batch of operations. This means you can easily catch and respond to errors without waiting for all promises to settle.

  • p-series:

    p-series will reject as soon as a promise in the series rejects, allowing you to handle errors in a linear fashion. This ensures that you can manage failures in a predictable sequence.

Use Cases

  • p-map:

    p-map is ideal for scenarios where you need to process a collection of items asynchronously, such as making API calls for each item in an array while controlling the number of concurrent requests to avoid hitting rate limits.

  • p-queue:

    p-queue is suitable for managing tasks that need to be processed in a specific order, such as processing a series of file uploads or database transactions where order and resource management are critical.

  • p-all:

    p-all is best used in scenarios where you need to fire off multiple independent asynchronous operations, such as fetching data from multiple APIs simultaneously without concern for their order.

  • p-series:

    p-series is perfect for situations where tasks are dependent on one another, such as a series of database migrations or sequential API calls that require the result of the previous call.

Performance Optimization

  • p-map:

    p-map strikes a balance between concurrency and resource management, allowing you to optimize performance while preventing overload by controlling the number of concurrent executions.

  • p-queue:

    p-queue optimizes performance by managing task execution order and concurrency limits, ensuring that resources are used efficiently and that tasks are not overwhelming the system.

  • p-all:

    p-all can lead to performance bottlenecks if too many promises are executed at once, especially when dealing with rate-limited APIs. It is important to monitor resource usage when using this package.

  • p-series:

    p-series can be less performant due to its sequential nature, but it is essential when task order is critical. It ensures that tasks complete in the correct sequence, which can be more important than raw speed.

Learning Curve

  • p-map:

    p-map may require a bit more understanding of concurrency concepts, especially when setting concurrency limits, but it remains accessible for developers with a basic knowledge of promises.

  • p-queue:

    p-queue introduces additional complexity with task management and queue handling, which may require a deeper understanding of asynchronous patterns, but offers powerful control over task execution.

  • p-all:

    p-all has a straightforward API, making it easy to learn and use for developers familiar with promises. Its simplicity allows for quick integration into existing codebases.

  • p-series:

    p-series is the simplest to understand for those familiar with synchronous programming, as it mimics a sequential execution flow, making it easy to grasp for developers transitioning from synchronous to asynchronous code.

How to Choose: p-map vs p-queue vs p-all vs p-series
  • p-map:

    Choose p-map when you want to run asynchronous operations in parallel but with a limit on the number of concurrent executions. This is particularly useful when dealing with APIs or resources that have rate limits, allowing you to control the flow of requests while still benefiting from concurrency.

  • p-queue:

    Choose p-queue when you need to manage a queue of asynchronous tasks and want to control their execution order. This is beneficial for scenarios where tasks must be processed in a specific sequence or when you want to limit the number of concurrent operations to avoid overwhelming resources.

  • p-all:

    Choose p-all when you need to execute multiple promises in parallel and want to wait for all of them to resolve or any to reject. It's ideal for scenarios where you want to handle a batch of asynchronous operations simultaneously without caring about their order.

  • p-series:

    Choose p-series when you need to execute asynchronous operations in a strict sequential order. This is useful for tasks that depend on the completion of previous tasks, ensuring that each operation finishes before the next one begins.

README for p-map

p-map

Map over promises concurrently

Useful when you need to run promise-returning & async functions multiple times with different inputs concurrently.

This is different from Promise.all() in that you can control the concurrency and also decide whether or not to stop iterating when there's an error.

Install

npm install p-map

Usage

import pMap from 'p-map';
import got from 'got';

const sites = [
	getWebsiteFromUsername('sindresorhus'), //=> Promise
	'https://avajs.dev',
	'https://github.com'
];

const mapper = async site => {
	const {requestUrl} = await got.head(site);
	return requestUrl;
};

const result = await pMap(sites, mapper, {concurrency: 2});

console.log(result);
//=> ['https://sindresorhus.com/', 'https://avajs.dev/', 'https://github.com/']

API

pMap(input, mapper, options?)

Returns a Promise that is fulfilled when all promises in input and ones returned from mapper are fulfilled, or rejects if any of the promises reject. The fulfilled value is an Array of the fulfilled values returned from mapper in input order.

pMapIterable(input, mapper, options?)

Returns an async iterable that streams each return value from mapper in order.

import {pMapIterable} from 'p-map';

// Multiple posts are fetched concurrently, with limited concurrency and backpressure
for await (const post of pMapIterable(postIds, getPostMetadata, {concurrency: 8})) {
	console.log(post);
};

input

Type: AsyncIterable<Promise<unknown> | unknown> | Iterable<Promise<unknown> | unknown>

Synchronous or asynchronous iterable that is iterated over concurrently, calling the mapper function for each element. Each iterated item is await'd before the mapper is invoked so the iterable may return a Promise that resolves to an item.

Asynchronous iterables (different from synchronous iterables that return Promise that resolves to an item) can be used when the next item may not be ready without waiting for an asynchronous process to complete and/or the end of the iterable may be reached after the asynchronous process completes. For example, reading from a remote queue when the queue has reached empty, or reading lines from a stream.

mapper(element, index)

Type: Function

Expected to return a Promise or value.

options

Type: object

concurrency

Type: number (Integer)
Default: Infinity
Minimum: 1

Number of concurrently pending promises returned by mapper.

backpressure

Only for pMapIterable

Type: number (Integer)
Default: options.concurrency
Minimum: options.concurrency

Maximum number of promises returned by mapper that have resolved but not yet collected by the consumer of the async iterable. Calls to mapper will be limited so that there is never too much backpressure.

Useful whenever you are consuming the iterable slower than what the mapper function can produce concurrently. For example, to avoid making an overwhelming number of HTTP requests if you are saving each of the results to a database.

stopOnError

Only for pMap

Type: boolean
Default: true

When true, the first mapper rejection will be rejected back to the consumer.

When false, instead of stopping when a promise rejects, it will wait for all the promises to settle and then reject with an AggregateError containing all the errors from the rejected promises.

Caveat: When true, any already-started async mappers will continue to run until they resolve or reject. In the case of infinite concurrency with sync iterables, all mappers are invoked on startup and will continue after the first rejection. Issue #51 can be implemented for abort control.

signal

Only for pMap

Type: AbortSignal

You can abort the promises using AbortController.

import pMap from 'p-map';
import delay from 'delay';

const abortController = new AbortController();

setTimeout(() => {
	abortController.abort();
}, 500);

const mapper = async value => value;

await pMap([delay(1000), delay(1000)], mapper, {signal: abortController.signal});
// Throws AbortError (DOMException) after 500 ms.

pMapSkip

Return this value from a mapper function to skip including the value in the returned array.

import pMap, {pMapSkip} from 'p-map';
import got from 'got';

const sites = [
	getWebsiteFromUsername('sindresorhus'), //=> Promise
	'https://avajs.dev',
	'https://example.invalid',
	'https://github.com'
];

const mapper = async site => {
	try {
		const {requestUrl} = await got.head(site);
		return requestUrl;
	} catch {
		return pMapSkip;
	}
};

const result = await pMap(sites, mapper, {concurrency: 2});

console.log(result);
//=> ['https://sindresorhus.com/', 'https://avajs.dev/', 'https://github.com/']

Related

  • p-all - Run promise-returning & async functions concurrently with optional limited concurrency
  • p-filter - Filter promises concurrently
  • p-times - Run promise-returning & async functions a specific number of times concurrently
  • p-props - Like Promise.all() but for Map and Object
  • p-map-series - Map over promises serially
  • More…