p-limit vs p-queue vs p-throttle
Node.js 并发控制库
p-limitp-queuep-throttle类似的npm包:
Node.js 并发控制库

在 Node.js 开发中,处理并发操作是一个常见的需求,尤其是在执行大量异步任务时。p-limit、p-queue 和 p-throttle 这三个库提供了不同的方式来控制并发任务的数量和速率,从而优化性能和资源使用。它们各自的设计理念和功能使得开发者能够根据具体的需求选择合适的工具,以实现更高效的异步操作管理。

npm下载趋势
3 年
GitHub Stars 排名
统计详情
npm包名称
下载量
Stars
大小
Issues
发布时间
License
p-limit173,392,0492,70711.7 kB52 个月前MIT
p-queue13,206,0884,05472.4 kB61 个月前MIT
p-throttle2,544,83150821.6 kB01 个月前MIT
功能对比: p-limit vs p-queue vs p-throttle

并发控制

  • p-limit:

    p-limit 允许你设置最大并发数,确保在任何时刻只会有指定数量的任务同时执行。这对于防止资源耗尽或过载非常有用。

  • p-queue:

    p-queue 提供了一个队列机制,允许你控制任务的执行顺序,并且可以设置并发数。它确保任务按照添加的顺序执行,适合需要顺序处理的场景。

  • p-throttle:

    p-throttle 通过限制在特定时间段内执行的任务数量来控制速率,确保不会超过设定的阈值。这在处理需要遵循速率限制的 API 请求时特别有效。

任务管理

  • p-limit:

    p-limit 适合简单的任务管理,主要关注并发数的限制,不涉及任务的优先级或顺序。

  • p-queue:

    p-queue 提供了更复杂的任务管理功能,包括任务优先级、取消任务等,适合需要更高控制力的场景。

  • p-throttle:

    p-throttle 主要关注任务的执行速率,而不是任务的顺序或优先级,适合需要控制速率的场景。

易用性

  • p-limit:

    p-limit 的 API 非常简单,易于上手,适合快速集成到项目中。

  • p-queue:

    p-queue 的 API 相对复杂一些,但提供了更多的功能和灵活性,适合需要精细控制的开发者。

  • p-throttle:

    p-throttle 的使用也很简单,适合需要快速实现速率限制的场景。

性能优化

  • p-limit:

    p-limit 通过限制并发数来优化性能,避免过多的并发任务导致的性能下降。

  • p-queue:

    p-queue 通过顺序执行和优先级管理来优化任务处理,确保高优先级任务能够及时执行。

  • p-throttle:

    p-throttle 通过控制任务执行的速率来优化性能,避免因过快的请求导致的 API 限制或资源耗尽。

适用场景

  • p-limit:

    适用于简单的并发控制场景,如批量处理文件或数据。

  • p-queue:

    适用于需要顺序执行的任务,如处理用户请求或消息队列。

  • p-throttle:

    适用于需要遵循速率限制的场景,如 API 调用或爬虫。

如何选择: p-limit vs p-queue vs p-throttle
  • p-limit:

    选择 p-limit 如果你需要限制并发操作的数量,但不需要对任务的执行顺序进行管理。它简单易用,适合快速限制并发任务。

  • p-queue:

    选择 p-queue 如果你需要在执行异步任务时保持任务的顺序,并且需要更复杂的队列管理功能。它支持优先级队列和任务的取消,适合需要精细控制任务执行的场景。

  • p-throttle:

    选择 p-throttle 如果你需要控制任务的执行速率,确保在特定时间内只执行有限数量的任务。它适合处理速率限制的场景,比如 API 请求。

p-limit的README

p-limit

Run multiple promise-returning & async functions with limited concurrency

Works in Node.js and browsers.

Install

npm install p-limit

Usage

import pLimit from 'p-limit';

const limit = pLimit(1);

const input = [
	limit(() => fetchSomething('foo')),
	limit(() => fetchSomething('bar')),
	limit(() => doSomething())
];

// Only one promise is run at once
const result = await Promise.all(input);
console.log(result);

API

pLimit(concurrency) default export

Returns a limit function.

concurrency

Type: number
Minimum: 1

Concurrency limit.

limit(fn, ...args)

Returns the promise returned by calling fn(...args).

fn

Type: Function

Promise-returning/async function.

args

Any arguments to pass through to fn.

Support for passing arguments on to the fn is provided in order to be able to avoid creating unnecessary closures. You probably don't need this optimization unless you're pushing a lot of functions.

limit.map(iterable, mapperFunction)

Process an iterable of inputs with limited concurrency.

The mapper function receives the item value and its index.

Returns a promise equivalent to Promise.all(Array.from(iterable, (item, index) => limit(mapperFunction, item, index))).

This is a convenience function for processing inputs that arrive in batches. For more complex use cases, see p-map.

limit.activeCount

The number of promises that are currently running.

limit.pendingCount

The number of promises that are waiting to run (i.e. their internal fn was not called yet).

limit.clearQueue()

Discard pending promises that are waiting to run.

This might be useful if you want to teardown the queue at the end of your program's lifecycle or discard any function calls referencing an intermediary state of your app.

Note: This does not cancel promises that are already running.

limit.concurrency

Get or set the concurrency limit.

limitFunction(fn, options) named export

Returns a function with limited concurrency.

The returned function manages its own concurrent executions, allowing you to call it multiple times without exceeding the specified concurrency limit.

Ideal for scenarios where you need to control the number of simultaneous executions of a single function, rather than managing concurrency across multiple functions.

import {limitFunction} from 'p-limit';

const limitedFunction = limitFunction(async () => {
	return doSomething();
}, {concurrency: 1});

const input = Array.from({length: 10}, limitedFunction);

// Only one promise is run at once.
await Promise.all(input);

fn

Type: Function

Promise-returning/async function.

options

Type: object

concurrency

Type: number
Minimum: 1

Concurrency limit.

FAQ

How is this different from the p-queue package?

This package is only about limiting the number of concurrent executions, while p-queue is a fully featured queue implementation with lots of different options, introspection, and ability to pause the queue.

Related

  • p-throttle - Throttle promise-returning & async functions
  • p-debounce - Debounce promise-returning & async functions
  • p-map - Run promise-returning & async functions concurrently with different inputs
  • p-all - Run promise-returning & async functions concurrently with optional limited concurrency
  • More…