agenda vs bee-queue vs bull vs bullmq vs kue vs node-resque
Node.js 任务队列库选型指南
agendabee-queuebullbullmqkuenode-resque类似的npm包:

Node.js 任务队列库选型指南

agendabee-queuebullbullmqkuenode-resque 都是 Node.js 生态中用于处理异步任务队列的流行库。它们允许开发者将耗时操作(如发送邮件、处理文件、调用外部 API)从主请求流程中解耦,放入后台队列异步执行,从而提升 Web 应用的响应速度和可靠性。这些库通常依赖 Redis 或 MongoDB 作为底层存储,提供任务调度、失败重试、延迟执行、并发控制等核心功能,是构建高可用后端服务的关键组件。

npm下载趋势

3 年

GitHub Stars 排名

统计详情

npm包名称
下载量
Stars
大小
Issues
发布时间
License
agenda09,651295 kB21 个月前MIT
bee-queue04,026107 kB454 个月前MIT
bull016,247309 kB1471 年前MIT
bullmq08,6442.34 MB3544 天前MIT
kue09,467-2889 年前MIT
node-resque01,409708 kB202 个月前Apache-2.0

Node.js 任务队列库深度对比:Agenda、Bee-Queue、Bull、BullMQ、Kue 与 node-resque

在构建现代 Web 应用时,我们常常需要将耗时操作(如发送邮件、生成报表、处理上传文件)从主请求流程中剥离出来,交给后台任务队列异步处理。Node.js 生态中有多个成熟的任务队列库,它们都基于 Redis 或 MongoDB 等存储后端,但在 API 设计、功能特性、维护状态和适用场景上存在显著差异。本文将从实际工程角度出发,深入比较 Agenda、Bee-Queue、Bull、BullMQ、Kue 和 node-resque,帮助你做出明智的技术选型。

⚠️ 首要提醒:部分库已不再维护

在深入技术细节前,必须明确一点:Kue 已被官方标记为废弃(deprecated)。其 GitHub 仓库 README 明确写道:“Kue is no longer maintained. We recommend using Bull instead.” 因此,任何新项目都不应再选用 Kue。同样,虽然 node-resque 仍在更新,但其社区活跃度和文档完整性远不如 Bull 系列,需谨慎评估。

🗄️ 存储后端:Redis 还是 MongoDB?

任务队列库的底层存储直接影响部署架构和运维复杂度。

  • Agenda 是唯一使用 MongoDB 的主流选项。如果你的应用已经重度依赖 MongoDB 且希望避免引入 Redis,Agenda 是自然选择。
// Agenda: 使用 MongoDB
const agenda = new Agenda({ db: { address: 'mongodb://127.0.0.1:27017/agenda' } });
agenda.define('send email', async (job) => {
  await sendEmail(job.attrs.data.to);
});
  • Bee-Queue、Bull、BullMQ、Kue、node-resque 均基于 Redis。Redis 的高性能和原子操作使其成为任务队列的理想存储。
// Bull: 使用 Redis
const queue = new Queue('send email', { redis: { port: 6379, host: '127.0.0.1' } });
queue.process('send email', async (job) => {
  await sendEmail(job.data.to);
});

// BullMQ: 使用 Redis(新版 API)
const worker = new Worker('send email', async (job) => {
  await sendEmail(job.data.to);
}, { connection: { host: '127.0.0.1', port: 6379 } });

// Bee-Queue: 使用 Redis
const queue = new Queue('send email', { redis: { host: '127.0.0.1', port: 6379 } });
queue.process(async (job) => {
  await sendEmail(job.data.to);
});

// node-resque: 使用 Redis
const queue = new Queue({ connection: { host: '127.0.0.1', port: 6379 } });
queue.enqueue('send email', 'sendEmail', [to]);

💡 建议:除非已有 MongoDB 技术栈,否则优先选择 Redis 方案 —— 它更轻量、更快,且专为这类场景优化。

🔧 API 设计哲学:面向对象 vs 函数式 vs 声明式

不同库对“如何定义和处理任务”有截然不同的抽象方式。

Agenda:面向 Job 实例

Agenda 将任务视为可调度的对象,支持复杂的重复计划(类似 cron)。

// Agenda: 定义任务并立即或定时执行
agenda.define('delete old users', async (job) => {
  await User.remove({ lastLoginAt: { $lt: twoDaysAgo } });
});

// 每天凌晨 2 点运行
await agenda.every('2 hours', 'delete old users');
// 或立即运行一次
await agenda.now('delete old users');

Bull / BullMQ:声明式队列与处理器分离

Bull 系列强调“队列”作为核心实体,任务(Job)通过名称提交,由独立的处理器消费。

// Bull: 提交任务
const job = await queue.add('send email', { to: 'user@example.com' });

// BullMQ: 更清晰的分离
const job = await queue.add('send email', { to: 'user@example.com' });
// 处理器在另一处定义(可能在不同进程)
const worker = new Worker('send email', async (job) => { /* ... */ });

Bee-Queue:简洁的链式 API

Bee-Queue 采用更函数式的风格,任务处理逻辑直接绑定到队列实例。

// Bee-Queue: 定义处理逻辑并提交任务
const emailQueue = new Queue('email');
emailQueue.process(async (job) => {
  return sendEmail(job.data.to);
});

// 提交任务
const job = await emailQueue.createJob({ to: 'user@example.com' }).save();

node-resque:仿 Ruby Resque 的命令式风格

node-resque 的 API 较为底层,需手动管理队列、任务注册和工作进程。

// node-resque: 注册任务函数
const jobs = {
  sendEmail: {
    perform: async (to) => await sendEmail(to)
  }
};

// 提交任务
await queue.enqueue('default', 'sendEmail', ['user@example.com']);

// 启动工作进程
const worker = new Worker({ connection: {}, queues: ['default'] }, jobs);
worker.start();

💡 建议:BullMQ 的 API 最符合现代工程实践 —— 清晰分离生产者与消费者,类型安全更好,适合大型项目。

⏱️ 任务调度与重试机制

可靠的任务队列必须处理失败重试和延迟执行。

延迟任务(Delayed Jobs)

// Agenda: 内置延迟支持
await agenda.schedule('in 10 minutes', 'send email', { to: 'user@example.com' });

// Bull: 通过 delay 选项
await queue.add('send email', { to: 'user@example.com' }, { delay: 10 * 60 * 1000 });

// BullMQ: 同 Bull
await queue.add('send email', { to: 'user@example.com' }, { delay: 600000 });

// Bee-Queue: 通过 delay() 方法
await emailQueue.createJob({ to: 'user@example.com' }).delay(600000).save();

// node-resque: 不直接支持延迟任务,需自行实现

重试策略(Retry Strategies)

// Bull: 内置重试,支持 backoff
await queue.add('send email', data, {
  attempts: 3,
  backoff: { type: 'exponential', delay: 1000 }
});

// BullMQ: 更强大的重试配置
await queue.add('send email', data, {
  attempts: 5,
  backoff: { type: 'custom', delay: (attemptsMade) => Math.pow(2, attemptsMade) * 1000 }
});

// Bee-Queue: 通过 retries 选项
await emailQueue.createJob(data).retries(3).save();

// Agenda: 通过 fail event 手动重试
agenda.on('fail', async (err, job) => {
  if (job.attrs.failures < 3) {
    await job.repeatEvery('5 minutes').save();
  }
});

💡 建议:BullMQ 提供最灵活的重试和退避策略,适合对可靠性要求高的场景。

📊 监控与可观测性

生产环境必须能监控队列状态、任务积压和失败率。

  • Bull / BullMQ 提供内置的 Web UI(通过 bull-board@bull-board/ui),可实时查看队列、任务状态、重试情况。
  • Agenda 无官方 UI,但可通过查询 MongoDB 监控任务。
  • Bee-Queuenode-resque 需自行实现监控逻辑,或依赖 Redis 监控工具。
// BullMQ + bull-board 示例
import { createBullBoard } from '@bull-board/api';
import { BullMQAdapter } from '@bull-board/api/bullMQAdapter';
import { ExpressAdapter } from '@bull-board/express';

const serverAdapter = new ExpressAdapter();
createBullBoard({
  queues: [new BullMQAdapter(myQueue)],
  serverAdapter
});
app.use('/admin/queues', serverAdapter.getRouter());

💡 建议:若需开箱即用的监控能力,BullMQ 是最佳选择。

🔄 迁移路径与未来演进

  • BullMQ 是 Bull 的现代化继任者,使用 TypeScript 重写,修复了 Bull 的许多设计缺陷(如内存泄漏、类型不安全)。官方推荐新项目直接使用 BullMQ。
  • Agenda 仍在维护,但更新较慢,适合 MongoDB 技术栈项目。
  • Bee-Queue 轻量简洁,适合小型项目或对依赖大小敏感的场景。
  • node-resque 适合已有 Ruby Resque 经验的团队,但学习曲线较陡。
  • Kue 已废弃,切勿用于新项目

📌 总结:如何选择?

场景推荐库理由
全新项目,追求现代化、类型安全、强大功能BullMQ完整的 TypeScript 支持、灵活的重试/延迟、优秀监控生态
已有 MongoDB 技术栈,不想引入 RedisAgenda唯一成熟的 MongoDB 任务队列方案
轻量级需求,简单任务处理,小团队快速迭代Bee-QueueAPI 简洁,代码量少,易于理解
从 Ruby Resque 迁移,或熟悉其模型node-resque概念一致,但需接受较弱的文档和生态
任何新项目避免 Kue官方已废弃,存在未修复 bug 和安全风险

最终,任务队列库的选择应基于你的技术栈现状、团队熟悉度和长期维护成本。对于绝大多数新项目,BullMQ 是最平衡、最面向未来的选择

如何选择: agenda vs bee-queue vs bull vs bullmq vs kue vs node-resque

  • agenda:

    选择 agenda 如果你的项目已经重度使用 MongoDB 且希望避免引入 Redis 依赖。它提供了类似 cron 的复杂调度能力,适合需要基于时间触发的周期性任务场景。但需注意其 API 较为老旧,且缺乏官方 Web UI 监控工具。

  • bee-queue:

    选择 bee-queue 如果你需要一个轻量、简洁、无额外依赖(仅需 Redis)的任务队列方案。它的 API 设计直观,适合小型项目或对 bundle size 敏感的场景。但功能相对基础,缺少高级重试策略和内置监控支持。

  • bull:

    选择 bull 如果你正在维护一个已使用它的老项目,或者需要一个成熟稳定的 Redis 队列方案。它拥有丰富的功能和活跃的社区插件(如 bull-board)。但新项目应优先考虑其继任者 BullMQ,因为 Bull 存在一些已知的设计缺陷和内存问题。

  • bullmq:

    选择 bullmq 作为新项目的首选。它是 Bull 的现代化重写,采用 TypeScript 开发,提供更强的类型安全、更灵活的重试/延迟配置、更低的内存占用和更好的监控集成。API 设计清晰,分离了生产者与消费者,适合中大型应用的长期维护。

  • kue:

    不要选择 kue 用于任何新项目。它已被官方明确标记为废弃(deprecated),不再接受功能更新或安全修复。现有项目应制定迁移计划,转向 BullMQ 或其他活跃维护的替代方案。

  • node-resque:

    选择 node-resque 仅当你有 Ruby Resque 的使用经验,或需要与现有 Resque 生态集成。它忠实复刻了 Resque 的概念模型,但文档和社区支持较弱,学习曲线较陡。对于大多数新项目,BullMQ 是更高效的选择。

agenda的README

Agenda

Agenda

A light-weight job scheduling library for Node.js

NPM Version NPM Downloads

Migrating from v5? See the Migration Guide for all breaking changes.

Agenda 6.x

Agenda 6.x is a complete TypeScript rewrite with a focus on modularity and flexibility:

  • Pluggable storage backends - Choose from MongoDB, PostgreSQL, Redis, or implement your own. Each backend is a separate package - install only what you need.

  • Pluggable notification channels - Move beyond polling with real-time job notifications via Redis, PostgreSQL LISTEN/NOTIFY, or other pub/sub systems. Jobs get processed immediately when saved, not on the next poll cycle.

  • Modern stack - ESM-only, Node.js 18+, full TypeScript with strict typing.

See the 6.x Roadmap for details and progress.

Features

  • Minimal overhead job scheduling
  • Pluggable storage backends (MongoDB, PostgreSQL, Redis)
  • TypeScript support with full typing
  • Scheduling via cron or human-readable syntax
  • Configurable concurrency and locking
  • Real-time job notifications (optional)
  • Sandboxed worker execution via fork mode
  • TypeScript decorators for class-based job definitions

Installation

Install the core package and your preferred backend:

# For MongoDB
npm install agenda @agendajs/mongo-backend

# For PostgreSQL
npm install agenda @agendajs/postgres-backend

# For Redis
npm install agenda @agendajs/redis-backend

Requirements:

  • Node.js 18+
  • Database of your choice (MongoDB 4+, PostgreSQL, or Redis)

Quick Start

import { Agenda } from 'agenda';
import { MongoBackend } from '@agendajs/mongo-backend';

const agenda = new Agenda({
  backend: new MongoBackend({ address: 'mongodb://localhost/agenda' })
});

// Define a job
agenda.define('send email', async (job) => {
  const { to, subject } = job.attrs.data;
  await sendEmail(to, subject);
});

// Start processing
await agenda.start();

// Schedule jobs
await agenda.every('1 hour', 'send email', { to: 'user@example.com', subject: 'Hello' });
await agenda.schedule('in 5 minutes', 'send email', { to: 'admin@example.com', subject: 'Report' });
await agenda.now('send email', { to: 'support@example.com', subject: 'Urgent' });

Official Backend Packages

PackageBackendNotificationsInstall
@agendajs/mongo-backendMongoDBPolling onlynpm install @agendajs/mongo-backend
@agendajs/postgres-backendPostgreSQLLISTEN/NOTIFYnpm install @agendajs/postgres-backend
@agendajs/redis-backendRedisPub/Subnpm install @agendajs/redis-backend

Backend Capabilities

BackendStorageNotificationsNotes
MongoDB (MongoBackend)Storage only. Combine with external notification channel for real-time.
PostgreSQL (PostgresBackend)Full backend. Uses LISTEN/NOTIFY for notifications.
Redis (RedisBackend)Full backend. Uses Pub/Sub for notifications.
InMemoryNotificationChannelNotifications only. For single-process/testing.

Backend Configuration

MongoDB

import { Agenda } from 'agenda';
import { MongoBackend } from '@agendajs/mongo-backend';

// Via connection string
const agenda = new Agenda({
  backend: new MongoBackend({ address: 'mongodb://localhost/agenda' })
});

// Via existing MongoDB connection
const agenda = new Agenda({
  backend: new MongoBackend({ mongo: existingDb })
});

// With options
const agenda = new Agenda({
  backend: new MongoBackend({
    mongo: db,
    collection: 'jobs'        // Collection name (default: 'agendaJobs')
  }),
  processEvery: '30 seconds', // Job polling interval
  maxConcurrency: 20,         // Max concurrent jobs
  defaultConcurrency: 5       // Default per job type
});

PostgreSQL

import { Agenda } from 'agenda';
import { PostgresBackend } from '@agendajs/postgres-backend';

const agenda = new Agenda({
  backend: new PostgresBackend({
    connectionString: 'postgresql://user:pass@localhost:5432/mydb'
  })
});

Redis

import { Agenda } from 'agenda';
import { RedisBackend } from '@agendajs/redis-backend';

const agenda = new Agenda({
  backend: new RedisBackend({
    connectionString: 'redis://localhost:6379'
  })
});

Real-Time Notifications

For faster job processing across distributed systems:

import { Agenda, InMemoryNotificationChannel } from 'agenda';
import { MongoBackend } from '@agendajs/mongo-backend';

const agenda = new Agenda({
  backend: new MongoBackend({ mongo: db }),
  notificationChannel: new InMemoryNotificationChannel()
});

Mixing Storage and Notification Backends

You can use MongoDB for storage while using a different system for real-time notifications:

import { Agenda } from 'agenda';
import { MongoBackend } from '@agendajs/mongo-backend';
import { RedisBackend } from '@agendajs/redis-backend';

// MongoDB for storage + Redis for real-time notifications
const redisBackend = new RedisBackend({ connectionString: 'redis://localhost:6379' });
const agenda = new Agenda({
  backend: new MongoBackend({ mongo: db }),
  notificationChannel: redisBackend.notificationChannel
});

This is useful when you want MongoDB's proven durability and flexible queries for job storage, but need faster real-time notifications across multiple processes.

API Overview

Defining Jobs

// Simple async handler
agenda.define('my-job', async (job) => {
  console.log('Processing:', job.attrs.data);
});

// With options
agenda.define('my-job', async (job) => { /* ... */ }, {
  concurrency: 10,
  lockLimit: 5,
  lockLifetime: 10 * 60 * 1000, // 10 minutes
  priority: 'high'
});

Defining Jobs with Decorators

For a class-based approach, use TypeScript decorators:

import { JobsController, Define, Every, registerJobs, Job } from 'agenda';

@JobsController({ namespace: 'email' })
class EmailJobs {
  @Define({ concurrency: 5 })
  async sendWelcome(job: Job<{ userId: string }>) {
    console.log('Sending welcome to:', job.attrs.data.userId);
  }

  @Every('1 hour')
  async cleanupBounced(job: Job) {
    console.log('Cleaning up bounced emails');
  }
}

registerJobs(agenda, [new EmailJobs()]);
await agenda.start();

// Schedule using namespaced name
await agenda.now('email.sendWelcome', { userId: '123' });

See Decorators Documentation for full details.

Scheduling Jobs

// Run immediately
await agenda.now('my-job', { userId: '123' });

// Run at specific time
await agenda.schedule('tomorrow at noon', 'my-job', data);
await agenda.schedule(new Date('2024-12-25'), 'my-job', data);

// Run repeatedly
await agenda.every('5 minutes', 'my-job');
await agenda.every('0 * * * *', 'my-job'); // Cron syntax

Job Control

// Cancel jobs (removes from database)
await agenda.cancel({ name: 'my-job' });

// Disable/enable jobs globally (by query)
await agenda.disable({ name: 'my-job' });  // Disable all jobs matching query
await agenda.enable({ name: 'my-job' });   // Enable all jobs matching query

// Disable/enable individual jobs
const job = await agenda.create('my-job', data);
job.disable();
await job.save();

// Progress tracking
agenda.define('long-job', async (job) => {
  for (let i = 0; i <= 100; i += 10) {
    await doWork();
    await job.touch(i); // Report progress 0-100
  }
});

Stopping / Draining

// Stop immediately - unlocks running jobs so other workers can pick them up
await agenda.stop();

// Drain - waits for running jobs to complete before stopping
await agenda.drain();

// Drain with timeout (30 seconds) - for cloud platforms with shutdown deadlines
const result = await agenda.drain(30000);
if (result.timedOut) {
    console.log(`${result.running} jobs still running after timeout`);
}

// Drain with AbortSignal - for external control
const controller = new AbortController();
setTimeout(() => controller.abort(), 30000);
await agenda.drain({ signal: controller.signal });

Use drain() for graceful shutdowns where you want in-progress jobs to finish their work.

Events

agenda.on('start', (job) => console.log('Job started:', job.attrs.name));
agenda.on('complete', (job) => console.log('Job completed:', job.attrs.name));
agenda.on('success', (job) => console.log('Job succeeded:', job.attrs.name));
agenda.on('fail', (err, job) => console.log('Job failed:', job.attrs.name, err));

// Job-specific events
agenda.on('start:send email', (job) => { /* ... */ });
agenda.on('fail:send email', (err, job) => { /* ... */ });

Custom Backend

For databases other than MongoDB, PostgreSQL, or Redis, implement AgendaBackend:

import { AgendaBackend, JobRepository } from 'agenda';

class SQLiteBackend implements AgendaBackend {
  readonly repository: JobRepository;
  readonly notificationChannel = undefined; // Or implement NotificationChannel

  async connect() { /* ... */ }
  async disconnect() { /* ... */ }
}

const agenda = new Agenda({
  backend: new SQLiteBackend({ path: './jobs.db' })
});

See Custom Backend Driver for details.

Documentation

Related Packages

Official Backend Packages:

Tools:

License

MIT