This document covers two distinct mechanisms for asynchronous task execution in NestJS applications:
@nestjs/bull with Redis-backed queues for persistent, distributed job processing@nestjs/schedule for cron-based periodic task executionThese systems enable applications to defer work from the request-response cycle, execute recurring maintenance tasks, and distribute workloads across multiple processes. For real-time event-driven communication patterns, see WebSocket Support. For message broker integrations (RabbitMQ, Kafka, etc.), see Microservices Architecture.
Sources: sample/26-queues/package.json22-26 sample/27-scheduling/package.json22-25
NestJS provides two complementary approaches to asynchronous task execution, each serving different use cases and architectural requirements.
Diagram: Task Execution System Architecture
| Feature | Bull Queues | Task Scheduling |
|---|---|---|
| Persistence | Redis-backed, survives restarts | In-memory, lost on restart |
| Distribution | Multiple workers, horizontal scaling | Single process per instance |
| Triggering | Programmatic (on-demand) | Time-based (cron patterns) |
| Use Cases | Email sending, data processing, async workflows | Cache warming, cleanup, reports |
| External Dependency | Redis server required | None |
| Package | @nestjs/bull + bull | @nestjs/schedule |
Sources: sample/26-queues/package.json22-26 sample/27-scheduling/package.json22-25
The Bull queue system provides Redis-backed job queues with support for job persistence, retries, rate limiting, and distributed processing across multiple worker processes.
Diagram: Bull Queue Module Structure
The Bull integration requires BullModule.forRoot() for Redis connection configuration and BullModule.registerQueue() to define named queues. Each queue operates independently with its own job storage and processing configuration.
Sources: sample/26-queues/package.json22-26
Job producers add work to queues using the @InjectQueue() decorator to inject a queue instance. Producers can specify job options including priority, delay, attempts, and backoff strategies.
Diagram: Job Producer Flow
Common job options include:
| Option | Type | Purpose |
|---|---|---|
priority | number | Job execution priority (lower values = higher priority) |
delay | number | Delay in milliseconds before job becomes available |
attempts | number | Maximum retry attempts on failure |
backoff | number/object | Delay between retry attempts (fixed or exponential) |
removeOnComplete | boolean/number | Remove job data after successful completion |
removeOnFail | boolean/number | Remove job data after final failure |
Sources: sample/26-queues/package.json22-26
Job consumers process queued work using the @Processor() decorator at the class level and @Process() decorator on handler methods. Multiple consumers can process the same queue for horizontal scaling.
Diagram: Job Consumer Architecture
The Job object provides access to:
job.data - The payload passed when creating the jobjob.id - Unique job identifierjob.attemptsMade - Number of processing attemptsjob.progress(percent) - Update job progress (0-100)job.log(message) - Add log entries to jobjob.remove() - Remove job from queueSources: sample/26-queues/package.json22-26
Bull provides automatic retry mechanisms with configurable backoff strategies. Failed jobs can be monitored and manually retried through queue event listeners.
Diagram: Job Error Handling and Retry Flow
Backoff strategies:
| Strategy | Configuration | Behavior |
|---|---|---|
| Fixed | backoff: 5000 | Fixed 5-second delay between retries |
| Exponential | backoff: { type: 'exponential', delay: 1000 } | Increasing delay: 1s, 2s, 4s, 8s, etc. |
| Custom | backoff: { type: 'custom' } | Custom backoff strategy implementation |
Sources: sample/26-queues/package.json22-26
The @nestjs/schedule package provides declarative cron-based scheduling using decorators, enabling periodic task execution without external dependencies.
The @Cron() decorator schedules methods using standard cron syntax or predefined expressions. Scheduled tasks run in the same process as the application.
Diagram: Cron Job Scheduling Architecture
Standard cron pattern format:
* * * * * *
│ │ │ │ │ │
│ │ │ │ │ └─ day of week (0-7, 0 or 7 is Sunday)
│ │ │ │ └─── month (1-12)
│ │ │ └───── day of month (1-31)
│ │ └─────── hour (0-23)
│ └───────── minute (0-59)
└─────────── second (0-59, optional)
Common CronExpression constants:
| Constant | Pattern | Description |
|---|---|---|
EVERY_SECOND | * * * * * * | Execute every second |
EVERY_5_SECONDS | */5 * * * * * | Execute every 5 seconds |
EVERY_MINUTE | 0 * * * * * | Execute every minute |
EVERY_HOUR | 0 0 * * * * | Execute every hour |
EVERY_DAY_AT_MIDNIGHT | 0 0 0 * * * | Execute at midnight |
EVERY_WEEK | 0 0 0 * * 0 | Execute every Sunday at midnight |
Sources: sample/27-scheduling/package.json22-25
Beyond cron patterns, the schedule module provides @Interval() for repeated execution and @Timeout() for one-time delayed execution.
Diagram: Interval and Timeout Scheduling
| Decorator | Timing | Use Case | Cancellable |
|---|---|---|---|
@Cron(pattern) | Cron expression | Specific times (e.g., midnight, every Monday) | Yes, via SchedulerRegistry |
@Interval(ms) | Fixed interval | Regular polling, metric collection | Yes, via SchedulerRegistry |
@Timeout(ms) | One-time delay | Delayed initialization, warmup tasks | Yes, via SchedulerRegistry |
Sources: sample/27-scheduling/package.json22-25
The SchedulerRegistry service enables runtime manipulation of scheduled tasks, allowing applications to dynamically add, remove, or modify scheduled jobs.
Diagram: Dynamic Scheduling Control Flow
The SchedulerRegistry provides methods for:
Sources: sample/27-scheduling/package.json22-25
The choice between Bull queues and task scheduling depends on workload characteristics, persistence requirements, and scaling needs.
| Aspect | Bull Queues | Task Scheduling |
|---|---|---|
| Trigger Mechanism | Programmatic (on-demand) | Time-based (cron/interval) |
| Persistence | Redis-backed, survives restarts | In-memory, lost on restart |
| Distribution | Multi-worker, cross-server | Single process only |
| Job Control | Priority, delay, rate limiting | Basic start/stop control |
| Failure Handling | Automatic retries with backoff | Manual error handling |
| Progress Tracking | Job progress and logging | No built-in progress |
| Resource Requirements | Redis server required | None |
| Horizontal Scaling | Yes, multiple workers | No, one instance per server |
| Job Inspection | Web UI available (bull-board) | No inspection UI |
| Use Case | Async workflows, user-triggered tasks | Periodic maintenance, reports |
Diagram: Task System Selection Decision Tree
Sources: sample/26-queues/package.json22-26 sample/27-scheduling/package.json22-25
Both Bull and Schedule integrate with NestJS's dependency injection system, allowing scheduled tasks and queue processors to access other services.
Diagram: Module Integration with Dependency Injection
Both systems integrate seamlessly with:
| System | Testing Approach | Test Utilities |
|---|---|---|
| Bull Queues | Mock queue instance with @InjectQueue(), test processors independently | @nestjs/testing with queue mocks |
| Schedule | Disable scheduling in tests, call methods directly | SchedulerRegistry for manual control |
Sources: sample/26-queues/package.json32-54 sample/27-scheduling/package.json31-53
NestJS provides two complementary systems for asynchronous task execution:
Bull Queues (@nestjs/bull) excel at:
Task Scheduling (@nestjs/schedule) excels at:
Both systems integrate with NestJS's dependency injection, support testing through @nestjs/testing, and can coexist in the same application for different use cases.
Sources: sample/26-queues/package.json sample/27-scheduling/package.json
Refresh this wiki