# Asynchronous Background Jobs & Task Queues

> Architectural guidelines for processing heavy tasks in the background with Redis task queues, worker pools, concurrency limits, and failure alerts.

Canonical: https://www.agentickit.co/skills/data-pipelines
Last updated: 2026-11-18

## Key practices
- Never execute long-running (>1s) tasks synchronously inside API request handlers
- Configure BullMQ jobs with exponential retry backoff and dead-letter queues
- Set concurrency limits per worker to avoid exhausting CPU and memory limits
- Emit progress events so users can monitor long-running background tasks in UI

## Related
- [data-engineer](https://www.agentickit.co/agents/data-engineer)
- [backend-architect](https://www.agentickit.co/agents/backend-architect)
- [/data-pipeline](https://www.agentickit.co/commands/data-pipeline)
- [/build-api](https://www.agentickit.co/commands/build-api)
