Asynchronous Background Jobs & Task Queues

What it does

Asynchronous Background Jobs & Task Queues is a Cursor skill: a SKILL.md playbook that AgenticKit installs at .cursor/skills/data-pipelines. Cursor's agent loads it when the task matches the skill description, instead of stuffing the same rules into every chat.

Architectural guidelines for processing heavy tasks in the background with Redis task queues, worker pools, concurrency limits, and failure alerts.

BullMQ, Redis queues, exponential retries, concurrency, and rate limiting. The file is domain knowledge, not a slash macro. You do not type it. The agent reads it when the job is data pipelines.

Why it exists

Default Cursor has no memory of how your team ships code, schema, and tests. Founders paste the same conventions into chat, then watch the model drift on the next turn. This skill exists so those conventions live on disk and load only when relevant.

It ships in the Engineering Kit. It is wired to the Data Pipeline & ETL Engineer (data-engineer) and Senior Backend Architect (backend-architect) agents. It is wired to the /data-pipeline: Background Job & Task Queue Architecture and /build-api: API Endpoint & Route Generator commands. That graph is the point: the skill is the standard, the agent is the role, the command is the trigger.

When to use it

• Use Asynchronous Background Jobs & Task Queues when the work is specifically about bullmq, redis queues, exponential retries, concurrency, and rate limiting.
• Load it before a long session that will touch this surface more than once. A one-line edit does not need the full playbook.
• Use it on production SaaS work (multi-tenant apps, paid features, anything that will be reviewed) rather than a throwaway prototype.

When not to use it

• Do not treat the skill as a replacement for a slash command. Skills teach. Commands run a pipeline.
• Do not paste the whole SKILL.md into chat. If Cursor is not loading it, fix the description or invoke the matching command.
• Do not use it as a generic 'write better code' rule. Scope is this domain only.

Example workflow

1. Install the Engineering Kit so .cursor/skills/data-pipelines lands in the repo.
2. Open a new Cursor agent chat pointed at the files this skill governs.
3. Ask the Data Pipeline & ETL Engineer (data-engineer) to read the skill, or run /data-pipeline: Background Job & Task Queue Architecture.
4. Review the first artifact against the practices below. If it violates one, stop and correct the file. Do not prompt 'just finish it'.
5. Commit the skill-guided files with the rest of the slice so the next session inherits the same standard.

Example usage

Example prompt: "Read .cursor/skills/data-pipelines and apply it to this change. Do not invent extra conventions."

Or trigger the pipeline: /data-pipeline: Background Job & Task Queue Architecture. That command is written to load this skill.

Check the output against: Never execute long-running (>1s) tasks synchronously inside API request handlers

Example output

• The skill itself does not write a single output file. It changes what the agent is allowed to produce in code, schema, and tests.
• When you run /data-pipeline: Background Job & Task Queue Architecture, expect repo files plus notes under docs/, not a chat-only answer.

Best practices

• Never execute long-running (>1s) tasks synchronously inside API request handlers
• Configure BullMQ jobs with exponential retry backoff and dead-letter queues
• Set concurrency limits per worker to avoid exhausting CPU and memory limits
• Emit progress events so users can monitor long-running background tasks in UI

Common mistakes

• Ignoring "Never execute long-running (>1s) tasks synchronously inside API request handlers" and hoping a later prompt will clean it up.
• Copying the skill into .cursor/rules as always-on. That burns context and fights Cursor's load-on-match design.
• Running two overlapping skills that contradict each other in the same turn.
• Letting the agent skip tests or types because 'the skill is about architecture'.