spinny:~/writing $ vim trigger-dev-background-jobs-guide.md
1~2Most production applications need work that does not fit the request/response cycle: sending emails, processing uploads, running AI pipelines, syncing third-party data, generating reports. The traditional answer is a queue (Redis, SQS, RabbitMQ), a worker fleet, a scheduler, and a fragile pile of glue code that breaks on every deploy.3~4[Trigger.dev](https://trigger.dev) collapses that stack into a single TypeScript SDK. You write functions, you call them from anywhere, and the platform handles queuing, retries, observability, scheduling, and durable execution. Tasks run for as long as they need - no 10-second serverless timeout, no lost work on redeploys.5~6## Why Trigger.dev7~8The shift in 2026 is durable execution. Workflows must survive restarts, crashes, deploys, and rate limits. They must also stream progress back to the UI in real time and pause for human input. Trigger.dev was rebuilt around these requirements with version 3 and continues to expand its AI infrastructure surface.9~10```mermaid11graph LR12 App[Your App] -->|trigger| API[Trigger.dev API]13 API --> Queue[Durable Queue]14 Queue --> Worker[Worker Container]15 Worker -->|run task| Task[Your Task Code]16 Task -->|metadata| Realtime[Realtime Stream]17 Realtime --> UI[React UI]18 Worker --> Storage[Run State Store]19```20~21The model is simple: you define tasks as exports, the SDK picks them up, the platform schedules and runs them in isolated containers, and the run state is persisted so you can resume, retry, and observe.22~23## Getting Started24~25### Initialize a project26~27```bash28npx trigger.dev@latest login29npx trigger.dev@latest init30```31~32This creates a `trigger.config.ts` file and a `trigger/` directory with example tasks. The config file is the source of truth for your project: which directories contain tasks, build settings, lifecycle hooks, and runtime options.33~34```typescript35// trigger.config.ts36import { defineConfig } from "@trigger.dev/sdk";37~38export default defineConfig({39 project: "proj_abc123",40 runtime: "node",41 logLevel: "log",42 maxDuration: 3600,43 retries: {44 enabledInDev: true,45 default: {46 maxAttempts: 3,47 factor: 2,48 minTimeoutInMs: 1000,49 maxTimeoutInMs: 30_000,50 },51 },52 dirs: ["./trigger"],53});54```55~56### Run tasks locally57~58```bash59npx trigger.dev@latest dev60```61~62The dev server connects to the cloud, registers your tasks, and streams runs through your local code. You set breakpoints in your editor and hit them on real triggers - the same loop you would use in any normal Node.js project.63~64## Defining a Task65~66A task is an object exported with a unique `id` and a `run` function. The SDK introspects exports across `dirs` and registers them automatically.67~68```typescript69// trigger/send-welcome-email.ts70import { task } from "@trigger.dev/sdk";71import { Resend } from "resend";72~73const resend = new Resend(process.env.RESEND_API_KEY);74~75export const sendWelcomeEmail = task({76 id: "send-welcome-email",77 retry: {78 maxAttempts: 5,79 factor: 1.8,80 minTimeoutInMs: 500,81 maxTimeoutInMs: 30_000,82 },83 run: async (payload: { email: string; name: string }) => {84 const { data, error } = await resend.emails.send({85 from: "hello@spinny.dev",86 to: payload.email,87 subject: `Welcome, ${payload.name}`,88 html: `<p>Glad you are here, ${payload.name}.</p>`,89 });90~91 if (error) throw error;92 return { messageId: data?.id };93 },94});95```96~97Three things to notice:98~991. **No timeout in the run body.** The platform manages execution time through `maxDuration` in config, not the runtime.1002. **Throws are retries.** The SDK catches exceptions and re-runs with exponential backoff according to the `retry` policy.1013. **The return value is persisted.** Other tasks and your frontend can read `run.output` from anywhere.102~103## Triggering Tasks104~105You call a task from your backend, your API routes, or another task.106~107```typescript108import { sendWelcomeEmail } from "@/trigger/send-welcome-email";109~110const handle = await sendWelcomeEmail.trigger(111 { email: "user@example.com", name: "Alex" },112 {113 idempotencyKey: `welcome-${userId}`,114 concurrencyKey: `tenant-${tenantId}`,115 queue: { name: "emails", concurrencyLimit: 50 },116 delay: "30s",117 ttl: "10m",118 }119);120~121console.log(handle.id); // run_xyz - use this to track or display progress122```123~124The options unlock a lot of behavior in one call:125~126- **`idempotencyKey`** - if a run with the same key already exists, the SDK returns the existing handle instead of duplicating work.127- **`concurrencyKey`** - serializes runs sharing the key so you do not overrun a per-tenant rate limit.128- **`queue.concurrencyLimit`** - global cap for the queue across all keys.129- **`delay`** - schedules the run for a future time.130- **`ttl`** - if the run has not started by then, expire it automatically.131~132### Batch trigger133~134For fan-out workloads, `batchTrigger` accepts up to 500 items per call and creates one run per item.135~136```typescript137await sendWelcomeEmail.batchTrigger(138 newUsers.map((u) => ({139 payload: { email: u.email, name: u.name },140 options: { idempotencyKey: `welcome-${u.id}` },141 }))142);143```144~145## Scheduled Tasks146~147Cron jobs become first-class declarations. The schedule itself is a separate object you can attach to a task multiple times.148~149```typescript150// trigger/daily-digest.ts151import { schedules } from "@trigger.dev/sdk";152~153export const dailyDigest = schedules.task({154 id: "daily-digest",155 cron: "0 9 * * *",156 run: async (payload) => {157 console.log("Scheduled at:", payload.timestamp);158 console.log("Last run:", payload.lastTimestamp);159 console.log("Timezone:", payload.timezone);160 console.log("Next 5 runs:", payload.upcoming);161~162 await sendDigestForDate(payload.timestamp);163 },164});165```166~167For per-tenant schedules - say, one cron per customer - you create them dynamically through the management API.168~169```typescript170import { schedules } from "@trigger.dev/sdk";171~172await schedules.create({173 task: "daily-digest",174 cron: "0 9 * * *",175 timezone: "America/New_York",176 externalId: `customer_${customerId}`,177 deduplicationKey: `digest-${customerId}`,178});179```180~181The `deduplicationKey` makes the call idempotent: re-running the same code at deploy time does not stack duplicate schedules.182~183## Queues, Concurrency, and Idempotency184~185Three primitives cover most rate-limiting and ordering needs.186~187```mermaid188graph TB189 Trigger[trigger payload] --> IK{idempotencyKey<br/>seen?}190 IK -->|yes| Reuse[Return existing run]191 IK -->|no| CK[concurrencyKey bucket]192 CK --> Q[Queue with<br/>concurrencyLimit]193 Q -->|slot available| Run[Run task]194 Q -->|slots full| Wait[Wait in queue]195```196~197A common pattern: a queue per tenant with a small per-key concurrency to respect a vendor's rate limit, plus an idempotency key to make retries safe.198~199```typescript200await syncShopifyOrders.trigger(201 { shopId },202 {203 queue: { name: `shopify-${shopId}`, concurrencyLimit: 2 },204 concurrencyKey: shopId,205 idempotencyKey: `sync-${shopId}-${Date.now() / 60_000 | 0}`,206 }207);208```209~210## Waits and Long-Running Work211~212Tasks can pause without holding a connection or burning compute. The platform persists state and resumes the function when the wait completes.213~214```typescript215import { wait } from "@trigger.dev/sdk";216~217export const onboarding = task({218 id: "onboarding",219 run: async (payload: { userId: string }) => {220 await sendWelcomeEmail.triggerAndWait({ userId: payload.userId });221 await wait.for({ days: 1 });222 await sendTipsEmail.trigger({ userId: payload.userId });223 await wait.until({ date: oneWeekFromSignup(payload.userId) });224 await sendUpgradeOffer.trigger({ userId: payload.userId });225 },226});227```228~229`triggerAndWait` is the killer feature: it triggers a child task and suspends the parent until the child completes. You compose tasks like async functions, but the orchestration runs durably across days or weeks.230~231### Human-in-the-loop with `wait.forToken`232~233For approval flows and AI gates, `wait.forToken` pauses until your application calls back with a result.234~235```typescript236import { task, wait } from "@trigger.dev/sdk";237~238export const publishPost = task({239 id: "publish-post",240 run: async (payload: { draftId: string }) => {241 const draft = await generateAIContent(payload.draftId);242~243 const token = await wait.createToken({ timeout: "7d" });244 await notifyEditor({ draftId: draft.id, token: token.id });245~246 const decision = await wait.forToken<{ approved: boolean; notes?: string }>(247 token.id248 );249~250 if (decision.approved) {251 return await publish(draft);252 }253 return await applyFeedback(draft, decision.notes);254 },255});256```257~258The editor opens a UI, reviews the draft, clicks Approve, and your backend completes the token. The task picks up where it left off - even if hours or days have passed.259~260## Lifecycle Hooks261~262You can attach `init`, `onStart`, `onSuccess`, and `onFailure` to a task or globally in `trigger.config.ts`. Use these for tracing, error reporting, and shared setup.263~264```typescript265// trigger.config.ts266export default defineConfig({267 // ...268 init: async () => {269 Sentry.init({ dsn: process.env.SENTRY_DSN });270 },271 onFailure: async ({ error, ctx }) => {272 Sentry.captureException(error, {273 tags: { taskId: ctx.task.id, runId: ctx.run.id },274 });275 },276});277```278~279`init` runs once per worker container at boot, not per run, so it is the right place to set up clients and pools.280~281## Realtime in the Frontend282~283Trigger.dev publishes run state changes - status, metadata, output - over a streaming API. The React hooks subscribe to that stream and re-render automatically.284~285```typescript286// trigger/process-video.ts287import { task, metadata } from "@trigger.dev/sdk";288~289export const processVideo = task({290 id: "process-video",291 run: async (payload: { videoId: string }) => {292 metadata.set("stage", "transcoding");293 await transcode(payload.videoId);294~295 metadata.set("stage", "thumbnails");296 await generateThumbnails(payload.videoId);297~298 metadata.set("stage", "uploading");299 const url = await uploadToCDN(payload.videoId);300~301 return { url };302 },303});304```305~306```tsx307// components/VideoStatus.tsx308"use client";309import { useRealtimeRun } from "@trigger.dev/react-hooks";310import type { processVideo } from "@/trigger/process-video";311~312export function VideoStatus({313 runId,314 publicAccessToken,315}: {316 runId: string;317 publicAccessToken: string;318}) {319 const { run, error } = useRealtimeRun<typeof processVideo>(runId, {320 accessToken: publicAccessToken,321 });322~323 if (error) return <p>Error: {error.message}</p>;324 if (!run) return <p>Loading...</p>;325~326 return (327 <div>328 <p>Status: {run.status}</p>329 <p>Stage: {String(run.metadata?.stage ?? "queued")}</p>330 {run.output?.url && <video src={run.output.url} controls />}331 </div>332 );333}334```335~336You generate the public access token server-side, scoped to a specific run, and ship it to the client. The hook handles auth, reconnection, and incremental updates.337~338For trigger-and-subscribe in one shot:339~340```tsx341import { useRealtimeTaskTrigger } from "@trigger.dev/react-hooks";342~343const { submit, run, isLoading } = useRealtimeTaskTrigger<typeof processVideo>(344 "process-video",345 { accessToken: publicAccessToken }346);347~348<button onClick={() => submit({ videoId })} disabled={isLoading}>349 Process video350</button>;351```352~353## AI Agents and Streaming354~355Trigger.dev has become a popular runtime for AI agents because the same primitives - durable execution, retries, waits, real-time metadata, human-in-the-loop - are exactly what agents need. You stream tokens from a model provider into `metadata` while the run is happening, the frontend renders them live, and the run survives long-running tool calls without burning a serverless timeout.356~357```typescript358import { task, metadata } from "@trigger.dev/sdk";359import { streamText } from "ai";360import { anthropic } from "@ai-sdk/anthropic";361~362export const researchAgent = task({363 id: "research-agent",364 maxDuration: 1800,365 run: async (payload: { question: string }) => {366 const result = streamText({367 model: anthropic("claude-opus-4-7"),368 system: "You are a research assistant. Use the web.",369 prompt: payload.question,370 tools: { webSearch },371 });372~373 let fullText = "";374 for await (const chunk of result.textStream) {375 fullText += chunk;376 metadata.set("partial", fullText);377 }378~379 return { answer: fullText, usage: await result.usage };380 },381});382```383~384The frontend uses `useRealtimeRun` and reads `run.metadata.partial` to render the streaming response, the same way you would render a chat completion - except this one survives a full page reload.385~386## Deploying387~388Deploys compile your tasks into a versioned bundle, build a container, and atomically swap traffic. Old in-flight runs keep using the previous version.389~390```bash391npx trigger.dev@latest deploy --env prod392```393~394In CI you typically wire this into the same workflow that ships your app:395~396```yaml397# .github/workflows/deploy.yml398- name: Deploy Trigger.dev399 env:400 TRIGGER_ACCESS_TOKEN: ${{ secrets.TRIGGER_ACCESS_TOKEN }}401 run: npx trigger.dev@latest deploy --env prod402```403~404For preview environments, pass `--env preview --branch ${{ github.head_ref }}` and Trigger.dev creates an isolated environment per branch, mirroring how Vercel handles preview deployments.405~406## Self-Hosting vs Cloud407~408Trigger.dev is open source under the Apache 2.0 license. You can self-host on any container platform (Docker Compose, Kubernetes, Fly.io) or use the managed cloud at trigger.dev.409~410| Aspect | Cloud | Self-hosted |411|--------|-------|-------------|412| **Setup** | Sign up, run `init` | Run docker-compose or Helm chart |413| **Scaling** | Automatic | Your responsibility |414| **Pricing** | Per run + per compute | Infra cost only |415| **Compliance** | SOC 2 | Whatever your environment provides |416| **Best for** | Most teams | Strict data residency, custom infra |417~418The SDK and CLI are identical between modes - you change a profile flag and point at your own instance.419~420## Best Practices421~422### 1. Make payloads small and serializable423~424Pass IDs and references, not full objects. Pull the data inside the task. This keeps the queue small, payloads cheap to log, and lets you change the data source without re-triggering.425~426### 2. Idempotency keys on every external call427~428Combine `idempotencyKey` on the task trigger with idempotency keys at your vendor APIs (Stripe, OpenAI, etc.). Retries will be safe end to end.429~430### 3. Use `triggerAndWait` for orchestration, not `Promise.all` of triggers431~432A parent that calls `triggerAndWait` durably composes child tasks. A parent that triggers and resolves immediately loses observability of the chain.433~434### 4. Tag runs435~436Add `tags` to triggers (`tags: ["user:123", "feature:onboarding"]`) so you can filter the dashboard and the management API by business dimensions.437~438### 5. Keep `init` idempotent439~440It runs on every cold start. Avoid migrations or one-shot side effects there.441~442## Conclusion443~444Trigger.dev removes the categories of work that used to require building a job system from scratch. You write async TypeScript, you call it from anywhere, and the platform gives you durable execution, scheduling, queues, retries, real-time updates, and human-in-the-loop patterns out of the box.445~446The same surface that powers a nightly cron is the surface that powers a multi-step AI agent that streams to the frontend and pauses for review. That convergence is what makes the framework worth a serious look in 2026, whether you are running a SaaS that needs reliable background work or shipping AI features that outlive a serverless timeout.447~448> **Getting Started Checklist:**449>450> - [x] Sign up at trigger.dev or run the self-hosted Docker stack451> - [x] `npx trigger.dev@latest init` in your project452> - [x] Define your first task with `task({ id, run })`453> - [x] Trigger it from your API and watch the run in the dashboard454> - [x] Add `idempotencyKey` and `concurrencyKey` for production safety455> - [x] Wire `useRealtimeRun` into a status component456> - [x] Deploy with `trigger.dev deploy --env prod` from CI457~
NORMAL · trigger-dev-background-jobs-guide.md [readonly]457 lines · :q to close