Cron jobs vs background workers vs queues

Use cron for work tied to the clock, a queue for work triggered by an event, and a worker as the process that consumes that queue. Most real apps combine them.

Updated 5 min read

Learning objectives

After reading this article you will be able to:

  • Tell cron jobs, workers and queues apart and what each one solves
  • Pick the right tool for a scheduled, event-driven or heavy task
  • Build a minimal queue with a retrying worker in Node.js
On this page

Sooner or later some task is too slow to finish before the user gets a response: sending an email, resizing an image, calling a flaky third-party API. Other tasks have no request at all, like a nightly cleanup. Three tools cover this ground, and people often mix them up: cron jobs, background workers and queues.

They are not rivals. They answer different questions: when does work start, who does it, and where does it wait.

when it startswhere it waitswho runs itClockno queueCron jobruns the taskRequestan eventCrontriggerQueuejobs in RedisWorker 1Worker 2retry
Cron decides when, the queue holds the job until its turn and the worker runs it, with retries.

What is each one?#

A cron job starts a task at times set by a schedule. Nothing triggers it except the clock. See what a cron job is for the syntax.

A background worker is a long-running process that does work outside the request cycle. It is defined by where it runs, not by how it gets work. A worker can poll a database, listen to a queue or run a loop.

A queue is a buffer of jobs. A producer (your web app) adds a job and moves on. A worker takes it later. Queues usually add retries, delays, priorities and concurrency limits.

How do they differ side by side?#

Cron job Background worker Queue
Starts because of The clock Whatever you code A job was added
Reacts to events No Yes, if it listens Yes
Retries on failure No, unless you code it Only if you code it Usually built in
Survives a restart mid-task No No Pending jobs survive, since they are stored
Handles bursts Poorly Limited by one process Well: jobs wait their turn
Extra infrastructure None A process A broker (for example Redis)

The last two rows matter most. A queue decouples producing work from doing it, so a burst of 1,000 jobs does not need 1,000 simultaneous processes.

When should you use cron?#

Use cron when the trigger is time and the task is self-contained: nightly cleanups, daily reports, token refreshes, syncing data every hour. It needs no broker and no extra moving parts.

Cron falls short when a run fails and must be retried, when a run lasts longer than the interval, or when each run has to fan out into many pieces. An in-app scheduler also fires once per instance, so if you run two copies of your app, both fire the job.

When should you use a queue and a worker?#

Use a queue when something happened and work must follow: a user signed up (send a welcome email), a file was uploaded (generate thumbnails), a payment was confirmed (issue the receipt). The request returns quickly, and the worker finishes the job.

Choose a queue when you need any of these:

  • Retries with backoff for flaky dependencies.
  • Rate control, for example no more than 5 calls per second to an API.
  • Scaling the work, by starting more worker processes.
  • No lost work when a deploy or crash interrupts the process, because pending jobs live in the broker.

A job can run more than once, for example when a worker crashes mid-job (BullMQ describes at-least-once as its worst case). That is why jobs should be idempotent: running the same job twice must not charge a customer twice.

What does a minimal queue look like in Node.js?#

BullMQ is a Node.js queue library built on Redis. Install it with npm install bullmq. The producer adds jobs:

js
// producer.js (needs "type": "module" in package.json)
import { Queue } from "bullmq";

const connection = {
  host: process.env.REDIS_HOST,
  port: Number(process.env.REDIS_PORT),
  password: process.env.REDIS_PASSWORD,
};

export const emailQueue = new Queue("emails", {
  connection,
  defaultJobOptions: {
    attempts: 3,
    backoff: { type: "exponential", delay: 1000 },
  },
});

await emailQueue.add("welcome", { to: "ana@example.com" });

The worker is a separate process that consumes them:

js
// worker.js
import { Worker } from "bullmq";

const connection = {
  host: process.env.REDIS_HOST,
  port: Number(process.env.REDIS_PORT),
  password: process.env.REDIS_PASSWORD,
};

const worker = new Worker(
  "emails",
  async (job) => {
    // throw an error here and BullMQ retries the job
    console.log("sending welcome email to", job.data.to);
  },
  { connection, concurrency: 5 },
);

worker.on("failed", (job, err) => {
  console.error(`job ${job?.id} failed:`, err.message);
});

With attempts: 3 and an exponential backoff of 1,000 ms, a failing job is retried after about 1 s and 2 s before it is marked failed. concurrency: 5 lets one worker process five jobs at once.

Two Redis details matter in production, according to the BullMQ docs: set maxmemory-policy to noeviction so Redis never silently drops queue keys, and keep maxRetriesPerRequest as null on worker connections (the default for Worker).

Can you combine them?#

Yes, and most mature apps do. Cron is a good trigger, a queue is a good buffer, and a worker is the doer. BullMQ's job schedulers add jobs to a queue on a cron pattern:

js
await emailQueue.upsertJobScheduler(
  "daily-digest",
  { pattern: "0 0 8 * * *", tz: "America/New_York" },
  { name: "digest", data: {} },
);

The pattern has six fields, with seconds first, as in the BullMQ docs examples (this one means 08:00:00 every day). Now the schedule lives in Redis, not in one process's memory, and the work gets retries and concurrency control like any other job.

How do you choose?#

  • A short, harmless task on a clock: cron, inside your app or on the system.
  • Work caused by an event, or work that can fail and must retry: queue plus worker.
  • A scheduled task that is heavy or must not be lost: cron trigger feeding a queue.
  • A task that is simply a long loop, such as a consumer for a stream: a worker on its own.

Start with the smallest option. Moving from cron to a queue later is easy if the job logic lives in a function that does not care who calls it. For the Redis side, see what Redis is and how to host a Node.js application.

How to do this on Square Cloud#

Workers and schedulers need processes that stay up. On Square Cloud, containers are always on, with no sleep mode, so a worker or an in-app scheduler keeps running. With automatic restart on (the default for dashboard deploys), the app comes back by itself when the process exits with an error.

For a queue, deploy the worker as its own application next to your web app. The plan's RAM is a pool shared across your applications, with a minimum of 256 MB per background application and 512 MB per web application. Redis hosting is available from the Standard plan, in the same datacenter as your apps, with TLS connections and persistence. The dashboard hands you the connection URL and the database certificate. Put the connection data in environment variables. Since TLS is mandatory and uses its own certificate, add a tls option with the downloaded CA to the connection object in the examples above (for example tls: { ca: fs.readFileSync("ca.crt") }).

Deploy from the dashboard, the CLI or GitHub. The squarecloud.app configuration file is optional: the dashboard generates one on deploy, with runtime, memory and main file. See the options for hosting APIs to get started.

API hosting

APIs in always-on containers, with no cold starts

Learn more

FAQs

Is a queue the same thing as a worker?

No. The queue stores jobs waiting to be done. The worker is the process that takes jobs from it and runs them. You need both for a queue to be useful.

Can I use cron instead of a queue?

Only for work that depends on time. If a task starts because something happened, such as a payment or an upload, a queue reacts at once and retries failures, which cron does not.

Do queues need Redis?

Not necessarily: brokers like RabbitMQ and Amazon SQS work without Redis. Libraries such as BullMQ for Node.js, on the other hand, store their jobs in Redis and so need a Redis server.

How do I run a recurring job with a queue?

BullMQ has job schedulers that add a job to the queue on a cron pattern or fixed interval. The schedule lives in Redis, and any worker can process the job.

Next articleWebSocket vs webhook vs SSEWebSocket is a two-way channel, SSE a stream from server to browser and a webhook an HTTP notice between servers. Compare them and choose, with code examples.