The problem a DLQ solves

Without a DLQ, a job that keeps failing has two bad outcomes:

1. Retry forever. It wastes resources, may spam a struggling endpoint, and clutters logs.
2. Give up silently. After N attempts it's dropped. The customer never got their invoice, and nobody knows.

A DLQ is a third option: after the last retry, move the job to a separate list with its payload, the error, and every attempt. Then alert someone.

What should end up there

Mostly failures that won't fix themselves:

• A 400 or 422 because the payload is invalid (a bug in the code that created the job)
• A 404 because the endpoint was renamed or deleted
• A 401 or 403 because credentials expired or a secret was rotated
• A handler that crashes on one specific record

Temporary failures (502, 503, timeouts, 429) usually succeed on retry and shouldn't reach the DLQ unless the outage lasts longer than the whole retry schedule.

Retry, then dead-letter

A typical policy:

attempt 1: immediately
attempt 2: +1 minute
attempt 3: +30 minutes
attempt 4: +3 hours
then: move to DLQ, alert

Spacing retries out (backoff) gives a struggling service time to recover. The total window, here a bit over three and a half hours, should be longer than a typical outage or deploy.

Some failures can skip straight to the DLQ. If a 400 means the payload is malformed, retrying it three more times won't help.

def should_retry(status):
    if status is None:          # timeout or connection error
        return True
    if status in (408, 425, 429) or status >= 500:
        return True
    return False                # 4xx: dead-letter now

What to store with each dead job

Enough to understand and replay it without digging through logs:

• Job ID and type
• Target URL and the full request (method, headers minus secrets, body)
• Every attempt: time, status code, response body (truncated), latency
• When it was created and when it was dead-lettered

Working the queue

A DLQ only helps if someone looks at it. A simple routine:

1. Alert on new entries, grouped so one bad deploy doesn't send a hundred messages.
2. Group by error. Fifty jobs with the same 404 have one cause.
3. Fix the cause: deploy the code fix, restore the endpoint, rotate the credential.
4. Replay the affected jobs. Because they were stored with their original payload, replay sends the same request again.
5. Discard jobs that no longer matter, deliberately, not by ignoring them.

Replay safely

Replaying sends a request that may have partly succeeded the first time, for example when the handler did its work but timed out before responding. Handlers should be idempotent, using a key that identifies the work, so a replay can't do it twice.

Replay in batches, too. Sending five hundred stored jobs at once to an endpoint that just recovered can knock it over again.

Watch the size

The number of jobs in the DLQ is a useful health signal. A steady zero is normal. A sudden jump usually means a deploy broke an endpoint. A slow, steady growth means a rare bug nobody has prioritised.

Do you need to build one?

If you run your own queue (SQS, RabbitMQ, a database table), you build the DLQ yourself or use the broker's built-in support. If you use a hosted scheduler for HTTP jobs, check whether it has one. delay.run retries jobs with backoff (by default after 1 minute, 30 minutes and 3 hours), keeps a dead letter queue with replay, and logs each attempt's request, response and status code.

Related reading

• How to send a reminder 24 hours after signup without a queue
• How to run a cron job on the last day of the month
• Make scheduled job handlers idempotent: safe to run twice

Option 1: a table plus a cron sweep

Store when the reminder is due, and run a job every few minutes that sends whatever is due.

CREATE TABLE reminders (
  id bigserial PRIMARY KEY,
  user_id bigint NOT NULL,
  due_at timestamptz NOT NULL,
  sent_at timestamptz
);
CREATE INDEX ON reminders (due_at) WHERE sent_at IS NULL;

At signup:

INSERT INTO reminders (user_id, due_at) VALUES ($1, now() + interval '24 hours');

Every five minutes:

UPDATE reminders SET sent_at = now()
WHERE id IN (
  SELECT id FROM reminders
  WHERE sent_at IS NULL AND due_at <= now()
  ORDER BY due_at
  LIMIT 200
  FOR UPDATE SKIP LOCKED
)
RETURNING id, user_id;

Send an email for each returned row.

Pros: no new infrastructure, easy to inspect, survives restarts. Cons: reminders go out up to five minutes late, and you need somewhere to run the sweep reliably. On serverless platforms, that "somewhere" is the hard part.

Option 2: a job queue with delayed jobs

Libraries like BullMQ (Node), Sidekiq (Ruby) and Celery (Python) support delayed jobs:

await reminderQueue.add("trial-reminder", { userId }, { delay: 24 * 60 * 60 * 1000 });

Pros: precise timing, built-in retries. Cons: you now run Redis or another broker, plus worker processes, and you have to monitor both. For one reminder email, that's a lot.

Option 3: ask a scheduler to call you back

The third approach moves the waiting outside your app. At signup, make one HTTP request to a scheduling service saying "call this URL in 24 hours with this payload". When the time comes, it calls your endpoint, and your handler sends the email.

// at signup (pseudo-code; check your scheduler's API for exact fields)
await fetch(SCHEDULER_URL, {
  method: "POST",
  headers: { Authorization: `Bearer ${TOKEN}`, "content-type": "application/json" },
  body: JSON.stringify({
    url: "https://app.example.com/hooks/trial-reminder",
    delay: "24h",
    body: { userId, key: `trial-reminder:${userId}` },
  }),
});

Your endpoint:

app.post("/hooks/trial-reminder", verifySignature, async (req, res) => {
  const { userId, key } = req.body;
  if (await alreadyProcessed(key)) return res.json({ skipped: true });
  const user = await db.users.find(userId);
  if (user && !user.hasCompletedSetup) await sendReminder(user);
  await markProcessed(key);
  res.json({ ok: true });
});

Pros: works on serverless platforms, no broker or workers, the scheduler handles retries. Cons: depends on an external service, and your endpoint must be reachable from the internet and verify requests.

Check conditions at send time, not at schedule time

Notice the !user.hasCompletedSetup check in the handler. Whatever option you choose, decide whether to send when the reminder fires, not when it was scheduled. In 24 hours, the user may have finished setup, deleted their account, or unsubscribed. Scheduling the check rather than the email avoids awkward messages.

Handle duplicates and failures

• Duplicates: retries happen. Use a key like trial-reminder:<userId> and skip if already processed.
• Failures: if the email provider is down, return an error so the job is retried later. Return success only after the send succeeded.
• Cancellations: if the user deletes their account, either cancel the scheduled job or make the handler check that the user still exists (simpler, and works with every option).

Which one to choose

A hosted option

delay.run supports exactly this pattern: one API call schedules a delayed job (up to 24 hours ahead) that calls your URL, retries with backoff if your endpoint fails, and logs each attempt. Requests are HMAC-signed, so your handler can verify them.

Related reading

• How to run a cron job on the last day of the month
• Make scheduled job handlers idempotent: safe to run twice
• How to verify HMAC-SHA256 signed webhooks in Node and Python
• Heartbeat checks: how to know when a scheduled job didn't run
• How to log cron job output so you can debug it later
• Cron vs systemd timers: which should run your scheduled jobs?
• Dead letter queues for HTTP jobs: what they are and how to use one

Why 0 23 31 * * doesn't work

That expression runs at 23:00 on the 31st, which only exists in seven months. In February, April, June, September and November it never runs. Listing 28-31 runs it up to four times in long months. Neither is what you want.

Option 1: run daily and check if tomorrow is the 1st

The most portable approach. Schedule the job for days 28 to 31 and let a shell test decide:

0 23 28-31 * * [ "$(date -d tomorrow +\%d)" = "01" ] && /usr/local/bin/month-end.sh

Notes:

• % has a special meaning in crontab lines (it becomes a newline), so it must be escaped as \%.
• date -d tomorrow is GNU date (Linux). On macOS or BSD, use date -v+1d +%d.
• Restricting to 28-31 means the check only runs four times a month at most.

If the line gets long, move the logic into the script itself:

#!/usr/bin/env bash
# month-end.sh
[ "$(date -d tomorrow +%d)" = "01" ] || exit 0
echo "Running month-end tasks for $(date +%Y-%m)"
# ... actual work

0 23 28-31 * * /usr/local/bin/month-end.sh

Option 2: run on the 1st and process the previous month

Often the simplest answer is to change the question. Instead of running at the end of a month, run just after it:

5 0 1 * * /usr/local/bin/report-previous-month.sh

Inside the script, calculate the previous month:

PREV=$(date -d "$(date +%Y-%m-01) -1 day" +%Y-%m)
echo "Building report for $PREV"

This has a real advantage: the month is actually over, so late-arriving data from the final evening is included. For reports and invoices, it's usually the better design.

Option 3: the L extension

Some cron implementations (Quartz in Java, some cloud schedulers and libraries) support L in the day-of-month field:

0 0 23 L * ?    # Quartz: 23:00 on the last day of every month

Standard Vixie cron and cronie on Linux don't support it. Check your scheduler's docs before relying on it. If your code may move between environments, options 1 and 2 are safer.

Option 4: systemd timers

If you use systemd timers instead of cron, OnCalendar supports a last-day syntax with ~:

[Timer]
OnCalendar=*-*~01 23:00:00
Persistent=true

~01 means "the first day counting backwards from the end", which is the last day. Persistent=true runs a missed job after the machine boots.

Time zones matter more than usual

"The last day of the month" depends on time zone. At 23:00 in UTC, it's already the 1st in parts of Asia and still the evening of the last day in the Americas. Decide which time zone defines the month for your business, and make sure the server or scheduler uses it. Many cron daemons accept CRON_TZ= at the top of a crontab:

CRON_TZ=Europe/Berlin
0 23 28-31 * * /usr/local/bin/month-end.sh

Make it safe to run twice

Month-end jobs are high-stakes, and a server restart or a retried run can trigger them again. Record each run by period, e.g. month-end:2026-11, and skip if it's already done. That turns an accidental double run into a no-op.

Know when it didn't run

A monthly job that silently fails won't be noticed for another month. Log the output, and alert on failure or on a missing run. A hosted scheduler like delay.run logs every run with status code, response body and latency, and alerts by email, Slack or Discord when a cron fails several times in a row.

Quick reference

Related reading

• Make scheduled job handlers idempotent: safe to run twice
• How to verify HMAC-SHA256 signed webhooks in Node and Python
• Heartbeat checks: how to know when a scheduled job didn't run
• How to log cron job output so you can debug it later
• Cron vs systemd timers: which should run your scheduled jobs?
• Dead letter queues for HTTP jobs: what they are and how to use one

The five fields

Standard (Unix) cron has five fields, separated by spaces:

┌──────── minute        (0–59)
│ ┌────── hour          (0–23)
│ │ ┌──── day of month  (1–31)
│ │ │ ┌── month         (1–12)
│ │ │ │ ┌ day of week   (0–6, Sunday = 0; 7 is also Sunday on most systems)
│ │ │ │ │
* * * * *

Four operators cover almost everything:

• * — every value
• , — a list: 1,15 means the 1st and the 15th
• - — a range: 9-17 means 9 through 17
• / — a step: */5 means every 5th value, starting from the lowest

The cheat sheet

Three mistakes that look right

1. Forgetting the minute in hourly schedules. * */2 * * * does not mean "every two hours". It means every minute of every second hour, which is 60 runs in a row. Pin the minute: 0 */2 * * *.

2. Steps that don't divide evenly. Steps restart at each boundary. */7 in the minute field gives :00, :07, :14 … :56, and then :00 again four minutes later. For an even 7-minute interval you need something other than cron. Steps that divide 60 (5, 10, 15, 20, 30) are safe for minutes; steps that divide 24 (2, 3, 4, 6, 8, 12) are safe for hours.

3. Day-of-month OR day-of-week. When both fields are restricted, standard cron runs the job if either matches. 0 9 13 * 5 runs on every 13th and on every Friday, not only on Friday the 13th. If you need "Friday the 13th", run on Fridays and check the date in your script.

Offsets: every 5 minutes, but not on the 0

If many jobs run at */5, they all start at :00, :05, :10 at the same moment. Spread them with a range and step:

2-59/5 * * * *    # :02, :07, :12 … :57

The same trick works for hours: 15 1-23/2 * * * runs at 01:15, 03:15, 05:15 and so on.

Only during working hours

Combine a step with an hour range and a weekday range:

*/5 9-17 * * 1-5   # every 5 minutes, 09:00–17:55, Monday to Friday

Note that 9-17 includes the whole 17:00 hour. If the last run should be at 17:00, add a separate line 0 18 * * 1-5 or use 9-16 plus 0 17.

“Cron every 5 minutes” with 6 fields

Different tools extend cron with extra fields, and copying between them is where most errors come from.

Spring (@Scheduled(cron = ...)) adds a seconds field at the front:

0 */5 * * * *       # second 0 of every 5th minute

Quartz also starts with seconds, and requires ? in either day-of-month or day-of-week:

0 0/5 * * * ?

AWS EventBridge / CloudWatch uses six fields with a year at the end, no seconds, and also needs ? in one of the day fields:

cron(0/5 * * * ? *)

So the same schedule has three different spellings. Before pasting an expression, check how many fields the tool expects and whether the first field is seconds or minutes.

What cron cannot do

• Sub-minute schedules. Standard cron's smallest unit is one minute. "Every 10 seconds" needs a loop, a systemd timer or a 6-field scheduler.
• "Every N days" across months. 0 0 */3 * * restarts on the 1st of each month, so the gap between the 31st (or 28th) and the 1st is shorter.
• Time zones per job in classic cron. The whole crontab uses the server's time zone unless your implementation supports CRON_TZ.

Testing an expression

Before you trust a schedule, list the next few run times. Many online cron explainers do this, and in Python you can use the croniter package:

from croniter import croniter
from datetime import datetime

it = croniter("2-59/5 9-17 * * 1-5", datetime.now())
for _ in range(5):
    print(it.get_next(datetime))

Seeing the next five timestamps catches most of the mistakes above in seconds.

Where the expression runs matters too

A correct expression is only half the job; something still has to run it reliably. On a single server that is the cron daemon, with all the quirks of its environment. If your task is an HTTP endpoint, a hosted scheduler such as delay.run takes the same standard five-field syntax, down to once a minute, and records the response of each run.

Keep this page handy, and when a schedule looks odd, read it field by field from left to right. The mistake is almost always in one of the three traps above.

Related reading

• How to log cron job output so you can debug it later
• How to verify HMAC-SHA256 signed webhooks in Node and Python
• How to send a reminder 24 hours after signup without a queue
• Cron vs systemd timers: which should run your scheduled jobs?