flashman
← All posts

Cron duplicate runs after system clock jumps

Debug cron duplicates after NTP corrections or VM clock jumps by aligning scheduler logs, monotonic durations, idempotency keys, and platform-specific behavior.

2026-08-27 · 6 min read · Rahul Chitturi

  • cron
  • scheduling
  • operations

Schedulers make decisions using wall-clock time, but wall clocks can move. A VM resume, manual correction, hypervisor issue, or large time synchronization adjustment may repeat a minute or skip ahead.

Whether a cron implementation reruns, skips, or compensates for the affected schedule is platform-specific. The resulting duplicate can look like a normal second invocation unless logs preserve both scheduler and host timing evidence.

Build one UTC incident timeline

Collect scheduler dispatch logs, job start times, host time-service events, VM lifecycle events, and downstream side effects. Convert all wall-clock timestamps to UTC before deciding which component duplicated work.

  • Look for backward or forward clock steps near the scheduled minute
  • Distinguish scheduler dispatch IDs from application request IDs
  • Use monotonic elapsed durations when available
  • Check whether failover started a second scheduler instance

Make repeated dispatch safe

No cron expression can guarantee exactly-once side effects. Derive a stable execution key from the job name and intended schedule window, then enforce uniqueness in durable storage before sending email, charging accounts, or publishing reports.

Avoid random IDs as the only deduplication key because each duplicate dispatch generates a different value. Random correlation IDs remain useful for tracing individual attempts.

A Flashman workflow

Use the cron tool to preview intended runs, the timestamp converter to normalize evidence, the diff tool to compare scheduler configuration, and the UUID generator for synthetic trace IDs in tests.

Document the scheduler's clock-jump behavior and add a test that dispatches the same logical window twice. Correctness should come from idempotency, not an assumption that time always advances smoothly.

Try these tools