2026-09-06 · 8 min read
- cron
- scheduling
- reliability
A calendar schedule states nominal run times but often does not state what should happen when the scheduler, control plane, queue, or workers are unavailable at those times. The platform's misfire or catch-up policy fills that gap.
Skipping, coalescing, and replaying every missed interval are all valid for different workloads. The correct choice follows the business meaning and recovery capacity, not merely the scheduler's default.
Define the logical run
Represent each run by the schedule identity and nominal interval, independently from dispatch attempts. That logical key supports duplicate suppression, audit history, retry limits, and controlled manual backfill.
- Record nominal time, dispatch time, start time, completion, and outcome.
- Keep all machine event timestamps in UTC.
- State timezone and daylight-saving rules for calendar interpretation.
- Distinguish retries from separate missed intervals.
Choose skip, coalesce, or replay
Skip stale work whose result has no remaining value, coalesce when one current run can restore state, and replay intervals only when each period carries required independent effects. Set a maximum useful age or starting deadline for every policy.
For long outages, bound the number and rate of recovered runs. Uncontrolled replay can overload databases, APIs, and queues immediately after they recover.
Preserve correctness during recovery
Use durable idempotency keys and explicit ordering where effects depend on sequence. Apply concurrency limits, queue backpressure, deadlines, and retry jitter after dispatch.
- Make duplicate delivery harmless or detect it before side effects.
- Prevent stale runs from overwriting newer state.
- Provide pause, inspect, resume, and cancel controls.
- Require authorization and an audit trail for manual backfills.
Exercise outage timelines
Use Flashman's cron helper to list nominal intervals, timestamp converter to align events, UUID generator for synthetic run IDs, JSON formatter for scheduler messages, and diff to review policy changes.
Test short and long outages, restarts on exact boundaries, duplicate dispatch, daylight-saving gaps and overlaps, queue saturation, worker loss, dependency rate limits, manual recovery, and history retention.