2026-09-06 · 6 min read · Rahul Chitturi
- sql
- database
- reliability
A deployment can time out after a database accepted some migration statements but before the runner recorded success. Retrying blindly may duplicate data, fail on an existing object, or hold locks while application versions disagree about the schema.
The first task is to establish observed database state, not to infer rollback from the runner's error message.
Reconstruct what committed
Collect the migration identifier, database engine and version, statement log, transaction boundaries, schema metadata, lock history, and UTC timestamps. Some schema operations commit implicitly or cannot be rolled back on particular engines.
- Compare expected and actual tables, columns, indexes, and constraints
- Check whether data backfills wrote partial batches
- Identify concurrent application or migration activity
- Preserve the original failure before attempting recovery
Make retries explicit
Design each step so a retry either detects its completed postcondition or fails safely before changing more state. Separate long data backfills from short schema changes and checkpoint progress with stable keys.
Do not hide broad errors behind IF EXISTS clauses without verifying object definitions. An existing index with the wrong columns is not a successful equivalent result.
A Flashman workflow
Use the SQL formatter to review statements, diff to compare intended and actual definitions, timestamp converter to align deploy events, JSON formatter for sanitized runner metadata, and UUID generator for public fixture IDs.
Rehearse clean apply, interrupted apply, retry, mixed application versions, lock contention, rollback or forward repair, and backups in a production-like database.