Zero-Downtime Database Migrations in Practice
· 6 min read
Use expand-and-contract delivery to change schemas safely under real traffic.
Deployments overlap
A schema migration rarely runs in isolation. Old and new application instances may serve traffic simultaneously, workers may process delayed jobs, and mobile clients may remain active for months. A safe migration assumes this overlap and keeps each intermediate state compatible.
The expand-and-contract pattern separates a risky replacement into reversible stages. First expand the schema or interface, then migrate behavior and data, observe the result, and only later remove the old path. The extra steps buy control at the moment it matters.
Expand without changing meaning
Add new nullable columns, tables, indexes, or endpoints in a way existing code can ignore. Avoid defaults or constraints that rewrite a large table under lock without understanding database behavior. Build large indexes concurrently when the engine supports it and monitor replication lag and lock duration.
Deploy code that can write both representations or populate the new model for newly created data. Dual writes introduce consistency risk, so keep the transition bounded, instrument divergence, and prefer a single transaction when both records share a database.
- Measure table size and lock behavior first
- Make migration commands restartable
- Throttle backfills under production load
- Record progress with stable checkpoints
Backfill as an operation
A production backfill is a workload, not a one-off script. Process deterministic batches, persist checkpoints, limit concurrency, and expose progress and failures. The job should be safe to stop and resume without duplicating effects.
Validate the new representation continuously. Compare counts, checksums, invariants, and sampled records rather than waiting until the end. If the migration transforms meaning, encode the expected mapping in executable checks reviewed by domain owners.
Move reads deliberately
Once new writes and historical data are ready, shift reads behind a feature flag or controlled rollout. Shadow reads can compare old and new results without changing the user response. Segment errors and latency by path so the decision to advance is based on evidence.
Rollback during this stage should usually mean switching reads back, not reversing the schema. Destructive rollback scripts can make a recoverable deployment much worse. Preserve the expanded state until confidence is high.
Contract only after evidence
Stop writing the old representation, wait for overlapping application versions and queued work to clear, then remove unused code. Confirm through telemetry that the old field or table is no longer read before dropping it in a separate deployment.
Zero downtime is not the absence of risk. It is a delivery shape that makes risk observable, limits blast radius, and preserves a safe decision at every stage.
Published on June 5, 2024 by Berktug Berke Ates.