Can You Migrate an Old Database Without Taking the Business Offline?

Most proposals for a migration maintenance window start from an unstated assumption: the old structure has to stop before the new one can start. For a lot of schema changes and a fair number of platform moves, that isn't true. Questioning that assumption before you sign off on a Saturday night outage is the most useful thing an operations lead can do.

This article covers the two techniques that avoid a hard stop, what each one costs, how each one fails, and the questions that show whether a vendor's plan is sound.

What is the expand-contract pattern?

Expand-contract (also called parallel change) never removes anything until the replacement has been proven. The common description has 4 phases:

1. Expand. Add the new column or table next to the old one. Both exist.
2. Dual-write. The application writes to both. Reads still come from the old structure.
3. Backfill. A background job copies historical data into the new structure.
4. Contract. Reads switch to the new structure. Writes to the old one stop, and the old one is dropped.

Some writers fold dual-write and backfill into a single "migrate" stage. The rule is the same either way: deprecated schema is removed only after all traffic has moved.

Here is a concrete example. Say you are replacing a free-text phone column with a normalized phone_e164 column in Postgres. The expand step is additive only:

-- Phase 1: expand. Nullable, no default, nothing removed.
ALTER TABLE customers ADD COLUMN phone_e164 text;

-- Index without locking writes
CREATE INDEX CONCURRENTLY idx_customers_phone_e164
  ON customers (phone_e164);

Postgres has CREATE INDEX CONCURRENTLY, and MySQL's default online DDL (or a tool like gh-ost) does the equivalent. If a vendor's plan builds indexes on a large table with a plain blocking statement, the plan has a problem.

The point that matters for a decision maker is that every phase before contract can be undone. The old column is still there and still being written. If the new path misbehaves, you flip reads back. The only step you can't reverse is the final drop, so a good plan schedules it last and puts it off until well after verification.

How do you dual-write without corrupting data?

Most of the risk in dual-writing comes from three failure modes:

• Partial writes. The application updates the old column, then fails before updating the new one. The fix is to write both in the same transaction so they commit or fail together.
• Writers you forgot about. Legacy systems almost always have more than one writer: the main app, a nightly import, an admin tool, a report script that "fixes" records. If only the main app dual-writes, the other writers leave the columns out of sync. A database trigger that fills in the new column catches every writer, at the cost of logic that lives in the database and is easy to overlook later.
• Transformation drift. The application and the backfill job transform data with slightly different code, so old rows and new rows disagree. Use one function for both.

Before any read switches over, run a verification query and require it to return zero:

SELECT count(*)
FROM customers
WHERE phone IS NOT NULL
  AND phone_e164 IS DISTINCT FROM normalize_phone(phone);

Run it again after reads have moved and before the contract step. A vendor who can't show you this kind of check is asking you to trust the migration rather than measure it.

This matters most for money. On commission and payout systems I hold to strict uptime targets (the ones I ran at Plondo Network held 99.9%). There, I'd also reconcile totals by account before and after the change, not only row by row. If your data is already a ledger, the migration is much easier to verify. The reasoning is in designing a commission engine that survives refunds and chargebacks.

Will the backfill slow down the live system?

It will if it's written as a single UPDATE across the whole table. That statement holds locks, bloats the transaction log, and competes with customer traffic for I/O. The standard approach is a throttled, resumable background job, commonly processed in batches of 5,000 to 10,000 rows with a short pause between batches to leave headroom for production queries.

-- Phase 3: one batch. The job records last_id, sleeps, repeats.
WITH batch AS (
  SELECT id FROM customers
  WHERE id > :last_id AND phone_e164 IS NULL
  ORDER BY id
  LIMIT 5000
)
UPDATE customers c
SET phone_e164 = normalize_phone(c.phone)
FROM batch
WHERE c.id = batch.id
RETURNING c.id;

How long it takes depends on row count, row width, index count, replica lag, and how hard you're willing to load the primary. That's why I won't quote a duration without seeing the table. The batch size is the setting you adjust: shrink it if latency rises, grow it if the database is idle at 2 a.m. Watch tail latency while it runs, not the average. The average can look fine while a small slice of requests stalls, as covered in why p95 and p99 latency matter.

When is a full parallel run worth the extra weeks?

Expand-contract has one hard requirement: you control the code that writes to the database. Many legacy systems fail that test. Examples include a packaged accounting or permitting product, a vendor-hosted system with no source access, or a move to a completely different data model. In those cases you run the old and new systems side by side. Both receive the same inputs, and you compare their outputs until you trust the new one.

It's slower and more expensive, and it still catches problems nothing else will. One logistics company ran parallel systems for 3 months and found 47 data quality issues after its official go-live date, with zero customer impact. In other words, the cutover was a period rather than a date.

Why the extra weeks usually pay for themselves

The failure numbers are real, though often garbled. The widely repeated claim that 83% of data migrations fail overstates the source. The traceable 2007 Bloor Research paper found that more than 60% overran on time, budget, or both. Another compilation puts database migrations at a 79% success rate and legacy and custom application migrations at 61%, with schema incompatibility and performance degradation as the main causes.

Now compare that with outage costs. Small-business estimates run $137 to $427 per minute, or $24,660 to $76,860 for a 3-hour outage. ITIC puts micro-SMBs under 25 employees at roughly $1,670 per minute. Running two structures for a few extra weeks costs hosting plus engineering time, and that varies with the system. For anything that takes payments, holds public records, or runs payroll, a single failed window usually costs more than those weeks.

A planned window is still the honest answer sometimes. Examples are a small internal database that restores in minutes, or a system nobody uses after 6 p.m. The test is whether the window is a deliberate choice or just the default.

What should you ask a vendor before approving a maintenance window?

Use this checklist on any migration plan, whether it comes from an outside firm in Monroe or Ruston or from your own team:

1. Why is downtime required? Ask for the specific step that can't run online. "That's how we always do it" doesn't answer the question.
2. Is every step before the final drop reversible? Ask how you get back to the previous state at each phase, and how long that takes.
3. How are reads switched? Prefer a feature flag or config switch with instant rollback over a deploy.
4. Who writes to these tables? Ask for the full list of writers, including imports, integrations, and scripts.
5. How is the backfill batched and throttled? Ask for batch size, pause, resumability, and what they monitor while it runs.
6. How will indexes and constraints be added? Look for concurrent or online methods on large tables.
7. What does verification look like? Ask for row-level mismatch queries, aggregate reconciliation for money, and who signs off.
8. When does the old structure get dropped? It should be days or weeks after cutover, not the same night.
9. If this is a platform replacement, why not a parallel run? Ask for the cost of a parallel run as an option, even if you decline it.
10. What's the abort criterion? Ask for the specific signal that stops the migration, and who has the authority to call it.

If a plan can't answer 2, 7, and 10, don't approve it, whatever the window length. This is the kind of review a fractional CTO does before the contract is signed rather than after the outage.

Where to start

• Inventory the system: tables involved, row counts, every process that writes to them, and the hours customers or staff depend on it.
• Decide whether you control the application code. If you do, expand-contract is likely on the table. If you don't, price a parallel run.
• Put your downtime cost per hour in writing, using your own revenue and staffing, so the extra weeks can be compared against it.
• Send the vendor the checklist above and ask for written answers before scheduling anything.
• If you're a small business in Ouachita, Morehouse, or Franklin parish, the LSBDC at ULM can help with the business planning around a systems change.

If you want a second opinion on a migration plan or help running one, email hello@josephleporati.com.