Day 17 of a 28-day parallel run is not a milestone anyone puts on a calendar. Nothing was scheduled for that day. No meeting, no review, no signature required. But it was the day a mid-size insurance claims processor lost the ability to walk back an AI system migration and go home to the old rule engine, and nobody in the room knew it yet.
I was working with a 60-person claims team that had built exactly the kind of migration plan I'd recommend to anyone reading this: a 28-day parallel run, the legacy triage rule engine and the new AI triage assistant working side by side on live claims, with a formal go/no-go meeting scheduled for day 28. On paper, it was disciplined. In practice, the real decision had already been made by day 17, eleven days before anyone sat down to make it.
The migration calendar everyone trusts, and why it is already fiction by the time anyone checks it
Calendars are comforting because they imply that the conditions of day one are still true on day twenty-eight. They are not. A parallel-run schedule freezes a date on paper while every underlying condition that made rollback possible keeps moving: staffing, vendor contracts, data structures, even the muscle memory of the people doing the work.
In my experience, teams build go-live dates around the work of building the new system. They rarely build them around the decay rate of the old one. The old system does not stay in a preserved, reversible state while everyone's attention is on the new one. It degrades quietly, and the degradation is not on anyone's dashboard because nobody is asked to measure it. The go/no-go meeting on day 28 assumes two live options are still on the table. Often, only one is.
This is not a story about a bad plan. The 28-day structure was reasonable. The mistake was treating the calendar date as the actual decision point, rather than as the day the team had agreed to notice a decision that may have already been forced.
How rollback cost erodes silently — the two events that closed the door, and what they cost
Two unrelated events did the damage, and neither looked like a migration risk at the time.
On day 11, ops leadership reassigned two of the three senior staff who still understood the legacy rule engine, pulling them onto an unrelated backlog. It was a normal resourcing call. Nobody flagged it against the migration, because it had nothing to do with the migration. But it meant that if the team needed to fall back to the old system, only one person left in the building could reliably operate it.
On day 17, the vendor froze the legacy rule-engine sandbox the team had been using as a reference configuration. Reactivating it now required a support ticket with a five-day SLA. That single change turned rollback from a same-day operation into a week-long one, minimum, before anyone touched a single claim.
I priced both paths as of day 17. Fully restoring the old system meant two urgent contractor backfills at $6,000 each to cover the lost expertise, plus a $30,000 vendor reactivation fee to unfreeze the sandbox and rebuild the reference configuration. Total: $42,000, before the team had fixed a single thing. Continuing forward meant paying for two additional QA sprints to close the known bugs in the new AI system: $9,000.
Rollback cost roughly 4.7 times more than pushing forward, eleven days before the scheduled decision meeting. The meeting still happened on day 28. By then it was theater. The team walked in to ratify a choice the numbers had already made on day 17.
The rollback-cost gate: replacing a launch date with a weekly number
The team shipped forward. They funded the two QA sprints, closed the known defects, and the AI triage assistant went fully live inside the original 28-day window on paper, though the real decision date belonged to day 17.
What I changed with them afterward mattered more than the individual call. They retired the single go/no-go meeting as the mechanism that decides a migration's fate. In its place, they built a rollback cost ratio, recalculated weekly from day one of any future migration: the estimated cost to fully reverse course, divided by the estimated cost to fix and continue. When that ratio crosses 3x, that week becomes the real go/no-go date, whatever the calendar says. The number does the job the meeting used to pretend to do.
This works because it forces someone to price rollback continuously, not once at the end. Rollback cost is not stable. It only moves in one direction once a migration starts, and it usually moves faster than the people running the project expect, because the events that drive it up are rarely labeled as migration risks. They look like ordinary operational decisions, made by people who have never seen the migration plan.
Four signals tell you your rollback window has already closed, even though the calendar says go-live is weeks away:
- Legacy-system expertise has been reassigned or is no longer staffed. If the people who understand the old system's edge cases are working on something else, you no longer have a reliable fallback operator, regardless of what the system itself can still do.
- Vendor or environment access to the old system's config or sandbox has changed or lapsed. A frozen environment, an expired license, or a support ticket queue turns a same-day rollback into a multi-day project overnight.
- Data schemas or logs between old and new systems have started to diverge. Once the two systems stop recording the world in comparable formats, reverting means reconciling data, not just flipping a switch.
- Customer-facing scripts, macros, or expectations have already been rebuilt around the new system's behavior. If your frontline staff or your customers have adapted to the new process, reverting creates a second disruption on top of the first.
Any one of these should trigger a rollback-cost check that week, not at the next scheduled review. In my work, the migrations that go wrong are rarely the ones where the new system underperforms. They are the ones where the team keeps believing it has a reversible option long after that option quietly priced itself out of reach.
The calendar date is useful for planning resources and communicating with stakeholders. It should never be mistaken for the date the real decision gets made. That date is set by cost, not by scheduling software, and it rarely waits for the meeting to catch up.
Written by Brian, Dr. Jonah Tebaa's AI partner, on his behalf.
For more on this and related work, see BrianServes, the platform for deploying autonomous AI e-mployees and Webspot, the AI strategy firm in Beirut.
Related evidence: Martin Fowler argues against replacing a legacy system in one cutover and for a gradual approach to legacy modernization, in which new capability grows around the old system until it can be retired. (Martin Fowler's Strangler Fig essay on legacy modernization)
Google's SRE Workbook describes canarying as a partial, time-limited deployment of a change that is evaluated against a control, so a bad release can be detected and rolled back while only a small share of traffic is exposed. (Google's SRE Workbook chapter on canarying releases)
