Nineteen points. That is how far approval accuracy fell on an AI returns-processing pilot, in the two weeks after the team that built it moved on to other work. Nobody touched the model. Not one weight changed, not one threshold, not one line of the prompt. What changed was who came in every morning to run it.
The case below is a composite, with details changed, but the pattern is one I have watched play out with enough regularity that it deserves a name. I call the composite company Meridian Retail, a mid-size retail chain that built an AI assistant to process customer returns and decide, case by case, when a refund should be approved automatically and when it should route to a human.
The Nineteen-Point Drop
Meridian's pilot team was three people: a data scientist, a product manager, and an operations lead on loan from the store network. For eight weeks they built and ran the returns assistant together, sitting close enough to hear each other's frustrations in real time. By the end of that period, the system was approving refund decisions with 91 percent accuracy against what a trained human reviewer would have decided.
That number was real, and the pilot was, by any reasonable measure, a success. At week eight, the pilot team handed the system to a four-person operations team who would run it permanently. The formal documentation covered five of the roughly thirty-five small judgment calls the pilot team had learned to make along the way. The other thirty were never written down, because nobody thought to write them down. They were just how the pilot team did the job.
Two weeks later, accuracy had fallen to 72 percent. Nineteen points, gone, with the model exactly as it had been on day one of the pilot. The operations team had not made an obvious mistake. They had simply run the system as documented, and the documentation was incomplete in ways nobody had flagged as a gap.
What Actually Left With the Pilot Team
Here is what some of those thirty undocumented behaviors looked like in practice. Every Monday, the inventory sync ran a few hours behind because of how the weekend batch job queued up, and the pilot team had learned to manually re-trigger it rather than let the assistant work from a stale count. Nobody wrote a rule that said "re-trigger the sync on Mondays," because the pilot team just did it, the way you might glance both directions before crossing a street you have walked a hundred times.
The pilot team had also noticed that refund requests containing phrases like "still deciding" in the customer's own words were a strong signal the case needed a human, even though nothing in the written rules said so. And they knew, from watching the system daily, that it consistently under-priced return credit on two or three specific product categories, so they nudged those cases by hand before approving them. None of this was hidden. It was simply lived knowledge, the kind that accumulates when three people spend eight weeks inside a system together.
I want to be clear that this is not a failure on the pilot team's part. It is close to inevitable. When you build something and operate it daily, your brain compresses hundreds of small observations into instinct, and instinct rarely announces itself as a rule worth writing down. In my work, I have found that a pilot's success is partly a property of the system and partly a property of the specific people running it during the pilot. A handoff that transfers only the system, and not the accumulated judgment of the people who ran it, is not actually a complete handoff. It only looks like one.
I explore this gap between a working system and a documented one at greater length in my book, Applied AI for Future-Ready Organizations, but the short version is the one that matters here: the riskiest day in most AI deployments is not launch day. It is the day the builders stop being the operators.
The Handoff Package
What Meridian needed, and what I now recommend to any team approaching the end of a pilot, is a handoff package with six specific components. This is not a governance framework and it is not about who has authority to approve what. It is a staffing and documentation checklist, built to move judgment out of people's heads and onto paper before those people move to their next project.
- The exception log, not just the exception count. Every case where a human overrode the AI's output should have the specific reason recorded at the time, in plain language, not folded into an aggregate percentage that hides the pattern behind it.
- The workaround list. Any manual step the pilot team performs "because we know to," such as re-running a job, double-checking a category, or ignoring a particular alert type, needs to become a numbered operating step, even if it feels too small or too obvious to write down.
- The named escalation path, not a ticket queue. For each class of failure, name the specific person and channel who owns it for the first ninety days after handoff. An amorphous "AI team" on a ticket form is not a handoff. It is a hope.
- The drift signal and its baseline number. Identify the one or two metrics the pilot team was watching informally, and hand over the actual number range they treated as normal, so the operations team has a figure to alarm on instead of a vague instruction to keep an eye on things.
- The freeze list and the adjustable list, named explicitly. Which prompts, thresholds, or fields are calibrated to a specific edge case and should not be casually edited, and which are safe to tune, such as tone or greeting copy. Without this distinction, everything looks equally editable, and calibrated fixes get undone by people who have no way of knowing they were calibrated at all.
- Who fixes a wrong output today. Not a policy document. An operational fact: when the system gives a wrong answer mid-shift, on an ordinary Thursday afternoon, which named person on the live team has the authority and the system access to correct it right now, without waiting for the original pilot team to be looped back in.
The One-Week Test
There is a simple way to check whether a handoff is actually complete before the pilot team disperses to other projects: have the operations team run the system solo for one full week while the pilot team is genuinely unreachable, not just theoretically available. If the operations team hits a judgment call they cannot resolve from the documentation alone, that gap gets written down and closed before anyone calls the handoff finished. It is a cheap test, and it catches almost everything that a five-item document review will miss.
None of this requires new technology or a bigger budget. It requires treating the moment the pilot team walks away with the same seriousness most organizations reserve for the moment the system goes live. Both moments carry risk. Only one of them usually gets a checklist.
Frequently asked questions
What exactly goes into a handoff package for an AI pilot?
A complete handoff package has six parts: an exception log that records the reason behind every human override, a written workaround list, a named escalation contact for each failure type, a documented baseline for the metrics the pilot team watched informally, a clear freeze list versus adjustable list, and a named person with the authority to fix a wrong output on any given shift.
How long should the pilot team stay reachable after handing off?
In my work, I recommend building in a short overlap window, roughly two to four weeks, where the pilot team is available for questions but not doing the daily work. The real test, though, is whether the operations team can run the system for a full week with the pilot team genuinely unreachable. If they cannot, the handoff was not finished.
What is an exception log, and how is it different from an error count?
An error count tells you how often a human overrode the AI. An exception log tells you why, recorded in plain language at the moment it happened. The count hides patterns; the log reveals them. I have seen teams track the number for months without ever noticing that the same three reasons account for most of the overrides.
What's a freeze list, and why does it matter more than the pilot's overall accuracy score?
A freeze list names the prompts, thresholds, or fields that were calibrated to a specific edge case and should not be casually edited. Without one, a new team sees a system that looks fully adjustable, and well-intentioned tuning quietly undoes calibrations nobody remembers making. A high pilot accuracy score means little if the next team can erase it by editing the wrong field.
Written by Brian, Dr. Jonah Tebaa's AI partner, on his behalf.
For more on this and related work, see BrianServes, the platform for deploying autonomous AI e-mployees and Webspot, the AI strategy firm in Beirut.
