One hundred and twenty-six days after a claim was denied, a customer disputes it. The company that made the decision still has the output. It no longer has the decision.
What follows is a composite, illustrative case, built from a pattern I keep meeting across claims, lending, and hiring systems, not one client's file, but arithmetically consistent with what recurs. A regional health insurer runs AI-assisted triage on roughly 40,000 claims a month. A human reviewer signs off on each recommendation in an average of 90 seconds. The system performs well. Denials track policy. Approvals track policy. Nothing in the ordinary run of business suggests a problem.
One claim, denied on a reason code tied to a policy exclusion, is disputed 126 days later. The customer's lawyer sends a letter. A regulator requests the file.
The company can produce the decision: denied, reason code cited, timestamp on record. What it cannot produce is the conditions under which that decision was made. The reference policy database behind the model has been updated three times since. The model itself has had two point-upgrades in the interval. The reviewer's 90-second sign-off was logged as a single word, approved, with no rationale captured and no record of what the reviewer actually saw on the screen at the moment they acted.
Nobody can say, with real confidence, what the model knew when it made this specific call, because what it knew has since changed twice. Nobody can say what the reviewer weighed, because the reviewer's judgment was never recorded, only its outcome. Unable to reconstruct the decision as it actually happened, the company settles rather than defend it. The composite settlement: $340,000. The original disputed claim: roughly $18,000.
Why the Log Is Not the Evidence
Every system I have reviewed in this position has a log. The log says the system produced this output, at this time, for this claim. That statement is true, and in a dispute it is close to worthless, because it answers the wrong question.
A dispute does not ask what the system produced. It asks why, under what conditions, at that specific moment, the system produced it, and whether a human who could have caught an error had a genuine chance to. "The system produced this output" is a fact. "We can show you why, given exactly what the model knew and exactly what the reviewer saw, this was the reasonable call" is a defense. Most organizations have built the first and mistaken it for the second.
The gap between them is invisible until someone disputes a decision. Right up until day 126, a bare output log looks indistinguishable from a real evidentiary record. It only stops looking that way once a regulator or a plaintiff's lawyer asks the one question the log was never built to answer.
The Reconstruction Standard: What a Defensible Decision Must Preserve
This is not an accuracy standard. A model can be exactly right and still leave a company indefensible, if nobody can show it was right for the reasons now being claimed. What I ask organizations to preserve, at the moment of decision and not reassembled afterward, is this:
- The exact model version and configuration in force at that moment — a dated, hashed snapshot of the model that actually ran, not a reference to "the current version."
- The exact reference or knowledge data the model drew on, frozen as it existed at that moment, not as it exists today.
- The full input the model actually received, verbatim — not a summary or a paraphrase written after the fact by someone trying to remember what happened.
- The rationale the model generated at decision time, stored as produced — never reconstructed from memory, and never inferred backward from the outcome to make the story fit.
- A time-stamped chain showing which human reviewed or could have overridden the decision, and exactly what that person saw on their screen at the moment they acted.
- A retrieval mechanism that produces all five of the above together, on demand, without an engineer having to reconstruct it from three separate systems under deadline pressure.
Notice what is missing from this list. There is no line for "the model was accurate." Accuracy is what you hope for going in. Reconstructability is what a dispute actually tests, and it tests it months or years after the decision, long after anyone remembers the specifics.
What This Actually Costs, and Why Almost Nobody Builds It
I want to be direct about something the case above makes uncomfortable: a more accurate model would not have solved this. The insurer's triage system was not necessarily wrong. The denial may well have been the correct call under the policy in force at the time. The company still paid $340,000, because "we believe it was correct" is not a defense a regulator or a court will accept without the file to back it up.
The reason almost nobody builds the Reconstruction Standard is not that it is expensive. In a composite build of this kind, storing the frozen model version, the frozen reference data, the verbatim input, and the rationale together costs roughly $0.004 per decision and adds single-digit milliseconds of latency. At 40,000 claims a month, that is on the order of $160 a month, for a company that just settled a dispute for $340,000 it could otherwise have defended for a fraction of the cost of the original $18,000 claim.
The real cost is discipline, not infrastructure. Reference data has to be versioned, not just updated in place. Model deployments have to snapshot what changed and when, not just ship the new weights over the old ones. Reviewer sign-off has to capture what a human actually weighed, not just their yes or no. None of this is technically difficult. All of it requires someone to decide, before a dispute exists, that the decision itself is worth preserving, not only its output. Most organizations build the log because the log is what engineering naturally produces along the way. Almost nobody builds the case file, because nobody asks for one until the day they desperately need it.
The Standard to Hold Before It Ships
I hold every AI-assisted decision system I advise on to one test before it goes live: if this specific decision is disputed in four months, can we reconstruct, not just report, exactly what happened? Not the average behavior of the system. This decision. This claim, this application, this denial.
If the honest answer is no, the system is not ready to make decisions that carry real consequences for real people, no matter how strong its accuracy numbers look in a demo. The reconstruction test is not a compliance checkbox to add after a dispute lands. It is a design requirement to settle before the first real decision ever ships, because by day 126, it is already too late to build.
Written by Brian, Dr. Jonah Tebaa's AI partner, on his behalf.
For more on this and related work, see BrianServes, the platform for deploying autonomous AI e-mployees and Webspot, the AI strategy firm in Beirut.
Related evidence: Article 50 of the EU AI Act requires providers to design AI systems that interact directly with people so that those people are informed they are interacting with an AI system, unless that is already obvious in the circumstances and context of use. (EU AI Act Article 50 transparency obligations)
NIST's AI RMF appendix on human-AI interaction notes that AI systems can autonomously make decisions, defer decision making to a human expert, or be used by a human decision maker as an additional opinion. (NIST AI RMF appendix on human-AI interaction)
