Tuesday, 4:10 pm. A customer asks a 14-person trading firm's AI assistant when 100 units of a catalogue item will arrive. The assistant answers: Thursday. The warehouse cannot deliver before the following Tuesday. The customer plans around Thursday, and by Friday morning there is a complaint in the owner's inbox. This is an illustrative composite, not a client engagement.
On Friday the owner gathers the team and asks, "Who approved this?" Nobody speaks. The meeting ends with a new rule: every AI reply must be read by a manager before it goes out. The firm answers about 40 customer chats a day, and within two weeks the rule is quietly ignored. The next mistake, when it comes, will have the same cause as this one.
Why "who did it?" fails
I understand the instinct. A date was promised, a customer is unhappy, and someone should answer for it. But in my work I have seen this question fail in the same two ways every time.
First, it makes people hide the facts. The person who last edited the delivery instructions will not volunteer that they did. The person who noticed the stock figure looked odd on Monday will not mention it. The facts you need most are held by the people with the strongest reason to keep them quiet.
Second, even a correct answer teaches you nothing about the system. Suppose you find the person. You have a name, and the instructions, the data and the checks are exactly as they were before. The mistake can happen again with a different name attached. A rule written to punish, such as "a manager reads everything", then adds work that nobody can sustain.
If you want to see what a good first response looks like, I wrote about the first day after a failure in the first 24 hours after an AI failure. This post starts later, when the fire is out and the facts are on the table. The question is how to judge them.
The Hindsight Split
I replace "who did it?" with one question: what was knowable then? The method is two columns on a whiteboard, and I call it the Hindsight Split.
- Knowable then. What the AI actually saw and was told on the day: its instructions, the data on its screen, the checks that existed, and what a careful person in the same seat would have known at 4:10 pm.
- Knowable only now. What only the outcome has shown: the complaint, the delay that happened afterwards, the reason the customer needed that date.
The rule that follows is the whole point. Fix only the first column. Every fact in it was available in time, so a change to instructions, data or checks would have caught it. The second column gets a short note and no new rule, because a rule built on something nobody could see will only slow down the ordinary work that was fine.
It also changes the room. Nobody has to defend a decision. The task is to rebuild Tuesday afternoon as it was and ask what could have been seen from there.

The split, run on the composite
Here is the trading firm's Tuesday, split. Five facts contributed to the wrong date. Three were knowable then, and two were not.
| Fact | Column | Response |
|---|---|---|
| 1. The assistant's instruction sheet promised delivery in 2 working days for every item. The warehouse had moved imported lines to 5 working days. | Knowable then | Fix: one named owner for the sheet, and a dated review whenever the warehouse changes a lead time. |
| 2. The stock screen showed 120 units available. Only 40 were on hand; 80 were still on order. | Knowable then | Fix: show on-hand and inbound as separate figures, and let the assistant quote only from on-hand. |
| 3. A check existed that sent date promises to a person, but only for orders above 5,000. This order was 3,800, so it never fired. | Knowable then | Fix: trigger the check on any date outside the standard lead time, whatever the order value. |
| 4. On Wednesday a supplier container was held at customs for four days. | Only now | Note: record it as a supplier risk. No new rule. |
| 5. The customer needed Thursday because of a trade show. The chat never said so. | Only now | Note: customers sometimes have a reason behind the date. No new rule. |
Look at what the split does to the Friday meeting. The owner's question, "Who approved this?", has an uncomfortable answer: nobody, because the one check that would have involved a person was set to ignore orders of this size. That is a design fact, not a personal failure. Facts 1, 2 and 3 were all on the screen or in the setup that afternoon. Three small changes, each with one owner, remove this class of error. The manager-reads-everything rule would have removed none of them.
Facts 4 and 5 are real, and they hurt. But no careful person at 4:10 pm could have known about a customs hold that began the next day, or a trade show the customer did not mention. If I wrote a rule for them, I would be asking the team to guard against the future. I would rather write a note, keep it where the next reviewer can see it, and watch whether either shows up again.
A caution: the split is only honest if the first column is built from records, not memory. If you cannot tell what the assistant was told or what it saw that day, then the first review finding is that you cannot reconstruct it. I wrote about that test in the AI decision reconstruction test.
Six questions, 45 minutes, within 10 days
I keep the review short on purpose. A long review becomes a trial, and a late one becomes a memory contest. Hold it within 10 days of the mistake, give it 45 minutes, and work through these in order.
- What did the AI see, and what was it told? Instructions, data and the customer's message, as they stood that day.
- What would a careful person have known that day? This is the line between the two columns.
- Which check existed, and why did it not fire? Most of the time a check existed and was set for something else.
- What is the smallest change that would have caught it? Small is the discipline. A big change is usually a reaction to the meeting, not the mistake.
- Who owns that change, and by when? One name and one date for each fix.
- What do we tell the customer, and who tells them? A person, not the assistant, and before the review is finished if it can be done sooner.
What the board should receive
A board does not need the whiteboard. For each material AI mistake I would send it half a page: what happened in two sentences, the two columns in short form, the fixes with owners and dates, and the customer outcome. No names of individuals, because the page is about the system.
I would also ask the board to read these pages together every quarter. One page tells you about an afternoon. A dozen show you which kind of fact keeps landing in the first column, and that is where the real investment belongs.
There is a commercial reason for all this beyond fairness. A business that can say, "Here is what we missed and here is what we changed," earns more trust from customers and regulators than one that claims it never errs. The claim of never erring is not believed, and the first mistake breaks it.
This week, pick your last AI mistake. Draw the two columns. Put in only what was knowable then, fix those items, and write a note for the rest.
Related evidence: Article 26(2) of the EU AI Act requires deployers of high-risk AI systems to assign human oversight to natural persons who have the necessary competence, training and authority, as well as the necessary support — so the oversight duty lands on a named person who must actually be empowered to act, not on a department. Article 26 sits in Chapter III, Section 3, whose date of application was moved by Regulation (EU) 2026/1744 to 2 December 2027 for Annex III high-risk systems and 2 August 2028 for Article 6(1) product-embedded systems. (Article 26(2) of the EU AI Act)
