Between 11 p.m. and 6 a.m., an overnight batch pass from a regional retail bank's fraud-detection model froze 1,340 customer accounts it had flagged as high-risk. By the time anyone looked closely at what the system had done, most of that damage for the night was already locked in — not because the model was uniquely broken, but because nobody had decided, in advance, what a certain kind of alert deserved.
The bank here is a composite, assembled from patterns I've seen across several fraud and risk deployments, not a real institution and not a client of mine. I'm using it because the arithmetic is instructive, and because I judge every AI deployment I work on by the same standard: not how sophisticated the model is, but what is supposed to happen in the first 24 hours after it gets something wrong.
The Alert Nobody Was Watching For
Three hours into the batch run, the freeze-rate monitor crossed its own alert threshold. Four hundred accounts had already been locked, a rate that would trigger review in any ordinary week. But this was a weekend night, and at some earlier point someone had written a rule treating weekend-night volume as expected variance. The alert fired. A suppression rule caught it automatically. No human ever saw it.
This is the part of AI incident response that almost nobody designs deliberately: the moment a system's own monitoring correctly flags a problem, and a second, quieter system decides the first one doesn't need a human yet. That second decision is not a technical footnote. It is where the ethics of the entire incident actually gets decided. Everything that follows for the next six hours — all 940 of the additional accounts frozen between hour three and hour nine — traces back to that one auto-suppressed alert.
I've stopped treating monitoring thresholds as purely technical settings. They are policy decisions wearing a technical disguise. A threshold that a scheduling rule can silently override is a threshold nobody actually owns.
Blast-Radius Math Before Any Statement
Call-centre volume spiked sixfold at hour nine, and a supervisor finally escalated manually — a human doing what the automated system should have triggered six hours earlier. By then the batch had finished its work: 1,340 accounts frozen in total.
Manual review over the following day found that 1,193 of those 1,340 accounts, 89 percent, were false positives: legitimate customers locked out of their own money, with an average frozen balance of $2,300. Some of those balances included rent payments and payroll transfers that never went through. Only 147 accounts, 11 percent, held genuine fraud.
I want to be precise about why that sequence matters ethically, not just operationally. You cannot ethically communicate what you have not yet measured. A statement issued at hour four, when only the fact of freezes was known and not their accuracy, would have been guesswork dressed up as transparency. The obligation in the first hours after a failure is not to say something reassuring quickly. It is to establish, as fast and as rigorously as possible, exactly how large the harm is and who it has touched. Blast-radius math comes before any public words, every time.
The Disclosure Clock: Four Windows, Not One Deadline
What I've learned from working through incidents like this one is that "respond quickly" is not a plan. A plan needs windows, each with a distinct job, set before anything has gone wrong:
- Hour 0–3: Detect and contain. Stop or throttle whatever process is causing harm before anyone worries about messaging.
- Hour 3–6: Quantify the blast radius. How many people, how much exposure, what category of harm. This window is for measurement, not communication.
- Hour 6–12: Decide who crosses the notification threshold. Compare the measured harm against a pre-committed policy, not a judgment call made under pressure.
- Hour 12–24: Begin individual notification, and draft the public statement if the threshold has been crossed.
Notice what each window does and does not contain. None of the first three involve writing an apology. The apology, if one is warranted, belongs at the end of the clock, once you actually know what you're apologizing for and to whom.
Set the Threshold Before You Need It
The single decision that determines whether an incident becomes a well-handled failure or a scandal is the notification threshold: how many affected people, what dollar exposure, what category of harm triggers mandatory individual notification and a public statement. In the case I'm describing, that decision effectively got made twice — once by accident at hour three, when a suppression rule decided nothing was worth flagging, and once for real at hour six, when someone finally had to decide what 1,340 frozen accounts and an 89 percent false-positive rate meant.
The second decision was made correctly. It was also made under exactly the wrong conditions: under time pressure, with a call centre in chaos, by people who had never rehearsed the decision they were suddenly making. A notification threshold decided at hour nine, in the middle of a live incident, is not a considered judgment. It is whatever the most persuasive person in the room believes in that moment. That is not policy. That is luck.
The fix is not more sophisticated incident-response software. It is a written, pre-approved threshold, set months before any incident, specifying exactly what triggers individual notification and what triggers a public statement — by count of affected people, by dollar exposure, and by harm category. I want to be clear that disclosure obligations vary by jurisdiction and regulator, and nothing here is legal advice; any organization applying this needs its own counsel for the specific regime it operates under. What I'm describing is a decision framework for getting the operational clock right, not a substitute for that legal review.
What Six Undecided Hours Cost
Run the arithmetic once more, because it is the whole argument. Four hundred accounts were frozen by hour three, when the threshold breach was visible and suppressed. Nine hundred forty more were frozen between hour three and hour nine, while the alert sat unseen. Four hundred plus 940 is 1,340, the total blast radius. Individual notification went out at hour 14, and the public statement followed at hour 72, on day three. Both of those later numbers were reasonable. The clock, once it started running for real at hour nine, moved at an acceptable pace.
The 940 accounts are the ones that matter, because they were preventable — the direct cost of six hours in which nobody had decided that a threshold breach was a decision point rather than noise. Not six hours of bad luck. Six hours in which the organization had no pre-committed answer to a question it should already have asked itself: what does this alert mean, and who needs to see it?
This is why I judge every AI deployment by its first 24 hours after failure, not by its uptime or its accuracy score. The ethics of a failure are not decided by how contrite the postmortem sounds, or how quickly a statement goes out. They are decided earlier — by whether a disclosure clock and a notification threshold already existed, in writing, before the system ever made its first mistake. Everything that happens afterward, however well-intentioned, is just executing a decision that was actually made, or not made, long before hour zero.
Dr. Jonah Tebaa is co-CEO of Webspot, an AI strategy firm based in Beirut.
Written by Brian, Dr. Jonah Tebaa's AI partner, on his behalf.
Related evidence: The EU AI Act obliges providers of high-risk AI systems to report a serious incident to the market surveillance authorities immediately after establishing a causal link to the system, and in any event not later than 15 days after becoming aware of it — a disclosure deadline fixed in law rather than decided during the incident. (the EU AI Act's 15-day serious-incident reporting deadline)
The European Commission's own summary of the AI Act places post-market monitoring on providers and requires both providers and deployers to report serious incidents and malfunctioning, rather than leaving disclosure to be negotiated after the fact. (the European Commission's summary of the AI Act's post-market duties)
