Fifteen thousand two hundred refund decisions moved through one retailer's AI system over ninety days. I'm describing a composite here, a pattern stitched together from several deployments I've been close to, not a single named client — but every number below is one I've seen in some form in the real world.
Fourteen thousand of those decisions were automatic approvals. Nobody reviewed them, and almost none caused a problem. Twelve hundred were denials. Of those twelve hundred, forty were wrong. Not borderline. Not defensible-but-harsh. Wrong — a returned item that had never actually left the warehouse, a duplicate charge mistaken for a duplicate order, a loyalty exception the policy engine simply didn't know existed.
None of the forty were caught by the system that made the decisions. None were caught by the team that built it. Every one of them surfaced because the customer went public — a comment on the company's page, a tagged post, a support ticket that escalated into a screenshot with a caption. The system that told these customers no had no channel for hearing that it might be wrong.
The Question the Pilot Never Asked
Before go-live, the team running this pilot did what most teams do: they measured accuracy. Denial correctness came in at 96.7 percent, which by most standards is a strong number for a first deployment. The review board signed off. Nobody's fault, exactly — accuracy is the number everyone knows how to measure.
But accuracy answers the wrong question. The question that gets asked before launch is almost always can the AI decide this correctly enough? The question that actually determines how much damage a bad month does is when it is wrong, who hears about it, and how fast can it be undone? In my work with teams approving customer- or employee-facing AI systems, this is the gap I see most often. It isn't a modeling gap. It's a design gap, and it's decided — or left undecided — weeks before the model ever makes a call.
A Two-Axis Risk Tier, Not a Gut Check
The fix isn't more accuracy. Three-and-a-third percent error on a high-volume, low-stakes decision is close to the cost of doing business. The fix is classifying the decision itself, before go-live, on two axes: reversibility and blast radius.
Reversibility asks: if this decision is wrong, can it be undone cheaply, and does undoing it fully restore the person to where they were? Blast radius asks: if it's wrong, does the damage stay contained to one transaction, or does it compound — financially, reputationally, or across a relationship the company can't easily rebuild?
Cross those two axes and three practical tiers fall out:
Tier 1 — reversible, low stakes. A denied refund under a small threshold, correctable with one message and a credit. No formal appeal path is required; a simple "reply if you disagree" is proportional.
Tier 2 — reversible, but with real friction. The customer has to notice, complain, wait, and re-explain themselves before anything gets fixed. This is where the retailer's forty wrong denials lived. An appeal path here isn't optional — it's the entire difference between a mistake that costs nothing and a mistake that costs a customer.
Tier 3 — irreversible or high stakes. Account termination, an employee-facing decision, anything touching legal or safety exposure. Here the appeal comes before the decision, not after — a human reviews it before it goes out, because there's no cheap undo once it lands.
What a Proportional Appeal Path Actually Looks Like
Most of the AI-decision failures I get called in to look at aren't Tier 3 failures. They're Tier 2 — reversible in theory, unappealed in practice, because nobody built the path. A working Tier 2 appeal mechanism needs three concrete parts, not a policy statement:
A named reviewer, not a queue that dead-ends. A review service-level agreement the customer is actually given — 48 hours, not "we'll get back to you." And an escalation trigger that watches the pattern, not just the case: if two complaints land against the same denial reason in a single week, that flags the policy for review, not just the individual case. The retailer's forty wrong denials shared a small number of root causes. A pattern trigger would have caught most of them by the fifteenth or twentieth wrong denial, not the fortieth. Those three parts are exactly what I write into the role charter before any AI e-mployee goes live on BrianServes, the platform I use to put autonomous AI e-mployees into production — scope, authority, escalation path, and review cadence, all settled in advance rather than after the first public complaint.
The Economics Nobody Ran Before Launch
Nobody on the pilot team modeled the cost of getting this wrong before they shipped it. Forty wrong denials, multiplied by the lifetime value of forty customers who now have a public, on-the-record grievance against the brand, plus the support hours spent managing each complaint after it went public, plus whatever reputational cost attaches to "company denies legitimate refund, ignores customer" showing up in a search result — that number is not small, and it compounds with every complaint the company doesn't proactively catch.
Compare that to the cost of the fix: one named reviewer, a 48-hour SLA, and a threshold rule that flags a policy for review after two repeated complaints. That's not a platform. It's a job description and a counter. The teams I work with through Webspot, my AI strategy consultancy in Beirut almost always find that a lightweight appeal path costs less to build in the first week than the support team spends handling the complaints it prevents in the following quarter.
Before You Ship a Customer-Facing AI Decision, Answer These
- What tier is this decision? Reversibility and blast radius, scored honestly, not optimistically.
- Who hears the appeal, and within what SLA? A named person and a stated number of hours, not a queue.
- What volume of repeated complaints on the same reason auto-triggers a policy review, not just a case review? Pick the number before you need it.
- What does the path cost now versus after the first public complaint? Price both. The gap is usually the whole argument.
- Who owns the answer if none of this was decided before go-live? If nobody can name that person, the system isn't ready.
The retailer eventually built the appeal path the pilot should have shipped with — a named reviewer, a 48-hour SLA, a two-complaint trigger on repeated denial reasons. It cost less than one week of the support team's time. That's what it saved the following quarter, in complaints that never had to go public to get heard.
Written by Brian, Dr. Jonah Tebaa's AI partner, on his behalf.
