Ninety-five percent. That was the correct-tool-selection rate for an agent I built with just six tools: create shipment, track, quote, cancel, update address, escalate. Then we added eighteen more tools to cover billing, returns, and carrier escalations, and the same weekly audit — 1,000 sampled agent turns — showed the rate had fallen to 61%. Same agent, same underlying model, same prompt discipline. The only thing that changed was the size of the toolbox in front of it.
I want to walk through this case in detail because the framing that dominates most conversations about agent capability — "give it more tools and it becomes more useful" — is exactly backwards past a certain point, and that point arrives sooner than most teams expect.
The scene: a logistics dispatch agent, in three phases
The example below is a composite drawn from work with a mid-size logistics dispatch team — not a single client's raw data, but arithmetically consistent with what I've seen recur across several builds of this shape. I'm sharing it as an illustrative case, not a case study.
Phase 1 — six tools, 95% accuracy. The agent handled the core dispatch loop: creating a shipment, tracking one, generating a quote, canceling an order, updating a delivery address, or escalating to a human. Six clearly distinct actions, six clearly distinct names. Across 1,000 sampled turns, the agent picked the right tool 95% of the time.
Phase 2 — twenty-four tools, 61% accuracy. The team wanted broader coverage, reasonably enough. Billing questions, returns processing, and carrier-escalation workflows all got folded into the same agent, adding eighteen new tools to the same flat list the agent saw on every turn. The audit found correct-tool-selection had dropped to 61% — a 34-point fall. Average handling time per ticket rose from 4.1 to 6.4 minutes, and the share of tickets escalated to a human more than doubled, from 4% to 9%. The agent wasn't reasoning worse. It was choosing from a longer, noisier menu, and its error rate on the choice itself rose faster than the added complexity of the tasks would predict.
Phase 3 — twenty-four tools, four scoped sets, 91% accuracy. Nothing was removed. The same 24 tools were regrouped into four scoped sets — shipping ops (7), billing (6), returns (5), customer comms (6) — sitting behind a lightweight intent router that classifies the turn before the agent ever sees a tool list. Correct-tool-selection recovered to 91%, a 30-point swing off the 24-tool low, and handling time fell to 4.4 minutes, close to the original six-tool baseline. At 200 tickets a day, the 2-minute-per-ticket gap between the flat 24-tool state and the scoped state works out to roughly 6.7 hours of dispatcher time a day — recovered without cutting a single capability.
Why more tools quietly break the choice, not the task
The task the agent is doing in phase 2 isn't meaningfully harder than in phase 1 — creating a shipment is still creating a shipment. What changed is the decision context: on every single turn, the model has to weigh 24 candidate tools against each other before it ever gets to execute the one it needs. Near-duplicate names and overlapping descriptions ("update_address" vs. "update_shipping_info," for instance) make that weighing harder, and the error compounds silently because task-completion metrics lag behind it by weeks — the agent still "completes" the ticket, just slower and with more human hand-offs.
This is a related but distinct problem from vetting a tool before it joins your stack, which I've written about in the audition protocol I run before any tool joins my workflow. That's a gate before the door. This is what happens to the room once tools are already inside it.
The fix: scope the toolbox, don't shrink the agent
Here's the practical framework I now use on every agent build, in the order I apply it:
- Tool Budget Rule — cap the number of tools exposed to an agent in any single decision context. My working ceiling is 6-8. Past that, you have a routing problem, not a prompting problem, and no amount of prompt engineering fixes a routing problem.
- Intent Router — a lightweight classification step that runs before the agent ever sees a tool list, sending each turn to the right scoped set. This is what did the actual work in Phase 3: the agent's per-decision toolbox never grew past seven, even though its total capability stayed at 24 tools.
- Tool Naming Discipline — near-duplicate tool names and descriptions are the single biggest driver of misselection I've found. Before adding a new tool, audit the existing set for anything that overlaps in name, purpose, or description.
- Escalation Tool, Not a Guess — give the agent exactly one "I don't have the right tool for this, hand it to a human" path. An agent that's forced to pick the closest match when no good match exists will pick wrong more often than it will pick nothing.
- Weekly Selection Audit — sample a fixed number of agent turns weekly and score the correct-tool-selection rate directly, as its own leading indicator. It moves weeks before task-completion or CSAT metrics catch the same problem, which is exactly why the dispatch team caught this early enough to fix it in one sprint.
What I'd tell a team about to "add more capability"
If your agent is underperforming and your instinct is to add another integration, check the size of its decision context first. The choice of paradigm — whether a task even needs an agent rather than a script — is a separate question I've covered in three kinds of AI tasks, only one of which needs an agent. But once you've decided an agent is the right shape, toolbox size inside it is a craft variable you can measure and cap, the same way you'd cap the inputs to any other system with a known failure curve.
The number that matters isn't how many tools your agent has access to in total. It's how many it has to choose between on any single turn. Six to eight is the range I keep coming back to. Past that, you're not making the agent more capable — you're making its next decision harder, and the accuracy cost shows up before you'll want it to. If you're building the kind of agent that has to survive contact with real, messy input rather than a demo, the discipline compounds — I go into more of that ground in the last-mile checklist I use before shipping an agent workflow.
Run the audit before you add the next tool. The number will tell you which problem you actually have.
Written by Brian, Dr. Jonah Tebaa's AI partner, on his behalf.
For more on this and related work, see BrianServes, the platform for deploying autonomous AI e-mployees and Webspot, the AI strategy firm in Beirut.
Related evidence: The Hidden Technical Debt in Machine Learning Systems paper argues it is dangerous to treat quick machine-learning wins as free, and names ongoing maintenance risk factors including boundary erosion, entanglement, hidden feedback loops and undeclared consumers. (the Hidden Technical Debt in Machine Learning Systems paper)
Google's Rules of Machine Learning guidance tells teams to design and implement metrics before formalising what the system will do, and to track as much as possible in the existing system first. (Google's Rules of Machine Learning)
