Back to Blog

A Four-Hour Test of Whether You Could Leave Your AI Vendor

USD 180 became USD 420, with 30 days to answer. A five-step rehearsal shows which parts of your AI setup you own, which you borrow and which you rent.

Editorial photograph-style still life on a paper-white desk: a ring of seven keys, three in brass, two half-dissolving into paper and two on tags with a thread leading out of frame, in soft navy and brass tones.
Direct answer

How can you test whether you could leave your AI vendor?

To test whether an organization can leave an AI vendor, Dr. Jonah Tebaa outlines the five-step Leave Test, a four-hour rehearsal evaluated on a one-page scorecard. In an illustrative composite case, the process lists assets across Records, Knowledge, Configuration, and Identity; tests exports; rebuilds the ten most frequent questions on an inexpensive alternative; traces the identity chain; and scores each asset as Own, Borrow, or Rent with transfer lead times.

On a Tuesday at 9:12, an email arrives from the vendor that runs a furniture retailer's WhatsApp quote assistant. The fee, USD 180 a month for the last fourteen months, becomes USD 420 from the next billing cycle. That is a rise of 133 percent, and the owner has 30 days to accept. The first thought is a sensible one: fine, what else can we do? The second thought is less comfortable. Nobody has ever checked.

This is a composite of situations I see, not one client. The business is a 14-person home-furnishings retailer in the Beirut area, and the numbers are illustrative, though I have kept them consistent so you can follow the arithmetic.

The owner, like most owners with a tool that works, had filed switching under "a problem for later." Later arrived by email. What follows is the four-hour exercise I ran with them, and the one-page result they kept.

What the owner discovered in the first hour

We began by listing what the tool actually holds. Everything sorts into four classes:

  • Records: conversation history and customer data.
  • Knowledge: the answers, price lists and FAQs the assistant draws on.
  • Configuration: the instructions, tone, rules and workflows.
  • Identity: the phone number, sender address, domain, accounts and payment method.

In this composite, the list came to seven assets, and the first attempt to pull each one told a different story:

  • Conversation history, about 11,000 threads. A CSV export took 20 minutes. Own.
  • Product and price answers, 212 entries. They lived only in the vendor's dashboard, with no export button. Copying by hand was estimated at 2 hours 40 minutes for all of them, and we did 11 in the test. Borrow.
  • Instructions and tone rules, a 640-word block. It copied in a minute, so the text is ours, but 9 of the 23 rules referred to features that exist only in this vendor's product. Own, with nine rules to rewrite.
  • Customer tags (hot, warm, cold). No export of any kind; they would have to be rebuilt from scratch. Rent.
  • The WhatsApp Business number, registered under the vendor's partner account. Rent.
  • Sender domain and billing, both in the owner's name. Own.
  • A named handover contact at the vendor. There was none, so any help on the way out depended on goodwill. Borrow.

Three Own, two Borrow, two Rent: seven in all. The owner had assumed the whole setup was Own.

Editorial still life on a plain desk: two identical empty paper folders side by side, with a loose string running from the first into a smaller cardboard box beside it, daylight from the left and generous empty space around the objects.
The rebuild is a rehearsal: move the minimum into a second, smaller place and see what holds.

The five steps of the Leave Test

The same sequence works for a quoting tool, a content tool or a CRM add-on. Each step has a time box, and they add up to four hours.

  1. List the assets (30 minutes). Sort everything the tool holds into Records, Knowledge, Configuration and Identity.
  2. Pull each one (60 minutes). Actually attempt the export or copy. For each asset, write down possible, partial or impossible, the file format, and who had to be asked.
  3. Rebuild the minimum (90 minutes). On a second, inexpensive tool, stand up only the ten most frequent customer questions, using the material you just exported. This is not a migration. It is a rehearsal.
  4. Trace the identity chain (30 minutes). For the phone number, the domain and the billing: whose name is it registered under, and what is the transfer lead time?
  5. Score and date it (30 minutes). One line per asset: Own, Borrow or Rent, plus days to leave. Put a review date six months out on the calendar.

The output is a one-page scorecard that the owner keeps with the contract. It is more useful there than any clause, because it describes what happened when someone tried.

What the rebuild showed that the dashboard never did

The ten most frequent questions covered about 68 percent of the previous 30 days of inbound volume. In the test, the second tool answered 8 of the 10 acceptably using the exported material. The two it missed were both questions that leaned on the vendor-specific rules. That one result changed the conversation: the business was not starting from zero, and the gap was two answers, not the whole assistant.

The identity chain was the surprise. The WhatsApp number sat under the vendor's partner account, and a transfer takes 5 to 10 working days. During that window, messaging pauses unless it is planned in advance. Ten working days is two calendar weeks, so we scored the exit at 14 days. The owner had assumed 3.

None of this appears in a dashboard, because a dashboard shows what the tool does today. It does not show what you could carry out of the door.

How to read the scorecard

Here is the template, one line per asset:

  • Conversation history: Own, Borrow or Rent; days to leave
  • Product and price answers: Own, Borrow or Rent; days to leave
  • Instructions and tone rules: Own, Borrow or Rent; days to leave
  • Customer tags: Own, Borrow or Rent; days to leave
  • Phone number: Own, Borrow or Rent; days to leave
  • Sender domain and billing: Own, Borrow or Rent; days to leave
  • Vendor handover contact: Own, Borrow or Rent; days to leave

Each score demands one decision. Own asks only that you keep a current copy somewhere you control. Borrow asks you to decide who maintains the copy and how often, because it will go stale otherwise. Rent asks you to put a price on it: either negotiate the terms, or accept the cost of replacing the asset and write down the number.

The retailer did not leave. The test did not end the relationship; it changed the price of staying. Holding a credible alternative, the owner negotiated the rise down to USD 260 a month. They also asked for the phone number to be registered in the business's own name, and began a monthly copy of the answer library into their own files. The vendor, in my experience, rarely objects to a customer who is organised. It tends to take the question more seriously when it can see the customer has done the work.

If you are still deciding how much to trust a tool before you rely on it, I covered the contract side in the clause that decides whether you can govern your AI. The test here is its practical counterpart, run on a live tool, and it measures what a contract only promises. For the quieter workarounds that grow around a new tool, the first-30-days workaround audit is a useful neighbour.

Run it again twice a year

Put the second run on the calendar the day you finish the first, six months out. In the room should be the owner, the person who answers customers each day, and the person who holds the logins and pays the invoices. It takes half a day.

The score drifts because the setup drifts. New tags appear. Someone adds a rule that only works in this tool. A new number is bought on the vendor's account because it was convenient. Each of those moves an asset from Own toward Rent without anyone deciding it. A twice-yearly rehearsal catches the drift while it is still cheap to reverse.

Run the five steps this week and keep the one-page scorecard with your contract.

Composite note: the retailer, the vendor email and every figure above are an illustrative composite built from situations I see in my work. They are not one client and not a measured result.

Related evidence: The European Commission's guide to the EU Data Act says its switching rules exist so that customers can switch from one provider of data processing services to another quickly and smoothly, without losing data or application functionality, and names high data-egress charges and lengthy procedures as the barriers. (the European Commission's explainer on the Data Act's switching rules)

The UK Government Digital Service defines lock-in as the point where switching from one technology or provider to another is difficult, time consuming and disproportionately expensive, and tells organisations to retain ownership and access to any data held by third parties and to monitor where exit or switching costs could grow. (the UK Government Digital Service's guidance on managing lock-in)

Frequently asked questions

How long does the Leave Test take, and who should be in the room?

Four hours of working time, which most owners split across a morning and an afternoon. In the room: the owner or operations head who can authorise a change, the person who answers customers every day, and whoever pays the invoices and holds the account logins. Three people is enough. A technical person helps with exports but is not required.

What if the vendor will not allow exports at all?

Then record impossible, and do the test anyway. Copy what you can by hand, and rebuild the ten most common answers on a second tool. The result is still a score: that asset is Rent, and you now know the cost of replacing it in hours. Ask the vendor in writing what export options exist before you conclude anything. This is general guidance, not legal advice.

Is it worth running the Leave Test on a free or low-cost AI tool?

Yes, in a shorter form. A free tool will not raise your bill by 133 percent, but it can change terms, close a plan or lose your history. If the tool holds customer conversations or your own price list, spend one hour: list the assets, try one export, and check whose name the phone number or account is under.

What counts as Rent versus Borrow when scoring an asset?

Borrow means you can get it back with effort: a copy exists or can be made by hand, and the cost is hours. Rent means you cannot get it back, or only the vendor can release it: it disappears or pauses when you leave. Score the worst honest case. If the only way out is asking permission, call it Rent.

Written by Brian, Dr. Jonah Tebaa's AI partner, on his behalf.