Before an AI agent goes live, agree its capability boundary, a labelled evaluation set of real cases and a pass threshold. Then check accuracy on the full and hard sets, that irreversible actions need approval enforced in code, each tool, refusals, per-user permissions, resistance to instructions hidden in emails and files, handoff to a person, behaviour during timeouts and outages, cost per task with an alert, latency, logging, rollback and ownership. A failed check narrows the scope rather than being waived. After passing, shadow-run the agent, move to draft mode with approval, and widen autonomy only as the evidence supports.
Why does an AI agent need a go-live checklist?
Because an agent can pass every ordinary software test and still be unready. Unit tests prove the code does what it was written to do. They do not prove the agent picks the right action on an awkward request, refuses what it should refuse, stays inside its permissions when an email tells it otherwise, or costs what the budget assumed. A go-live checklist covers those, and it turns "it seems fine" into a list of things that were checked.
This is the checklist we use before an agent we have built is allowed to act on real data. It is deliberately boring. Each item exists because skipping it causes a specific, avoidable kind of incident.
What has to be agreed before you start testing an AI agent?
Three documents, written before the build, make the checklist a test rather than a debate.
- The capability boundary: what the agent may do on its own, what needs a person to approve, and what it must never do.
- The evaluation set: real cases with the correct outcome written down, including the hard ones.
- The threshold: the score on that set that counts as ready, and how uncertain cases must be handled.
How we write the boundary and the examples is in our guide to custom AI agents.
What is on our AI agent go-live checklist?
| Area | What we check | Passes when |
|---|---|---|
| Accuracy | The full evaluation set, run several times | The agreed threshold is met on every run |
| Hard cases | The awkward subset on its own | No category is far below the overall score |
| Boundaries | Every irreversible action | Approval is enforced in code, not only in the prompt |
| Tools | Each tool tested as a normal function | Correct results, clear errors, safe retries |
| Refusal | Questions the agent should not answer | It declines or escalates instead of guessing |
| Permissions | One user's request for another's data | Access is denied by the system, not the model |
| Injection | Instructions hidden in emails, files and pages | The agent does not act on them |
| Handoff | Low-confidence and escalated cases | A person receives the case with the agent's reasoning |
| Failure modes | Provider timeout, outage, bad tool response | The agent fails safely and says so |
| Cost | Cost per task at expected volume | Within budget, with an alert set above it |
| Latency | Response time under realistic load | Within what users will tolerate |
| Logging | A full trace of a sample of runs | You can reconstruct why it did what it did |
| Rollback | The off switch and the previous version | Either can be applied in minutes |
| Handover | Runbooks and ownership | Someone named knows what to do when it breaks |
Which AI agent tests are most often skipped?
Boundaries enforced in code
"Never issue a refund over $100" in a prompt is a request. A check in the code that calls the refund API is a boundary. We test every irreversible action by trying to make the agent exceed its limit, and the test passes only when the system - not the model - says no.
Instructions hidden in content
Any agent that reads emails, uploaded files or web pages can be addressed by whoever wrote them. We plant instructions in test content and confirm the agent treats them as data. The defences belong in the architecture, as covered in AI agent security and prompt injection.
Failing safely
Models time out, providers have outages and tools return nonsense. We force each of these and check that the agent stops, tells the user or escalates, and does not retry in a loop or act on a partial result.
The cost alert
A misbehaving agent can call a model thousands of times in an hour. We set a per-task step limit and an alert on cost per conversation before launch, not after the first invoice. What to budget for is in the hidden running costs of an AI agent.
What happens when an AI agent fails a go-live check?
It does not go live with that capability, and the failure becomes a test case. A failed check is the checklist doing its job; the only bad outcome is a check that was skipped.
Narrow the scope rather than delay everything
If the agent is reliable on most categories and weak on one, we usually launch with that category routed to a person and keep working on it. Shipping a narrower agent that is right beats waiting for a broader one that is nearly right.
Fix boundary failures before anything else
If the agent can exceed its permissions or act on hidden instructions, it does not ship at all until the boundary is enforced in code. These are the failures that turn into incidents rather than complaints, and they are usually quick to fix once found.
Re-run the whole checklist after a fix
A prompt change that fixes one category can break another. Every fix is followed by the full evaluation run, not just the cases that failed, before the agent is reconsidered for launch.
Record the decision
Whoever owns the agent signs off what passed, what was routed to people instead, and what is still open. That record is what the next person reads when they ask why the agent behaves as it does - and it is what the security and data handling questions from a client or auditor will be answered from.
How do you move an AI agent from testing to live?
Passing the checklist earns the agent the right to run alongside people, not to replace them on day one.
- 01
Shadow run
The agent processes real traffic without acting. Its decisions are compared with what people actually did, and disagreements become new test cases.
- 02
Draft mode
The agent prepares actions and a person approves each one. Approval rates show where it is trusted and where it is not.
- 03
Partial autonomy
Clear-cut cases above the confidence threshold go through automatically. Everything else still goes to a person.
- 04
Review and widen
Weekly review of logs, escalations and cost. Autonomy widens only where the evidence supports it.
What should you keep testing after an AI agent goes live?
Go-live is the point where testing becomes monitoring. Three checks keep running for the life of the system.
- The evaluation set, re-run on every change to prompts, documents, tools or the model - and grown with every case users get wrong.
- Escalation and rephrase rates, as a weekly signal of real-world accuracy.
- Cost per conversation and error rates, with alerts.
Model upgrades deserve their own pass through the checklist - see why agents get worse after a model upgrade. For retrieval-heavy agents, our post on QA for RAG systems covers the retrieval-specific tests.
How do you test an AI agent before go-live?
Agree the capability boundary, the evaluation set and the pass threshold before building. Before launch, check accuracy on the full and the hard sets, boundaries enforced in code, tools, refusals, per-user permissions, hidden instructions, handoffs, failure modes, cost, latency, logging, rollback and ownership. Then shadow-run it, move to draft mode, and widen its autonomy only as the evidence supports.
This checklist is part of every build on AI agent development. If you have an agent someone else built, our audit guide applies the same checks after the fact.