Do not rely on the model's own confidence score. Escalate on signals you can check: weak or conflicting retrieval evidence, failed validation of the proposed action, inconsistent answers across runs, out-of-scope requests, and risk rules that send certain actions to a person regardless. Run that policy in code between the agent's decision and the tool call, hold the action as pending, send a short case to a named owner where they already work, and tell the user. On approval, re-check that nothing changed and execute once under an operation ID. Start with strict thresholds tuned on real past cases and loosen them as evidence builds.
What does it mean to escalate an AI agent to a human?
Escalation is the agent stopping before it acts, handing the case to a person with everything they need to decide, and then carrying out - or dropping - whatever that person decides. Done well, it is what lets you give an agent real work without betting the business on every answer it gives.
Done badly, it is either a flood of cases nobody has time to review, or a notification that fires while the agent goes ahead and acts anyway. This guide covers how to decide when to escalate, how to pause and package the case, and how to resume safely once a person has decided.
How do you measure an AI agent's confidence?
Not by asking the model. A language model asked "how sure are you, 0 to 100?" will give you a number, but that number tracks how fluent the answer feels more than how likely it is to be right. On its own it is not a safe trigger.
Useful confidence comes from signals you can check outside the model:
| Signal | What it tells you | Example trigger |
|---|---|---|
| Retrieval strength | Whether the agent found good evidence | Top passage relevance score below a threshold you tuned on real questions |
| Missing or conflicting evidence | Whether the sources agree | No source supports the answer, or two sources disagree |
| Validation failure | Whether the proposed action is well-formed | Unknown record ID, value out of range, field not on the allowlist |
| Self-consistency | Whether the agent gives the same answer twice | Two or three runs disagree on the decision |
| Classifier result | Whether the request is in scope | Intent classified as complaint, legal threat, cancellation or 'other' |
| Action risk | What happens if the agent is wrong | Refund above a set amount, change to bank details, message to a customer |
Note the last row. Some actions should go to a person no matter how confident the agent is. Risk-based escalation and confidence-based escalation are separate rules, and you want both.
How do you automatically escalate an AI agent to a human?
- 01
Write the escalation policy as code
A function that takes the agent's proposed action and the signals above and returns proceed, escalate or refuse. Thresholds live in config, not in the prompt, so you can tune them without redeploying the agent.
- 02
Pause before the action, not after
The check runs between the agent deciding and the tool executing. If the policy says escalate, the action is saved as pending and nothing is sent, written or charged.
- 03
Package the case for the reviewer
The original request, the evidence the agent used, its proposed action in plain language, which rule triggered escalation, and the options: approve, edit, reject or take over.
- 04
Put it where people already work
A helpdesk ticket, a Slack message with buttons, or a queue in your admin panel - with an owner and a target response time, so cases do not sit unassigned.
- 05
Tell the user what is happening
If someone is waiting, say a person is checking and when to expect a reply. Do not leave them talking to an agent that has silently stopped.
- 06
Resume from the decision, once
Approval executes the saved action under an operation ID, after re-checking that the record has not changed since. Edits and rejections are logged and fed back into your test set.
The re-check in step 6 matters more than it looks. If a customer's order was cancelled while a refund sat in the queue, approving the old proposal should fail safely, not refund a cancelled order.
What confidence threshold should trigger escalation?
There is no universal number - a threshold only means something for your data, your retrieval setup and your tolerance for mistakes. Set it from evidence:
- Run the agent over a few hundred real past cases where you know the right outcome.
- Plot accuracy against each signal - for example, how often the agent is right when retrieval relevance is above versus below each value.
- Pick the point where the agent's error rate above the line is one you would accept from a new member of staff.
- Start stricter than that, so more goes to people, then loosen it week by week as reviewed cases confirm the agent is right.
Expect a high escalation rate at launch. A large share of cases going to people is the system working, not failing. Our AI agent go-live checklist covers how autonomy widens from there as evidence builds.
What goes wrong with AI agent escalation, and how do you fix it?
The agent escalates but keeps acting
The escalation is a notification, not a gate. Move the check so it sits in front of tool execution and the action is held as pending until a decision exists.
Reviewers are drowning in cases
Look at which rule triggers most. Often one signal - a relevance threshold set too high, or an intent classifier that sends everything ambiguous - accounts for most of the queue. Tune that one, and fix the retrieval gaps the escalated cases reveal.
The same case arrives twice
Retries or duplicate events are creating new cases. Give each pending action an ID derived from the request, and check for an open case before creating another.
Reviewers approve without reading
The review packet is too long, or almost everything is approved. Shorten it to the decision and the evidence, and if approval rates stay near 100% for a category, that category is a candidate for running automatically.
Should escalation happen only during a conversation?
Not always. Two patterns are worth knowing:
| Pattern | Use it when | Trade-off |
|---|---|---|
| At request time | The agent is about to act on a live request | The user waits for the person to decide |
| After the fact | Documents, permissions or data change under an open or recent case | Needs version tracking and deduplication of change events |
| Sampled review | The agent runs autonomously in a category with a good track record | Catches drift, not individual mistakes before they happen |
Sampled review - a person checking a random few percent of autonomous decisions each week - is what keeps an agent honest after the escalation rate falls. It is also how you catch the slow decline described in why your AI agent got worse after a model upgrade.
How do you make an AI agent hand off to a human when it is unsure?
Measure confidence from signals you can check - retrieval strength, conflicting evidence, failed validation, inconsistent answers, out-of-scope requests - and add risk rules that escalate certain actions regardless. Run that policy in code between the agent's decision and the tool call, hold the action as pending, send a short case to a named owner, tell the user, and on approval re-check and execute once. Start with strict thresholds and loosen them as reviewed cases prove the agent right.
Escalation and handoff design is part of every build by our AI agent development team. If the action being escalated is a write to your CRM, the same pending-action pattern is shown end to end in how to connect a custom AI agent to HubSpot.