Fail-Closed by Default: Designing AI That Stops When Unsure
When an AI agent isn't sure, the safe default is to stop, not to guess. Here's why that matters.
Most AI support demos are built to impress. They answer everything, instantly, with total confidence. That confidence is the problem. The question that actually decides whether you can trust an AI agent in front of customers is the opposite one: what does it do when it doesn't know?
The safe answer is nothing customer-affecting. It stops and hands off. That design choice is called fail-closed, and it is the single most important thing separating an AI agent you can ship from one you can only demo.
What fail-closed AI actually means
A fail-closed AI agent treats uncertainty as a signal to stop, not to improvise. When confidence is low, when the data it needs is missing, or when a system it depends on is down, it does not take its best guess and proceed. It halts the action and routes the conversation to a person.
The opposite is fail-open: keep going no matter what, fill the gap with a plausible-sounding answer, take the action anyway. Fail-open looks better in a demo because it always produces output. In production it produces wrong answers and wrong actions on real customers, and you find out after the damage is done.
The distinction is borrowed from safety engineering. A fail-closed door locks when the power dies. A fail-open one swings open. You want the lock.
The cost of an agent that does too little is a slightly slower handoff. The cost of one that does too much is a wrong action on a real customer.
When an agent should stop
Stopping is not failure. It is the agent working as designed. There are a handful of moments where a well-built agent should always pull back and get a person involved.
- Low confidence in the answer. If the model isn't sure it's right, "I'll connect you with someone" beats a confident wrong answer every time.
- Missing or stale data. If the order record didn't load or the account status is unknown, the agent has no business acting on a guess about it.
- An action above its limit. A refund over the ceiling you set, a cancellation, anything irreversible — the agent proposes, a person disposes.
- A topic outside its scope. Legal, medical, or financial judgment calls, or anything you've marked as human-only, route straight to a person.
- A dependency is down. If the system the agent needs to act through is unavailable, it hands off instead of pretending the action went through.
Each of these is a place where guessing is cheap to do and expensive to be wrong about. Fail-closed makes the cheap-but-wrong path unavailable on purpose.
Why this beats a smarter model
It is tempting to think the fix for uncertainty is a better model. Make it smart enough and it won't be unsure. But no model is right all the time, and the dangerous cases are exactly the ones where it's confidently wrong. You cannot raise your way out of that with model quality alone.
Fail-closed design works regardless of how good the model is. A modest model that stops when unsure is safer than a brilliant one that never does. That's why the behavior belongs in the system around the model, not in the model itself.
Here's the trade-off, stated plainly:
| Design | When unsure it... | Best case | Worst case |
|---|---|---|---|
| Fail-open | Guesses and proceeds | Fast answer | Wrong action on a real customer |
| Fail-closed | Stops and hands off | Fast answer | Slightly slower handoff |
The worst case is the whole game. Fail-closed caps your downside at "a person had to step in." Fail-open leaves it uncapped.
Building it so it can't be skipped
For fail-closed to mean anything, it has to be structural, not a polite suggestion the model can talk itself out of. A few principles make it real:
- Separate proposing from doing. The model proposes an action. A deterministic check decides whether it runs. The model never acts directly, so it can't act past its limits.
- Default to no action. Absent a clear, validated reason to proceed, the system does nothing customer-affecting. Inaction is the safe resting state.
- Make the handoff graceful. Stopping shouldn't dead-end the customer. The agent should pass the conversation to a person with full context, not drop it.
- Leave a receipt. Even a decision to stop should be recorded — what the agent was unsure about, and why it handed off — so you can review and tune the thresholds.
Done this way, fail-closed isn't a feature you toggle. It's the shape of the system.
Where BearScope fits
BearScope is fail-closed by default. Every AI action is proposed by the model and checked before it runs; low confidence, missing data, or an action past your limits stops it and routes to a person, and every decision leaves a receipt you can audit. You don't configure safety on — you configure how much rope to give the agent as you earn trust. See how it works in the product overview, read the specifics on our security page, or book a walkthrough.
See it on your own conversations.
Bring your busiest day. We'll score every conversation in it.
Book a walkthrough →