An agent that can only answer questions is a feature. An agent that can send an email, update a record or quote a price is a member of staff with no manager, no contract and no memory of what it agreed yesterday. Almost everything written about building them is about the first kind. This is about the second, and about the machinery you have to put around it before it is allowed near a customer.
We can publish this in full because we own the system it describes. That is rarer than it should be in this subject.
The failure mode is not what people prepare for
The anticipated failure is the agent saying something false. Real, and largely solvable by grounding it in your own material and having it decline when it does not know.
The failure that actually causes damage is different: the agent doing something reasonable that it was not authorised to do. Offering a discount to close a deal. Promising a delivery date. Answering a question about a contract. Every one of those is a defensible action in isolation, and every one is a commitment made on your behalf by something that cannot be held to it.
Agents are named and scoped, never general
There is no single agent. There are several, each with one job, each with access to only the tools that job requires. The agent that drafts outreach cannot read the contract repository. The agent that answers technical questions cannot send anything externally. The agent that qualifies an inbound enquiry cannot write to the CRM without a pass through the compliance step.
This is unglamorous and it is most of the safety. A general agent with every tool has a combinatorial number of ways to act badly, and you cannot enumerate them. Five narrow agents have five surfaces you can each describe in a sentence, and a reviewer can hold all five in their head at once.
Autonomy is declared, per action, not per agent
Every action an agent can take carries a level, and the level is a property of the action rather than of the agent or of anyone's confidence in the model. It does not change because the system has been behaving well.
| Level | Behaviour | Typical action |
|---|---|---|
| Observe | Reads, never writes, never sends | Enriching a record, summarising a thread |
| Draft | Produces output that a person must approve before it moves | Outreach copy, a proposed reply, a suggested price |
| Act, reversible | Executes alone; the effect can be undone | Adding a tag, creating an internal task, updating a status |
| Act, irreversible | Requires a named human approval every time, with no exceptions and no batching | Sending anything external, committing to a price or a date, changing a customer record |
The line that matters is reversibility. Anything a person could undo in a minute can run unattended. Anything that leaves the building, touches money, or would require an apology to reverse takes a human name. We have never found a case where relaxing that saved enough time to be worth the exposure.
Nothing leaves without a compliance pass
Every outbound artefact goes through a separate check before it is allowed to leave — a distinct step with a distinct prompt and, importantly, no ability to edit what it is reviewing. It approves or it rejects with a reason. Letting the reviewer rewrite the thing collapses two roles into one and you lose the check.
What it looks for is specific rather than general "is this appropriate": claims we cannot substantiate, numbers that do not appear in our own published material, commitments to dates or prices, anything resembling legal or financial advice, and tone that misrepresents the relationship — familiarity with someone we have never spoken to, or certainty about a project we have not scoped.
It rejects more than people expect, and the rejections are the most useful output of the whole system. They are a live readout of where the generation step is drifting, and they arrive before a customer sees it rather than after.
One person holds the switch
A single named individual can stop every agent immediately, and that authority is not shared, not delegated, and not behind a deployment. Not a config change, not a pull request — a switch.
This sounds like a small implementation detail and it is the thing that makes the rest defensible. Every other control assumes the system is behaving as designed. The switch is the one that works when it is not, and the reason it is one person rather than a team is that in the situation where you need it, "who is calling it" must not be a question anyone has to ask.
The log is a record, not telemetry
Every action records what was attempted, which agent attempted it, what it was working from, whether it was approved or rejected and by whom, and what actually happened. Written before the action, not after — a log written on success does not exist for the cases you most need it.
The requirement to design against is not debugging. It is being able to answer, six months later, why a specific customer received a specific message. If you cannot reconstruct that from the record, you do not have a governed system — you have an unsupervised one with good intentions.
What we got wrong
We started with one capable agent and a broad toolset, because it was faster to build and demonstrated better. It was ungovernable in exactly the way described above: every new tool multiplied the behaviours we would have had to enumerate to review it honestly. Splitting into narrow named agents felt like a step backwards in capability and was the change that made the system shippable.
We also under-built the approval interface at first. If approving a drafted action is slow or awkward, the human in the loop becomes a rubber stamp within a fortnight — they stop reading and start clicking, and you now have an unsupervised system that produces an audit trail claiming otherwise, which is worse than having no approval step at all. Approval has to be faster than the alternative or it decays into theatre.
Compliance rejection rate, and the top three rejection reasons
From the runtime logs. This is the number that demonstrates the check is real rather than decorative.
Median time from drafted to approved
The measure of whether the approval step is genuinely usable or quietly decaying into rubber-stamping.
The question is not whether the model is good enough to be trusted. It is what happens on the day it is not, and whether anyone finds out before the customer does.