The refusal is not the hard part
Input classifiers, output schema validation, a deny list for actions, a check that the generated SQL touches only the tenant's own schema — none of this is difficult engineering. Most teams have it working within a fortnight. Then the feature meets real traffic and the interesting problem arrives: three to six per cent of requests get stopped, and those requests belong to people with work to do.
If there is no path for them, the guardrail becomes a wall and users route around it — back to email, back to the spreadsheet, back to ringing the operations desk. We have watched a well-built assistant lose two thirds of its weekly active users in a month, not because it was wrong, but because being blocked with no recourse is worse than never asking.
Do the staffing arithmetic before you pick a threshold
A support assistant at Tinsmith Retail handled roughly 12,000 requests a day. A 5 per cent escalation rate is 600 human reviews. At 45 seconds of median handling time that is seven and a half hours — one reviewer, plus cover for absence and peak, so call it two heads on staggered shifts. That arithmetic has to be done before the threshold is chosen, not after.
When the sums do not work, one of three things is wrong: the threshold is too aggressive, the reviewer tooling is too slow, or the feature should not exist in that form. We have taken all three decisions on different projects. What we do not do is ship a guardrail whose escalations land in a shared inbox nobody owns.
Give the queue a service level and publish it. Ours is typically 15 minutes during business hours for anything that blocks a user in-session, and four hours for asynchronous work. Reviewers need the original request, the retrieved sources, the blocked output and a one-click correct-and-send, or the 45 seconds becomes three minutes and the model breaks.
Constrain the blast radius, not the wording
Most of the useful controls live below the prompt. Tools are read-only by default; a write tool is added deliberately, with a scope. Monetary actions carry hard limits in the tool layer — an agent at Tinsmith could issue a credit note up to 500 pounds and above that could only draft one for a supervisor. Tenancy is enforced by the data access layer, never by an instruction in a system prompt.
Treat every model output as untrusted input, in the same way you would treat a form field from the public internet. If the model summarises a supplier's PDF and that summary is then passed to a tool, the PDF's author has an input channel into your tool calls. Indirect prompt injection through retrieved documents is the failure mode we see most often in enterprise deployments, and it is invisible to any check that only looks at what the user typed.
The design test we apply: if the model were replaced by an adversary with the same permissions, what is the worst thing that happens in one call, and in a hundred? If the answer is unbounded, the permissions are wrong, and no amount of prompt hardening fixes that.
Log what you would need in a dispute
At Aldridge Risk the requirement was set by a regulator's question rather than by an engineer: show me why this customer was told that, on that date. Satisfying it means retaining the full prompt, the identifiers and content hashes of every retrieved document, the model identifier and version, every tool call with arguments and results, the guardrail verdicts, and the reviewer's decision with their name against it.
That record is roughly 8 to 20 kilobytes per interaction before compression. At a million interactions a year it is trivial storage and enormous relief. Retention was set at 24 months to match the complaints window, with personal data separated so that a deletion request removes the customer's content without destroying the audit chain.
Write the log before the response is streamed to the user, not after. Logging on the success path only means the failures — the ones that generate the disputes — are precisely what you cannot reconstruct.
Thresholds are tuned forever
The metric that matters most is the false-block rate: legitimate requests stopped for no good reason. Nobody instruments it by default because it requires the reviewer to record that the block was unnecessary, which takes one extra click. Add the click. A guardrail that wrongly blocks one request in five will be worked around within a month, and you will not hear about it until usage has already gone.
We publish four numbers to the delivery team every week: block rate, false-block rate, median time in queue, and the count of items that breached the service level. Three of those are about the humans. That ratio is about right, because the classifier was never the difficult part.