Verifying AI decisions before they act: the case for a verification layer

An AI verification layer is the part of an agentic system that checks a decision before it takes effect. It screens what goes into the model, enforces policy on what an agent proposes to do, validates the output against rules and evidence, routes decisions that need a person to human approval, and records every check. Nothing consequential reaches the real world without passing it.

This article makes the case for treating verification as a dedicated layer rather than a few prompts and filters, and describes what that layer needs to contain.

Why agents need verification

Three properties of agents make verification necessary rather than optional.

• Agents act. Their output is a change in a system — a record, a payment, a machine state — not a paragraph someone can ignore.
• Agents read text they did not write. Documents, emails, web pages and tool results all flow into the model, and any of them can carry instructions. Research on indirect prompt injection has shown how content retrieved by an application can steer the model that reads it.
• Models are probabilistic. The same input can produce different outputs, and a fluent answer is not necessarily a correct one.

None of these can be fixed by a better prompt. They are fixed by architecture: checks that sit outside the model and do not depend on the model behaving well.

What a verification layer checks

Input screening

User input and retrieved content are checked for instruction-like patterns, attempts to override the system's rules, and data that policy says must not be sent to a particular model. Screening reduces risk; it does not remove it, which is why the following layers exist.

Least privilege and tool policy

Each agent has an explicit allow-list of tools, and each tool call is checked against policy: which agent, which system, which action, which limits. If an agent is manipulated into wanting something it was never granted, the call fails. Least privilege limits the blast radius of every other failure.

Output validation

Before an output is used, it is checked against rules and evidence. A grounded answer should be traceable to the passages it came from; a proposed value should fall within expected bounds; a structured result should match its schema. Outputs that fail are replaced with a safe fallback or escalated.

Human approval

Good systems decide in advance which actions need a person, route them to the right person with the context needed to decide, and record the decision. Asking a human to approve everything is not oversight; it is a queue people learn to click through.

The audit trace

Every check, approval and outcome is written to a trace. The trace is how you debug, how you evaluate a change before rolling it out, and how you answer an auditor, a customer or your own team when they ask why something happened.

Layers, not a single gate

No single control is reliable enough on its own. Screening misses things; models can be confidently wrong; people approve under time pressure. Verification works because the layers are independent: an injection that slips past screening still meets least-privilege tools; an action within the allow-list still meets policy limits; a consequential action still meets approval; and everything is recorded. The OWASP Top 10 for LLM Applications (opens in a new tab) catalogues the failure modes these layers are designed to contain.

Verification in the RUUT system

In the RUUT Agent Loop, verification is its own stage. RUUT FORGE sits on both sides of RUUT BRAIN: it guards what enters reasoning and checks what reasoning proposes before any agent acts. For example, when RUUT FOREMAN observes a safety violation, an alert to the shift supervisor can proceed within policy, but stopping the line is routed to the supervisor for approval — and both paths are recorded.

This website applies the same idea at small scale. ATLAS, our site concierge, screens each question, answers only from published content, checks its answer against what it retrieved, and hands off to a person when it cannot ground a reply. The agent trace you see in the ATLAS panel is that pipeline, step by step.

Common objections

"Won't verification slow the agent down?"

Most checks are fast, deterministic code — allow-lists, schema validation, policy rules — and add little time compared with a model call. The checks that take real time, such as human approval, are reserved for actions where speed matters less than being right. Fast where it can be, careful where it must be.

"Isn't human approval a bottleneck?"

Only if everything needs it. The point of a policy is to decide in advance which actions are routine and which are consequential. Routine actions proceed within limits; consequential ones go to the right person with the context to decide quickly. Approval stays rare enough to be taken seriously.

"Won't a better model make this unnecessary?"

Better models make fewer mistakes, but agents still read untrusted text and still act in systems where one error is costly. Verification does not assume the model is weak; it assumes that any component can fail and that the consequences of action deserve an independent check.

Questions to ask of any AI system that acts

1. What is checked before a consequential action executes, and who defines the rules?
2. What is each agent prevented from doing, regardless of its instructions?
3. Which actions require a person, and is that enforced by the system rather than by convention?
4. How are retrieved documents and tool results screened before the model reads them?
5. Can you reconstruct exactly why a given action happened?

References

1. Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (Greshake et al.) (opens in a new tab) — arXiv
2. OWASP Top 10 for Large Language Model Applications (opens in a new tab) — OWASP Gen AI Security Project
3. AI Risk Management Framework (opens in a new tab) — National Institute of Standards and Technology (NIST)
4. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al.) (opens in a new tab) — arXiv