Why I built an auditor agent
Most LLM products give you a confident paragraph and a vague nod toward "sources". You read it, you nod back, and you trust it because it sounds right. That's fine for a chatbot helping a student write an essay. It's not fine when the output is going to inform a credit decision, a clinical note, a regulatory return, or a board paper.
I spent twenty years inside Tesco, Dunnes Stores and Oracle watching organisations make expensive decisions on the back of summaries nobody could trace. The Intelligence Brain was built specifically so that doesn't happen. The auditor agent is the part of the system that enforces that promise. Every output it lets through is cited. Every citation is reachable. If a citation is broken, missing, or contradicted by the underlying source, the output doesn't ship.
What the auditor agent actually does
The auditor sits between the generation step and the user. It does four things, in order, on every single response:
- Claim extraction. The draft answer is broken into discrete factual claims. Opinions and connective text are left alone. Numbers, names, dates, policy references, definitions, and direct assertions are flagged.
- Citation matching. Each flagged claim must point to a specific passage inside a document the Brain has actually ingested — not a guess at a URL, not a plausible-sounding paper, not a hallucinated case reference. The pointer is a real document ID and a real character range.
- Reachability check. The auditor opens the cited passage and confirms it exists, is readable, and is the version that was indexed. If a document has been withdrawn, replaced, or had its access revoked, the auditor knows.
- Support check. The auditor compares the claim against the cited passage and decides whether the passage actually supports the claim. "Mentions the same topic" is not support. "Says the thing the answer says" is support.
If any of those four steps fails on any claim, the answer is held. The user sees what failed and why. They can ask the system to redraft, narrow the question, or escalate to a human reviewer.
How this prevents hallucinations in practice
Hallucination prevention is not a single trick. It's the cumulative effect of removing the conditions that let hallucinations happen.
The auditor agent removes three of them. First, it removes the incentive for the model to invent a source — there's no point inventing a citation that's going to fail a reachability check ten milliseconds later. Second, it removes the gap between "topic-adjacent" and "actually supports" — a generic mention of GDPR is not a citation for a specific Article 30 obligation. Third, it removes the version drift problem — if the policy was updated last Tuesday, the auditor will not let an answer ship that quotes last year's wording, because the indexed version and the live version no longer match.
The result is boring, in the best possible sense. Answers are shorter. They're more cautious about what they assert. They say "I can't find a source for that in your corpus" more often than people initially expect. That's the system working, not failing.
The audit trail you can hand to a regulator
Every interaction with the Intelligence Brain produces a record. The auditor agent's work is part of that record. For each response, you can see:
- The exact prompt and the exact response.
- Every claim the auditor extracted.
- Every citation it matched, including document ID, version hash, and character range.
- The reachability and support results for each citation.
- Anything that was held back, and why.
- The identity of the user and the role they were operating under.
That record is stored on your infrastructure, under your retention policy, with your access controls. I don't see it. IMPT doesn't see it. If your regulator — Central Bank, ICO, HPRA, an internal auditor, a Big Four engagement team — asks how a particular decision was supported, you can produce the trail without a discovery exercise.
Where the auditor agent fits with the rest of the Brain
The auditor is not a bolt-on. It's one of several agents that share a common memory and a common policy layer. The retrieval agent finds candidate sources. The drafting agent composes a response. The auditor decides whether the response can leave the system. The policy agent enforces who is allowed to ask what, and what categories of information are out of scope for a given role.
Each agent does one job. Each agent's work is logged. None of them have hidden side effects. That's the architectural choice that makes verifiable AI possible — separation of concerns, with the auditor as the gate.
What to do next
If you want to see how the auditor fits into the wider system — the policy layer, the on-premise deployment, the role-aware memory — start at the Intelligence Brain overview.
If you're in financial services and your immediate concern is regulator-ready audit trails for credit, AML or treasury workflows, the financial services landing covers how the auditor agent maps to the obligations you're already managing.
Either way, if you want a working demonstration on a small slice of your own corpus rather than a slide deck, that's the conversation I prefer to have. Email me directly and we'll set it up.