Audit trails and provenance — if you cannot reconstruct the decision, it did not happen.
Governance begins with one unforgiving question: months later, can you prove exactly why this agent took that action? Not approximately — exactly: which model build, which prompt version, which retrieved documents, which tool outputs, which policy was in force. An audit trail is not a log file; it is the evidentiary record that lets a regulator, an auditor, or an incident reviewer replay a contested decision and reach the same conclusion you did. This essay covers what to capture, how to make it tamper-evident, how long to keep it, and the provenance chain that makes any of it credible.
Capture enough to reconstruct any single decision.
The test for an audit record is reconstruction: from the record alone, a reviewer must be able to explain why the agent did what it did, without re-running anything. That means capturing the decision inputs (the rendered prompt sent to the model, not the template), the decision (raw model output, parsed tool call, arguments), the provenance (model id and build, prompt version, retrieved document ids and hashes, tool versions), the context (which policy version was enforced, who the principal was, what authority was delegated), and the effect (what the tool actually did, the external receipt or transaction id). A trace built for debugging (covered under Evaluation & Observability) is the substrate; the audit trail is that trace with provenance and an integrity guarantee added so it survives scrutiny by someone who does not trust you.
Make it tamper-evident, not merely stored.
An audit trail you can silently edit is worth nothing in a dispute — the other party will simply assert you changed it. Tamper-evidence means any modification or deletion is detectable after the fact. The standard primitive is a hash chain: each record carries the hash of the previous one, so altering an old entry breaks every link after it. Append-only, write-once storage and an external anchor (a periodic signed digest published where you cannot rewrite it) raise the bar further. The goal is not unbreakable cryptography — it is that nobody, including you, can quietly rewrite history.
# hash-chained, append-only audit record prev = store.last_hash() rec = { "ts": now, "actor": "agent:fin-ops", "decision": decision, "provenance": prov, "policy_version": "pol@2026-05-12", "prev_hash": prev, } rec["hash"] = sha256(canonical(rec)) store.append(rec) # WORM; never update in place
Hash-chaining is cheap and proves order and integrity. Reserve heavier machinery (external timestamping authorities, transparency logs) for the records a regulator or court will actually examine — financial actions, data deletions, anything irreversible.
Provenance: the model, the prompt, the tools, the data — all four.
A decision is only explainable if you can name every input that shaped it. Agent provenance has four strands and a gap in any one breaks the chain. The model: exact build or snapshot id, not "GPT-class". The prompt: the versioned system prompt and the exact assembled context, content-addressed by hash. The tools: which tool versions were callable and which were invoked. The data: for every retrieved document, a stable id and content hash so you can prove what the agent actually read — not what that source says today. Documentation conventions like model cards and data cards exist precisely so the model and dataset strands are recorded in a comparable form rather than reinvented per team.
- Content-address the context — hash the assembled prompt so "what did it see" is answerable byte-for-byte, not reconstructed from a template.
- Pin retrieved sources — store the document version/hash at read time; a knowledge base mutates and "what it says now" is not evidence.
- Stamp the triple on every run — (model, prompt, tools) on the record links a decision to a specific, reproducible behavior.
Retention is a policy decision, not a disk-space decision.
How long you keep audit records is set by obligation, not convenience. Regulated domains carry statutory minimums (often measured in years); data-protection law pulls the other way with minimization and the right to erasure. These collide: you may be required to retain proof that a decision was made while also being required to delete the personal data inside it. The resolution is structural separation — keep the immutable decision skeleton (what, when, which policy, which model) for the long statutory period, and hold linked personal data under a shorter, separately-governed lifecycle so it can be redacted or tombstoned without breaking the hash chain.
"Keep everything forever" is not a retention policy — it is an unbounded liability and a discovery target. Undefined retention is a finding in its own right.
Design the trail for the person who does not trust you.
The audience for an audit trail is adversarial by default: an external auditor, a regulator after an incident, opposing counsel, a customer disputing a charge. That reframes every design choice. Records must be intelligible to someone without your codebase — stable identifiers, human-readable decision summaries, the policy text that was in force, not just a version number. The trail must be queryable along the questions they will ask: every action affecting this person, every decision under this policy, everything this agent did in this window. If answering "what did the agent do to account X and why" requires a forensic engineering project, the trail has already failed its only real test.
The honest tradeoff.
Evidentiary-grade trails cost real money: storage of full context grows fast, content can carry PII and secrets that need redaction at the boundary, and tamper-evidence adds write-path complexity. Capturing less is cheaper right up to the day you need to prove a decision and cannot — at which point the absence of the record is the adverse finding. Decide deliberately which decisions are audit-grade (irreversible, regulated, high-consequence) and make those reconstructable and tamper-evident; do not pretend a debug log is an audit trail.