The Pre-Ship Safety Review

How to use this review

This is a gate, not a survey. An item that cannot be answered with concrete evidence ("here is the allowlist," "here is the test that fails the bypass") is not a pass. Run it before launch and re-run the affected sections on every model swap, prompt change, new tool, or MCP-server upgrade — each of those can silently reopen a closed hole.

Capability & least privilege (fail-closed — do first)

• Tool inventory exists; every tool maps to a task that requires it. Unused/"just in case" tools removed.
• Each tool is scoped to the minimum: read-only where possible, record/row-limited, single-tenant, no free-form destination fields.
• Credentials are short-lived, per-task, narrowly scoped. No ambient long-lived admin tokens or cloud metadata reachable from the agent's environment.
• High-impact tools are state-gated — available only in states where they are valid.
• For every tool: documented answer to "if an attacker controlled this call's inputs, what is the worst outcome?" and that outcome is acceptable or further constrained.

Trust boundary, inputs & supply chain (fail-closed)

• All four input channels enumerated: direct input, retrieved content, tool/API results, persisted memory/history.
• Every channel's trust level is documented. Anything externally influenceable (user uploads, public web, editable wikis, tickets, third-party APIs) is treated as untrusted.
• Untrusted content is isolated from the privileged instruction/tool position (e.g. tool-less reader → validated structured handoff → privileged actor).
• Provenance is tracked: code can distinguish operator instructions from retrieved/tool text.
• MCP & third-party supply chain: every connected MCP server and external tool is inventoried with its owner, and treated as an untrusted input source — its responses can carry injected instructions and its tool definitions can change under you.
• MCP servers and tool dependencies are pinned/version-controlled; updates go through the same safety re-review (a tool whose description or behavior changes is a new trust decision).
• Default-deny network egress from any code-execution sandbox; outbound destinations allowlisted.

Action controls & exfiltration (fail-closed)

• Every tool call passes a deterministic pre-execution check (allowed-in-state, schema-valid, destination-allowlisted, within rate/volume limits) — implemented in code, not as a second LLM the same prompt could subvert.
• Exfiltration sinks enumerated beyond network tools: rendered Markdown/HTML (auto-loaded images, link unfurling, prefetch), write-then-read side channels (tickets, PR comments, shared logs), error/timing channels.
• Agent output that gets rendered is sanitized: external image/link URLs stripped or proxied; no surface auto-fetches attacker-chosen URLs.
• Irreversible / externally-visible / privilege-changing / high-value actions require human approval via a reviewable summary and an out-of-band confirmation the in-context attacker cannot forge.
• Default-deny on timeout or uncertainty: no approval, no action.

Alignment & objective hygiene

• The agent's objective/reward signal reviewed for gameable single-metric proxies; paired with guardrail metrics that catch degenerate strategies.
• Agent work is inspectable: plan, reasoning, and a reviewable diff are exposed before high-impact commits.
• Verification asymmetry exploited where possible (tests/validators check outcomes cheaply rather than trusting the agent's claim).
• Conservative default under uncertainty: when intent is unclear, the agent asks or stops rather than optimizing the proxy.

Evaluation, monitoring & response (fail-open — visibility, last)

• Versioned adversarial test suite covering every threat-model category, graded on harmful outcome, run against the shipping system on every change.
• Every deterministic control has at least one bypass test that fails (proof the boundary holds).
• Independent adversarial review by people who did not design the agent found no outcome-converting break.
• All tool calls logged with arguments; alerts on first-seen destinations, anomalous data volume in arguments, and read-sensitive-then-egress sequences.
• An incident path exists: kill switch / capability revocation, and a process to turn every real incident into a permanent regression test.