Human-in-the-loop & least privilege: bounded autonomy by design.
Autonomy is the value proposition of agents and also their primary risk multiplier. The resolution is not "no autonomy" but bounded autonomy: least privilege as the default, and human approval gates placed exactly where they buy the most safety per unit of friction. This essay turns those two ideas into concrete design decisions.
Least privilege as the foundational control
Least privilege is the oldest principle in security and the highest-leverage one for agents, because it attacks impact rather than chasing triggers. Every prompt-injection essay ends in the same place: the attack that matters is the one that produces a harmful action, and an agent that lacks the capability cannot perform the harmful action no matter how thoroughly its context is poisoned.
Applied to agents, least privilege has four dimensions:
- Tool minimality: the agent has only the tools this task needs, not the full catalog "in case."
- Scope minimality: each tool's reach is narrowed — read-only where possible, row/record-limited, destination-allowlisted, single-tenant.
- Credential minimality: short-lived, per-task, narrowly-scoped credentials; no ambient long-lived admin tokens in the agent's environment.
- Temporal minimality: a capability is present only during the state where it is valid, then withdrawn.
The fastest security win for most production agents is closing the gap between "tools the agent has" and "tools this task actually requires." It shrinks every impact category simultaneously and needs no model changes.
Why autonomy needs a checkpoint
A human reviewing each step is a slow but extremely strong control: a competent reviewer rejects the email to attacker@example.com that an injected agent was about to send. Multi-step autonomy is valuable precisely because it removes that reviewer — and removes the control with it. Human-in-the-loop design is about re-introducing the checkpoint selectively, only where its value exceeds its cost.
The failure mode at both extremes:
- Too little: fully autonomous agents with broad tools and no gate — silent compromise, discovered after the damage.
- Too much: approval prompts on every step — operators rubber-stamp out of fatigue, and the gate becomes theater that provides assurance without protection.
Where to place approval gates
Gate by consequence, not by step count. A useful rubric: require human approval when an action is irreversible, externally visible, privilege-changing, or high-value. Let reversible, internal, low-value actions run autonomously.
# Consequence-based gating, not step-based read_internal_doc # auto — reversible, internal draft_reply # auto — no external effect yet send_external_email # GATE — externally visible delete_records # GATE — irreversible grant_access / merge_pr # GATE — privilege / production issue_refund > threshold # GATE — high-value
Design the gate so it actually carries information. A good approval request states what action, with what arguments, why the agent chose it, and what it will affect — and makes "reject" as easy as "approve." A dialog that only says "Allow the agent to continue? [Yes]" trains the exact rubber-stamping it was meant to prevent.
An approval gate that the user cannot meaningfully evaluate is worse than none: it manufactures false assurance and shifts blame to a human who never had the information to decide. If you cannot present a reviewable summary, the action probably should not be automatable at all.
Patterns that keep gates effective
- Batch and summarize: one reviewable plan ("send these 3 emails, update these 2 records") beats 30 isolated pop-ups. Summarization reduces fatigue without reducing oversight.
- Tiered autonomy: earn scope. Low-risk operations auto-run; the agent proposes higher-risk ones; the riskiest always require explicit human action.
- Dry-run / preview: show the diff or the exact request before commit, so the human reviews effects, not intentions.
- Out-of-band confirmation: approve high-impact actions through a channel the in-context attacker cannot also forge — not a "click yes" the agent could be steered to synthesize.
- Default-deny on timeout/uncertainty: if approval is not granted or the agent is unsure, the safe outcome is "do not act," not "proceed."
Gate by consequence, not by step, and the cost collapses: the overwhelming majority of an agent's actions are reversible internal reads that run untouched. Friction lands only on the handful of irreversible or externally-visible actions — exactly where an unattended mistake or a successful injection is most expensive. Pair it with least privilege so most actions never reach a gate at all because the capability is scoped narrow enough to be safe autonomously.
It can advise, never decide. A self-assessment shares the agent's prompt-injection failure mode — the same poisoned context that triggers the bad action can also produce a confident "this is safe, no approval needed." The gate decision must be deterministic and external: a code-level policy keyed to the tool and its arguments, not the model's opinion of its own safety.