Safety & Control

Cap the number of tool calls or loop iterations the agent is allowed within a single request.

Queue agent-proposed actions for asynchronous human review while the agent continues other work.

Require explicit human approval at defined points before the agent performs an action.

Transfer the entire conversation thread from agent to human operator, with state transfer and return primitive.

Validate inputs before they reach the model and outputs before they reach the user.

Provide an out-of-band control plane to halt running agent instances without redeploy.

Express agent stop criteria as small single-purpose conditions composed with AND/OR into one explicit termination contract instead of ad-hoc loop guards.

Define rules the agent reads every turn but cannot modify, encoding inviolable boundaries.

Pair every irreversible-looking agent action with a compensating action that can undo or counteract it.

Treat pause, resume, and cancel as a first-class control surface on every long-running agent so users can halt expensive or off-track trajectories mid-task while state is preserved for resumption.

Block actions whose expected cost exceeds a threshold without explicit user (or operator) acknowledgement.

Cap the number of requests, tokens, or tool calls per user (or session) within a time window.

Classify every agent action by risk/cost and route each tier to a different approval policy, bounding the autonomy surface per-action instead of by one global flag.

Tag user-supplied or tool-supplied content as untrusted and refuse to follow instructions found inside it.

Catch and react to predictable failure modes (tool errors, rate limits, validation failures) with structured recovery paths.

Explicitly refuse requests that fall outside the agent's scope, capability, or policy boundaries.

Expose agent autonomy as a continuous adjustable parameter so the same codebase can span scripted assistant to fully autonomous worker without re-architecting.

Run the entire agent stack (model weights, inference, tool layer, vector stores, logs) inside a jurisdictional and operational boundary the operator controls, so no request, prompt, or output crosses…

Treat tool output as untrusted content and apply instruction-stripping plus per-tool trust labels.

Design the agent so being shut down or overridden by a human carries positive expected value, because the human's intervention is itself evidence the current objective is mis-specified.

Agent treats its own reward/objective as a hidden variable to be inferred from human behaviour, not a fixed target.

When operating outside the distribution the reward was designed for, treat the specified objective as a noisy proxy and plan conservatively across plausible true objectives.

Cap how strongly the agent optimises its inferred objective — sample from the top quantile of acceptable actions rather than the argmax, or stop improving once the objective is good enough.

Detect and remove personally identifiable information from inputs to and outputs from the model.

Define an explicit programmatic predicate that decides when the agent's loop should terminate.

Eliminate the feedback channel from tool outputs back into the agent's reasoning step by having the agent select actions from a fixed catalog rather than free-form generation over tool output.

Have the agent emit code in a sandbox DSL whose values are statically tagged trusted/tainted via dataflow analysis before execution, enabling per-value policy enforcement.

Reduce untrusted input to a strictly formatted interface (typed fields, max lengths, allow-listed enums) before it reaches any LLM.

Treat the agent's planned step sequence as a trusted control-flow graph that tool outputs, retrieved content, and user-supplied data cannot redirect at runtime.

Detect when the agent is about to emit a near-duplicate of its own recent output and either drop, replace, or escalate to a stronger model rather than ship the loop.

Have an agent act for a principal using scoped, short-lived, revocable delegated credentials rather than the principal's own static secrets, so each action stays attributable across the principal-to-…

Simulate planned actions (and their projected side effects) without committing them, surfacing a reviewable diff before any commit.

Split agent work between a privileged model that holds tool access and a quarantined model that reads untrusted content, exchanging only opaque references between them.

Block prompt-injection-driven exfiltration by ensuring no single agent execution path holds all three of: access to private data, exposure to untrusted content, and an outbound communication channel.

Process each untrusted document in its own sealed sub-agent and merge only structured outputs, so an injection in one document cannot steer the processing of others.

Input and output guardrails that operate across modalities (vision, audio, file) rather than text only — handling e.g. malicious instructions embedded in image OCR or audio transcription.

Evaluate every proposed agent action against externally-managed machine-readable policies before dispatch, so compliance authorship lives outside the prompt and outside the agent code.

Each agent action passes through a policy gate (NIS2, EU the agent Act, BSI rules) and is tagged with Run ID + Model Digest + Policy Hash for WORM-audit reconstruction.

Pre-define how the agent must resolve specific classes of goal conflicts via a human-authored lookup table — transforming the agent from a decision-maker (where it fails on competing objectives) into…

Grant tool permissions on a need-to-use basis, starting minimum and expanding only as the agent proves competency, mirroring how humans earn system access.

Ensure the model never receives secrets in plaintext; tools resolve credentials from references at runtime.

Before issuing an irreversible action, run a deterministic simulation that computes pre-conditions, invariants, and expected deltas; require a verifier — automated or human — to green-light the simul…

Supervisor controller that validates and gates LLM outputs against deterministic checks before they commit to side-effects.

Agent synchronously emits its full execution plan for user confirmation before any side-effect step, and provides asynchronous operation recordings for post-hoc review.

Place exactly two human-in-the-loop checkpoints in agentic pipelines: one at content selection and one at final review before publication.

Define a single source of truth for machine-readable refusal codes across all guard surfaces, so refusals can be triaged mechanically rather than by string-grepping ad-hoc human-readable messages.

Wrap system/developer instructions in cryptographically signed blocks that user-generated text cannot reproduce; train or scaffold the model to refuse instructions lacking a valid signature.

Require multiple consecutive ticks (or runs) to agree before a mutation to durable state lands.