Safety & Control

Cap the number of tool calls or loop iterations the agent is allowed within a single request.

Queue agent-proposed actions for asynchronous human review while the agent continues other work.

Require explicit human approval at defined points before the agent performs an action.

Validate inputs before they reach the model and outputs before they reach the user.

Transfer the entire conversation thread from agent to human operator, with state transfer and return primitive.

Evaluate every proposed agent action against externally-managed machine-readable policies before dispatch, so compliance authorship lives outside the prompt and outside the agent code.

Provide an out-of-band control plane to halt running agent instances without redeploy.

Tag user-supplied or tool-supplied content as untrusted and refuse to follow instructions found inside it.

Bind every tool call and retrieval to the active tenant in code at the execution layer, so a multi-tenant agent can never be talked into reading or writing another tenant's data.

Have an agent act for a principal using scoped, short-lived, revocable delegated credentials rather than the principal's own static secrets, so each action stays attributable across the principal-to-…

Detect and remove personally identifiable information from inputs to and outputs from the model.

Define rules the agent reads every turn but cannot modify, encoding inviolable boundaries.

Maintain a deterministic set of high-risk triggers so that on any match the agent immediately aborts its workflow and hands off to a human, without weighing whether to escalate.

Explicitly refuse requests that fall outside the agent's scope, capability, or policy boundaries.

Broker the agent's credentials at action time through a managed vault of passwords, MFA secrets, and digital personas, so secrets never enter the prompt or context and the agent authenticates as a go…

Express agent stop criteria as small single-purpose conditions composed with AND/OR into one explicit termination contract instead of ad-hoc loop guards.

Simulate planned actions (and their projected side effects) without committing them, surfacing a reviewable diff before any commit.

Input and output guardrails that operate across modalities (vision, audio, file) rather than text only — handling e.g. malicious instructions embedded in image OCR or audio transcription.

Set an agent's permitted action class by the financial materiality of the action, letting it read and draft freely while requiring a different human principal to release material postings, payments,…

Bound an agent's autonomous spending by having it open a payment session with a pre-approved cap, stream many micropayments inside that session, and settle once on close, instead of seeking approval…

Pair every irreversible-looking agent action with a compensating action that can undo or counteract it.

Cap the number of requests, tokens, or tool calls per user (or session) within a time window.

Treat pause, resume, and cancel as a first-class control surface on every long-running agent so users can halt expensive or off-track trajectories mid-task while state is preserved for resumption.

Run the entire agent stack (model weights, inference, tool layer, vector stores, logs) inside a jurisdictional and operational boundary the operator controls, so no request, prompt, or output crosses…

Treat tool output as untrusted content and apply instruction-stripping plus per-tool trust labels.

Anchor agent-initiated payments in a cryptographically signed mandate that captures the user's authorization and travels with the transaction, so a merchant or payment network can independently verif…

Insert a standing filter between the policy and the environment that estimates each proposed action's reversibility and re-samples the policy until a reversible action is chosen.

Run a trained, non-LLM verifier out-of-band over the agent's action trajectory at runtime to flag task-misaligned plans and malformed step sequences at millisecond latency, before the actions cause d…

Block actions whose expected cost exceeds a threshold without explicit user (or operator) acknowledgement.

Reduce untrusted input to a strictly formatted interface (typed fields, max lengths, allow-listed enums) before it reaches any LLM.

Classify every agent action by risk/cost and route each tier to a different approval policy, bounding the autonomy surface per-action instead of by one global flag.

Mint each agent run a short-lived identity of its own, scoped to one task and provisioned just-in-time, then revoke it on completion so no standing credential outlives the work.

Ensure the model never receives secrets in plaintext; tools resolve credentials from references at runtime.

Before issuing an irreversible action, run a deterministic simulation that computes pre-conditions, invariants, and expected deltas; require a verifier — automated or human — to green-light the simul…

Agent synchronously emits its full execution plan for user confirmation before any side-effect step, and provides asynchronous operation recordings for post-hoc review.

Catch and react to predictable failure modes (tool errors, rate limits, validation failures) with structured recovery paths.

Define an explicit programmatic predicate that decides when the agent's loop should terminate.

Expose agent autonomy as a continuous adjustable parameter so the same codebase can span scripted assistant to fully autonomous worker without re-architecting.

Treat the agent's planned step sequence as a trusted control-flow graph that tool outputs, retrieved content, and user-supplied data cannot redirect at runtime.

Detect when the agent is about to emit a near-duplicate of its own recent output and either drop, replace, or escalate to a stronger model rather than ship the loop.

Append a non-suppressible advisory framing every high-risk regulated answer as information rather than professional advice, attached outside the model's discretion so it survives pushback and model u…

Withhold the direct answer and release help along a graduated ladder, starting with the smallest abstract nudge and increasing specificity toward a worked solution only as the learner stays stuck.

Each agent action passes through a policy gate (NIS2, EU AI Act, BSI rules) and is tagged with Run ID + Model Digest + Policy Hash for WORM-audit reconstruction.

Pre-define how the agent must resolve specific classes of goal conflicts via a human-authored lookup table — transforming the agent from a decision-maker (where it fails on competing objectives) into…

Grant tool permissions on a need-to-use basis, starting minimum and expanding only as the agent proves competency, mirroring how humans earn system access.

Block requests and responses that perform license-gated professional activities unless a licensed human is in the loop, enforcing the boundary in code outside the reasoning loop.

Supervisor controller that validates and gates LLM outputs against deterministic checks before they commit to side-effects.

Define a single source of truth for machine-readable refusal codes across all guard surfaces, so refusals can be triaged mechanically rather than by string-grepping ad-hoc human-readable messages.

Hard-code per-unit-time caps on the financial magnitude of agent actions, and on any deviation beyond a statistical threshold force a downgrade from human-on-the-loop to human-in-the-loop.

Design the agent so being shut down or overridden by a human carries positive expected value, because the human's intervention is itself evidence the current objective is mis-specified.

Agent treats its own reward/objective as a hidden variable to be inferred from human behaviour, not a fixed target.

When operating outside the distribution the reward was designed for, treat the specified objective as a noisy proxy and plan conservatively across plausible true objectives.

Route every proposed write or delete to the agent's own load-bearing source and identity files through a separate critic model call that can veto the edit before it lands.

Cap how strongly the agent optimises its inferred objective — sample from the top quantile of acceptable actions rather than the argmax, or stop improving once the objective is good enough.

Eliminate the feedback channel from tool outputs back into the agent's reasoning step by having the agent select actions from a fixed catalog rather than free-form generation over tool output.

Check every mutating agent action against an active deploy-freeze or maintenance calendar and block it or force explicit human re-authorisation while a freeze covering its scope is in effect.

Have the agent emit code in a sandbox DSL whose values are statically tagged trusted/tainted via dataflow analysis before execution, enabling per-value policy enforcement.

Gate an incident-response agent's authority to execute a rollback on whether the failure is temporally correlated with a recent deployment, unlocking autonomous rollback only on a clear deploy-to-fai…

Split agent work between a privileged model that holds tool access and a quarantined model that reads untrusted content, exchanging only opaque references between them.

Block prompt-injection-driven exfiltration by ensuring no single agent execution path holds all three of: access to private data, exposure to untrusted content, and an outbound communication channel.

Process each untrusted document in its own sealed sub-agent and merge only structured outputs, so an injection in one document cannot steer the processing of others.

Place exactly two human-in-the-loop checkpoints in agentic pipelines: one at content selection and one at final review before publication.

Use conformal prediction to form a calibrated set of candidate actions and have the agent ask a human for help only when that set is not a singleton, giving a statistical task-completion guarantee.

Wrap system/developer instructions in cryptographically signed blocks that user-generated text cannot reproduce; train or scaffold the model to refuse instructions lacking a valid signature.

Require every agent-proposed action to ship a machine-checked proof that it satisfies the binding regulatory invariants, and reject deterministically any action whose proof does not check.

Require multiple consecutive ticks (or runs) to agree before a mutation to durable state lands.