AI代理安全模式

AI代理安全模式是代理操作的硬性限制：它无法超过的预算、未经批准不能采取的行动、无法遵循的提示、无法逃离的环境、触发杀死开关的行为。它们是故意消极的——每个模式的约束槽命名了被禁止的内容——因为这就是强大生成器无法绕过的唯一类型的约束。

LLM系统中的安全性不是单一机制，而是一个堆栈。步骤预算限制失控循环；沙盒隔离包含工具副作用；审批队列控制破坏性行为；宪法章程以书面形式声明政策，代理在每次回合中阅读；致命三重威胁模型命名了将普通代理转变为外泄风险的能力组合（不受信输入 + 私密数据 + 外发行动）。下面的模式是该堆栈的命名部分。

Field-tested patterns to start with

• Step Budget — Cap the number of tool calls or loop iterations the agent is allowed within a single request.
• Kill Switch — Provide an out-of-band control plane to halt running agent instances without redeploy.
• Constitutional Charter — Define rules the agent reads every turn but cannot modify, encoding inviolable boundaries.
• Approval Queue — Queue agent-proposed actions for asynchronous human review while the agent continues other work.
• Sandbox Isolation — Run agent-emitted code or actions in a contained environment with restricted filesystem, network, and process privileges.
• Input/Output Guardrails — Validate inputs before they reach the model and outputs before they reach the user.
• Refusal — Explicitly refuse requests that fall outside the agent's scope, capability, or policy boundaries.
• Lethal Trifecta Threat Model — Block prompt-injection-driven exfiltration by ensuring no single agent execution path holds all three of: access to private data, exposure to untrusted content, and an outbound communication channel.
• Rate Limiting — Cap the number of requests, tokens, or tool calls per user (or session) within a time window.
• PII Redaction — Detect and remove personally identifiable information from inputs to and outputs from the model.
• Human-in-the-Loop — Require explicit human approval at defined points before the agent performs an action.
• Policy-as-Code Gate — Evaluate every proposed agent action against externally-managed machine-readable policies before dispatch, so compliance authorship lives outside the prompt and outside the agent code.
• Compensating Action — Pair every irreversible-looking agent action with a compensating action that can undo or counteract it.

Recommended reading

• Safety & Control — 72 patterns
• Governance & Observability — 46 patterns

Or open the full contents for all 562 patterns in 14 books.

Related guides

• AI Agents Patterns — AI agents patterns: named, reusable shapes for building AI agents that reason, use tools, coordinate, and stay safe — single-agent loops an…
• AI Agents Patterns Catalog — The AI agents patterns catalog: a complete, GoF-formal pattern language for AI agents across reasoning, planning, tool use, retrieval, memo…
• LLM Agent Design Patterns — A GoF-formal catalog of LLM agent design patterns: ReAct, tool use, plan-and-execute, reflection, step budget, and more. Each pattern decom…
• Agentic Design Patterns — A GoF-formal catalog of agentic design patterns — named, reusable shapes for building autonomous AI agents: agent loops, tool use, planning…
• Agentic AI Design Patterns — Agentic AI design patterns for systems already in production — what to ship, what to observe, what to budget, what to gate. Augmented LLM,…
• AI Agent Design Patterns — How to build an AI agent: the named shapes you reach for during design and implementation — reasoning (ReAct, plan-and-execute, reflection)…
• Agent Design Patterns — Agent design patterns treat the agent loop as a software-engineering primitive: an observe→reason→act cycle wrapped in tools, memory, super…
• Agentic Patterns — A complete pattern language for agentic systems, organised in Alexander-style books across reasoning, planning, tool use, retrieval, verifi…
• Agentic AI Architecture — How to structure agentic AI: the architectural patterns that hold an LLM-powered system together. Supervisor, orchestrator-workers, augment…
• RAG Agent Patterns — Patterns for building retrieval-augmented generation agents: naive RAG, agentic RAG, hybrid search, cross-encoder reranking, contextual ret…
• Multi-Agent Patterns — Patterns for coordinating multiple LLM agents: supervisor, orchestrator-workers, handoff, debate, hierarchical agents, swarm, role assignme…