{"$schema":"https://github.com/agentpatternscatalog/patterns","composition_count":250,"compositions":[{"id":"openclaw-rl","name":"OpenClaw-RL","kind":"framework","category":"agent-sdk","build_surface":"full-code","vendor":"Gen-Verse","language":"Python","license":"Apache-2.0","status":"active","first_released":"2026-02-26","url":"https://openclaw.ai","repo":"https://github.com/Gen-Verse/OpenClaw-RL","docs_url":"https://github.com/Gen-Verse/OpenClaw-RL","intent":"Train personalised LLM agents by turning live multi-turn conversations into fully-asynchronous RL training signals across terminal, GUI, software-engineering, and tool-call settings.","description":"OpenClaw-RL is an Apache-2.0 reinforcement-learning framework from Gen-Verse that wraps a self-hosted model as an OpenAI-compatible API, intercepts live conversations through the OpenClaw plugin, and runs four async loops (agent serving, rollout collection, PRM/judge evaluation, policy training) that continuously optimise the policy without interrupting usage. Two paradigms are supported: Binary RL (GRPO with a Process Reward Model) and On-Policy Distillation (OPD via a judge model that emits textual hints), plus a Hybrid combination. Technical report on arXiv (2603.10165, 2026-03-10) reached #1 on HuggingFace Daily Papers.","primary_use_cases":["personalising a self-hosted agent from a single user's conversational feedback","scaling RL training across terminal, GUI, SWE, and tool-call agent environments","continuously updating a deployed policy without taking the inference endpoint offline"],"agent_loop_shape":"Four independent asynchronous loops (serving, rollout, judge, training) instead of a single synchronous agent loop. Conversation traffic flowing through an OpenAI-compatible wrapper feeds the rollout collector; the trainer updates the policy in the background while serving and judging continue concurrently.","key_concepts":[{"name":"OpenClaw plugin","summary":"OpenAI-compatible wrapper around a self-hosted model that intercepts live multi-turn conversations and forwards them into the RL training pipeline.","url":"https://openclaw.ai"},{"name":"PRM / Judge","summary":"Process Reward Model and Judge component that scores each turn asynchronously, with majority voting for robustness.","maps_to_pattern":"agent-as-judge","url":"https://github.com/Gen-Verse/OpenClaw-RL"},{"name":"Binary RL (GRPO)","summary":"Per-turn scalar reward from the PRM combined with GRPO advantage estimation and a PPO-style clipped surrogate loss.","url":"https://github.com/Gen-Verse/OpenClaw-RL"},{"name":"On-Policy Distillation (OPD)","summary":"Judge extracts a textual hint from next-state hindsight; the token-level log-prob gap between teacher and student becomes a directional advantage signal.","url":"https://github.com/Gen-Verse/OpenClaw-RL"},{"name":"Fully-asynchronous 4-component architecture","summary":"Serving, rollout, judge, and training run as independent loops that do not block one another.","maps_to_pattern":"evaluator-optimizer","url":"https://github.com/Gen-Verse/OpenClaw-RL"}],"pattern_composition":"flowchart TD\n  USER[User conversation] --> PLUGIN[OpenClaw plugin<br/>OpenAI-compatible wrapper]\n  PLUGIN --> SERVE[Async loop: Agent serving]\n  SERVE --> ROLLOUT[Async loop: Rollout collection]\n  ROLLOUT --> JUDGE[Async loop: PRM / Judge<br/>scores each turn + majority voting]\n  JUDGE -->|Binary RL signal| TRAINER[Async loop: Policy training]\n  JUDGE -->|OPD textual hints| TRAINER\n  TRAINER -->|updated policy| SERVE\n  PLUGIN -.->|tool calls| TOOLS[(Terminal / GUI / SWE / tool-call envs)]\n  TOOLS -.->|next-state observation| ROLLOUT","members":[{"pattern":"agent-as-judge","role":"core","note":"An asynchronous Process Reward Model / judge model scores each turn; majority voting is used when needed for robust scoring. Judge-emitted hints also drive the OPD training signal.","evidence":[{"type":"repo","title":"OpenClaw-RL README — Features","url":"https://github.com/Gen-Verse/OpenClaw-RL","quote":"Runs PRM/judge evaluation asynchronously, with majority voting when needed for more robust scoring","accessed":"2026-05-20"},{"type":"repo","title":"OpenClaw-RL README — OPD method","url":"https://github.com/Gen-Verse/OpenClaw-RL","quote":"a judge model extracts a textual hint. This hint augments the original prompt to create an enhanced teacher","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"evaluator-optimizer","role":"core","note":"Evaluator (judge) and optimizer (trainer) are decoupled into independent async loops, so judging happens concurrently with new interactions and training runs in the background while serving continues.","evidence":[{"type":"repo","title":"OpenClaw-RL README — Fully Asynchronous 4-Component Architecture","url":"https://github.com/Gen-Verse/OpenClaw-RL","quote":"OpenClaw-RL decouples agent serving, rollout collection, PRM/judge evaluation, and policy training into independent async loops. None of them block one another","accessed":"2026-05-20"},{"type":"paper","title":"OpenClaw-RL Technical Report (arXiv 2603.10165)","url":"https://arxiv.org/abs/2603.10165","quote":"continuously optimises the policy in the background — all without interrupting usage","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"event-driven-agent","role":"first-class","note":"Live multi-turn conversation traffic, intercepted through the OpenClaw OpenAI-compatible plugin, is the event source feeding rollout collection and downstream training; nothing is batch-scheduled.","evidence":[{"type":"repo","title":"OpenClaw-RL README — TL;DR","url":"https://github.com/Gen-Verse/OpenClaw-RL","quote":"wraps your self-hosted model in OpenClaw as an OpenAI-compatible API, intercepts live multi-turn conversations, and continuously optimizes the policy in the background — all without interrupting your usage","accessed":"2026-05-20"}],"evidence_status":"limited"},{"pattern":"process-reward-model","role":"first-class","note":"Per-turn scalar reward from a Process Reward Model drives the Binary RL paradigm; GRPO advantage estimation plus a PPO-style clipped surrogate loss train the policy.","evidence":[{"type":"repo","title":"OpenClaw-RL README — Binary RL / PRM","url":"https://github.com/Gen-Verse/OpenClaw-RL","quote":"Per-turn scalar reward from the PRM combined with GRPO advantage estimation and a PPO-style clipped surrogate loss","accessed":"2026-05-20"}],"evidence_status":"limited"},{"pattern":"tool-use","role":"supported","note":"Track 2 explicitly targets tool-call agents (alongside terminal, GUI, SWE) for real-world settings; supported as a training environment rather than a runtime feature.","evidence":[{"type":"repo","title":"OpenClaw-RL README — Roadmap / Track 2","url":"https://github.com/Gen-Verse/OpenClaw-RL","quote":"scalable RL implementations for general agent settings across terminal, GUI, SWE, and tool-call scenarios","accessed":"2026-05-20"}],"evidence_status":"limited"}],"alternatives":[{"composition":"openclaw","relation":"complements","note":"Same brand and openclaw.ai homepage; OpenClaw-RL is the RL training framework that wraps a model in the OpenClaw plugin so OpenClaw can intercept and learn from conversations."}],"references":[{"type":"paper","title":"OpenClaw-RL Technical Report","url":"https://arxiv.org/abs/2603.10165","quote":"Ranked #1 on HuggingFace Daily Papers","accessed":"2026-05-20"},{"type":"blog","title":"OpenClaw-RL project blog page","url":"https://yinjjiew.github.io/projects/openclawrl1","accessed":"2026-05-20"},{"type":"repo","title":"Gen-Verse/OpenClaw-RL","url":"https://github.com/Gen-Verse/OpenClaw-RL","accessed":"2026-05-20"}],"last_analyzed":"2026-05-20","verification_status":"partial","last_updated":"2026-05-24","date_added":"2026-05-20","status_in_practice":"experimental","tags":["open-source","reinforcement-learning","training-framework","personalised-agent"]},{"id":"claude-agent-sdk","name":"Claude Agent SDK","aliases":["Claude Code SDK"],"kind":"framework","category":"agent-sdk","build_surface":"full-code","vendor":"Anthropic","language":"Python, TypeScript","license":"MIT (Python), proprietary (TypeScript — Anthropic Commercial Terms)","status":"active","first_released":"2025-06-11","url":"https://docs.claude.com/en/docs/agent-sdk/overview","repo":"https://github.com/anthropics/claude-agent-sdk-python","docs_url":"https://docs.claude.com/en/docs/agent-sdk/overview","intent":"Embed Claude Code's autonomous agent loop — same tools, same context management — as a programmable library in Python or TypeScript so production agents can read files, run commands, edit code, and execute multi-step workflows.","description":"The Claude Agent SDK is Anthropic's Python and TypeScript library for building agents that run Claude Code's loop inside your own process. It gives you the same tools, agent loop, and context management that power Claude Code, programmable in either language. The SDK exposes built-in tools (Read/Write/Edit/Bash/Glob/Grep/WebSearch/WebFetch/Agent/Skill), MCP servers (in-process and external), filesystem-defined Agents, Skills and Slash Commands under .claude/, hooks, and a canUseTool approval callback. It is the renamed successor of the Claude Code SDK; distinct from the lower-level Anthropic Client SDK (where you implement the tool loop) and from Managed Agents (Anthropic-hosted sandbox).","primary_use_cases":["production coding agents that mirror the Claude Code shape","agents that need filesystem + bash + browser tools out of the box","agents with mid-run approval gates via canUseTool","agents using Anthropic skills and MCP servers"],"agent_loop_shape":"Imperative query() / ClaudeSDKClient loop that wraps the Claude Code harness. Receive prompt → evaluate and respond → execute tools → repeat → return result. Turns continue until Claude produces output with no tool calls. Caller caps the loop with max_turns / maxTurns, max_budget_usd / maxBudgetUsd, effort, and permission_mode. Subagents spawn via the Agent tool, run with their own fresh context, and return a single final message. Sessions persist as JSONL under ~/.claude/projects/ and can be resumed, continued, or forked.","key_concepts":[{"name":"Agent loop","summary":"Same execution loop that powers Claude Code; Receive → Evaluate → Execute tools → Repeat → Return result.","url":"https://docs.claude.com/en/docs/agent-sdk/overview"},{"name":"Subagents","summary":"Separate agent instances the main agent can spawn for focused subtasks; their context starts fresh and intermediate tool calls stay inside the subagent.","maps_to_pattern":"subagent-isolation","url":"https://docs.claude.com/en/docs/agent-sdk/subagents"},{"name":"Agent Skills","summary":"Specialized capabilities packaged as SKILL.md files that Claude autonomously invokes when relevant; descriptions load at session start, full content only when used.","maps_to_pattern":"agent-skills","url":"https://docs.claude.com/en/docs/agent-sdk/skills"},{"name":"MCP servers","summary":"Three transports — stdio, http/sse, and in-process SDK MCP servers; mcp_servers / mcpServers is a top-level option.","maps_to_pattern":"mcp","url":"https://docs.claude.com/en/docs/agent-sdk/mcp"},{"name":"Permission modes + canUseTool","summary":"Permission modes (default / dontAsk / acceptEdits / bypassPermissions / plan / auto-TS-only) layered with canUseTool callback that pauses execution until the host returns a decision.","maps_to_pattern":"approval-queue","url":"https://docs.claude.com/en/docs/agent-sdk/permissions"},{"name":"Hooks","summary":"Callback functions that run host code in response to agent events (PreToolUse, PostToolUse, PermissionRequest, Stop).","url":"https://docs.claude.com/en/docs/agent-sdk/hooks"}],"pattern_composition":"flowchart TD\n  USER[Host process<br/>query() / ClaudeSDKClient] --> LOOP[Agent loop<br/>Receive → Evaluate → Execute → Repeat]\n  LOOP --> MODEL[Claude model]\n  MODEL -->|tool calls| PERM{Permission check<br/>hooks → deny → mode → allow → canUseTool}\n  PERM -- approved --> TOOLS[(Built-in tools<br/>Read/Write/Edit/Bash/Glob/Grep/<br/>WebSearch/WebFetch/Agent/Skill)]\n  PERM -- canUseTool pause --> HOST[Host approval callback]\n  HOST -.->|resume with decision| PERM\n  TOOLS --> OBS[Observation]\n  OBS --> LOOP\n  LOOP -.->|Agent tool spawn| SUB[Subagent<br/>fresh context, no nested spawn]\n  SUB -.->|final message only| LOOP\n  LOOP --> SKILLS[(Agent Skills<br/>.claude/skills/*/SKILL.md)]\n  LOOP --> MCP[(MCP servers<br/>stdio / HTTP / in-process)]\n  LOOP --> SESS[(Sessions on disk<br/>JSONL — resume/continue/fork)]\n  LOOP -->|max_turns or max_budget_usd| OUT[Final result]","members":[{"pattern":"agent-resumption","role":"first-class","note":"Sessions persist as JSONL on disk; resume by sessionId restores full context. continue (TS) / continue_conversation (Py) and fork_session also supported. Subagents themselves are resumable.","evidence":[{"type":"doc","title":"Sessions — resume","url":"https://docs.claude.com/en/docs/agent-sdk/sessions","quote":"Resume takes a specific session ID. You track the ID.","accessed":"2026-05-20"},{"type":"doc","title":"Sessions — disk persistence","url":"https://docs.claude.com/en/docs/agent-sdk/sessions","quote":"The SDK writes it to disk automatically so you can return to it later.","accessed":"2026-05-20"},{"type":"doc","title":"Subagents — resumable","url":"https://docs.claude.com/en/docs/agent-sdk/subagents","quote":"Subagents can be resumed to continue where they left off. Resumed subagents retain their full conversation history","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"agent-skills","role":"first-class","note":"Skills are markdown procedures loaded on demand from .claude/skills/ and ~/.claude/skills/; descriptions surface at session start, full content only when invoked.","evidence":[{"type":"doc","title":"Agent SDK — Skills","url":"https://docs.claude.com/en/docs/agent-sdk/skills","quote":"Agent Skills extend Claude with specialized capabilities that Claude autonomously invokes when relevant. Skills are packaged as `SKILL.md` files","accessed":"2026-05-20"},{"type":"doc","title":"Agent SDK — Skills loading","url":"https://docs.claude.com/en/docs/agent-sdk/skills","quote":"Skills are loaded from filesystem locations governed by `settingSources` (TypeScript) or `setting_sources` (Python)","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"approval-queue","role":"first-class","note":"canUseTool callback pauses the loop on tool requests; permission evaluation order is hooks → deny rules → permission mode → allow rules → canUseTool. Callbacks can stay pending indefinitely.","evidence":[{"type":"doc","title":"Agent SDK — User input / canUseTool","url":"https://docs.claude.com/en/docs/agent-sdk/user-input","quote":"Claude requests user input in two situations: when it needs permission to use a tool ... Both trigger your `canUseTool` callback, which pauses execution until you return a response.","accessed":"2026-05-20"},{"type":"doc","title":"Agent SDK — pause semantics","url":"https://docs.claude.com/en/docs/agent-sdk/user-input","quote":"The callback can stay pending indefinitely. Execution remains paused until your callback returns","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"extended-thinking","role":"supported","note":"ThinkingConfig and effort parameter expose Claude's reasoning behaviour. On Opus 4.7 manual extended thinking is no longer supported; adaptive thinking with effort replaces it. Downgraded from first-class because the SDK surfaces a model-level capability rather than implementing its own thinking pattern.","evidence":[{"type":"doc","title":"Agent SDK — TypeScript reference (thinking)","url":"https://docs.claude.com/en/docs/agent-sdk/typescript","quote":"thinking | ThinkingConfig | { type: 'adaptive' } for supported models | Controls Claude's thinking/reasoning behavior.","accessed":"2026-05-20"},{"type":"doc","title":"Agent loop — effort vs extended thinking","url":"https://docs.claude.com/en/docs/agent-sdk/agent-loop","quote":"effort trades latency and token cost for reasoning depth within each response. Extended thinking is a separate feature that produces visible chain-of-thought blocks in the output.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"Three transports — stdio, HTTP, and in-process SDK MCP servers; mcp_servers / mcpServers top-level option.","evidence":[{"type":"doc","title":"Agent SDK — MCP","url":"https://docs.claude.com/en/docs/agent-sdk/mcp","quote":"The Model Context Protocol (MCP) is an open standard for connecting AI agents to external tools and data sources.","accessed":"2026-05-20"},{"type":"doc","title":"Agent SDK — MCP transports","url":"https://docs.claude.com/en/docs/agent-sdk/mcp","quote":"MCP servers can run as local processes, connect over HTTP, or execute directly within your SDK application.","accessed":"2026-05-20"},{"type":"repo","title":"claude-agent-sdk-python README","url":"https://github.com/anthropics/claude-agent-sdk-python","quote":"Custom tools are implemented in-process MCP servers that run directly within your Python application, eliminating the need for separate processes","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"skill-library","role":"first-class","note":"Skill selection at session start via Pass-a-list or 'all'; description-first loading lets a large skill catalogue stay cheap.","evidence":[{"type":"doc","title":"Agent SDK — skill selection","url":"https://docs.claude.com/en/docs/agent-sdk/skills","quote":"Pass a list of skill names, \"all\", or [] to control which are available in the session","accessed":"2026-05-20"},{"type":"doc","title":"Agent SDK — agent-loop / skills","url":"https://docs.claude.com/en/docs/agent-sdk/agent-loop","quote":"Skill descriptions | Session start, via setting sources | Short summaries; full content loads only when invoked","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"step-budget","role":"first-class","note":"Caller caps the loop with max_turns / maxTurns (counts tool-use turns only) and max_budget_usd / maxBudgetUsd; exceeding emits a ResultMessage with error_max_turns or error_max_budget_usd.","evidence":[{"type":"doc","title":"Agent SDK — max_turns","url":"https://docs.claude.com/en/docs/agent-sdk/agent-loop","quote":"You can cap the loop with `max_turns` / `maxTurns`, which counts tool-use turns only.","accessed":"2026-05-20"},{"type":"doc","title":"Agent SDK — max_budget_usd","url":"https://docs.claude.com/en/docs/agent-sdk/agent-loop","quote":"You can also use `max_budget_usd` / `maxBudgetUsd` to cap turns based on a spend threshold.","accessed":"2026-05-20"},{"type":"doc","title":"Agent SDK — budget error subtypes","url":"https://docs.claude.com/en/docs/agent-sdk/agent-loop","quote":"When either limit is hit, the SDK returns a `ResultMessage` with a corresponding error subtype (`error_max_turns` or `error_max_budget_usd`).","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"structured-output","role":"first-class","note":"outputFormat / output_format option accepts a JSON Schema (or Zod / Pydantic via .toJSONSchema / .model_json_schema); SDK validates output, re-prompts on mismatch, surfaces error_max_structured_output_retries on failure.","evidence":[{"type":"doc","title":"Agent SDK — TypeScript reference (outputFormat)","url":"https://docs.claude.com/en/docs/agent-sdk/typescript","quote":"outputFormat | { type: 'json_schema', schema: JSONSchema } | undefined | Define output format for agent results.","accessed":"2026-05-20"},{"type":"doc","title":"Agent SDK — Structured outputs (validation and re-prompt)","url":"https://code.claude.com/docs/en/agent-sdk/structured-outputs","quote":"Define a JSON Schema for the structure you need, and the SDK validates the output against it, re-prompting on mismatch.","accessed":"2026-05-24"},{"type":"doc","title":"Agent SDK — Structured outputs (retry-limit error)","url":"https://code.claude.com/docs/en/agent-sdk/structured-outputs","quote":"error_max_structured_output_retries | Agent couldn't produce valid output after multiple attempts","accessed":"2026-05-24"},{"type":"doc","title":"Agent SDK — Structured outputs (Zod/Pydantic)","url":"https://code.claude.com/docs/en/agent-sdk/structured-outputs","quote":"For full type safety, use Zod (TypeScript) or Pydantic (Python) to define your schema and get strongly-typed objects back.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"subagent-isolation","role":"first-class","note":"Each subagent runs in its own fresh conversation; intermediate tool calls/results stay inside and only the final message returns to the parent. Subagents cannot spawn subagents.","evidence":[{"type":"doc","title":"Agent SDK — Subagents (fresh context)","url":"https://docs.claude.com/en/docs/agent-sdk/subagents","quote":"Each subagent runs in its own fresh conversation. Intermediate tool calls and results stay inside the subagent; only its final message returns to the parent.","accessed":"2026-05-20"},{"type":"doc","title":"Agent SDK — Subagents (parent → child channel)","url":"https://docs.claude.com/en/docs/agent-sdk/subagents","quote":"A subagent's context window starts fresh (no parent conversation) but isn't empty. The only channel from parent to subagent is the Agent tool's prompt string","accessed":"2026-05-20"},{"type":"doc","title":"Agent SDK — Subagents (no nesting)","url":"https://docs.claude.com/en/docs/agent-sdk/subagents","quote":"Subagents cannot spawn their own subagents.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-result-caching","role":"limited","note":"Prompt caching of the running context (system prompt, tool definitions, CLAUDE.md, accumulated history) is automatic; this is not deduplicated caching keyed on individual tool calls.","evidence":[{"type":"doc","title":"Agent SDK — Cost tracking (automatic prompt caching)","url":"https://docs.claude.com/en/docs/agent-sdk/cost-tracking","quote":"The Agent SDK automatically uses prompt caching to reduce costs on repeated content. You do not need to configure caching yourself.","accessed":"2026-05-20"},{"type":"doc","title":"Agent SDK — agent-loop / cached content","url":"https://docs.claude.com/en/docs/agent-sdk/agent-loop","quote":"Content that stays the same across turns (system prompt, tool definitions, CLAUDE.md) is automatically prompt cached","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Built-in tools (Read/Write/Edit/Bash/Glob/Grep/WebSearch/WebFetch/Agent/Skill/AskUserQuestion/Monitor) + custom tools via @tool / create_sdk_mcp_server + external MCP servers.","evidence":[{"type":"doc","title":"Agent SDK — built-in tools","url":"https://docs.claude.com/en/docs/agent-sdk/overview","quote":"Your agent can read files, run commands, and search codebases out of the box. Key tools include: Read ... Write ... Edit ... Bash ... Monitor ... Glob ... Grep ... WebSearch ... WebFetch ... AskUserQuestion","accessed":"2026-05-20"},{"type":"doc","title":"Agent SDK — tools rationale","url":"https://docs.claude.com/en/docs/agent-sdk/agent-loop","quote":"Tools give your agent the ability to take action. Without tools, Claude can only respond with text. With tools, Claude can read files, run commands, search code, and interact with external services.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"augmented-llm","role":"core","note":"Anthropic defines the foundational agentic building block as an LLM enhanced with retrieval, tools, and memory that the model itself drives.","evidence":[{"type":"doc","title":"Building effective agents","url":"https://www.anthropic.com/research/building-effective-agents","quote":"The basic building block of agentic systems is an LLM enhanced with augmentations such as retrieval, tools, and memory.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"mcp-as-code-api","role":"core","note":"Anthropic presents MCP servers as a filesystem of typed code modules the model imports on demand, so intermediate tool data stays in the execution environment instead of the context window.","evidence":[{"type":"blog","title":"Code execution with MCP: building more efficient agents","url":"https://www.anthropic.com/engineering/code-execution-with-mcp","quote":"With code execution environments becoming more common for agents, a solution is to present MCP servers as code APIs rather than direct tool calls. ... When agents use code execution with MCP, intermediate results stay in the execution environment by default.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"tool-result-eviction","role":"first-class","note":"The clear_tool_uses strategy clears the oldest tool results once context grows past a threshold and replaces each with placeholder text so the model still knows the call happened.","evidence":[{"type":"doc","title":"Context editing - Claude API Docs","url":"https://platform.claude.com/docs/en/build-with-claude/context-editing","quote":"When activated, the API automatically clears the oldest tool results in chronological order. The API replaces each cleared result with placeholder text so Claude knows it was removed.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"lead-researcher","role":"core","note":"A LeadResearcher analyzes the query, develops a strategy, and spawns subagents in parallel to explore different aspects simultaneously, then synthesizes their returned findings.","evidence":[{"type":"blog","title":"How we built our multi-agent research system","url":"https://www.anthropic.com/engineering/built-multi-agent-research-system","quote":"the lead agent analyzes it, develops a strategy, and spawns subagents to explore different aspects simultaneously","accessed":"2026-06-17"},{"type":"blog","title":"How we built our multi-agent research system","url":"https://www.anthropic.com/engineering/built-multi-agent-research-system","quote":"subagents return findings to the LeadResearcher. The LeadResearcher synthesizes these results and decides whether more research is needed","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"context-compaction","role":"first-class","note":"Claude's compaction feature automatically summarizes older context when nearing the context-window limit and replaces that stale span with a concise model-written summary so long-running tasks continue.","evidence":[{"type":"doc","title":"Compaction - Claude API Docs","url":"https://platform.claude.com/docs/en/build-with-claude/compaction","quote":"Compaction extends the effective context length for long-running conversations and tasks by automatically summarizing older context when approaching the context window limit. ... Compaction keeps the active context focused and performant by replacing stale content with concise summaries.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"reasoning-trace-carry-forward","role":"core","note":"Claude requires thinking/reasoning blocks to be passed back unmodified across tool-use turns to preserve reasoning continuity, while a non-tool-result user block ends the episode and strips prior thinking on older models.","evidence":[{"type":"doc","title":"Building with extended thinking - Claude API Docs","url":"https://platform.claude.com/docs/en/build-with-claude/extended-thinking","quote":"During tool use, you must pass `thinking` blocks back to the API, and you must include the complete unmodified block back to the API. This is critical for maintaining the model's reasoning flow and conversation integrity.","accessed":"2026-06-17"},{"type":"doc","title":"Building with extended thinking - Claude API Docs","url":"https://platform.claude.com/docs/en/build-with-claude/extended-thinking","quote":"When a non-tool-result user block is included: on Opus 4.5+ and Sonnet 4.6+, previous thinking blocks are kept; on earlier Opus/Sonnet models and all Haiku models, all previous thinking blocks are ignored and stripped from context.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"planner-generator-evaluator-harness","role":"core","note":"A planner expands a short prompt into a full spec, a generator works one feature per sprint against that spec, and a separate evaluator grades each sprint against criteria the generator never sees — a GAN-style loop.","evidence":[{"type":"blog","title":"Harness design for long-running application development","url":"https://www.anthropic.com/engineering/harness-design-long-running-apps","quote":"a planner agent that took a simple 1-4 sentence prompt and expanded it into a full product spec ... instructing the generator to work in sprints, picking up one feature at a time from the spec ... by separating frontend generation from frontend grading, we can create a feedback loop that drives the generator toward stronger outputs.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"anti_patterns_avoided":[{"pattern":"tool-output-poisoning","note":"Hooks + deny rules + canUseTool allow explicit interception of every tool invocation before execution; the SDK warns that allowed_tools does not constrain bypassPermissions and that an allow-list paired with dontAsk is the safer pattern.","evidence":[{"type":"doc","title":"Agent SDK — permissions warning","url":"https://docs.claude.com/en/docs/agent-sdk/permissions","quote":"allowed_tools does not constrain bypassPermissions","accessed":"2026-05-20"}]},{"pattern":"agent-privilege-escalation","note":"Least-privilege tool allowlists: anything not explicitly approved is denied outright rather than prompted, blocking capability escalation.","evidence":[{"type":"doc","title":"Claude Agent SDK — permissions","url":"https://code.claude.com/docs/en/agent-sdk/permissions","quote":"For a locked-down agent, pair allowedTools with permissionMode: \"dontAsk\". Listed tools are approved; anything else is denied outright instead of prompting","accessed":"2026-06-19"}]},{"pattern":"context-anxiety","note":"Context-awareness gives the model its real remaining token budget so it persists to the end instead of guessing how many tokens remain and wrapping up early.","evidence":[{"type":"doc","title":"Claude — context windows","url":"https://platform.claude.com/docs/en/build-with-claude/context-windows","quote":"Claude is trained to use this context precisely, persisting in the task until the very end rather than guessing how many tokens remain.","accessed":"2026-06-19"}]},{"pattern":"tool-over-broad-scope","note":"Scoped deny rules (e.g. Bash(rm *)) enforce fine-grained least-privilege limits inside an otherwise-available tool, in every permission mode.","evidence":[{"type":"doc","title":"Claude Agent SDK — permissions","url":"https://code.claude.com/docs/en/agent-sdk/permissions","quote":"disallowed_tools=[\"Bash(rm *)\"]: Bash stays available. Calls matching rm * are denied in every permission mode, including bypassPermissions.","accessed":"2026-06-19"}]}],"instantiates":["modern-coding-agent"],"alternatives":[{"composition":"claude-code","relation":"same-vendor","note":"Same capabilities, different interface — Claude Code is the CLI/IDE product; Claude Agent SDK is the library that exposes the same loop."},{"composition":"openai-agents-sdk","relation":"competes-with","note":"Cross-vendor SDK in the same agent-SDK category."},{"composition":"google-adk","relation":"competes-with","note":"Same category, Google equivalent."},{"composition":"vercel-ai-sdk","relation":"competes-with","note":"TypeScript-first competitor in the same category."},{"composition":"openai-swarm","relation":"competes-with","note":"Same category, different vendor."},{"composition":"taskweaver","relation":"similar-shape"}],"references":[{"type":"doc","title":"Claude Agent SDK overview","url":"https://docs.claude.com/en/docs/agent-sdk/overview","accessed":"2026-05-20"},{"type":"repo","title":"anthropics/claude-agent-sdk-python","url":"https://github.com/anthropics/claude-agent-sdk-python","accessed":"2026-05-20"},{"type":"repo","title":"anthropics/claude-agent-sdk-typescript","url":"https://github.com/anthropics/claude-agent-sdk-typescript","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-06-19","date_added":"2026-04-30","status_in_practice":"emerging","tags":["sdk","claude-code-shape","filesystem-tools","mcp","subagent-tree"]},{"id":"google-adk","name":"Agent Development Kit (ADK)","kind":"framework","category":"agent-sdk","build_surface":"full-code","vendor":"Google","language":"Python, Java","license":"Apache-2.0","status":"active","first_released":"2025-03-17","url":"https://adk.dev/","repo":"https://github.com/google/adk-python","docs_url":"https://adk.dev/","intent":"Provide a code-first, model-agnostic Python and Java framework for composing LLM agents with deterministic workflow agents into multi-agent applications that can be evaluated and deployed at enterprise scale.","description":"Agent Development Kit (ADK) is Google's open-source, Apache-2.0 framework for building, evaluating, and deploying AI agents from a single LlmAgent up to multi-agent workflows. The 2.0 release adds a graph-based Workflow Runtime that interleaves deterministic execution flows with LLM reasoning, plus a Task API for structured agent-to-agent delegation. Core abstractions include LlmAgent (instructions + model + tools), the workflow agents Sequential / Parallel / Loop, sub_agents trees, MCP tool integration via McpToolset, an evaluation harness (AgentEvaluator, eval sets, tool-trajectory and response-match criteria), and first-class deployment to Cloud Run, GKE, and Vertex AI Agent Runtime.","primary_use_cases":["production multi-agent workflows on Google Cloud (Agent Runtime, Cloud Run, GKE)","deterministic orchestration via SequentialAgent / ParallelAgent / LoopAgent","LLM-coordinated dynamic routing across sub-agents","evaluation-driven agent development with eval sets and trajectory matching","MCP-tool integration into Gemini and other LLMs"],"agent_loop_shape":"Hierarchical agent tree with a graph-based Workflow Runtime. An LlmAgent runs the standard reason-act-observe loop over its tools; workflow agents (Sequential / Parallel / Loop) compose sub_agents with fixed execution logic; in collaborative workflows an LlmAgent coordinator dynamically routes work across sub_agents. ADK 2.0 adds a graph-based execution engine that interleaves deterministic nodes with LLM agents inside one run.","key_concepts":[{"name":"LlmAgent","summary":"Self-contained execution unit: model + instructions + tools. The core building block, often aliased simply as Agent.","maps_to_pattern":"tool-use","url":"https://adk.dev/agents/llm-agents/"},{"name":"Workflow Agents (Sequential / Parallel / Loop)","summary":"Specialised agents that control execution flow over sub-agents with predefined, deterministic logic.","maps_to_pattern":"orchestrator-workers","url":"https://adk.dev/agents/workflow-agents/"},{"name":"Collaborative workflow","summary":"Single LlmAgent acts as dynamic coordinator over a set of sub_agents.","maps_to_pattern":"supervisor","url":"https://adk.dev/workflows/"},{"name":"McpToolset","summary":"Bridges an MCP server into ADK: connects, lists tools, converts them to ADK BaseTool instances, manages shutdown.","maps_to_pattern":"mcp","url":"https://adk.dev/tools-custom/mcp-tools/"},{"name":"output_schema / output_key","summary":"output_schema enforces a JSON shape on the final response; output_key auto-saves the response to session state.","maps_to_pattern":"structured-output","url":"https://adk.dev/agents/llm-agents/"},{"name":"AgentEvaluator + eval sets","summary":"Evaluation harness comparing actual trajectories to expected ones via tool_trajectory_avg_score, response_match_score, rubric_based_final_response_quality_v1, and similar criteria.","maps_to_pattern":"eval-harness","url":"https://adk.dev/evaluate/"}],"pattern_composition":"flowchart TD\n  HOST[Host app / runner] --> ROOT[Root LlmAgent<br/>instructions + model + tools]\n  ROOT -->|tool calls| TOOLS[(Function / OpenAPI / Built-in tools)]\n  ROOT -->|MCP| MCP[McpToolset<br/>stdio + SSE]\n  ROOT -->|sub_agents tree| WF{Workflow Agent}\n  WF --> SEQ[SequentialAgent]\n  WF --> PAR[ParallelAgent]\n  WF --> LOOP[LoopAgent]\n  WF --> COORD[Collaborative coordinator<br/>dynamic routing]\n  SEQ --> CHILD[(sub_agents)]\n  PAR --> CHILD\n  LOOP --> CHILD\n  COORD --> CHILD\n  ROOT -.->|output_schema| OBJ[Structured JSON]\n  ROOT -.->|output_key| STATE[(Session state)]\n  HOST -.->|AgentEvaluator + eval sets| EVAL[(Trajectory + response-match criteria)]","members":[{"pattern":"tool-use","role":"first-class","note":"Tools (function tools, OpenAPI tools, MCP tools, built-in tools) attach to an LlmAgent via its tools list and the LLM invokes them through the standard reason-act-observe loop.","evidence":[{"type":"doc","title":"ADK — LlmAgent tools","url":"https://adk.dev/agents/llm-agents/","quote":"Tools give your `LlmAgent` capabilities beyond the LLM's built-in knowledge or reasoning","accessed":"2026-05-20"},{"type":"doc","title":"ADK — agent definition","url":"https://adk.dev/agents/","quote":"The basic components of an **_Agent_** are an artificial intelligence (AI) model, task instructions, and optionally, a set of tools to be used by the agent.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"McpToolset connects to an MCP server, lists tools, converts them to ADK BaseTool instances and manages graceful shutdown; both StdioServerParameters and SSE-style connections are supported.","evidence":[{"type":"doc","title":"ADK — MCP tools","url":"https://adk.dev/tools-custom/mcp-tools/","quote":"When you include an McpToolset instance in your agent's tools list, it automatically handles the interaction with the specified MCP server.","accessed":"2026-05-20"},{"type":"doc","title":"ADK — MCP tool discovery","url":"https://adk.dev/tools-custom/mcp-tools/","quote":"Once connected, McpToolset queries the MCP server for its available tools (via the list_tools MCP method). It then converts the schemas of these discovered MCP tools into ADK-compatible BaseTool instances.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"structured-output","role":"first-class","note":"LlmAgent.output_schema constrains the final response to a JSON shape; output_key writes the response into session state for downstream agents.","evidence":[{"type":"doc","title":"ADK — LlmAgent output_schema","url":"https://adk.dev/agents/llm-agents/","quote":"Define a schema representing the desired output structure. If set, the agent's final response must be JSON","accessed":"2026-05-20"},{"type":"doc","title":"ADK — LlmAgent output_key","url":"https://adk.dev/agents/llm-agents/","quote":"If set, the text content of the agent's final response will be automatically saved to the session's state","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"supervisor","role":"first-class","note":"Collaborative workflow places one LlmAgent in a dynamic coordinator role over a set of sub_agents that automatically return to the coordinator after task completion; orchestration is LLM-driven rather than predefined.","evidence":[{"type":"doc","title":"ADK — Workflows (collaborative)","url":"https://adk.dev/workflows/","quote":"This workflow type allows a single agent to act in a dynamic coordinator role to accomplish tasks with a set of specified sub-agents","accessed":"2026-05-20"},{"type":"doc","title":"ADK — Collaborative coordinator","url":"https://adk.dev/workflows/collaboration/","quote":"a coordinator agent handles delegation of tasks to one or more subagents","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"orchestrator-workers","role":"first-class","note":"Workflow agents (Sequential / Parallel / Loop) execute sub-agents under fixed deterministic logic, yielding orchestrator-worker shape.","evidence":[{"type":"doc","title":"ADK — Workflow agents","url":"https://adk.dev/agents/workflow-agents/","quote":"Specialized agents that control the execution flow of one or more sub-agents.","accessed":"2026-05-20"},{"type":"doc","title":"ADK — Workflow agents (deterministic)","url":"https://adk.dev/agents/workflow-agents/","quote":"Template workflow agents operate based on predefined logic...This approach results in deterministic and predictable execution patterns.","accessed":"2026-05-20"},{"type":"doc","title":"ADK — Workflow agents (composition)","url":"https://adk.dev/workflows/","quote":"These pre-built workflows are extended from **_BaseAgent_** and provide fixed execution logic structures including sequences, loops, and parallel execution","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"handoff","role":"first-class","note":"ADK 2.0 ships a Task API for structured agent-to-agent delegation; combined with sub_agents trees and the collaborative coordinator this realises the handoff pattern.","evidence":[{"type":"repo","title":"google/adk-python README — Task API","url":"https://github.com/google/adk-python","quote":"Task API: Structured agent-to-agent delegation with multi-turn task mode","accessed":"2026-05-20"},{"type":"doc","title":"ADK — Workflows scope","url":"https://adk.dev/workflows/","quote":"Agent Development Kit (ADK) supports building sophisticated agent applications by composing multiple agents and executable nodes into _agent workflows_","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"evaluator-optimizer","role":"first-class","note":"AgentEvaluator with eval sets compares actual trajectories to expected ones; multiple built-in criteria (tool_trajectory_avg_score, response_match_score, rubric_based_final_response_quality_v1) drive an evaluation -> iteration cycle.","evidence":[{"type":"doc","title":"ADK — Evaluate (trajectory)","url":"https://adk.dev/evaluate/","quote":"The trajectory is just a list of steps the agent took before it returned to the user. We can compare that against the list of steps we expect the agent to have taken.","accessed":"2026-05-20"},{"type":"doc","title":"ADK — Evaluate (built-in criteria)","url":"https://adk.dev/evaluate/","quote":"ADK provides several built-in criteria for evaluating agent performance, ranging from tool trajectory matching to LLM-based response quality assessment.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"subagent-isolation","role":"limited","note":"sub_agents trees give an LlmAgent named children it can delegate to with automatic return-to-parent behaviour after each task; ADK does not document a hard context-isolation boundary at the level Anthropic's Agent tool does, so isolation is structural rather than enforced.","evidence":[{"type":"doc","title":"ADK — Workflows (composition)","url":"https://adk.dev/workflows/","quote":"Workflows: multi-agent, multi-node applications","accessed":"2026-05-20"},{"type":"doc","title":"ADK — Collaborative (subagents return to parent)","url":"https://adk.dev/workflows/collaboration/","quote":"subagents defined to handle specific tasks, and automatic return to the parent after completing a task","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"agentic-rag","role":"supported","note":"ADK ships the vertex_ai_rag_retrieval tool that lets an LlmAgent perform private-data retrieval via Vertex AI Knowledge Engine; agentic RAG is a first-party integration but not a top-level abstraction.","evidence":[{"type":"doc","title":"ADK — LlmAgent (tool surface)","url":"https://adk.dev/agents/llm-agents/","quote":"Tools give your `LlmAgent` capabilities beyond the LLM's built-in knowledge or reasoning","accessed":"2026-05-20"},{"type":"doc","title":"ADK — vertex_ai_rag_retrieval","url":"https://adk.dev/integrations/knowledge-engine/","quote":"The `vertex_ai_rag_retrieval` tool allows the agent to perform private data retrieval using Knowledge Engine.","accessed":"2026-05-24"}],"evidence_status":"full"}],"alternatives":[{"composition":"claude-agent-sdk","relation":"competes-with","note":"Same agent-sdk category; ADK is Python/Java + Google-Cloud-first, Claude Agent SDK is Anthropic-only with built-in filesystem tools."},{"composition":"openai-agents-sdk","relation":"competes-with","note":"Cross-vendor competitor; OpenAI is handoff-centric, ADK is workflow-agent-centric with explicit deterministic orchestration."},{"composition":"vercel-ai-sdk","relation":"competes-with","note":"TypeScript-first competitor in the same category."}],"references":[{"type":"doc","title":"ADK homepage","url":"https://adk.dev/","quote":"Build production agents, not prototypes.","accessed":"2026-05-20"},{"type":"repo","title":"google/adk-python README","url":"https://github.com/google/adk-python","quote":"An open-source, code-first Python framework for building, evaluating, and deploying sophisticated AI agents with flexibility and control.","accessed":"2026-05-20"},{"type":"repo","title":"google/adk-python README — license","url":"https://github.com/google/adk-python","quote":"This project is licensed under the Apache 2.0 License","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","google-cloud","multi-agent","workflow-agents","evaluation"]},{"id":"instructor","name":"Instructor","kind":"framework","category":"agent-sdk","build_surface":"full-code","vendor":"Jason Liu / community","language":"Python, TypeScript, Go, Ruby, Elixir, Rust","license":"MIT","status":"active","first_released":"2023-07-28","url":"https://python.useinstructor.com/","repo":"https://github.com/instructor-ai/instructor","docs_url":"https://python.useinstructor.com/","intent":"Get reliable, type-safe structured data from any LLM by patching the provider client to accept a Pydantic response_model, validate the response, and retry with validation feedback when the model violates the schema.","description":"Instructor is the MIT-licensed library that turns Pydantic models into the contract between application code and an LLM. It patches each supported provider's chat-completion client to add response_model, max_retries, and context parameters, then wraps the call: when validation fails, the library reasks the model with the validation error attached so the next attempt converges on a valid object. Modes (TOOLS, JSON, MD_JSON, FUNCTIONS) select the underlying provider mechanism; from_provider exposes a unified entry point across 15+ providers (OpenAI, Anthropic, Google, Mistral, Cohere, Ollama, DeepSeek, Groq, and more). Instructor is positioned as an extraction library, not an agent framework: 'Instructor for extraction, PydanticAI for agents.'","primary_use_cases":["extracting typed objects from LLM responses","schema-validated tool-call arguments before execution","multi-provider structured-output code that does not change when swapping models","self-correcting JSON output via validation-feedback retries"],"agent_loop_shape":"Single-call extraction with internal retry, not an agent loop. The patched create() runs the request, parses the response into the response_model, validates it, and on ValidationError reasks the model up to max_retries times. There is no multi-step tool loop, no handoff, no session memory; agentic behaviour requires composing Instructor inside a higher-level framework.","key_concepts":[{"name":"Patching","summary":"Adds response_model, max_retries, and context to the provider's create() method without changing original client code.","url":"https://python.useinstructor.com/concepts/patching/"},{"name":"response_model","summary":"Pydantic BaseModel that defines the expected output shape; docstrings and field annotations feed the prompt.","maps_to_pattern":"structured-output","url":"https://python.useinstructor.com/concepts/models/"},{"name":"Modes (TOOLS / JSON / MD_JSON / FUNCTIONS)","summary":"Selects which provider-side mechanism Instructor uses to coerce the model into the schema. OpenAI / Anthropic / Google default to TOOLS.","url":"https://python.useinstructor.com/concepts/patching/"},{"name":"max_retries with validation feedback","summary":"On Pydantic ValidationError Instructor retries with the validation error fed back to the model.","maps_to_pattern":"self-refine","url":"https://python.useinstructor.com/concepts/retrying/"},{"name":"from_provider","summary":"Unified entry point across all supported providers; recommended over manual patching.","url":"https://python.useinstructor.com/concepts/patching/"}],"pattern_composition":"flowchart TD\n  APP[Application] --> CLIENT[Patched provider client<br/>via patch() or from_provider()]\n  CLIENT --> CREATE[create(response_model=Model, max_retries=N)]\n  CREATE -->|TOOLS / JSON / MD_JSON| LLM[Provider LLM]\n  LLM --> RESP[Raw response]\n  RESP --> VAL{Pydantic validate}\n  VAL -- ok --> OUT[Typed Pydantic instance]\n  VAL -- ValidationError --> REASK[Reask with error feedback]\n  REASK --> CREATE","members":[{"pattern":"structured-output","role":"core","note":"Core purpose: response_model is a Pydantic class that defines the LLM output shape, drives the prompt via docstrings and field annotations, and is used to validate the response.","evidence":[{"type":"doc","title":"Instructor homepage — tagline","url":"https://python.useinstructor.com/","quote":"Extract structured data from any LLM with type safety, validation, and automatic retries.","accessed":"2026-05-20"},{"type":"doc","title":"Instructor concepts — models","url":"https://python.useinstructor.com/concepts/models/","quote":"Define LLM output schemas using `pydantic.BaseModel`.","accessed":"2026-05-20"},{"type":"doc","title":"Instructor concepts — patching","url":"https://python.useinstructor.com/concepts/patching/","quote":"New parameters: `response_model`, `max_retries`, and `context` to completion methods","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"self-refine","role":"first-class","note":"On Pydantic ValidationError Instructor reasks the model with the validation feedback attached, up to max_retries times; this is the self-refine loop applied to structured-output extraction.","evidence":[{"type":"doc","title":"Instructor concepts — patching (validation)","url":"https://python.useinstructor.com/concepts/patching/","quote":"Validates responses: Checks LLM outputs against your Pydantic model. Handles retries: Retries with validation feedback if needed","accessed":"2026-05-20"},{"type":"doc","title":"Instructor concepts — retrying","url":"https://python.useinstructor.com/concepts/retrying/","quote":"Failed attempts are automatically propagated to reask handlers, enabling contextual error messages and progressive corrections.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"limited","note":"Instructor does not implement an agent tool loop; it uses the provider's tool-calling API (TOOLS mode) as the transport for structured output. Building a true tool-loop agent requires wrapping Instructor inside another framework.","evidence":[{"type":"doc","title":"Instructor concepts — patching modes","url":"https://python.useinstructor.com/concepts/patching/","quote":"Tool Calling (TOOLS): Uses the provider's function/tool calling API","accessed":"2026-05-20"},{"type":"doc","title":"Instructor homepage — positioning","url":"https://python.useinstructor.com/","quote":"Instructor for extraction, PydanticAI for agents. Instructor shines when you need fast, schema-first extraction without extra agents.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"multi-model-routing","role":"supported","note":"from_provider exposes a single API surface across 15+ providers (OpenAI, Anthropic, Google, Mistral, Cohere, Ollama, DeepSeek, Groq, ...) so the same response_model code works across vendors.","evidence":[{"type":"doc","title":"Instructor homepage — providers","url":"https://python.useinstructor.com/","quote":"works seamlessly with 15+ popular LLM providers","accessed":"2026-05-20"},{"type":"doc","title":"Instructor concepts — from_provider","url":"https://python.useinstructor.com/concepts/patching/","quote":"Use `from_provider` instead of manual patching. It works the same way across all providers.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"workflow-success-business-invalid","role":"supported","note":"Its llm_validator plugs an LLM-based business-rule check into the same Pydantic model and re-asks the model on failure, so a returned result is gated on validity rather than merely on a clean call.","evidence":[{"type":"doc","title":"Enhancing AI Validations with Pydantic's Framework - Instructor","url":"https://python.useinstructor.com/concepts/reask_validation/","quote":"LLM-based validation can also be plugged into the same Pydantic model. Here, if the answer attribute contains content that violates the rule 'don't say objectionable things,' Pydantic will raise a validation error.","accessed":"2026-06-14"}],"evidence_status":"full"},{"pattern":"prompt-response-optimiser","role":"core","note":"Instructor is built entirely around transforming free-form LLM output into a predefined, constraint-checked shape: you declare a Pydantic model and Instructor handles prompting, parsing, validation and automatic retry-on-failure so every response conforms to the template you specified.","evidence":[{"type":"repo","title":"567-labs/instructor README","url":"https://github.com/567-labs/instructor","quote":"Built on Pydantic for validation, type safety, and IDE support.","accessed":"2026-06-17"},{"type":"doc","title":"Instructor - Multi-Language Library for Structured LLM Outputs","url":"https://python.useinstructor.com/","quote":"Instructor is built on Pydantic for schema definitions, abstracting away everything - prompting, parsing, retrying, and more.","accessed":"2026-06-17"}],"evidence_status":"full"}],"alternatives":[{"composition":"vercel-ai-sdk","relation":"similar-shape","note":"Vercel's generateObject covers the structured-output slice in TypeScript with Zod schemas."},{"composition":"openai-agents-sdk","relation":"complements","note":"OpenAI Agents SDK uses Pydantic output_type for structured agent results; Instructor is the standalone extraction library."}],"references":[{"type":"doc","title":"Instructor homepage","url":"https://python.useinstructor.com/","quote":"Extract structured data from any LLM with type safety, validation, and automatic retries.","accessed":"2026-05-20"},{"type":"repo","title":"instructor-ai/instructor","url":"https://github.com/instructor-ai/instructor","quote":"Get reliable JSON from any LLM. Built on Pydantic for validation, type safety, and IDE support.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-20","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","structured-output","pydantic","multi-provider","extraction"]},{"id":"openai-agents-sdk","name":"OpenAI Agents SDK","kind":"framework","category":"agent-sdk","build_surface":"full-code","vendor":"OpenAI","language":"Python, TypeScript","license":"MIT","status":"active","first_released":"2025-03-04","url":"https://openai.github.io/openai-agents-python/","repo":"https://github.com/openai/openai-agents-python","docs_url":"https://openai.github.io/openai-agents-python/","intent":"Provide a lightweight, production-ready Python and TypeScript framework for building multi-agent workflows around four primitives: Agents, handoffs, guardrails, and sessions, with built-in tracing for the entire run.","description":"The OpenAI Agents SDK is the MIT-licensed successor to Swarm, shipped in March 2025. Its agent loop is small and explicit: Runner.run(agent, input) calls the LLM, executes any tool calls, follows any handoffs, and re-runs until the agent emits a final_output or max_turns is exceeded (MaxTurnsExceeded). Agents are LLMs configured with instructions, tools, optional output_type, handoffs, and guardrails. Handoffs are exposed to the model as transfer_to_<agent> tools with optional input_filter to reshape the conversation history. MCP support ships with three transports (Stdio, SSE, Streamable HTTP). SQLiteSession is the default session-memory backing store and supports run resumption across pauses.","primary_use_cases":["multi-agent workflows where specialised agents hand off control","agents that call MCP servers alongside Python function tools","agents with input/output guardrails and Pydantic-typed outputs","production agents that need built-in tracing and session memory","TypeScript agents sharing the same primitive set as the Python SDK"],"agent_loop_shape":"Runner.run loop: call LLM -> if final_output, return; if handoff, swap current agent and re-run; if tool calls, execute and append results then re-run. Loop is capped by max_turns (None disables). Sessions automatically prepend prior history before each run and store new items afterwards.","key_concepts":[{"name":"Agent","summary":"LLM configured with instructions, tools, optional output_type, handoffs, guardrails.","maps_to_pattern":"tool-use","url":"https://openai.github.io/openai-agents-python/agents/"},{"name":"Runner","summary":"Runs the agent loop via Runner.run / run_sync / run_streamed; capped by max_turns, raises MaxTurnsExceeded.","maps_to_pattern":"step-budget","url":"https://openai.github.io/openai-agents-python/running_agents/"},{"name":"Handoffs","summary":"Sub-agent delegation exposed as transfer_to_<agent> tools; handoff() customises tool name, description, on_handoff callback, input_filter.","maps_to_pattern":"handoff","url":"https://openai.github.io/openai-agents-python/handoffs/"},{"name":"Guardrails","summary":"Input guardrails run on the first agent's input; output guardrails on the last agent's final output. Input guardrails can run in parallel or block.","maps_to_pattern":"input-output-guardrails","url":"https://openai.github.io/openai-agents-python/guardrails/"},{"name":"Sessions","summary":"Built-in session memory (SQLiteSession default) that prepends history before each run and stores new items afterwards; supports resumption after approval pauses.","maps_to_pattern":"agent-resumption","url":"https://openai.github.io/openai-agents-python/sessions/"},{"name":"MCP transports","summary":"MCPServerStdio (subprocess), MCPServerSse (HTTP+SSE), MCPServerStreamableHttp (self-managed connection); tools exposed identically to function tools.","maps_to_pattern":"mcp","url":"https://openai.github.io/openai-agents-python/mcp/"},{"name":"output_type","summary":"Optional Pydantic / typed output declaration that switches the model into structured-outputs mode.","maps_to_pattern":"structured-output","url":"https://openai.github.io/openai-agents-python/agents/"}],"pattern_composition":"flowchart TD\n  CALLER[Caller] --> RUN[Runner.run / run_sync / run_streamed]\n  RUN --> AGENT[current_agent: Agent<br/>instructions + tools + output_type]\n  AGENT --> LLM[LLM call]\n  LLM --> DEC{output?}\n  DEC -- final_output --> OUT[RunResult]\n  DEC -- tool calls --> TOOLS[(Function tools<br/>+ MCP tools)]\n  TOOLS --> RUN\n  DEC -- transfer_to_X --> RUN\n  RUN -.->|max_turns exceeded| ERR[MaxTurnsExceeded]\n  RUN -.->|input guardrails| GIN[Input guardrails<br/>first agent only]\n  OUT -.->|output guardrails| GOUT[Output guardrails<br/>last agent only]\n  RUN -.->|prepend / append| SESS[(SQLiteSession or custom)]\n  RUN -.->|tracing| TRACE[(Built-in tracing)]\n  TOOLS -.->|MCPServerStdio<br/>MCPServerSse<br/>MCPServerStreamableHttp| MCP[(MCP servers)]","members":[{"pattern":"tool-use","role":"first-class","note":"Agent.tools accepts function tools (auto-schemaed from Python signatures) and MCP-exposed tools; the runner's loop executes tool calls and appends results before re-running.","evidence":[{"type":"doc","title":"Agents SDK — Agent class","url":"https://openai.github.io/openai-agents-python/agents/","quote":"An agent is a large language model (LLM) configured with instructions, tools, and optional runtime behavior such as handoffs, guardrails, and structured outputs.","accessed":"2026-05-20"},{"type":"doc","title":"Agents SDK — Runner loop","url":"https://openai.github.io/openai-agents-python/running_agents/","quote":"If the LLM produces tool calls, we run those tool calls, append the results, and re-run the loop.","accessed":"2026-05-20"},{"type":"repo","title":"openai/openai-agents-python README","url":"https://github.com/openai/openai-agents-python","quote":"Agents: LLMs configured with instructions, tools, guardrails, and handoffs","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"handoff","role":"first-class","note":"Handoffs are surfaced to the LLM as transfer_to_<agent> tools; handoff() exposes tool_name_override, on_handoff, input_type, input_filter, is_enabled. The runner swaps current_agent on handoff and continues the same run.","evidence":[{"type":"doc","title":"Agents SDK — Handoffs as tools","url":"https://openai.github.io/openai-agents-python/handoffs/","quote":"Handoffs are represented as tools to the LLM. So if there's a handoff to an agent named `Refund Agent`, the tool would be called `transfer_to_refund_agent`.","accessed":"2026-05-20"},{"type":"doc","title":"Agents SDK — Handoff effect","url":"https://openai.github.io/openai-agents-python/running_agents/","quote":"If the LLM does a handoff, we update the current agent and input, and re-run the loop.","accessed":"2026-05-20"},{"type":"doc","title":"Agents SDK — Handoff delegation","url":"https://openai.github.io/openai-agents-python/agents/","quote":"Handoffs are sub‑agents the agent can delegate to. When a handoff occurs, the delegated agent receives the conversation history and takes over.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"Three transports ship in the box: MCPServerStdio for subprocess servers, MCPServerSse for HTTP+SSE, MCPServerStreamableHttp for self-managed connections. MCP tools appear identically to function tools.","evidence":[{"type":"doc","title":"Agents SDK — MCP stdio","url":"https://openai.github.io/openai-agents-python/mcp/","quote":"For MCP servers that run as local subprocesses, use MCPServerStdio.","accessed":"2026-05-20"},{"type":"doc","title":"Agents SDK — MCP SSE","url":"https://openai.github.io/openai-agents-python/mcp/","quote":"If the MCP server implements the HTTP with SSE transport, instantiate MCPServerSse.","accessed":"2026-05-20"},{"type":"doc","title":"Agents SDK — MCP streamable HTTP","url":"https://openai.github.io/openai-agents-python/mcp/","quote":"When you want to manage the network connection yourself, use MCPServerStreamableHttp.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"step-budget","role":"first-class","note":"max_turns argument on Runner.run / run_sync / run_streamed caps the loop; exceeding it raises MaxTurnsExceeded. Pass None to disable.","evidence":[{"type":"doc","title":"Agents SDK — max_turns","url":"https://openai.github.io/openai-agents-python/running_agents/","quote":"If we exceed the `max_turns` passed, we raise a `MaxTurnsExceeded` exception. Pass `max_turns=None` to disable this turn limit.","accessed":"2026-05-20"},{"type":"doc","title":"Agents SDK — MaxTurnsExceeded source","url":"https://openai.github.io/openai-agents-python/running_agents/","quote":"MaxTurnsExceeded: This exception is raised when the agent's run exceeds the `max_turns` limit passed to the `Runner.run`, `Runner.run_sync`, or `Runner.run_streamed` methods.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"structured-output","role":"first-class","note":"Agent.output_type accepts a Pydantic / typed return; when set the SDK switches the model into structured-outputs mode.","evidence":[{"type":"doc","title":"Agents SDK — output_type","url":"https://openai.github.io/openai-agents-python/agents/","quote":"By default, agents produce plain text (i.e. `str`) outputs. If you want the agent to produce a particular type of output, you can use the `output_type` parameter.","accessed":"2026-05-20"},{"type":"doc","title":"Agents SDK — structured outputs mode","url":"https://openai.github.io/openai-agents-python/agents/","quote":"When you pass an `output_type`, that tells the model to use structured outputs instead of regular plain text responses.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"agent-resumption","role":"first-class","note":"Sessions persist conversation history; the runner prepends prior items before each run and stores new items afterwards. Runs that pause for approval can resume on the same session instance.","evidence":[{"type":"doc","title":"Agents SDK — Sessions overview","url":"https://openai.github.io/openai-agents-python/sessions/","quote":"The Agents SDK provides built-in session memory to automatically maintain conversation history across multiple agent runs","accessed":"2026-05-20"},{"type":"doc","title":"Agents SDK — Sessions resume","url":"https://openai.github.io/openai-agents-python/sessions/","quote":"If a run pauses for approval, resume it with the same session instance (or another session instance that points at the same backing store) so the resumed turn continues the same stored conversation history.","accessed":"2026-05-20"},{"type":"doc","title":"Agents SDK — SQLiteSession","url":"https://openai.github.io/openai-agents-python/sessions/","quote":"The default, lightweight session implementation using SQLite","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"supervisor","role":"first-class","note":"Two supported supervisor shapes are documented: (a) a manager agent that keeps the conversation and calls specialists through Agent.as_tool, and (b) decentralised peer handoff via transfer_to_*.","evidence":[{"type":"doc","title":"Agents SDK — decentralised handoff","url":"https://openai.github.io/openai-agents-python/agents/","quote":"Peer agents hand off control to a specialized agent that takes over the conversation. This is decentralized.","accessed":"2026-05-20"},{"type":"doc","title":"Agents SDK — manager agent (agents as tools)","url":"https://openai.github.io/openai-agents-python/multi_agent/","quote":"A manager agent keeps control of the conversation and calls specialist agents through `Agent.as_tool()`.","accessed":"2026-05-24"},{"type":"doc","title":"Agents SDK — triage agent as router","url":"https://openai.github.io/openai-agents-python/multi_agent/","quote":"A triage agent routes the conversation to a specialist, and that specialist becomes the active agent for the rest of the turn.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"subagent-isolation","role":"limited","note":"Handoffs transfer control rather than spawn an isolated subagent; the delegated agent receives the conversation history (optionally reshaped via input_filter). Nested handoffs are an opt-in beta that collapses prior transcript into a single assistant summary message.","evidence":[{"type":"doc","title":"Agents SDK — handoff receives history","url":"https://openai.github.io/openai-agents-python/agents/","quote":"When a handoff occurs, the delegated agent receives the conversation history and takes over.","accessed":"2026-05-20"},{"type":"doc","title":"Agents SDK — nested handoff","url":"https://openai.github.io/openai-agents-python/handoffs/","quote":"Nested handoffs are available as an opt-in beta... the runner collapses the prior transcript into a single assistant summary message.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"input-output-guardrails","role":"first-class","note":"Input guardrails run only for the first agent in the chain; output guardrails run only for the agent producing the final output. Input guardrails can run in parallel with the agent (default) or block before it starts.","evidence":[{"type":"doc","title":"Agents SDK — input guardrails","url":"https://openai.github.io/openai-agents-python/guardrails/","quote":"Input guardrails run on the initial user input","accessed":"2026-05-20"},{"type":"doc","title":"Agents SDK — output guardrails","url":"https://openai.github.io/openai-agents-python/guardrails/","quote":"Output guardrails run on the final agent output","accessed":"2026-05-20"},{"type":"doc","title":"Agents SDK — guardrail execution modes","url":"https://openai.github.io/openai-agents-python/guardrails/","quote":"Parallel execution (default, `run_in_parallel=True`): The guardrail runs concurrently with the agent's execution","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"supported","note":"The Runner loop has the reason-act-observe shape (LLM produces output, either final or tool calls, which are executed and fed back). Upstream docs do not use the term 'ReAct'; downgraded from first-class to supported because the pattern is implemented but not named.","evidence":[{"type":"doc","title":"Agents SDK — loop steps","url":"https://openai.github.io/openai-agents-python/running_agents/","quote":"The runner then runs a loop: 1. We call the LLM for the current agent, with the current input. 2. The LLM produces its output.","accessed":"2026-05-20"},{"type":"doc","title":"Agents SDK — loop termination","url":"https://openai.github.io/openai-agents-python/running_agents/","quote":"If the LLM returns a `final_output`, the loop ends and we return the result.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"reversibility-aware-action-filter","role":"limited","note":"Tools marked needsApproval record an approval interruption instead of executing, pausing the run before side effects such as edits, shell commands, or sensitive MCP actions; reversibility is a recommended risk axis rather than a built-in classifier.","evidence":[{"type":"doc","title":"Guardrails and human review | OpenAI API","url":"https://developers.openai.com/api/docs/guides/agents/guardrails-approvals","quote":"The run records an approval interruption instead of executing the tool. The result returns `interruptions` plus a resumable `state`. Your application approves or rejects the pending items.","accessed":"2026-06-14"}],"evidence_status":"limited"},{"pattern":"large-reasoning-model-paradigm","role":"core","note":"OpenAI's reasoning models spend reasoning tokens to think before answering, breaking down the prompt and considering multiple approaches; reasoning.effort routes how much inference-time deliberation a query gets.","evidence":[{"type":"doc","title":"Reasoning models — OpenAI API","url":"https://developers.openai.com/api/docs/guides/reasoning","quote":"The models use these reasoning tokens to \"think,\" breaking down the prompt and considering multiple approaches to generating a response.","accessed":"2026-06-17"},{"type":"doc","title":"Reasoning models — OpenAI API","url":"https://developers.openai.com/api/docs/guides/reasoning","quote":"The reasoning.effort parameter guides the model on how much to think when performing a task.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"test-time-compute-scaling","role":"core","note":"OpenAI documents that o-series quality keeps climbing with more inference-time reasoning — the same more-compute-equals-better trend extended into RL and test-time thinking — exposed to developers via the reasoning.effort control.","evidence":[{"type":"blog","title":"Introducing OpenAI o3 and o4-mini","url":"https://openai.com/index/introducing-o3-and-o4-mini/","quote":"the models' performance continues to improve the more they're allowed to think","accessed":"2026-06-17"},{"type":"doc","title":"Reasoning models — OpenAI API","url":"https://developers.openai.com/api/docs/guides/reasoning","quote":"Lower effort favors speed and lower token usage, while at higher effort the model thinks more completely to provide higher quality responses.","accessed":"2026-06-17"}],"evidence_status":"full"}],"alternatives":[{"composition":"openai-swarm","relation":"successor","note":"Production-ready evolution of Swarm; same primitive set (agents + handoffs) plus guardrails, sessions, tracing, MCP."},{"composition":"claude-agent-sdk","relation":"competes-with","note":"Same agent-sdk category, different vendor; Anthropic's SDK is filesystem-tool-first, OpenAI's is handoff-first."},{"composition":"google-adk","relation":"competes-with","note":"Google's competitor in the agent-sdk category."},{"composition":"vercel-ai-sdk","relation":"competes-with","note":"TypeScript-first competitor in the same category."},{"composition":"strands-agents","relation":"similar-shape","note":"Peer minimal agent SDK with a model-driven loop."},{"composition":"instructor","relation":"complements","note":"OpenAI Agents SDK uses Pydantic output_type for structured agent results; Instructor is the standalone extraction library."}],"references":[{"type":"doc","title":"OpenAI Agents SDK — docs root","url":"https://openai.github.io/openai-agents-python/","quote":"The OpenAI Agents SDK enables you to build agentic AI apps in a lightweight, easy-to-use package with very few abstractions.","accessed":"2026-05-20"},{"type":"repo","title":"openai/openai-agents-python","url":"https://github.com/openai/openai-agents-python","quote":"A lightweight, powerful framework for multi-agent workflows","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-06-19","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","openai","multi-agent","handoff","mcp","tracing"],"anti_patterns_avoided":[{"pattern":"authorized-tool-misuse","note":"Human-in-the-loop approval gates sensitive tool calls: needs_approval pauses execution until a person approves or rejects, so an authorised tool is not silently misused.","evidence":[{"type":"doc","title":"OpenAI Agents SDK — human-in-the-loop","url":"https://openai.github.io/openai-agents-python/human_in_the_loop/","quote":"Use the human-in-the-loop (HITL) flow to pause agent execution until a person approves or rejects sensitive tool calls. ... Set needs_approval to True to always require approval or provide an async function that decides per call.","accessed":"2026-06-19"}]}]},{"id":"openai-swarm","name":"OpenAI Swarm","kind":"framework","category":"agent-sdk","build_surface":"full-code","vendor":"OpenAI","language":"Python","license":"MIT","status":"deprecated","successor":"openai-agents-sdk","first_released":"2024-10-10","url":"https://github.com/openai/swarm","repo":"https://github.com/openai/swarm","docs_url":"https://github.com/openai/swarm","intent":"Explore an ergonomic, minimal model of multi-agent orchestration where agents are functions, control transfer is just returning another Agent from a tool, and a single client.run() drives the whole loop.","description":"OpenAI Swarm is an MIT-licensed educational framework (not for production) from the OpenAI Solution team. Its API has two primitives: an Agent (name, model, instructions, functions, tool_choice) and a handoff that happens when a function returns another Agent. context_variables flow through every call and can be updated via a Result return. The library is no longer recommended for new code; the README states 'Swarm is now replaced by the OpenAI Agents SDK, which is a production-ready evolution of Swarm. We recommend migrating to the Agents SDK for all production use cases.'","primary_use_cases":["studying minimal multi-agent orchestration as a teaching reference","small experimental routines where agents transfer control by returning each other","porting designs into the OpenAI Agents SDK"],"agent_loop_shape":"Single client.run() loop, analogous to chat.completions.create(). On each turn the current Agent is called, any tool functions execute; if a function returns an Agent, the runner swaps current_agent and continues; if it returns a Result, context_variables update. The loop terminates when the agent produces a plain message with no further handoff or tool call.","key_concepts":[{"name":"Agent","summary":"Bundle of name, model, instructions (string or callable), functions, tool_choice. The unit of behaviour.","maps_to_pattern":"tool-use","url":"https://github.com/openai/swarm"},{"name":"Handoff via function return","summary":"Returning another Agent from a function transfers control to that Agent for the next turn.","maps_to_pattern":"handoff","url":"https://github.com/openai/swarm"},{"name":"context_variables","summary":"Dict-shaped state passed into instructions and functions; persists across handoffs and can be updated via a Result return.","url":"https://github.com/openai/swarm"},{"name":"client.run()","summary":"Single-entry runner that drives the loop, analogous to chat.completions.create.","url":"https://github.com/openai/swarm"}],"pattern_composition":"flowchart TD\n  CALLER[Caller] --> RUN[client.run<br/>messages + context_variables + current_agent]\n  RUN --> AGENT[current_agent<br/>name + model + instructions + functions]\n  AGENT --> LLM[LLM call]\n  LLM --> DEC{output}\n  DEC -- plain message --> OUT[Final response]\n  DEC -- function call --> FN[Run function]\n  FN -- returns str --> RUN\n  FN -- returns Agent --> SWAP[Swap current_agent]\n  SWAP --> RUN\n  FN -- returns Result --> CTX[Update context_variables]\n  CTX --> RUN","members":[{"pattern":"handoff","role":"core","note":"Handoff is the framework's defining primitive: an Agent transfers control by being returned from a function call. There is no router, no graph, no scheduler.","evidence":[{"type":"repo","title":"Swarm README — primitives","url":"https://github.com/openai/swarm","quote":"An `Agent` encompasses `instructions` and `tools`, and can at any point choose to hand off a conversation to another `Agent`.","accessed":"2026-05-20"},{"type":"repo","title":"Swarm README — handoff via return","url":"https://github.com/openai/swarm","quote":"An `Agent` can hand off to another `Agent` by returning it in a `function`.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Functions are Python callables on the Agent; type hints and docstrings generate the schema. Functions can also read context_variables and return either a string result or a new Agent (handoff) or a Result (updates context).","evidence":[{"type":"repo","title":"Swarm README — Agent fields","url":"https://github.com/openai/swarm","quote":"An `Agent` encompasses `instructions` and `tools`","accessed":"2026-05-20"},{"type":"repo","title":"Swarm README — client.run","url":"https://github.com/openai/swarm","quote":"analogous to the `chat.completions.create()` function in the Chat Completions API","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"supervisor","role":"limited","note":"There is no dedicated supervisor primitive: any Agent can become the supervisor by routing other Agents through its functions. The triage_agent example demonstrates a supervisor by handoff. The pattern is achievable, not first-class. Downgraded from first-class.","evidence":[{"type":"repo","title":"Swarm README — coordination scope","url":"https://github.com/openai/swarm","quote":"Swarm focuses on making agent **coordination** and **execution** lightweight, highly controllable, and easily testable.","accessed":"2026-05-20"},{"type":"repo","title":"Swarm triage_agent example","url":"https://github.com/openai/swarm/tree/main/examples/triage_agent","quote":"This example is a Swarm containing a triage agent, which takes in user inputs and chooses whether to respond directly, or triage the request to a sales or refunds agent.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"structured-output","role":"limited","note":"Swarm has no Pydantic output_type or response_model; the README documents only two primitive abstractions (Agents and handoffs), and structured output is only available through tool/function call signatures or by the application parsing the final string.","evidence":[{"type":"repo","title":"Swarm README — minimal primitives","url":"https://github.com/openai/swarm","quote":"Two primitive abstractions: Agents and handoffs","accessed":"2026-05-20"},{"type":"repo","title":"Swarm README — Agent fields (no output_type)","url":"https://github.com/openai/swarm","quote":"An `Agent` encompasses `instructions` and `tools`","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"decentralized-swarm-handoff","role":"core","note":"An agent transfers control by returning the next Agent from a called function, so handoffs are agent-initiated with no central router.","evidence":[{"type":"repo","title":"openai/swarm README","url":"https://github.com/openai/swarm","quote":"An `Agent` can hand off to another `Agent` by returning it in a `function`.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"alternatives":[{"composition":"openai-agents-sdk","relation":"successor","note":"Direct successor; production-ready evolution of the same primitive set."},{"composition":"claude-agent-sdk","relation":"competes-with","note":"Same category, different vendor."},{"composition":"autogen","relation":"similar-shape","note":"Handoff-based swarm pattern."},{"composition":"taskweaver","relation":"competes-with","note":"Both are code-first agent frameworks; Swarm uses agent-as-tool handoffs, TaskWeaver uses stateful Python sessions."}],"references":[{"type":"repo","title":"openai/swarm README","url":"https://github.com/openai/swarm","quote":"Educational framework exploring ergonomic, lightweight multi-agent orchestration. Managed by OpenAI Solution team.","accessed":"2026-05-20"},{"type":"repo","title":"openai/swarm README — deprecation","url":"https://github.com/openai/swarm","quote":"Swarm is now replaced by the OpenAI Agents SDK, which is a production-ready evolution of Swarm.","accessed":"2026-05-20"},{"type":"repo","title":"openai/swarm README — migration recommendation","url":"https://github.com/openai/swarm","quote":"We recommend migrating to the Agents SDK for all production use cases.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-06-19","date_added":"2026-04-30","status_in_practice":"deprecated","tags":["deprecated","educational","multi-agent","handoff"],"anti_patterns_avoided":[{"pattern":"orchestrator-as-bottleneck","note":"Agents hand off directly to other agents by returning them, with no central stateful orchestrator that all traffic must funnel through.","evidence":[{"type":"doc","title":"OpenAI Swarm","url":"https://github.com/openai/swarm","quote":"An Agent can hand off to another Agent by returning it in a function... Swarm runs (almost) entirely on the client and, much like the Chat Completions API, does not store state between calls.","accessed":"2026-06-19"}]}]},{"id":"vercel-ai-sdk","name":"Vercel AI SDK","kind":"framework","category":"agent-sdk","build_surface":"full-code","vendor":"Vercel","language":"TypeScript","license":"Apache-2.0","status":"active","first_released":"2023-05-23","url":"https://ai-sdk.dev/","repo":"https://github.com/vercel/ai","docs_url":"https://ai-sdk.dev/docs","intent":"Provide a free, TypeScript-first toolkit that standardises calls to any model provider with a single API for text generation, structured-object generation, streaming UIs, tool-calling, and multi-step agent loops in TypeScript/JavaScript runtimes including browser, Node.js, and Vercel Edge.","description":"The Vercel AI SDK is the Apache-2.0 TypeScript toolkit from Vercel split into AI SDK Core ('A unified API for generating text, structured objects, tool calls, and building agents with LLMs') and AI SDK UI ('A set of framework-agnostic hooks for quickly building chat and generative user interface'). Core exposes generateText, streamText, and the structured-output siblings generateObject / streamObject driven by Zod, Valibot, or JSON Schema. The agent loop is opt-in via stopWhen (default stepCountIs(20)). UI layer ships useChat and useCompletion hooks for React (plus Vue/Svelte equivalents). Designed for edge runtimes alongside Node.js.","primary_use_cases":["TypeScript agents over multiple LLM providers behind one API","streaming chat UIs via useChat / streamText","Zod-typed structured outputs","MCP tool integration in TypeScript"],"agent_loop_shape":"Imperative TypeScript loop. Caller invokes generateText({ model, tools, stopWhen, ... }) or streamText(...). In each step the model either generates text (loop ends) or calls a tool (executed in-process, observation feeds back in for another generation). Loop terminates when stopWhen is satisfied; default cap stepCountIs(20). No graph DSL, no class hierarchy — just async functions, plain objects, and AbortSignal.","key_concepts":[{"name":"generateText / streamText","summary":"Core agent-loop entry points; streamText returns a token stream.","url":"https://ai-sdk.dev/docs/ai-sdk-core/generating-text"},{"name":"tools","summary":"A tool consists of three properties: description, inputSchema (Zod or JSON schema), and execute.","maps_to_pattern":"tool-use","url":"https://ai-sdk.dev/docs/foundations/tools"},{"name":"stopWhen / stepCountIs","summary":"Loop-termination predicate; stepCountIs(20) is the default safety cap.","maps_to_pattern":"step-budget","url":"https://ai-sdk.dev/docs/agents/loop-control"},{"name":"generateObject + Zod","summary":"Structured-output generation with schema validation.","maps_to_pattern":"structured-output","url":"https://ai-sdk.dev/docs/ai-sdk-core/generating-structured-data"},{"name":"Providers (language model spec)","summary":"Standardised interface abstracting provider differences (OpenAI / Anthropic / Google / xAI / Bedrock / Vertex / Gateway).","url":"https://ai-sdk.dev/docs/foundations/providers-and-models"},{"name":"useChat hook","summary":"React UI hook for conversational interfaces (framework-agnostic equivalents for Vue/Svelte).","url":"https://ai-sdk.dev/docs/ai-sdk-ui/chatbot"},{"name":"AbortSignal / onAbort","summary":"Cancellation lifecycle — onAbort fires when streams are aborted via AbortSignal; onFinish is skipped.","maps_to_pattern":"stop-cancel","url":"https://ai-sdk.dev/docs/ai-sdk-core/error-handling"}],"pattern_composition":"flowchart TD\n  CALLER[Caller / Route handler] --> GT[generateText / streamText]\n  GT -->|loop until stopWhen| MODEL[LLM via provider]\n  MODEL --> DEC{tool call?}\n  DEC -- yes --> TOOL[tool.execute]\n  TOOL --> GT\n  DEC -- no --> OUT[Final text or object]\n  GT -.->|tools| TOOLS[(tools: description+inputSchema+execute)]\n  GT -.->|MCP| MCP[MCP client tools]\n  GT -.->|stopWhen| BUDGET[stepCountIs / hasToolCall]\n  GT -.->|AbortSignal| ABRT[AbortController → onAbort]\n  GT -.->|providerMetadata| SRC[(sources stream)]\n  GT -.->|schema| OBJ[generateObject / Output.object]\n  OUT --> UI[useChat / useCompletion]\n  UI -.->|stop()| ABRT","members":[{"pattern":"citation-streaming","role":"first-class","note":"Sources surface as typed source stream-parts (sourceType: 'url'); each entry has id, url, title, providerMetadata. Upstream uses the word 'sources' rather than 'citations'.","evidence":[{"type":"doc","title":"AI SDK — sources from generateText","url":"https://ai-sdk.dev/docs/ai-sdk-core/generating-text","quote":"Some providers such as Perplexity and Google Generative AI include sources in the response.","accessed":"2026-05-20"},{"type":"doc","title":"AI SDK — url source shape","url":"https://ai-sdk.dev/docs/ai-sdk-core/generating-text","quote":"Each `url` source contains the following properties: `id`, `url`, `title`, and `providerMetadata`.","accessed":"2026-05-20"},{"type":"doc","title":"AI SDK — sources via fullStream","url":"https://ai-sdk.dev/docs/ai-sdk-core/generating-text","quote":"When you use `streamText`, you can access the sources using the `fullStream` property.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"Three transports — HTTP (recommended), SSE, stdio. client.tools() adapts MCP tool definitions to AI SDK tool definitions.","evidence":[{"type":"doc","title":"AI SDK — MCP overview","url":"https://ai-sdk.dev/docs/ai-sdk-core/tools-and-tool-calling#mcp-tools","quote":"The AI SDK supports connecting to Model Context Protocol (MCP) servers to access their tools. MCP enables your AI applications to discover and use tools across various services through a standardized interface.","accessed":"2026-05-20"},{"type":"doc","title":"AI SDK — MCP transports","url":"https://ai-sdk.dev/docs/ai-sdk-core/mcp-tools","quote":"Create an MCP client using one of the following transport options: HTTP transport (Recommended), SSE (Server-Sent Events), or stdio","accessed":"2026-05-20"},{"type":"doc","title":"AI SDK — MCP tools adapter","url":"https://ai-sdk.dev/docs/ai-sdk-core/mcp-tools","quote":"The client's tools method acts as an adapter between MCP tools and AI SDK tools.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"supported","note":"The agent loop IS the ReAct shape (reason → act → observe → repeat), but upstream docs never use the term 'ReAct'. Downgraded from first-class to supported because the pattern is implemented but not named.","evidence":[{"type":"doc","title":"AI SDK — Agents overview","url":"https://ai-sdk.dev/docs/agents/overview","quote":"Agents are large language models (LLMs) that use tools in a loop to accomplish tasks.","accessed":"2026-05-20"},{"type":"doc","title":"AI SDK — Building agents (step shape)","url":"https://ai-sdk.dev/docs/agents/building-agents","quote":"In each step, the model either generates text or calls a tool. If it generates text, the agent completes.","accessed":"2026-05-20"},{"type":"doc","title":"AI SDK — Building agents (loop)","url":"https://ai-sdk.dev/docs/agents/building-agents","quote":"After each tool execution, the agent triggers a new generation where the model can call another tool or generate text.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"step-budget","role":"first-class","note":"Default stepCountIs(20); stopWhen accepts any predicate (stepCountIs, hasToolCall, custom).","evidence":[{"type":"doc","title":"AI SDK — Loop control default cap","url":"https://ai-sdk.dev/docs/agents/loop-control","quote":"By default, agents stop after 20 steps using `stepCountIs(20)`.","accessed":"2026-05-20"},{"type":"doc","title":"AI SDK — Loop control safety framing","url":"https://ai-sdk.dev/docs/agents/loop-control","quote":"This default is a safety measure to prevent runaway loops that could result in excessive API calls and costs.","accessed":"2026-05-20"},{"type":"doc","title":"AI SDK — stopWhen multi-step","url":"https://ai-sdk.dev/docs/ai-sdk-core/tools-and-tool-calling","quote":"With the `stopWhen` setting, you can enable multi-step calls in `generateText` and `streamText`.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"stop-cancel","role":"first-class","note":"Streams accept AbortSignal; onAbort callback fires on abort (onFinish skipped). useChat exposes a stop() helper.","evidence":[{"type":"doc","title":"AI SDK — Error handling (abort)","url":"https://ai-sdk.dev/docs/ai-sdk-core/error-handling","quote":"When streams are aborted (e.g., via chat stop button), you may want to perform cleanup operations like updating stored messages in your UI.","accessed":"2026-05-20"},{"type":"doc","title":"AI SDK — onAbort callback","url":"https://ai-sdk.dev/docs/ai-sdk-core/error-handling","quote":"The `onAbort` callback is called when a stream is aborted via `AbortSignal`, but `onFinish` is not called.","accessed":"2026-05-20"},{"type":"doc","title":"AI SDK — useChat stop button","url":"https://ai-sdk.dev/docs/ai-sdk-ui/chatbot","quote":"When the user clicks the 'Stop' button, the fetch request will be aborted.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"structured-output","role":"first-class","note":"generateObject / streamObject accept Zod, Valibot, or JSON Schema; schemas drive both generation and validation. Composable with tool calling.","evidence":[{"type":"doc","title":"AI SDK — Generating structured data (schemas)","url":"https://ai-sdk.dev/docs/ai-sdk-core/generating-structured-data","quote":"You can use Zod schemas, Valibot, or JSON schemas to specify the shape of the data that you want.","accessed":"2026-05-20"},{"type":"doc","title":"AI SDK — Structured output + tool calling","url":"https://ai-sdk.dev/docs/ai-sdk-core/generating-structured-data","quote":"Structured output generation is part of the generateText and streamText flow. This means you can combine it with tool calling.","accessed":"2026-05-20"},{"type":"doc","title":"AI SDK — Schema validation","url":"https://ai-sdk.dev/docs/ai-sdk-core/generating-structured-data","quote":"The schema is also used to validate the generated data, ensuring type safety and correctness.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Tools are objects with description, inputSchema (Zod/JSON Schema), execute; passed to generateText / streamText via tools param.","evidence":[{"type":"doc","title":"AI SDK — Tool shape","url":"https://ai-sdk.dev/docs/foundations/tools","quote":"A tool consists of three properties: `description`, `inputSchema`, and `execute`.","accessed":"2026-05-20"},{"type":"doc","title":"AI SDK — Passing tools","url":"https://ai-sdk.dev/docs/foundations/tools","quote":"You can use tools with `generateText` and `streamText` by passing one or more tools to the `tools` parameter.","accessed":"2026-05-20"},{"type":"doc","title":"AI SDK — Schema dual-use","url":"https://ai-sdk.dev/docs/ai-sdk-core/tools-and-tool-calling","quote":"A Zod schema or a JSON schema that defines the input parameters. The schema is consumed by the LLM, and also used to validate the LLM tool calls","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"generative-ui","role":"first-class","note":"The AI SDK's streamUI / render API maps a model's tool calls to React Server Components at runtime and streams those components into the response, so the rendered interface follows the model's output instead of a hardcoded layout.","evidence":[{"type":"blog","title":"Introducing AI SDK 3.0 with Generative UI support","url":"https://vercel.com/blog/ai-sdk-3-generative-ui","quote":"you can use the new render method to map specific calls to React Server Components.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"anti_patterns_avoided":[{"pattern":"vendor-lock-in","note":"Provider abstraction is a stated design goal: a unified language-model spec lets callers swap providers without rewriting application code.","evidence":[{"type":"doc","title":"AI SDK — Providers framing","url":"https://ai-sdk.dev/docs/foundations/providers-and-models","quote":"Each provider typically has its own unique method for interfacing with their models, complicating the process of switching providers and increasing the risk of vendor lock-in.","accessed":"2026-05-20"}]},{"pattern":"hallucinated-tools","note":"NoSuchToolError fires when the model calls a tool not in the defined tool set, and experimental_repairToolCall can repair it, so an invented tool name does not silently execute.","evidence":[{"type":"doc","title":"Vercel AI SDK — tools and tool calling","url":"https://ai-sdk.dev/docs/ai-sdk-core/tools-and-tool-calling","quote":"NoSuchToolError ... the model tries to call a tool that is not defined in the tools object ... experimental_repairToolCall function to attempt to repair the tool call with a custom function.","accessed":"2026-06-19"}]}],"instantiates":["streaming-ux-stack","structured-output-stack"],"alternatives":[{"composition":"claude-agent-sdk","relation":"competes-with","note":"Same agent-sdk category; Vercel is TS-first multi-provider, Claude Agent SDK is Anthropic-only with built-in filesystem tools."},{"composition":"openai-agents-sdk","relation":"competes-with","note":"Both ship TS + Python flavours; OpenAI is Handoff-centric while Vercel is loop-centric."},{"composition":"google-adk","relation":"competes-with","note":"Same category, different host language (Python/Java vs TypeScript)."},{"composition":"mastra","relation":"similar-shape","note":"TypeScript-native agent framework."},{"composition":"langchain","relation":"competes-with","note":"LangChain.js covers similar territory in TS."},{"composition":"genkit","relation":"similar-shape","note":"Peer provider-neutral TS-first AI app framework."},{"composition":"instructor","relation":"similar-shape","note":"Vercel's generateObject covers the structured-output slice in TypeScript with Zod schemas."}],"references":[{"type":"repo","title":"vercel/ai","url":"https://github.com/vercel/ai","quote":"The AI Toolkit for TypeScript. From the creators of Next.js, the AI SDK is a free open-source library for building AI-powered applications and agents","accessed":"2026-05-20"},{"type":"doc","title":"AI SDK — Introduction","url":"https://ai-sdk.dev/docs/introduction","quote":"The TypeScript toolkit designed to help developers build AI-powered applications and agents","accessed":"2026-05-20"}],"last_analyzed":"2026-05-20","verification_status":"needs-verification","last_updated":"2026-06-19","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","typescript","multi-provider","streaming","edge-runtime"]},{"id":"strands-agents","name":"Strands Agents","kind":"framework","category":"agent-sdk","build_surface":"full-code","vendor":"Strands Agents (AWS-originated, community-run)","language":"Python, TypeScript","license":"Apache-2.0","status":"active","url":"https://strandsagents.com","repo":"https://github.com/strands-agents/sdk-python","docs_url":"https://strandsagents.com/latest/documentation/docs/","intent":"A model-driven approach to building AI agents in just a few lines of code.","description":"Lightweight, model-driven agent SDK: define an agent with a model, system prompt and tools, and let the model drive the loop. Native MCP server support, 13+ model providers (Amazon Bedrock default, plus OpenAI/Anthropic/Gemini/Ollama and others), multi-agent and autonomous-agent primitives, bidirectional audio streaming, Python-decorator tools and hot-reloadable tool directories. AWS-originated, now an open community project.","primary_use_cases":["building agents in a few lines with a model-driven loop","MCP-native tool integration across many providers","multi-agent and autonomous-agent systems, including real-time audio"],"agent_loop_shape":"Model-driven loop: the model decides which tools to call and when to stop, given a model, system prompt and a set of decorator-defined tools. Supports multi-agent composition and bidirectional streaming for audio.","key_concepts":[{"name":"Model-driven loop","summary":"The model decides which tools to call and when to stop.","url":"https://strandsagents.com/"},{"name":"Tool decorators","summary":"Python decorators register tools; tool directories hot-reload.","maps_to_pattern":"tool-use"},{"name":"MCP","summary":"Native Model Context Protocol server support.","maps_to_pattern":"mcp"}],"pattern_composition":"flowchart TD\n  GOAL[Task] --> LOOP[Model-driven loop]\n  LOOP --> DECIDE{Call tool?}\n  DECIDE -->|yes| TOOL[Decorator tool / MCP tool]\n  TOOL --> LOOP\n  DECIDE -->|done| OUT[Result]\n  LOOP -.multi-agent.-> WORKERS[Sub-agents]\n  LOOP -.audio.-> STREAM[Bidirectional streaming]","members":[{"pattern":"tool-use","role":"first-class","note":"Easily build tools using Python decorators","evidence":[{"type":"repo","title":"strands-agents/sdk-python README","url":"https://github.com/strands-agents/sdk-python","quote":"Easily build tools using Python decorators","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"mcp","role":"first-class","note":"Native support for Model Context Protocol (MCP) servers, enabling access to thousands of pre-built tools","evidence":[{"type":"repo","title":"strands-agents/sdk-python README","url":"https://github.com/strands-agents/sdk-python","quote":"Native support for Model Context Protocol (MCP) servers, enabling access to thousands of pre-built tools","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"multi-model-routing","role":"first-class","note":"Model Agnostic: Support for Amazon Bedrock, Anthropic, Gemini, LiteLLM, Llama, Ollama, OpenAI, Writer, and custom providers","evidence":[{"type":"repo","title":"strands-agents/sdk-python README","url":"https://github.com/strands-agents/sdk-python","quote":"Model Agnostic: Support for Amazon Bedrock, Anthropic, Gemini, LiteLLM, Llama, Ollama, OpenAI, Writer, and custom providers","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"orchestrator-workers","role":"supported","note":"Advanced Capabilities: Multi-agent systems, autonomous agents, and streaming support","evidence":[{"type":"repo","title":"strands-agents/sdk-python README","url":"https://github.com/strands-agents/sdk-python","quote":"Advanced Capabilities: Multi-agent systems, autonomous agents, and streaming support","accessed":"2026-06-04"}],"evidence_status":"limited"}],"instantiates":["multi-agent-coordination"],"alternatives":[{"composition":"openai-agents-sdk","relation":"similar-shape","note":"Peer minimal agent SDK with a model-driven loop."},{"composition":"pydantic-ai","relation":"similar-shape","note":"Peer Python-first typed agent SDK."},{"composition":"bedrock-agentcore","relation":"complements","note":"AgentCore runs Strands agents as a first-class supported framework."}],"references":[{"type":"repo","title":"strands-agents/sdk-python","url":"https://github.com/strands-agents/sdk-python","accessed":"2026-06-04"},{"type":"doc","title":"Strands Agents documentation","url":"https://strandsagents.com/latest/documentation/docs/","accessed":"2026-06-04"}],"last_analyzed":"2026-06-04","verification_status":"partial","last_updated":"2026-06-04","date_added":"2026-06-04","status_in_practice":"emerging","tags":["aws","python","typescript","mcp","agent-sdk","open-source"]},{"id":"copilotkit","name":"CopilotKit","kind":"framework","category":"agent-sdk","vendor":"CopilotKit","language":"TypeScript","license":"MIT","status":"active","first_released":"2023","url":"https://www.copilotkit.ai/","repo":"https://github.com/CopilotKit/CopilotKit","docs_url":"https://docs.copilotkit.ai/","intent":"CopilotKit gives a web frontend the components and protocol to connect to an agent backend so the agent can stream text, render UI, share state, and pause for the user.","description":"CopilotKit is an open-source frontend stack for agent-powered user interfaces, built by the CopilotKit team. It supplies React components and hooks plus the AG-UI protocol, which carries events in both directions between the application and any agentic backend. Over that channel the agent streams responses, renders React components as tool output, syncs shared state with the page, and can pause execution to ask the user for input or approval.","primary_use_cases":["in-app copilot and chat interfaces","generative UI rendered from agent tool calls","shared state between frontend and agent","human-in-the-loop approval and confirmation flows","connecting React/Angular/mobile frontends to agent backends"],"agent_loop_shape":"A bidirectional event stream between frontend and agent: the application POSTs user input and state to the agent, and the agent emits a stream of typed events (text, tool calls, state deltas) over AG-UI. The agent's loop runs on the backend; CopilotKit drives the UI from the event stream and can send the agent back into the loop after a human responds.","build_surface":"full-code","members":[{"pattern":"bidirectional-impulse-channel","role":"first-class","note":"AG-UI is one event-streaming channel carrying both directions: the frontend POSTs user prompts/state into the agent and listens to the same SSE stream for the agent's text, tool calls and state deltas; session/cancellation events let the agent pause mid-run to ask the user for input.","evidence":[{"type":"doc","title":"AG-UI Protocol","url":"https://www.copilotkit.ai/ag-ui","quote":"AG-UI (Agent–User Interaction) is the general-purpose, bi-directional connection between a user-facing application and any agentic backend.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"generative-ui","role":"first-class","note":"Agents render real React components in the chat by calling them as tools; CopilotKit lets the agent generate and update UI components at runtime based on user intent and agent state rather than returning only text.","evidence":[{"type":"repo","title":"CopilotKit/CopilotKit — README (Features: Generative UI)","url":"https://raw.githubusercontent.com/CopilotKit/CopilotKit/main/README.md","quote":"Generative UI – Allows agents to generate and update UI components dynamically at runtime based on user intent and agent state.","accessed":"2026-06-17"},{"type":"repo","title":"CopilotKit/CopilotKit — README (tagline)","url":"https://github.com/CopilotKit/CopilotKit","quote":"The Frontend Stack for Agents & Generative UI. React, Angular, Mobile, Slack, and more.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"human-in-the-loop","role":"first-class","note":"CopilotKit lets the agent pause its run and hand control to the user to request input, confirmation, or edits before continuing, then resumes the agent loop with the user's response over the AG-UI channel.","evidence":[{"type":"repo","title":"CopilotKit/CopilotKit — README (Features: Human-in-the-Loop)","url":"https://raw.githubusercontent.com/CopilotKit/CopilotKit/main/README.md","quote":"Human-in-the-Loop – Lets agents pause execution to request user input, confirmation, or edits before continuing.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"streaming-typed-events","role":"core","note":"AG-UI carries the agent run to the frontend as a stream of typed events (text-message, tool-call, state-delta, reasoning, and others), each delivered incrementally so the UI updates as the agent works rather than after it finishes.","evidence":[{"type":"doc","title":"Events — AG-UI Protocol documentation","url":"https://docs.ag-ui.com/concepts/events","quote":"Events are the fundamental units of communication between agents and frontends, enabling real-time, structured interaction.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"blackboard","role":"first-class","note":"CopilotKit's Shared State is a single synchronized state layer both the agent and the UI components read from and write to, so the page and the backend agent collaborate over the same workspace rather than passing copies of state back and forth.","evidence":[{"type":"repo","title":"CopilotKit/CopilotKit — README (Features: Shared State)","url":"https://raw.githubusercontent.com/CopilotKit/CopilotKit/main/README.md","quote":"Shared State – A synchronized state layer that both agents and UI components can read from and write to in real time.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"AG-UI Protocol","summary":"The open event protocol CopilotKit defines and maintains for connecting a user-facing application to any agentic backend; the agent and the app exchange typed events over it in both directions.","maps_to_pattern":"bidirectional-impulse-channel","url":"https://docs.ag-ui.com/"},{"name":"useCopilotAction","summary":"A React hook that registers a frontend action the agent can call as a tool; the action can render a React component as its result, which is how generative UI reaches the page.","maps_to_pattern":"generative-ui","url":"https://docs.copilotkit.ai/"},{"name":"useCopilotReadable","summary":"A hook that exposes a slice of application state to the agent as readable context, so the agent reasons over what the user currently sees on the page.","maps_to_pattern":"blackboard","url":"https://docs.copilotkit.ai/"},{"name":"CoAgents","summary":"CopilotKit's integration layer for agentic backends (LangGraph, CrewAI, and others) that streams the agent's intermediate state into the UI and syncs shared state between the running agent and the page.","maps_to_pattern":"blackboard","url":"https://docs.copilotkit.ai/"}],"references":[{"type":"doc","title":"CopilotKit — The Frontend Stack for Agents","url":"https://www.copilotkit.ai/","accessed":"2026-06-17"},{"type":"repo","title":"CopilotKit/CopilotKit — GitHub repository","url":"https://github.com/CopilotKit/CopilotKit","quote":"The Frontend Stack for Agents & Generative UI. React, Angular, Mobile, Slack, and more.","accessed":"2026-06-17"},{"type":"doc","title":"AG-UI Protocol documentation","url":"https://docs.ag-ui.com/","accessed":"2026-06-17"}],"alternatives":[{"composition":"vercel-ai-sdk","relation":"competes-with","note":"Vercel AI SDK also provides React hooks and streaming primitives for building agent-driven UIs, but ships its own streaming protocol rather than the cross-vendor AG-UI channel CopilotKit centres on."},{"composition":"langgraph","relation":"complements","note":"LangGraph is a common backend CopilotKit's CoAgents connect to; LangGraph runs the agent loop and CopilotKit renders it in the frontend over AG-UI."}],"verification_status":"partial","status_in_practice":"emerging","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"CopilotKit\"]\n  fw --> p1[\"Streaming Typed Events<br/>(core)\"]\n  fw --> p2[\"Bidirectional Impulse Channel<br/>(first-class)\"]\n  fw --> p3[\"Generative UI<br/>(first-class)\"]\n  fw --> p4[\"Human-in-the-Loop<br/>(first-class)\"]\n  fw --> p5[\"Blackboard<br/>(first-class)\"]","tags":["agent-sdk","streaming-ux","safety-control","multi-agent","full-code"]},{"id":"cloudflare-code-mode","name":"Cloudflare Code Mode","kind":"framework","category":"agent-sdk","vendor":"Cloudflare","language":"TypeScript","license":"proprietary","status":"active","first_released":"2025","url":"https://blog.cloudflare.com/code-mode/","docs_url":"https://blog.cloudflare.com/code-mode/","intent":"Convert MCP tools into a TypeScript API and have the model write code that calls that API, then run the code in an isolated sandbox on Cloudflare Workers.","description":"Code Mode is Cloudflare's technique for invoking tools from an agent by generating TypeScript that calls a tool API rather than emitting discrete tool-call messages. It converts Model Context Protocol tools into a TypeScript surface and asks the model to write code against it. The generated code runs in a Worker isolate created on demand through the Worker Loader API and discarded after the run. The isolate is cut off from the open Internet so the generated code cannot reach arbitrary network destinations.","primary_use_cases":["calling MCP tools through generated code instead of direct tool calls","running model-written TypeScript in a short-lived sandbox","composing multiple tool calls within a single code execution"],"agent_loop_shape":"The host converts available MCP tools into a TypeScript API, the model writes a code snippet that calls that API, and the snippet is run in a freshly created Worker isolate that is isolated from the Internet and thrown away after the run. Results from the executed code flow back to the model.","build_surface":"full-code","members":[{"pattern":"mcp-as-code-api","role":"supported","note":"Code Mode materializes MCP tools as a TypeScript API the model writes code against, rather than exposing each tool as a separate tool-call schema.","evidence":[{"type":"blog","title":"Code Mode: the better way to use MCP","url":"https://blog.cloudflare.com/code-mode/","quote":"Convert the MCP tools into a TypeScript API, and then ask an LLM to write code that calls that API.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"code-execution","role":"core","note":"The model emits a code snippet, which Code Mode runs in an isolated Worker sandbox created on demand and discarded after the run; the run result is treated as the action's output.","evidence":[{"type":"blog","title":"Code Mode: the better way to use MCP","url":"https://blog.cloudflare.com/code-mode/","quote":"The code is then executed in a secure sandbox. The sandbox is totally isolated from the Internet.","accessed":"2026-06-17"},{"type":"blog","title":"Code Mode: the better way to use MCP","url":"https://blog.cloudflare.com/code-mode/","quote":"Just create it, on demand, run the code, and throw it away. It all happens so fast that the overhead is negligible.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"sandbox-isolation","role":"core","note":"The model-written code runs in a V8 isolate that is denied general network access — global fetch() and connect() throw — and is instead handed bindings only for the specific MCP servers it may reach, so the generated code is confined to a contained environment with restricted network privileges.","evidence":[{"type":"blog","title":"Code Mode: the better way to use MCP","url":"https://blog.cloudflare.com/code-mode/","quote":"In Code Mode, we prohibit the sandboxed worker from talking to the Internet. The global `fetch()` and `connect()` functions throw errors.","accessed":"2026-06-17"},{"type":"blog","title":"Code Mode: the better way to use MCP","url":"https://blog.cloudflare.com/code-mode/","quote":"In Code Mode, we give the sandbox access to bindings representing the MCP servers it is connected to. Thus, the agent can specifically access those MCP servers _without_ having network access in general.","accessed":"2026-06-17"}],"evidence_status":"full"}],"key_concepts":[{"name":"Worker Loader API","summary":"The Cloudflare API that loads Worker code on demand, so a fresh isolate can be created to run the model's generated code and then discarded; this is what makes per-call sandboxes cheap enough to use.","maps_to_pattern":"code-execution","url":"https://blog.cloudflare.com/code-mode/"},{"name":"V8 Isolate","summary":"Cloudflare's lightweight execution unit, far cheaper than a container, that starts in milliseconds and a few megabytes; Code Mode runs each code snippet in its own isolate.","maps_to_pattern":"sandbox-isolation","url":"https://blog.cloudflare.com/code-mode/"},{"name":"Bindings","summary":"Capability handles given to the sandbox that provide an already-authorized client to a specific MCP server; the sandbox can reach those servers but has no general network access, and credentials never enter the generated code.","maps_to_pattern":"sandbox-isolation","url":"https://blog.cloudflare.com/code-mode/"}],"references":[{"type":"blog","title":"Code Mode: the better way to use MCP","url":"https://blog.cloudflare.com/code-mode/","quote":"Convert the MCP tools into a TypeScript API, and then ask an LLM to write code that calls that API.","accessed":"2026-06-17"},{"type":"doc","title":"Cloudflare Agents — Model Context Protocol (MCP) documentation","url":"https://developers.cloudflare.com/agents/model-context-protocol/","accessed":"2026-06-17"}],"alternatives":[{"composition":"claude-agent-sdk","relation":"competes-with","note":"Anthropic's agent SDK documents a comparable code-execution approach to MCP, where the model writes code that imports tools instead of issuing many discrete tool calls; Code Mode binds that idea to Cloudflare's Worker isolate runtime."}],"verification_status":"partial","status_in_practice":"emerging","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Cloudflare Code Mode\"]\n  fw --> p1[\"Code Execution<br/>(core)\"]\n  fw --> p2[\"Sandbox Isolation<br/>(core)\"]\n  fw --> p3[\"MCP-as-Code-API<br/>(supported)\"]","tags":["agent-sdk","tool-use-environment","full-code"]},{"id":"baml","name":"BAML","kind":"framework","category":"agent-sdk","vendor":"Boundary (BoundaryML)","language":"Rust","license":"Apache-2.0","status":"active","first_released":"2023-10-01","url":"https://www.boundaryml.com","repo":"https://github.com/BoundaryML/baml","docs_url":"https://docs.boundaryml.com/home","intent":"BAML is a domain-specific language for writing LLM functions that return type-safe structured outputs and are version-controlled and testable alongside application code.","description":"BAML is a domain-specific language for defining and testing LLM functions. Each prompt becomes a typed function that produces structured outputs from an LLM, with generated client code that calls the endpoint, parses the output, fixes broken JSON, and handles errors. Prompts are version controlled with the application code and can be viewed and run directly within the editor. BAML works across model providers, including open-source models, and is released under Apache 2.0.","primary_use_cases":["defining LLM functions as typed code","extracting type-safe structured outputs from LLMs","version-controlling and testing prompts with application code","running prompts directly in the editor"],"agent_loop_shape":"A developer declares an LLM function in BAML with a typed return shape and prompt. BAML generates client code that sends the prompt to the configured model, parses the response, repairs malformed JSON, and returns a typed value to the application. The function can be run and tested directly in the editor, and lives in source control with the rest of the code.","build_surface":"full-code","members":[{"pattern":"own-your-prompts","role":"core","note":"BAML makes every prompt a typed function defined in source files in the application repo, version-controlled with the code and testable in-editor, so prompts are owned and tested by the team rather than inherited as a framework default.","evidence":[{"type":"doc","title":"What is BAML? — BAML Documentation","url":"https://docs.boundaryml.com/home","quote":"BAML is a domain-specific language to write and test LLM functions... Prompts are version controlled with your code.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"structured-output","role":"core","note":"BAML's purpose is generating structured outputs from LLMs: every LLM function declares a typed return shape, and the generated client parses the model output into that type with Schema-Aligned Parsing, repairing broken JSON, so downstream code consumes a typed value rather than free-form text.","evidence":[{"type":"doc","title":"What is BAML? — BAML Documentation","url":"https://docs.boundaryml.com/home","quote":"BAML is a domain-specific language to generate structured outputs from LLMs -- with the best developer experience.","accessed":"2026-06-17"},{"type":"doc","title":"What is BAML? — BAML Documentation","url":"https://docs.boundaryml.com/home","quote":"Fully type-safe outputs, even when streaming structured data (that means autocomplete!)","accessed":"2026-06-17"},{"type":"blog","title":"Prompting vs JSON Mode vs Function Calling vs Constrained Generation vs SAP — BAML Blog","url":"https://boundaryml.com/blog/schema-aligned-parsing","quote":"Instead of relying on the model to strictly understand our desired format, write a parser that generously reads the output text and applies error correction techniques with knowledge of the original schema.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"provider-fallback","role":"supported","note":"A BAML client can be declared as a fallback strategy over an ordered list of provider clients; if the first provider fails, the generated client tries the next, so a function can fail over across model providers without changing call sites.","evidence":[{"type":"doc","title":"fallback — BAML LLM client strategies documentation","url":"https://docs.boundaryml.com/ref/llm-client-strategies/fallback","quote":"A fallback will attempt to use the first client, and if it fails, it will try the second client, and so on.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"eval-harness","role":"supported","note":"BAML functions carry test blocks that run the function against a model with fixed inputs and check the result with @@assert / @@check; tests run in the editor playground or from the CLI, so prompt changes are exercised against expected outputs before release.","evidence":[{"type":"doc","title":"Testing functions — BAML Documentation","url":"https://docs.boundaryml.com/guide/baml-basics/testing-functions","quote":"You can test your BAML functions in the VSCode Playground by adding a `test` snippet into a BAML file","accessed":"2026-06-17"},{"type":"doc","title":"Testing functions — BAML Documentation","url":"https://docs.boundaryml.com/guide/baml-basics/testing-functions","quote":"You can also run your tests from the command line using the BAML CLI","accessed":"2026-06-17"}],"evidence_status":"full"}],"key_concepts":[{"name":"LLM function","summary":"BAML's central unit: a named function with typed parameters, a typed return shape, a client, and a prompt template, compiled into a generated client your application calls like an ordinary function.","maps_to_pattern":"own-your-prompts","url":"https://docs.boundaryml.com/home"},{"name":"Schema-Aligned Parsing (SAP)","summary":"BAML's Rust parser that reads whatever the model returns and coerces it into the declared schema, fixing markdown fences, trailing commas, and unquoted values instead of requiring strictly valid JSON.","maps_to_pattern":"structured-output","url":"https://boundaryml.com/blog/schema-aligned-parsing"},{"name":"baml_client","summary":"The generated client package BAML emits for your target language (Python, TypeScript, Ruby, Go, and more) that wraps each LLM function with type-safe call, parse, and error handling.","maps_to_pattern":"structured-output","url":"https://docs.boundaryml.com/home"},{"name":"test block","summary":"An in-file test case for an LLM function with fixed inputs and @@assert / @@check expectations, runnable in the VSCode/JetBrains playground or via the BAML CLI.","maps_to_pattern":"eval-harness","url":"https://docs.boundaryml.com/guide/baml-basics/testing-functions"}],"references":[{"type":"doc","title":"What is BAML? — BAML Documentation","url":"https://docs.boundaryml.com/home","quote":"BAML is a domain-specific language to generate structured outputs from LLMs -- with the best developer experience.","accessed":"2026-06-17"},{"type":"repo","title":"BoundaryML/baml — GitHub repository","url":"https://github.com/BoundaryML/baml","quote":"BAML is a simple prompting language for building reliable **AI workflows and agents**.","accessed":"2026-06-17"}],"alternatives":[{"composition":"instructor","relation":"competes-with","note":"Instructor also turns LLM calls into type-safe structured outputs, but works as a library inside the host language using Pydantic/Zod models, whereas BAML defines prompts in its own DSL and generates client code from them."},{"composition":"dspy","relation":"competes-with","note":"DSPy and BAML both separate prompt definition from application code, but DSPy compiles and optimises prompts programmatically in Python while BAML keeps prompts as hand-authored, version-controlled functions in its DSL."}],"verification_status":"partial","status_in_practice":"emerging","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"BAML\"]\n  fw --> p1[\"Own Your Prompts (12-Factor Agents)<br/>(core)\"]\n  fw --> p2[\"Structured Output<br/>(core)\"]\n  fw --> p3[\"Provider Fallback<br/>(supported)\"]\n  fw --> p4[\"Eval Harness<br/>(supported)\"]","tags":["agent-sdk","governance-observability","structure-data","routing-composition","full-code"]},{"id":"stainless-mcp","name":"Stainless (API MCP server architecture)","kind":"framework","category":"agent-sdk","vendor":"Stainless Software, Inc.","language":"TypeScript","license":"proprietary","status":"active","first_released":"2024-01-01","url":"https://www.stainless.com/mcp/","docs_url":"https://www.stainless.com/mcp/api-mcp-server-architecture-guide/","intent":"Stainless generates and prescribes an architecture for MCP servers that turn an existing REST or OpenAPI surface into tools an agent can call.","description":"Stainless generates MCP servers from an API specification and publishes guidance on how to structure the resulting tool surface. The architecture guide describes several tool-exposure strategies, from mapping every endpoint one-to-one to a tool, to composite tools that bundle several API calls, to dynamic meta-tools that let the model discover endpoints on demand. The guidance also covers filtering which tools are exposed based on the user's permissions or OAuth scopes. Stainless targets API providers that want their service to be callable by agents.","primary_use_cases":["generating MCP servers from an existing API","exposing API endpoints as agent-callable tools","bundling multi-step API workflows into single composite tools","scoping the exposed tool set to user permissions"],"agent_loop_shape":"An agent connects to a Stainless-generated MCP server that fronts an existing API. The server presents the API as tools, either one tool per endpoint, composite tools that run a sequence of calls, or dynamic meta-tools the model uses to discover and invoke endpoints on demand. The agent selects a tool, the server executes the underlying API call or sequence of calls, and the result returns to the agent's context for the next step.","build_surface":"full-code","members":[{"pattern":"composite-service-mcp","role":"supported","note":"Stainless's MCP architecture guidance prescribes composite tools that perform a sequence of API calls as a single higher-level tool for multi-step workflows.","evidence":[{"type":"doc","title":"API MCP Server Architecture Guide for API Providers","url":"https://www.stainless.com/mcp/api-mcp-server-architecture-guide/","quote":"Composite tools: Create higher-level tools that perform a sequence of API calls.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"direct-api-wrapper","role":"first-class","note":"The default and recommended starting strategy maps every API endpoint one-to-one to a tool, exposing a stable API as agent-callable with minimal wrapper logic; the guide also documents filtering the exposed tool list by the user's permissions or OAuth scopes.","evidence":[{"type":"doc","title":"API MCP Server Architecture Guide for API Providers","url":"https://www.stainless.com/mcp/api-mcp-server-architecture-guide/","quote":"Expose every endpoint as a tool. This is a great starting point for simple APIs or for quickly validating which tools an LLM uses effectively.","accessed":"2026-06-17"},{"type":"doc","title":"API MCP Server Architecture Guide for API Providers","url":"https://www.stainless.com/mcp/api-mcp-server-architecture-guide/","quote":"Yes, you can implement logic in your remote server's authentication flow to filter the list of tools based on the user's permissions or OAuth scopes.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"mcp","role":"core","note":"Stainless generates the MCP server itself from the provider's existing OpenAPI specification, so a service that already publishes an OpenAPI surface for its SDKs becomes callable over MCP by any conformant agent without hand-writing the server.","evidence":[{"type":"doc","title":"API MCP Server Architecture Guide for API Providers","url":"https://www.stainless.com/mcp/api-mcp-server-architecture-guide/","quote":"This entire MCP layer can be automatically generated from the same OpenAPI specification you already use for your SDKs, and you can generate an MCP server from an OpenAPI spec turning a complex integration into a simple configuration step.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"tool-discovery","role":"supported","note":"Rather than exposing every endpoint as a static tool, the guide prescribes three meta-tools — list_api_endpoints, get_api_endpoint_schema, invoke_api_endpoint — so the model discovers, inspects, and calls endpoints at runtime instead of carrying the whole tool list in context.","evidence":[{"type":"doc","title":"API MCP Server Architecture Guide for API Providers","url":"https://www.stainless.com/mcp/api-mcp-server-architecture-guide/","quote":"Instead of exposing every endpoint, you provide just three meta-tools: `list_api_endpoints`, `get_api_endpoint_schema`, and `invoke_api_endpoint`. This allows the LLM to discover, learn about, and call endpoints on demand, neatly sidestepping context window limitations.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"Meta-tools (list/get/invoke endpoint)","summary":"A trio of dynamic tools — list_api_endpoints, get_api_endpoint_schema, invoke_api_endpoint — that let the model browse and call API endpoints on demand instead of being handed one tool per endpoint.","maps_to_pattern":"tool-discovery","url":"https://www.stainless.com/mcp/api-mcp-server-architecture-guide/"},{"name":"Composite tools","summary":"Hand-designed higher-level tools that each run a fixed sequence of underlying API calls, so the agent invokes one task-level operation instead of orchestrating several endpoints itself.","maps_to_pattern":"composite-service-mcp","url":"https://www.stainless.com/mcp/api-mcp-server-architecture-guide/"},{"name":"OpenAPI-driven generation","summary":"The Stainless build step that derives the MCP server, and its tool surface, from the same OpenAPI specification the provider already uses to generate SDKs.","maps_to_pattern":"mcp","url":"https://www.stainless.com/mcp/api-mcp-server-architecture-guide/"},{"name":"Scope-based tool filtering","summary":"Logic in the server's authentication flow that trims the advertised tool list to what the user's permissions or OAuth scopes allow.","maps_to_pattern":"direct-api-wrapper","url":"https://www.stainless.com/mcp/api-mcp-server-architecture-guide/"}],"references":[{"type":"doc","title":"MCP servers for your API — Stainless","url":"https://www.stainless.com/mcp/","accessed":"2026-06-17"},{"type":"doc","title":"API MCP Server Architecture Guide for API Providers","url":"https://www.stainless.com/mcp/api-mcp-server-architecture-guide/","accessed":"2026-06-17"}],"alternatives":[{"composition":"claude-agent-sdk","relation":"complements","note":"Stainless produces MCP servers; an agent built on a runtime such as the Claude Agent SDK is the consumer that connects to those servers and calls their tools."}],"verification_status":"partial","status_in_practice":"emerging","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Stainless (API MCP server architecture)\"]\n  fw --> p1[\"Model Context Protocol<br/>(core)\"]\n  fw --> p2[\"Direct API Wrapper<br/>(first-class)\"]\n  fw --> p3[\"Composite Service<br/>(supported)\"]\n  fw --> p4[\"Tool Discovery<br/>(supported)\"]","tags":["agent-sdk","tool-use-environment","full-code"]},{"id":"vllm","name":"vLLM","kind":"framework","category":"agent-sdk","vendor":"vLLM Project (originally UC Berkeley Sky Computing Lab)","language":"Python","license":"Apache-2.0","status":"active","first_released":"2023-06-20","url":"https://docs.vllm.ai","repo":"https://github.com/vllm-project/vllm","docs_url":"https://docs.vllm.ai","intent":"vLLM is an open-source LLM inference and serving engine the operator runs on its own hardware to serve models behind an OpenAI-compatible API.","description":"vLLM is an open-source library for LLM inference and serving that the operator deploys on its own GPUs or CPUs. It manages attention key and value memory with PagedAttention and batches incoming requests continuously to raise throughput. It exposes an OpenAI-compatible API server so existing clients point at a self-hosted endpoint. The project supports a range of hardware including NVIDIA, AMD, and CPU targets, and is released under Apache 2.0.","primary_use_cases":["self-hosted LLM inference and serving","high-throughput batched model serving","OpenAI-compatible endpoints on operator-owned hardware","running open-weight models across diverse accelerators"],"agent_loop_shape":"vLLM is the inference backend, not the agent loop itself. An agent or application sends generation requests to a vLLM server through its OpenAI-compatible API; vLLM batches incoming requests continuously, manages attention key and value memory with PagedAttention, and returns generations to the caller, who runs the surrounding agent orchestration.","build_surface":"full-code","members":[{"pattern":"sovereign-inference-stack","role":"core","note":"vLLM is an open-source LLM inference and serving engine the operator deploys on its own GPUs/CPUs, keeping the entire inference stack inside infrastructure the operator controls.","evidence":[{"type":"repo","title":"vllm-project/vllm README — Hardware support","url":"https://raw.githubusercontent.com/vllm-project/vllm/main/README.md","quote":"Support for NVIDIA GPUs, AMD GPUs, and x86/ARM/PowerPC CPUs. Additionally, diverse hardware plugins such as Google TPUs, Intel Gaudi, IBM Spyre, Huawei Ascend, Rebellions NPU, Apple Silicon, MetaX GPU, and more.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"prompt-caching","role":"supported","note":"vLLM serves prefix caching as a built-in serving feature, reusing the cached unchanging prompt prefix across requests so repeated prefixes cut recomputation; the engine is the provider side that makes prompt-prefix caching available to callers.","evidence":[{"type":"repo","title":"vllm-project/vllm README — Features","url":"https://raw.githubusercontent.com/vllm-project/vllm/main/README.md","quote":"Continuous batching of incoming requests, chunked prefill, prefix caching","accessed":"2026-06-17"},{"type":"repo","title":"vllm-project/vllm README — Efficient memory management","url":"https://raw.githubusercontent.com/vllm-project/vllm/main/README.md","quote":"Efficient management of attention key and value memory with PagedAttention","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"degenerate-output-detection","role":"supported","note":"vLLM exposes repetition_penalty to penalise tokens already present in the generated text, countering degenerate repetition.","evidence":[{"type":"doc","url":"https://docs.vllm.ai/en/v0.6.4/dev/sampling_params.html","quote":"Float that penalizes new tokens based on whether they appear in the prompt and the generated text so far. Values > 1 encourage the model to use new tokens, while values < 1 encourage the model to repeat tokens.","accessed":"2026-06-19"}],"evidence_status":"limited"}],"key_concepts":[{"name":"PagedAttention","summary":"vLLM's memory-management technique that stores the attention key/value cache in non-contiguous pages, so concurrent requests share GPU memory and common prefixes are reused instead of duplicated.","maps_to_pattern":"prompt-caching","url":"https://docs.vllm.ai"},{"name":"Continuous batching","summary":"The scheduler that admits and retires requests at the token step rather than at the batch boundary, keeping the accelerator busy and raising request throughput under mixed load.","url":"https://docs.vllm.ai"},{"name":"OpenAI-compatible API server","summary":"A drop-in server exposing /v1/chat/completions and related endpoints, so clients written against the OpenAI SDK point at a self-hosted vLLM endpoint without code changes.","url":"https://docs.vllm.ai"},{"name":"LLM / engine entrypoint","summary":"vLLM's in-process generation interface (the LLM class and the underlying engine) for offline batched inference, distinct from the long-running API server.","url":"https://docs.vllm.ai"}],"references":[{"type":"doc","title":"vLLM documentation","url":"https://docs.vllm.ai","accessed":"2026-06-17"},{"type":"repo","title":"vllm-project/vllm — GitHub repository","url":"https://github.com/vllm-project/vllm","quote":"vLLM is a fast and easy-to-use library for LLM inference and serving.","accessed":"2026-06-17"},{"type":"paper","title":"Efficient Memory Management for Large Language Model Serving with PagedAttention","url":"https://arxiv.org/abs/2309.06180","accessed":"2026-06-17"}],"alternatives":[{"composition":"openrouter","relation":"competes-with","note":"OpenRouter routes requests to hosted model providers behind one API, whereas vLLM is the engine an operator runs on its own hardware to be one of those endpoints; they sit on opposite sides of the serve/route boundary."}],"verification_status":"needs-verification","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-19","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"vLLM\"]\n  fw --> p1[\"Sovereign Inference Stack<br/>(core)\"]\n  fw --> p2[\"Prompt Caching<br/>(supported)\"]","tags":["agent-sdk","safety-control","tool-use-environment","full-code"],"anti_patterns_avoided":[{"pattern":"missing-max-tokens-cap","note":"SamplingParams.max_tokens caps generation length per output sequence, preventing unbounded output.","evidence":[{"type":"doc","title":"vLLM — sampling params","url":"https://docs.vllm.ai/en/v0.5.0/dev/sampling_params.html","quote":"max_tokens -- Maximum number of tokens to generate per output sequence.","accessed":"2026-06-19"}]}]},{"id":"x402","name":"x402","kind":"framework","category":"agent-sdk","vendor":"Coinbase","language":"TypeScript","license":"Apache-2.0","status":"active","first_released":"2025-05-06","url":"https://docs.cdp.coinbase.com/x402/welcome","repo":"https://github.com/coinbase/x402","docs_url":"https://docs.cdp.coinbase.com/x402/welcome","intent":"x402 is an open payment protocol that revives the HTTP 402 status code so an agent can pay per request over HTTP without accounts or manual payment flows.","description":"x402 is an open payment protocol from Coinbase that uses the HTTP 402 Payment Required status code to settle stablecoin payments directly over HTTP. When a resource requires payment, the server returns 402 with payment instructions, and the buyer retries the request carrying a signed payment. It targets both human developers and AI agents that need to access paid services without accounts or manual payment flows. The protocol is an open standard not tied to a single provider, and supports stablecoin payments such as USDC across several chains.","primary_use_cases":["agent payments per HTTP request","monetizing APIs and digital content with pay-as-you-go billing","autonomous service access without accounts or manual checkout","stablecoin micropayments over HTTP"],"agent_loop_shape":"An agent (or any HTTP client) requests a paid resource and receives a 402 Payment Required response carrying payment instructions in a header. The agent constructs and sends a signed payment payload, then retries the request; once the payment is verified, the server returns the resource. This lets the agent settle a payment mid-request and continue, with no per-transaction human checkout.","build_surface":"full-code","members":[{"pattern":"session-scoped-payment-authorization","role":"supported","note":"x402 lets an agent pay programmatically per HTTP request over the 402 Payment Required flow, so an agent can stream many autonomous micropayments for service access without per-transaction human approval.","evidence":[{"type":"doc","title":"x402 Overview - Coinbase Developer Documentation","url":"https://docs.cdp.coinbase.com/x402/welcome","quote":"Buyers: Human developers and AI agents seeking to access paid services without accounts or manual payment flows.","accessed":"2026-06-17"},{"type":"doc","title":"x402 Overview - Coinbase Developer Documentation","url":"https://docs.cdp.coinbase.com/x402/welcome","quote":"Let AI agents pay and access services autonomously with no keys or human input needed.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"agent-initiated-payment","role":"core","note":"When a resource needs payment, the server returns 402 with payment instructions and the buyer responds by sending a signed payment payload and retrying, so the agent settles a payment mid-request to unlock the resource by answering a payment-required challenge with a verifiable proof.","evidence":[{"type":"doc","title":"x402 Overview - Coinbase Developer Documentation","url":"https://docs.cdp.coinbase.com/x402/welcome","quote":"If payment is required, the server responds with a 402 Payment Required, including payment instructions in the `PAYMENT-REQUIRED` header","accessed":"2026-06-17"},{"type":"doc","title":"x402 Overview - Coinbase Developer Documentation","url":"https://docs.cdp.coinbase.com/x402/welcome","quote":"The buyer constructs and sends a payment payload via the `PAYMENT-SIGNATURE` header","accessed":"2026-06-17"}],"evidence_status":"full"}],"key_concepts":[{"name":"HTTP 402 Payment Required","summary":"The long-dormant HTTP status code x402 revives: a server answers a request for a paid resource with 402 plus machine-readable payment instructions, turning payment into a step of the normal request/retry cycle.","maps_to_pattern":"agent-initiated-payment","url":"https://docs.cdp.coinbase.com/x402/welcome"},{"name":"Facilitator","summary":"An x402 service that verifies and settles a buyer's payment on chain so the seller does not have to run its own blockchain infrastructure to accept payments.","url":"https://docs.cdp.coinbase.com/x402/welcome"},{"name":"Payment payload","summary":"The signed proof the buyer attaches when retrying the request (carried in a payment header), built from token authorizations such as EIP-3009 or Permit2, that the server or facilitator verifies before releasing the resource.","maps_to_pattern":"agent-initiated-payment","url":"https://docs.cdp.coinbase.com/x402/welcome"}],"references":[{"type":"doc","title":"x402 Overview - Coinbase Developer Documentation","url":"https://docs.cdp.coinbase.com/x402/welcome","quote":"By reviving the HTTP 402 Payment Required status code, x402 lets services monetize APIs and digital content onchain.","accessed":"2026-06-17"},{"type":"repo","title":"coinbase/x402 — GitHub repository","url":"https://github.com/coinbase/x402","accessed":"2026-06-17"}],"alternatives":[{"composition":"claude-agent-sdk","relation":"complements","note":"x402 is the payment rail an agent runtime calls when a tool or resource returns 402; an SDK such as the Claude Agent SDK hosts the agent loop that drives the request/retry, while x402 settles the payment."}],"verification_status":"partial","status_in_practice":"emerging","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"x402\"]\n  fw --> p1[\"Agent-Initiated Payment<br/>(core)\"]\n  fw --> p2[\"Session-Scoped Payment Authorization<br/>(supported)\"]","tags":["agent-sdk","safety-control","tool-use-environment","full-code"]},{"id":"ax","name":"Ax (AxACE optimizer)","kind":"framework","category":"agent-sdk","vendor":"ax-llm","language":"TypeScript","license":"Apache-2.0","status":"active","first_released":"2023-12-01","url":"https://axllm.dev/","repo":"https://github.com/ax-llm/ax","docs_url":"https://axllm.dev/","intent":"Ax is a DSPy port that compiles typed declarative signatures into LLM calls and optimizes the resulting prompts and demonstrations against an evaluation metric.","description":"Ax is a framework for building LLM applications around declarative signatures rather than hand-written prompts. A signature declares the typed inputs and outputs of a step, and Ax turns it into the prompt, the output parser, the validators, and the retry loop, returning typed data. Ax includes optimizers, among them AxACE, which mirrors the Agentic Context Engineering paper's Generator-Reflector-Curator loop and a Genetic-Pareto optimizer that tunes prompts and demonstrations against evals. It is released under Apache 2.0.","primary_use_cases":["building LLM pipelines from typed signatures","automatic prompt and few-shot optimization","agentic context engineering with an evolving playbook","type-safe structured outputs from LLMs"],"agent_loop_shape":"A developer declares a signature stating what goes in and what comes out. Ax compiles the signature into a prompt, schema, parser, validators, and retry loop, runs it against the model, and returns typed output. Given example input/output pairs and a metric, an optimizer such as GEPA or AxACE searches for better prompts and demonstrations; AxACE additionally maintains a structured bullet playbook updated through add/update/remove deltas at inference time.","build_surface":"full-code","members":[{"pattern":"agentic-context-engineering-playbook","role":"first-class","note":"AxACE mirrors the ACE Generator-Reflector-Curator loop, holds context as a structured bullet playbook, and applies add/update/remove deltas (offline compile and online inference-time updates) to resist context collapse.","evidence":[{"type":"doc","title":"Agentic Context Engineering (ACE) - Ax","url":"https://axllm.dev/ace/","quote":"AxACE is an optimizer that mirrors the ACE paper's Generator → Reflector → Curator loop. It represents context as structured bullets, applies incremental deltas, and returns a serialized playbook you can save, load, and keep updating at inference time.","accessed":"2026-06-17"},{"type":"repo","title":"ax-llm/ax index.ts (AxACE / AxACEOptimizedProgram exports)","url":"https://github.com/ax-llm/ax","quote":"export { AxACE }; export { AxACEOptimizedProgram };","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"dspy-signatures","role":"core","note":"Ax is a port of DSPy in which behaviour is specified as a declarative typed signature rather than a prompt; Ax compiles the signature into the prompt, parser, validators, and retry loop and returns typed data, and an optimizer tunes the prompts and demonstrations from example pairs against an eval.","evidence":[{"type":"doc","title":"Ax — DSPy in TypeScript","url":"https://axllm.dev/","quote":"A one-line signature declares what goes in and what comes out. Ax turns it into the prompt, the parser, the validators, and the retry loop — and hands back typed data your code can trust.","accessed":"2026-06-17"},{"type":"doc","title":"Ax — Optimization","url":"https://axllm.dev/","quote":"GEPA, the Genetic-Pareto optimizer, tunes prompts, demos, flows, and agents against evals.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"structured-output","role":"core","note":"A signature declares typed outputs and Ax validates the generated fields against them, accepting a string DSL, a fluent builder, or a Standard Schema validator such as Zod, so downstream code receives typed structured data rather than free text.","evidence":[{"type":"repo","title":"ax-llm/ax README","url":"https://github.com/ax-llm/ax","quote":"Signatures for typed structured generation: string DSL, fluent `f()` builder, or any **Standard Schema v1** validator — Zod, Valibot, ArkType.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"tool-use","role":"supported","note":"Ax exposes typed host functions and MCP servers as callable capabilities and runs a ReAct-style tool loop, letting a signature-driven program invoke functions with typed arguments and typed returns during generation.","evidence":[{"type":"doc","title":"Ax — Tools and capabilities","url":"https://axllm.dev/","quote":"Expose typed host functions, MCP servers, runtimes, flows, and child agents as callable capabilities.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"agent-as-tool-embedding","role":"supported","note":"AxAgent turns a signature into a long-running tool-using actor whose child agents are exposed as typed capabilities behind the same signature contract, so a parent agent calls a sub-agent the same way it calls any other function.","evidence":[{"type":"doc","title":"Ax — Agents","url":"https://axllm.dev/","quote":"Built-in memory, skills, MCP tools, and child agents become typed capabilities behind the same signature contract.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"multi-model-routing","role":"supported","note":"A single ai() call selects OpenAI, Claude, Gemini, a gateway, or a local OpenAI-compatible endpoint while signatures, tools, traces, and outputs stay unchanged, so the same program runs against different providers.","evidence":[{"type":"doc","title":"Ax — Providers","url":"https://axllm.dev/","quote":"Pick OpenAI, Claude, Gemini, a gateway, or a local OpenAI-compatible endpoint in `ai()`. Your signatures, tools, traces, and outputs stay the same.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"streaming-typed-events","role":"supported","note":"Streaming is the default: Ax parses signature fields as they arrive, runs streaming assertions, and can fail and cancel an in-flight stream before the model finishes, returning typed structured results incrementally.","evidence":[{"type":"repo","title":"ax-llm/ax README","url":"https://github.com/ax-llm/ax","quote":"Streaming is the default because it lets Ax do useful work before the model finishes: parse fields as they arrive, run streaming assertions, fail early, cancel the in-flight stream, and start correction without spending tokens on an output that is already known to be invalid.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"Signature","summary":"A one-line declaration of a step's typed inputs and outputs that Ax compiles into the prompt, output parser, validators, and retry loop, so the developer states what goes in and out rather than writing a prompt.","maps_to_pattern":"dspy-signatures","url":"https://axllm.dev/"},{"name":"GEPA optimizer","summary":"The Genetic-Pareto optimizer that searches over prompts, demonstrations, flows, and agents against an evaluation metric to improve a signature-driven program without manual prompt edits.","maps_to_pattern":"evaluator-optimizer","url":"https://axllm.dev/"},{"name":"AxACE","summary":"An optimizer that mirrors the Agentic Context Engineering Generator-Reflector-Curator loop, representing context as structured bullets and applying incremental add/update/remove deltas into a serialized playbook updatable at inference time.","maps_to_pattern":"agentic-context-engineering-playbook","url":"https://axllm.dev/ace/"},{"name":"AxAgent","summary":"A three-stage pipeline that turns a signature into a long-running, tool-using actor whose memory, skills, MCP tools, and child agents are exposed as typed capabilities behind the signature contract.","maps_to_pattern":"agent-as-tool-embedding","url":"https://axllm.dev/"},{"name":"ai() provider handle","summary":"The single constructor that binds a program to a model provider — OpenAI, Claude, Gemini, a gateway, or a local OpenAI-compatible endpoint — leaving signatures, tools, traces, and outputs unchanged when the provider is swapped.","maps_to_pattern":"multi-model-routing","url":"https://axllm.dev/"}],"references":[{"type":"doc","title":"Ax — DSPy in TypeScript (documentation home)","url":"https://axllm.dev/","quote":"Ax compiles typed signatures into reliable LLM calls — validation, streaming, tools, agents, audio, and optimization.","accessed":"2026-06-17"},{"type":"repo","title":"ax-llm/ax (source repository)","url":"https://github.com/ax-llm/ax","accessed":"2026-06-17"},{"type":"doc","title":"Agentic Context Engineering (ACE) — Ax","url":"https://axllm.dev/ace/","accessed":"2026-06-17"}],"alternatives":[{"composition":"dspy","relation":"predecessor","note":"Ax is a TypeScript port of DSPy; both declare behaviour as typed signatures and optimize prompts and demonstrations against an eval rather than hand-writing prompts."},{"composition":"instructor","relation":"similar-shape","note":"Both return typed structured data from LLM calls with validation and retry, but Instructor centres on schema-bound extraction while Ax adds signature compilation and prompt optimization."},{"composition":"pydantic-ai","relation":"competes-with","note":"Both are typed agent frameworks with structured outputs, tools, and multi-provider model selection; Pydantic AI builds on Pydantic models whereas Ax builds on DSPy signatures and optimizers."}],"verification_status":"partial","status_in_practice":"emerging","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Ax (AxACE optimizer)\"]\n  fw --> p1[\"DSPy Signatures<br/>(core)\"]\n  fw --> p2[\"Structured Output<br/>(core)\"]\n  fw --> p3[\"Agentic Context Engineering Playbook<br/>(first-class)\"]\n  fw --> p4[\"Tool Use<br/>(supported)\"]\n  fw --> p5[\"Agent-as-Tool Embedding<br/>(supported)\"]\n  fw --> p6[\"Multi-Model Routing<br/>(supported)\"]\n  fw --> p7[\"Streaming Typed Events<br/>(supported)\"]","tags":["agent-sdk","verification-reflection","structure-data","tool-use-environment","multi-agent","routing-composition","streaming-ux","full-code"]},{"id":"jason-bdi","name":"Jason (AgentSpeak interpreter)","kind":"framework","category":"agent-sdk","vendor":"Jason / AgentSpeak community (R. Bordini, J. Hubner et al.)","language":"Java","license":"LGPL-3.0","status":"active","first_released":"2007","url":"https://jason-lang.github.io/","repo":"https://github.com/jason-lang/jason","docs_url":"https://jason-lang.github.io/doc/","intent":"Jason is an interpreter for an extended AgentSpeak that runs BDI agents with explicit beliefs, goals, and plans, and supports building distributed multi-agent systems with speech-act communication.","description":"Jason implements the operational semantics of AgentSpeak(L), the canonical BDI agent language. An agent holds a belief base, adopts goals, and commits to plans (intentions), reasoning by reconciling the three over a reasoning cycle. Jason multi-agent systems can be distributed over a network and exchange messages through speech-act based inter-agent communication. It is a long-standing, maintained implementation of the BDI architecture rather than an LLM-era reinterpretation.","primary_use_cases":["building BDI cognitive agents with explicit beliefs, goals, and plans","distributed multi-agent systems","speech-act based agent communication","research and teaching of agent-oriented programming"],"agent_loop_shape":"Each Jason agent runs a reasoning cycle: it perceives the environment to update its belief base, processes received messages, selects an applicable plan for a triggering event (a new belief or goal), and executes one step of a committed intention. Plans are reactive — triggered by events and goals — and multiple agents run concurrently, exchanging speech-act messages across a possibly distributed system.","build_surface":"full-code","members":[{"pattern":"bdi-agent","role":"core","note":"Jason is a maintained interpreter for AgentSpeak, the canonical BDI agent language: agents hold an explicit belief base, adopt desires/goals, and commit to intentions (plans), reasoning by reconciling the three in the operational semantics of the language. This is the direct, named implementation of the BDI architecture rather than an LLM-era reinterpretation.","evidence":[{"type":"doc","title":"Jason, a BDI agent programming language","url":"https://jason-lang.github.io/","quote":"Jason is an interpreter for an extended version of AgentSpeak. It implements the operational semantics of that language","accessed":"2026-06-17"},{"type":"doc","title":"BDI architecture (Jason / AgentSpeak overview)","url":"https://jason-lang.github.io/","quote":"The BDI architecture models behavior based on the agent's Beliefs about the world, Desires representing its goals and Intentions, the plans it commits to in order to achieve those goals.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"inter-agent-communication","role":"first-class","note":"Jason provides speech-act based messaging between agents, with belief annotations recording the information source, and a multi-agent system can run distributed across a network so agents exchange tasks and results across process boundaries.","evidence":[{"type":"doc","title":"Jason, a BDI agent programming language","url":"https://jason-lang.github.io/","quote":"speech-act based inter-agent communication (and belief annotations on information sources);","accessed":"2026-06-17"},{"type":"doc","title":"Jason, a BDI agent programming language","url":"https://jason-lang.github.io/","quote":"A Jason multi-agent system can be distributed over a network effortlessly.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"performative-message","role":"first-class","note":"Inter-agent messages in Jason carry an explicit illocutionary force (a KQML-like performative such as tell, achieve, or askone) that determines how the receiver treats the content, so a sender's intent — assert a belief, request a goal, query — is encoded in the message type rather than inferred from prose.","evidence":[{"type":"doc","title":"Jason, a BDI agent programming language","url":"https://jason-lang.github.io/","quote":"speech-act based inter-agent communication (and belief annotations on information sources);","accessed":"2026-06-17"},{"type":"paper","title":"On the Formal Semantics of Speech-Act Based Communication in an Agent-Oriented Programming Language","url":"https://arxiv.org/abs/1111.0041","quote":"a computationally grounded semantics for (the core) performatives used in speech-act based agent communication languages","accessed":"2026-06-17"}],"evidence_status":"full"}],"key_concepts":[{"name":"Belief base","summary":"The set of facts (and their annotations recording the information source) that an agent holds true about its world and itself, revised by perception and by received messages on each reasoning cycle.","maps_to_pattern":"bdi-agent","url":"https://jason-lang.github.io/"},{"name":"Plan","summary":"A reactive rule of the form triggering-event : context <- body that an agent selects and runs when a matching event (a new belief or goal) occurs, becoming a committed intention while it executes.","maps_to_pattern":"bdi-agent","url":"https://jason-lang.github.io/doc/"},{"name":"Internal action","summary":"A user-extensible operation, written in Java, that a plan body can invoke for behaviour outside AgentSpeak itself, giving the interpreter a customisation surface for perception, belief revision, and acting.","url":"https://jason-lang.github.io/"},{"name":".send / performative","summary":"The communication action that sends a message to another agent tagged with an illocutionary force (tell, achieve, askone, and other KQML-like performatives) that fixes how the receiver acts on the content.","maps_to_pattern":"performative-message","url":"https://jason-lang.github.io/"}],"references":[{"type":"doc","title":"Jason, a BDI agent programming language (project home)","url":"https://jason-lang.github.io/","quote":"Jason is an interpreter for an extended version of AgentSpeak. It implements the operational semantics of that language","accessed":"2026-06-17"},{"type":"repo","title":"jason-lang/jason (source repository)","url":"https://github.com/jason-lang/jason","accessed":"2026-06-17"},{"type":"doc","title":"Jason documentation","url":"https://jason-lang.github.io/doc/","accessed":"2026-06-17"}],"alternatives":[{"composition":"camel-ai","relation":"competes-with","note":"Both are agent-construction frameworks, but Jason runs logic-programmed BDI agents with explicit beliefs and plans whereas CAMEL-AI builds LLM agents that coordinate through role-playing conversation."},{"composition":"agentverse","relation":"similar-shape","note":"Both build multi-agent systems with explicit communication between members; AgentVerse coordinates LLM agents in a simulation environment whereas Jason runs distributable speech-act-communicating BDI agents."}],"verification_status":"partial","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Jason (AgentSpeak interpreter)\"]\n  fw --> p1[\"BDI Agent<br/>(core)\"]\n  fw --> p2[\"Inter-Agent Communication<br/>(first-class)\"]\n  fw --> p3[\"Performative Message<br/>(first-class)\"]","tags":["agent-sdk","cognition-introspection","multi-agent","full-code"]},{"id":"anthropic-computer-use","name":"Anthropic Computer Use","kind":"framework","category":"browser-computer-use","build_surface":"full-code","vendor":"Anthropic","language":"API","license":"proprietary","status":"active","first_released":"2024-10-22","url":"https://docs.claude.com/en/docs/agents-and-tools/computer-use","docs_url":"https://docs.claude.com/en/docs/agents-and-tools/computer-use","intent":"Anthropic API beta tool that lets Claude see a desktop via screenshots and drive mouse/keyboard, running inside a developer-supplied sandbox driven by an agent loop the application implements.","description":"Computer Use is a beta API capability where Claude returns abstract tool calls (screenshot, click, type, key, scroll) for a virtual display and the developer's program executes them in a sandboxed environment (the reference implementation runs Xvfb + Mutter + Tint2 inside Docker). The model is paired with text_editor and bash tools for end-to-end automation. Anthropic ships a reference quickstart that bundles the container, the tool implementations, the agent loop, and a web UI. The feature is gated behind a beta header and Anthropic recommends running with classifiers, allowlisted domains, and human confirmation for consequential actions.","primary_use_cases":["desktop GUI automation via Claude controlling mouse, keyboard and screenshots","browser navigation and form filling inside a sandboxed VM","agentic workflows combining the computer, text_editor and bash tools","research and prototyping of GUI agents on benchmarks like WebArena"],"agent_loop_shape":"Stateful multi-turn agent loop driven by the developer. Each turn the model emits one or more tool_use blocks (computer / text_editor / bash); the application executes them in a sandboxed environment, captures results (a screenshot, command output, file diff), and returns them as tool_result blocks. The loop continues until Claude responds without further tool_use blocks or a max-iteration safeguard fires.","key_concepts":[{"name":"computer tool","summary":"Screenshot capture plus mouse and keyboard control over a virtual display.","maps_to_pattern":"computer-use","url":"https://docs.claude.com/en/docs/agents-and-tools/computer-use"},{"name":"text_editor and bash tools","summary":"Companion tools for file editing and shell commands inside the same sandbox.","maps_to_pattern":"code-execution","url":"https://docs.claude.com/en/docs/agents-and-tools/computer-use"},{"name":"Agent loop","summary":"Developer-owned cycle that ships tool_use results back to Claude until the task ends.","maps_to_pattern":"react","url":"https://github.com/anthropics/anthropic-quickstarts/tree/main/computer-use-demo"},{"name":"Sandboxed computing environment","summary":"Xvfb virtual display, Mutter window manager and Tint2 panel inside a Docker container.","maps_to_pattern":"sandbox-isolation","url":"https://docs.claude.com/en/docs/agents-and-tools/computer-use"},{"name":"Prompt-injection classifier","summary":"Server-side classifier flags screenshot prompt injections and steers Claude to ask for confirmation.","maps_to_pattern":"approval-queue","url":"https://docs.claude.com/en/docs/agents-and-tools/computer-use"}],"pattern_composition":"flowchart TD\n  user[User task] --> claude[Claude with computer + text_editor + bash tools]\n  claude --> req[tool_use request]\n  req --> classify[Prompt-injection classifier]\n  classify -->|flag| confirm[Ask user to confirm]\n  classify -->|clear| exec[Application executes in sandbox]\n  confirm --> exec\n  exec --> shot[Screenshot or command output]\n  shot --> result[tool_result back to Claude]\n  result --> claude\n  claude --> done[Final assistant text]","members":[{"pattern":"computer-use","role":"first-class","note":"Headline capability: screenshot, mouse, keyboard via the computer tool.","evidence":[{"type":"doc","title":"Anthropic docs — computer use overview","url":"https://docs.claude.com/en/docs/agents-and-tools/computer-use","quote":"Computer use is a beta feature that enables Claude to interact with desktop environments. This tool provides: Screenshot capture: See what's currently displayed on screen; Mouse control: Click, drag, and move the cursor; Keyboard input: Type text and use keyboard shortcuts","accessed":"2026-05-20"},{"type":"blog","title":"Anthropic news — computer use launch","url":"https://www.anthropic.com/news/3-5-models-and-computer-use","quote":"Available today on the API, developers can direct Claude to use computers the way people do","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"Documented agent loop is the canonical model-acts-then-observes-result shape.","evidence":[{"type":"doc","title":"Anthropic docs — agent loop","url":"https://docs.claude.com/en/docs/agents-and-tools/computer-use","quote":"The repetition of steps 3 and 4 without user input is referred to as the \"agent loop\" (that is, Claude responding with a tool use request and your application responding to Claude with the results of evaluating that request).","accessed":"2026-05-20"},{"type":"doc","title":"Anthropic docs — agentic loop","url":"https://docs.claude.com/en/docs/agents-and-tools/computer-use","quote":"The core of computer use is the \"agent loop\": a cycle where Claude requests tool actions, your application runs them, and returns results to Claude.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Three first-class tools: computer_*, text_editor_*, bash_*.","evidence":[{"type":"doc","title":"Anthropic docs — computer + text_editor + bash","url":"https://docs.claude.com/en/docs/agents-and-tools/computer-use","quote":"While computer use can be augmented with other tools such as bash and text editor for more comprehensive automation workflows, computer use specifically refers to the computer use tool's capability to see and control desktop environments.","accessed":"2026-05-24"},{"type":"doc","title":"Anthropic docs — adding tools to the same array","url":"https://docs.claude.com/en/docs/agents-and-tools/computer-use","quote":"To add other tools alongside computer use, include them in the same `tools` array. The Quick start section shows this pattern with the bash tool and text editor tool. You can add your own custom tool definitions the same way.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"code-execution","role":"first-class","note":"bash and text_editor tools execute commands and edit files in the sandbox.","evidence":[{"type":"doc","title":"Anthropic docs — bash tool persistent session","url":"https://docs.claude.com/en/docs/agents-and-tools/tool-use/bash-tool","quote":"The bash tool enables Claude to execute shell commands in a persistent bash session, allowing system operations, script execution, and command-line automation.","accessed":"2026-05-24"},{"type":"doc","title":"Anthropic docs — text editor tool","url":"https://docs.claude.com/en/docs/agents-and-tools/tool-use/text-editor-tool","quote":"Claude can use an Anthropic-schema text editor tool to view and modify text files, helping you debug, fix, and improve your code or other text documents.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"sandbox-isolation","role":"first-class","note":"Reference implementation runs Xvfb + Mutter + Tint2 inside Docker; Anthropic recommends a dedicated VM/container.","evidence":[{"type":"doc","title":"Anthropic docs — computing environment","url":"https://docs.claude.com/en/docs/agents-and-tools/computer-use","quote":"Computer use requires a sandboxed computing environment where Claude can safely interact with applications and the web.","accessed":"2026-05-20"},{"type":"doc","title":"Anthropic docs — dedicated VM","url":"https://docs.claude.com/en/docs/agents-and-tools/computer-use","quote":"Using a dedicated virtual machine or container with minimal privileges to prevent direct system attacks or accidents.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"approval-queue","role":"first-class","note":"Prompt-injection classifier steers the model to ask for confirmation before risky next actions; Anthropic recommends human confirmation for consequential actions.","evidence":[{"type":"doc","title":"Anthropic docs — classifier confirmation","url":"https://docs.claude.com/en/docs/agents-and-tools/computer-use","quote":"When these classifiers identify potential prompt injections in screenshots, they will automatically steer the model to ask for user confirmation before proceeding with the next action.","accessed":"2026-05-20"},{"type":"doc","title":"Anthropic docs — human confirmation","url":"https://docs.claude.com/en/docs/agents-and-tools/computer-use","quote":"Asking a human to confirm decisions that might result in meaningful real-world consequences and any tasks requiring affirmative consent, such as accepting cookies, completing financial transactions, or agreeing to terms of service.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"human-in-the-loop","role":"supported","note":"Anthropic encourages low-risk tasks and explicit user consent; not a built-in workflow primitive.","evidence":[{"type":"blog","title":"Anthropic news — low-risk exploration","url":"https://www.anthropic.com/news/3-5-models-and-computer-use","quote":"we encourage developers to begin exploration with low-risk tasks.","accessed":"2026-05-24"},{"type":"doc","title":"Anthropic docs — require human oversight","url":"https://docs.claude.com/en/docs/agents-and-tools/computer-use","quote":"Always carefully review and verify Claude's computer use actions and logs. Do not use Claude for tasks requiring perfect precision or sensitive user information without human oversight.","accessed":"2026-05-24"},{"type":"doc","title":"Anthropic docs — inform users and obtain consent","url":"https://docs.claude.com/en/docs/agents-and-tools/computer-use","quote":"Inform end users of relevant risks and obtain their consent prior to enabling computer use in your own products.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"dual-system-gui-agent","role":"limited","note":"The computer tool alone is single-system (vision + action by one model). Anthropic's own docs explicitly frame computer use as a single-agent capability, and separation of planner/executor is left to the application; no first-party documented dual-system primitive.","evidence":[{"type":"doc","title":"Anthropic docs — single-agent framing on WebArena","url":"https://platform.claude.com/docs/en/agents-and-tools/computer-use","quote":"Claude achieves state-of-the-art results among single-agent systems, demonstrating strong ability to complete multi-step browser tasks end to end.","accessed":"2026-05-24"}],"evidence_status":"limited"}],"alternatives":[{"composition":"openai-operator","relation":"competes-with","note":"First-party browser-only GUI agent from OpenAI, discontinued Aug 2025 and replaced by ChatGPT agent."},{"composition":"browser-use","relation":"competes-with","note":"Open-source browser agent that can use Computer Use as one of its model backends."},{"composition":"stagehand","relation":"competes-with","note":"Browserbase SDK that exposes computer-use models as one of several agent backends."},{"composition":"bytebot","relation":"similar-shape","note":"Peer computer-use approach driving a full desktop/GUI."},{"composition":"mobile-agent","relation":"competes-with","note":"Anthropic's computer-use is desktop-first and Western; Mobile-Agent is mobile-first and Chinese, with a unified cross-platform abstraction."},{"composition":"manus","relation":"competes-with","note":"Anthropic's computer-use loop targets the same hosted-VM-plus-browser shape; Manus 1.5 explicitly integrates the Agent Skills standard."}],"references":[{"type":"doc","title":"Anthropic docs — Computer use tool","url":"https://docs.claude.com/en/docs/agents-and-tools/computer-use","accessed":"2026-05-20"},{"type":"blog","title":"Anthropic — Introducing computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku","url":"https://www.anthropic.com/news/3-5-models-and-computer-use","quote":"looking at a screen, moving a cursor, clicking buttons, and typing text","accessed":"2026-05-20"},{"type":"repo","title":"anthropic-quickstarts/computer-use-demo","url":"https://github.com/anthropics/anthropic-quickstarts/tree/main/computer-use-demo","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"emerging","tags":["proprietary","api","computer-use","beta","sandboxed"]},{"id":"browser-use","name":"Browser Use","kind":"framework","category":"browser-computer-use","build_surface":"full-code","vendor":"Browser Use (Magnus Mueller, Gregor Zunic)","language":"Python","license":"MIT","status":"active","first_released":"2024-10-31","url":"https://browser-use.com","repo":"https://github.com/browser-use/browser-use","docs_url":"https://docs.browser-use.com","intent":"Open-source Python library that wraps a Playwright-controlled browser into an agent loop driven by any of 15+ LLM providers, with a paid stealth-browser cloud as the production tier.","description":"Browser Use exposes three primary objects — Agent, Browser and a chat client — that together let a language model navigate, click, type, scroll and extract data from real web pages. The agent extracts an annotated DOM, optionally adds screenshots for vision, and emits structured actions executed by the underlying Playwright browser. The library is MIT-licensed and integrates with OpenAI, Anthropic, Google Gemini, Azure, Bedrock, Groq, Ollama, DeepSeek, OpenRouter and others. A paid managed cloud at api.browser-use.com layers stealth Chromium, residential proxies in 195+ countries and CAPTCHA solving on top.","primary_use_cases":["natural-language web automation and form filling","agent-driven data extraction from authenticated sites","browser testing and monitoring via LLM-written tasks","production scraping behind a stealth cloud with residential proxies"],"agent_loop_shape":"Per-step Playwright-backed loop. Each turn the Agent snapshots the page (annotated DOM plus optional screenshot), the LLM emits up to max_actions_per_step structured actions, the Controller executes them through Playwright, the result feeds back into the next observation. The loop runs under agent.run() until the task completes or max_steps is hit.","key_concepts":[{"name":"Agent","summary":"High-level entry point that owns the task, LLM and step loop.","maps_to_pattern":"browser-agent","url":"https://docs.browser-use.com/quickstart"},{"name":"Browser","summary":"Playwright-controlled browser instance the Agent drives.","url":"https://docs.browser-use.com/quickstart"},{"name":"Annotated DOM + optional vision","summary":"DOM elements are extracted and indexed; screenshots are included when the vision mode requests them.","url":"https://docs.browser-use.com/customize/agent-settings"},{"name":"max_actions_per_step","summary":"Per-turn action budget; defaults to 4 so the model can submit a whole form in one step.","url":"https://docs.browser-use.com/customize/agent-settings"},{"name":"Multi-provider LLM support","summary":"Native integrations for 15+ providers plus LiteLLM and OpenAI-compatible base_url.","maps_to_pattern":"multi-model-routing","url":"https://docs.browser-use.com/customize/supported-models"},{"name":"Browser Use Cloud","summary":"Paid stealth-browser API with residential proxies, CAPTCHA solving and managed infrastructure.","url":"https://browser-use.com"}],"pattern_composition":"flowchart TD\n  task[User task] --> agent[Agent]\n  agent --> snap[Annotated DOM + optional screenshot]\n  snap --> llm[LLM emits up to N actions]\n  llm --> ctrl[Controller / Playwright]\n  ctrl --> browser[Real Chromium]\n  browser --> obs[New page state]\n  obs --> snap\n  agent --> done{max_steps or done?}\n  done -->|yes| out[Final answer]\n  done -->|no| snap\n  cloud[Browser Use Cloud] -.stealth + proxies.-> browser","members":[{"pattern":"browser-agent","role":"first-class","note":"Core abstraction — Agent class drives a real browser to complete natural-language tasks.","evidence":[{"type":"repo","title":"browser-use README","url":"https://github.com/browser-use/browser-use","quote":"Make websites accessible for AI agents. Automate tasks online with ease.","accessed":"2026-05-20"},{"type":"repo","title":"browser-use README tagline","url":"https://github.com/browser-use/browser-use","quote":"Tell your computer what to do, and it gets it done.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Structured browser actions (click, type, scroll, navigate, extract) are the LLM's tool surface; MCP registry adds external tools.","evidence":[{"type":"doc","title":"browser-use docs — max_actions_per_step","url":"https://docs.browser-use.com/customize/agent-settings","quote":"max_actions_per_step (default: 4): Maximum actions per step, e.g. for form filling the agent can output 4 fields at once","accessed":"2026-05-20"},{"type":"repo","title":"browser-use README — MCP","url":"https://github.com/browser-use/browser-use","quote":"MCP Registry New Integrate external tools","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"multi-model-routing","role":"first-class","note":"15+ provider integrations plus LiteLLM and any OpenAI-compatible endpoint.","evidence":[{"type":"doc","title":"browser-use docs — supported models","url":"https://docs.browser-use.com/customize/supported-models","quote":"Browser Use natively supports 15+ LLM providers","accessed":"2026-05-20"},{"type":"doc","title":"browser-use docs — OpenAI-compatible","url":"https://docs.browser-use.com/customize/supported-models","quote":"Any provider with an OpenAI-compatible endpoint works via ChatOpenAI with a custom base_url","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"Browser Use ships a documented MCP server surface that exposes Autonomous Agent Tools, Direct Browser Control (browser_navigate / browser_click / browser_type / browser_get_state), Tab Management, Content Extraction and Session Management to any MCP client. Launched via `uvx --from 'browser-use[cli]' browser-use --mcp`. README also references the MCP Registry for inbound external tools.","evidence":[{"type":"doc","title":"browser-use docs — MCP server overview","url":"https://docs.browser-use.com/open-source/customize/integrations/mcp-server","quote":"Run browser-use as a local Model Context Protocol server. Connect AI models to browser automation through the MCP standard.","accessed":"2026-05-24"},{"type":"doc","title":"browser-use docs — MCP server CLI entry point","url":"https://docs.browser-use.com/open-source/customize/integrations/mcp-server","quote":"uvx --from 'browser-use[cli]' browser-use --mcp","accessed":"2026-05-24"},{"type":"repo","title":"browser-use README — MCP Registry","url":"https://github.com/browser-use/browser-use","quote":"MCP Registry New Integrate external tools","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"Per-step observe (DOM + optional screenshot) → act (structured actions) → observe again loop.","evidence":[{"type":"doc","title":"browser-use docs — max_steps loop","url":"https://docs.browser-use.com/customize/agent/all-parameters","quote":"max_steps (default: 100): Maximum number of steps an agent can take.","accessed":"2026-05-24"},{"type":"doc","title":"browser-use docs — max_actions_per_step","url":"https://docs.browser-use.com/customize/agent-settings","quote":"max_actions_per_step (default: 4): Maximum actions per step, e.g. for form filling the agent can output 4 fields at once.","accessed":"2026-05-24"},{"type":"doc","title":"browser-use docs — vision mode","url":"https://docs.browser-use.com/customize/agent-settings","quote":"use_vision (default: \"auto\"): Vision mode - \"auto\" includes screenshot tool but only uses vision when requested","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"computer-use","role":"limited","note":"Honest downgrade — browser-use primarily acts via the DOM, not coordinate-based screen control. Vision is opt-in supplement, not the headline modality.","evidence":[{"type":"doc","title":"browser-use docs — vision optional","url":"https://docs.browser-use.com/customize/agent-settings","quote":"use_vision (default: \"auto\"): Vision mode - \"auto\" includes screenshot tool but only uses vision when requested","accessed":"2026-05-24"}],"evidence_status":"limited"},{"pattern":"plan-and-execute","role":"limited","note":"Re-verified 2026-05-24 against docs.browser-use.com customize/agent-settings — no planner, planning mode or plan_steps primitive is documented; max_actions_per_step is a within-step batch, not an explicit plan stage. Honest gap retained.","evidence":[],"evidence_status":"none"},{"pattern":"session-isolation","role":"supported","note":"Browser Use Cloud advertises stealth Chromium sessions per agent run; local library uses standard Playwright contexts.","evidence":[{"type":"doc","title":"browser-use homepage — undetectable browsers","url":"https://browser-use.com","quote":"The Way AI uses the web. Agents at scale. Undetectable browsers. The API for any website.","accessed":"2026-05-24"},{"type":"doc","title":"browser-use homepage — anti-detect + proxies","url":"https://browser-use.com","quote":"Anti-detect, CAPTCHA solving, 195+ country proxies. Zero config.","accessed":"2026-05-24"}],"evidence_status":"full"}],"alternatives":[{"composition":"stagehand","relation":"competes-with","note":"Peer open-source browser agent SDK, TypeScript, built on Playwright."},{"composition":"browserbase","relation":"competes-with","note":"Browserbase is browser-infrastructure-as-a-service; Browser Use Cloud is the comparable managed tier."},{"composition":"anthropic-computer-use","relation":"wraps","note":"Browser Use can use Claude (including computer-use-capable models) as a backend."},{"composition":"skyvern","relation":"competes-with","note":"Peer open-source browser agent."},{"composition":"notte","relation":"competes-with","note":"Peer web-agent framework."},{"composition":"mobile-agent","relation":"complements","note":"Browser-Use is browser-only; Mobile-Agent's web environment is one of three it supports."},{"composition":"multion","relation":"competes-with","note":"Open-source browser agent occupying the niche MultiOn vacated."},{"composition":"openai-operator","relation":"competes-with","note":"Open-source alternative for browser automation."}],"references":[{"type":"repo","title":"browser-use/browser-use","url":"https://github.com/browser-use/browser-use","accessed":"2026-05-20"},{"type":"doc","title":"browser-use docs","url":"https://docs.browser-use.com","accessed":"2026-05-20"},{"type":"doc","title":"browser-use homepage","url":"https://browser-use.com","quote":"The Way AI uses the web. Agents at scale. Undetectable browsers. The API for any website.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","python","playwright","browser-agent","mit"]},{"id":"browserbase","name":"Browserbase","kind":"framework","category":"browser-computer-use","build_surface":"full-code","vendor":"Browserbase, Inc.","language":"API","license":"proprietary","status":"active","url":"https://www.browserbase.com","docs_url":"https://docs.browserbase.com","intent":"Managed browser-infrastructure platform for AI agents — isolated Chromium sessions with stealth fingerprints, residential proxies, CAPTCHA solving, persistent contexts and HLS session-replay recordings.","description":"Browserbase runs real Chromium browsers as a service for agents that need to act on the web. Every session is automatically recorded and replayable as HLS through a Session Inspector or embedded player; contexts persist cookies and auth across sessions; the Verified browser ships real fingerprints that bot-protection partners recognise; residential and datacenter proxies route traffic by geography; and the platform exposes integrations for Anthropic Computer Use and OpenAI Computer-Using Agent so a model can drive a Browserbase session directly. The company also publishes the Stagehand SDK as its agent layer.","primary_use_cases":["browser sessions as infrastructure for agent runs","session replay and debugging via Session Inspector","stealth navigation with residential proxies and CAPTCHA solving","running OpenAI or Anthropic computer-use models against a managed browser"],"agent_loop_shape":"Session-as-a-service model. The developer (or Stagehand, or a computer-use model) opens a Browserbase session; the agent drives it via CDP or higher-level SDKs; every frame is recorded and an HLS replay is produced. Contexts persist auth and cookies across sessions for resumed work. Browserbase itself does not host the agent loop — it hosts the browser the loop drives.","key_concepts":[{"name":"Sessions","summary":"Isolated Chromium browser sessions created and observed via API.","maps_to_pattern":"session-isolation","url":"https://docs.browserbase.com/features/sessions"},{"name":"Session replay","summary":"Every session is recorded by default; streamed as HLS for inspection and embedding.","maps_to_pattern":"replay-time-travel","url":"https://docs.browserbase.com/features/session-replay"},{"name":"Contexts","summary":"Reuse cookies, authentication and application data across sessions.","maps_to_pattern":"agent-resumption","url":"https://docs.browserbase.com"},{"name":"Verified / Stealth","summary":"Purpose-built Chromium with real browser fingerprints recognised by bot-protection partners.","url":"https://docs.browserbase.com/features/stealth-mode"},{"name":"Proxies and CAPTCHA solving","summary":"Residential and datacenter proxies plus automatic CAPTCHA resolution.","url":"https://docs.browserbase.com/features/stealth-mode"},{"name":"Computer Use integrations","summary":"First-party adapters wiring Anthropic Computer Use and OpenAI CUA to a Browserbase session.","maps_to_pattern":"computer-use","url":"https://docs.browserbase.com"}],"pattern_composition":"flowchart TD\n  dev[Developer or Stagehand or CUA model] --> create[POST /sessions]\n  create --> sess[Isolated Chromium session]\n  sess --> stealth[Verified fingerprint + proxies + CAPTCHA solve]\n  stealth --> drive[Agent drives via CDP / SDK]\n  drive --> page[Real web]\n  page --> rec[Auto session recording]\n  rec --> hls[HLS replay manifest]\n  hls --> inspect[Session Inspector or embedded player]\n  ctx[Context] -.cookies + auth.-> sess\n  cua[Anthropic CU / OpenAI CUA integrations] -.connect.-> drive","members":[{"pattern":"browser-agent","role":"first-class","note":"Core product is a browser-as-a-service for agents.","evidence":[{"type":"doc","title":"Browserbase docs — platform","url":"https://docs.browserbase.com","quote":"Browserbase is the complete platform to build and deploy agents that browse and interact with the web like humans.","accessed":"2026-05-20"},{"type":"doc","title":"Browserbase homepage","url":"https://www.browserbase.com","quote":"Give your agent a real browser to use the web like a human.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"session-isolation","role":"first-class","note":"Browser sessions are the unit of isolation; created, controlled and observed via API.","evidence":[{"type":"doc","title":"Browserbase — session definition","url":"https://docs.browserbase.com/fundamentals/create-browser-session","quote":"A browser session represents a single browser instance running in the cloud.","accessed":"2026-05-24"},{"type":"doc","title":"Browserbase — sessions as building block","url":"https://docs.browserbase.com/fundamentals/create-browser-session","quote":"It's the fundamental building block of Browserbase, providing an isolated environment for your web automation tasks.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"sandbox-isolation","role":"first-class","note":"Each session runs in an isolated managed Chromium environment with verified fingerprints.","evidence":[{"type":"doc","title":"Browserbase — verified browser","url":"https://docs.browserbase.com/features/stealth-mode","quote":"Purpose-built Chromium browser, maintained by the Browserbase team, with real browser fingerprints","accessed":"2026-05-24"},{"type":"blog","title":"Browserbase blog — ephemeral sandboxed VMs","url":"https://www.browserbase.com/blog/internal-agents","quote":"an isolated cloud sandbox, an ephemeral Linux VM with its own filesystem, network stack, and process tree","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"replay-time-travel","role":"first-class","note":"Every session recorded by default; HLS replays power Session Inspector and custom embeds.","evidence":[{"type":"doc","title":"Browserbase — session recording default","url":"https://docs.browserbase.com","quote":"Every session is automatically recorded so you can inspect actions and debug issues","accessed":"2026-05-20"},{"type":"doc","title":"Browserbase — HLS replay","url":"https://docs.browserbase.com/features/session-replay","quote":"Stream session replays as HLS to embed playback in your own application.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"agent-resumption","role":"first-class","note":"Contexts persist cookies, auth and application data across sessions.","evidence":[{"type":"doc","title":"Browserbase docs — Contexts overview","url":"https://docs.browserbase.com/features/contexts","quote":"Contexts allow you to persist user data across multiple browser sessions, enabling smoother automation, seamless authentication, and faster end-to-end workflows.","accessed":"2026-05-24"},{"type":"doc","title":"Browserbase docs — Contexts persisted data","url":"https://docs.browserbase.com/features/contexts","quote":"A Context persists the contents of the Chromium user data directory, including: Cookies, localStorage, IndexedDB, Session Storage, Service Workers, Web Data, Browser preferences.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"computer-use","role":"first-class","note":"Documented integrations for Anthropic Computer Use and OpenAI Computer-Using Agent against a Browserbase session.","evidence":[{"type":"doc","title":"Browserbase — Anthropic managed agents","url":"https://docs.browserbase.com","quote":"Run Claude agents in Anthropic's managed runtime and give them a Browserbase session to drive the web","accessed":"2026-05-20"},{"type":"doc","title":"Browserbase — OpenAI CUA","url":"https://docs.browserbase.com","quote":"Integrate OpenAI CUA with Browserbase for scalable browser agents","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"multi-model-routing","role":"supported","note":"Provider-agnostic: model selection is the developer's; Browserbase is infrastructure under any model.","evidence":[{"type":"doc","title":"Browserbase — any model","url":"https://www.browserbase.com","quote":"Models: Use any model with a single API key","accessed":"2026-05-24"},{"type":"doc","title":"Browserbase — choose your model","url":"https://www.browserbase.com","quote":"Choose your model: Connect AI to the browser session so it can interpret pages, make decisions, and drive interactions autonomously.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"limited","note":"Browserbase itself doesn't ship a tool catalogue — tools are exposed by the agent SDK (Stagehand) or model (CUA). The docs frame the platform as 'tool use'-enabling infrastructure and document LangChain integration as a 'browser tools' surface, but no first-party Browserbase tool primitives.","evidence":[{"type":"doc","title":"Browserbase docs — tool use positioning","url":"https://docs.browserbase.com","quote":"Power AI agents with real browser interactions and tool use.","accessed":"2026-05-24"},{"type":"doc","title":"Browserbase docs — LangChain browser tools integration","url":"https://docs.browserbase.com","quote":"Composable LLM chains and agents with browser tools.","accessed":"2026-05-24"}],"evidence_status":"full"}],"alternatives":[{"composition":"browser-use","relation":"competes-with","note":"Browser Use Cloud is the comparable open-core hosted browser tier."},{"composition":"stagehand","relation":"complements","note":"Stagehand is Browserbase's agent SDK that runs against Browserbase sessions."},{"composition":"notte","relation":"competes-with","note":"Peer managed browser infrastructure for agents."}],"references":[{"type":"doc","title":"Browserbase docs","url":"https://docs.browserbase.com","accessed":"2026-05-20"},{"type":"doc","title":"Browserbase homepage","url":"https://www.browserbase.com","quote":"Browserbase makes the web as reliable and programmable as APIs","accessed":"2026-05-20"},{"type":"doc","title":"Browserbase — session replay","url":"https://docs.browserbase.com/features/session-replay","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"mature","tags":["proprietary","api","browser-infrastructure","session-replay","stealth"]},{"id":"mobile-agent","name":"Mobile-Agent / GUI-Owl","kind":"framework","category":"browser-computer-use","build_surface":"full-code","vendor":"Alibaba Qwen / X-PLUG","language":"Python","license":"Apache-2.0","status":"active","url":"https://x-plug.github.io/MobileAgent/","repo":"https://github.com/X-PLUG/MobileAgent","intent":"Cross-platform multi-agent GUI automation framework (mobile / desktop / browser) built on the GUI-Owl native VLM family, with planning, progress management, reflection, and memory as distinct cooperating agents.","description":"Mobile-Agent is X-PLUG / Alibaba Qwen's GUI agent family. The v3 (2025) release became cross-platform — mobile, desktop, and browser — and shifted from a single-agent (v1) and mobile-only multi-agent (v2) design to a multi-agent framework over the GUI-Owl native VLM models (2B/4B/8B/32B/235B, built on Qwen3-VL). The framework decomposes a GUI task into planning, progress management, reflection, and memory roles, each backed by a specialised model call, and orchestrates them around grounded screenshot + DOM observation. Won the CCL 2025 best demo award. Distinct from the Western browser-first computer-use bias (Anthropic computer-use, OpenAI Operator) by being mobile-first and treating cross-platform grounding as a foundational model concern (GUI-Owl) rather than an integration layer.","primary_use_cases":["mobile UI automation on Android with native screenshot + tap grounding","cross-platform GUI tasks spanning mobile, desktop, and browser in one framework","research baseline for VLM-as-grounder for GUI actions","evaluation against 20+ GUI benchmarks (state-of-the-art on most per published numbers)"],"agent_loop_shape":"Multi-agent loop on every GUI step. A Planner emits the next high-level action given task and history. A Grounder (the GUI-Owl VLM) localises the action target on the current screenshot. An Executor performs the action (tap, swipe, type, scroll). A Reflector reads the post-action screenshot to verify success and writes to Memory. Across turns, the Planner consumes Memory plus current state to choose the next action.","key_concepts":[{"name":"GUI-Owl native VLM family","summary":"Foundation VLM (Qwen3-VL based, 2B-235B) trained natively for GUI grounding and action.","url":"https://github.com/X-PLUG/MobileAgent"},{"name":"Planner / Grounder / Executor / Reflector","summary":"Four-role decomposition per GUI step.","maps_to_pattern":"hierarchical-agents"},{"name":"Cross-platform unified abstraction","summary":"Single framework spans mobile, desktop, browser environments."},{"name":"Progress + memory roles","summary":"Dedicated agents track multi-step progress and persist relevant observations.","maps_to_pattern":"episodic-memory"}],"pattern_composition":"flowchart TD\n  TASK[GUI task] --> PLANNER[Planner agent<br/>next high-level action]\n  PLANNER --> GROUNDER[Grounder agent<br/>GUI-Owl VLM]\n  GROUNDER --> LOC[Localise target<br/>on current screenshot]\n  LOC --> EXECUTOR[Executor agent<br/>tap / swipe / type / scroll]\n  EXECUTOR --> ENV[(Mobile / desktop / browser<br/>cross-platform)]\n  ENV --> SHOT[Post-action screenshot]\n  SHOT --> REFLECTOR[Reflector agent<br/>verify success]\n  REFLECTOR --> MEM[(Memory<br/>observations + outcomes)]\n  MEM --> PLANNER\n  REFLECTOR -->|fail| PLANNER\n  REFLECTOR -->|done| RESULT[Task complete]","members":[{"pattern":"computer-use","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"browser-agent","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"dual-system-gui-agent","role":"first-class","note":"Planner over Grounder/Executor is a dual-system shape.","evidence":[],"evidence_status":"none"},{"pattern":"hierarchical-agents","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"orchestrator-workers","role":"first-class","note":"Planner orchestrates Grounder/Executor/Reflector workers.","evidence":[],"evidence_status":"none"},{"pattern":"react","role":"first-class","note":"Reflector closes the observation loop.","evidence":[],"evidence_status":"none"},{"pattern":"reflection","role":"first-class","note":"Reflector is a named agent.","evidence":[],"evidence_status":"none"},{"pattern":"tool-use","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"episodic-memory","role":"supported","note":"Memory role tracks observations and outcomes.","evidence":[],"evidence_status":"none"},{"pattern":"vector-memory","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"structured-output","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"plan-and-execute","role":"first-class","evidence":[],"evidence_status":"none"}],"alternatives":[{"composition":"anthropic-computer-use","relation":"competes-with","note":"Anthropic's computer-use is desktop-first and Western; Mobile-Agent is mobile-first and Chinese, with a unified cross-platform abstraction."},{"composition":"openai-operator","relation":"competes-with","note":"OpenAI Operator is browser-focused; Mobile-Agent spans mobile/desktop/browser."},{"composition":"browser-use","relation":"complements","note":"Browser-Use is browser-only; Mobile-Agent's web environment is one of three it supports."},{"composition":"stagehand","relation":"similar-shape","note":"Stagehand is a browser-control SDK; both work on the perception-action loop over GUIs."}],"last_analyzed":"2026-05-22","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-05-22","status_in_practice":"emerging","tags":["browser-computer-use","tool-use-environment","multi-agent","planning-control-flow","verification-reflection","memory","structure-data","full-code"],"references":[{"type":"doc","title":"Mobile-Agent / GUI-Owl — Homepage","url":"https://x-plug.github.io/MobileAgent/"},{"type":"repo","title":"Mobile-Agent / GUI-Owl — Source repository","url":"https://github.com/X-PLUG/MobileAgent"}]},{"id":"multion","name":"MultiOn","kind":"framework","category":"browser-computer-use","build_surface":"full-code","vendor":"AGI, Inc. (formerly MultiOn / MULTI·ON)","language":"API","license":"proprietary","status":"rebranded","url":"https://theagi.company","intent":"Originally a hosted browser-agent API that let developers send a natural-language instruction and have it executed on real websites; the company has since rebranded to AGI, Inc. and pivoted away from browser automation.","description":"MultiOn shipped a browser-agent API and consumer product centred on autonomous web tasks (booking, shopping, form filling) driven by natural language. As of 2026-05 the multion.ai domain 301-redirects to theagi.company and the rebranded entity, AGI, Inc., positions itself as an edge-AI lab for on-device mobile agents (AGI-0 \"personalized, proactive AI co-worker\") with no public mention of the prior browser product. The original MULTI-ON GitHub organisation returns 404; the legacy SDK repositories are no longer publicly accessible.","primary_use_cases":["(historic) browser-agent API for booking, ordering and form filling","(historic) MultiOn Retrieve / API SDK for developers","(current rebrand) edge-side mobile AI agents under the AGI-0 product"],"agent_loop_shape":"(Historic) Hosted browser-agent loop: client sends an instruction and a starting URL, MultiOn drives a remote browser, optionally streams progress, and returns a structured result. Current AGI Inc. focus is on-device mobile agents rather than browser automation; the prior loop is no longer publicly documented.","key_concepts":[{"name":"AGI, Inc. rebrand","summary":"multion.ai 301-redirects to theagi.company; the new positioning is edge-AI, not browser automation.","url":"https://theagi.company"},{"name":"AGI-0","summary":"Current flagship product: an on-device personalised mobile AI co-worker, in early access.","url":"https://theagi.company"}],"pattern_composition":"flowchart TD\n  past[Historic MultiOn API] -.deprecated.-> agentapi[Hosted browser agent]\n  agentapi -.no longer documented.-> rebrand[AGI, Inc. theagi.company]\n  rebrand --> agi0[AGI-0 mobile agent]\n  note[GitHub MULTI-ON org returns 404] -.evidence.-> rebrand","members":[{"pattern":"browser-agent","role":"limited","note":"Historic product was a browser agent (docs.multion.ai still describes the Agent API); current AGI Inc. positioning has pivoted to on-device mobile agents.","evidence":[{"type":"doc","title":"MultiOn legacy docs — Welcome","url":"https://docs.multion.ai/welcome","quote":"MultiOn is the Motor Cortex layer for AI, enabling autonomous actions on the web using natural language commands","accessed":"2026-05-24"},{"type":"doc","title":"MultiOn legacy docs — Agent API quickstart","url":"https://docs.multion.ai/quick-start","quote":"Create your first web browsing agent with the Agent API","accessed":"2026-05-24"},{"type":"doc","title":"theagi.company — current homepage (post-redirect from multion.ai)","url":"https://theagi.company","quote":"We are an AI lab bringing superintelligence to the edge, on a mission to make it 100% secure, private and accessible.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"computer-use","role":"limited","note":"Historic browser-agent product acted over websites but never claimed coordinate-grounded computer use; the rebrand targets mobile devices instead. Honest 'limited' — no first-party source documents a screen-coordinate model.","evidence":[{"type":"doc","title":"theagi.company — AGI-0 mobile agent","url":"https://theagi.company","quote":"AGI-0 is a personalized, proactive AI co-worker that gets things done on your smartphone.","accessed":"2026-05-24"}],"evidence_status":"limited"}],"alternatives":[{"composition":"browser-use","relation":"competes-with","note":"Open-source browser agent occupying the niche MultiOn vacated."},{"composition":"openai-operator","relation":"similar-shape","note":"Operator was the higher-profile hosted browser agent over the same period; also discontinued."},{"composition":"stagehand","relation":"competes-with","note":"Browserbase-backed open-source browser agent SDK."}],"references":[{"type":"doc","title":"AGI, Inc. (multion.ai redirect target)","url":"https://theagi.company","accessed":"2026-05-24"},{"type":"doc","title":"MultiOn legacy docs (still reachable at docs.multion.ai)","url":"https://docs.multion.ai/welcome","accessed":"2026-05-24"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"deprecated","tags":["rebranded","browser-agent","historic","edge-ai"]},{"id":"openai-operator","name":"OpenAI Operator","kind":"framework","category":"browser-computer-use","build_surface":"closed-application","vendor":"OpenAI","language":"Web product","license":"proprietary","status":"discontinued","discontinued_date":"2025-08-31","first_released":"2025-01-23","url":"https://en.wikipedia.org/wiki/OpenAI_Operator","intent":"OpenAI's hosted browser-agent product (Jan 2025–Aug 2025) powered by the Computer-Using Agent (CUA) model, autonomously performing web tasks for ChatGPT Pro users; deprecated and shut down on 2025-08-31, succeeded by ChatGPT agent.","description":"Operator was a research-preview consumer product at operator.chatgpt.com that paired ChatGPT with a remote browser and the Computer-Using Agent (CUA) model so it could autonomously fill forms, place orders, schedule appointments and perform other repetitive web tasks. It was a Pro-tier ChatGPT feature. After the July 2025 launch of ChatGPT agent — which combines and extends Operator's browser control with deep research and tool use — Operator was deprecated and the operator.chatgpt.com domain was shut down on 2025-08-31. Visits now 308-redirect to chatgpt.com in agent mode.","primary_use_cases":["(historic) autonomous browser navigation for ChatGPT Pro users","(historic) form filling, online orders, appointment scheduling","(now) superseded by ChatGPT agent"],"agent_loop_shape":"Computer-Using Agent loop: CUA model perceived the browser via screenshots, emitted mouse/keyboard actions, observed the next screenshot, and continued until the task finished or the user intervened. Operator hosted the browser, the CUA model and the UI; the user supervised via the Operator chat panel.","key_concepts":[{"name":"Computer-Using Agent (CUA)","summary":"Vision+action model behind Operator that drives a browser via screenshots and synthetic mouse/keyboard.","maps_to_pattern":"computer-use","url":"https://en.wikipedia.org/wiki/OpenAI_Operator"},{"name":"Hosted browser session","summary":"Operator ran a managed browser the model controlled; the user watched and could take over.","maps_to_pattern":"human-in-the-loop","url":"https://en.wikipedia.org/wiki/OpenAI_Operator"},{"name":"Successor: ChatGPT agent","summary":"Replaced Operator; combines virtual computer control with deep research and tools.","url":"https://en.wikipedia.org/wiki/ChatGPT"}],"pattern_composition":"flowchart TD\n  user[ChatGPT Pro user] --> ui[Operator panel]\n  ui --> cua[CUA model]\n  cua --> shot[Screenshot of hosted browser]\n  shot --> act[Mouse / keyboard action]\n  act --> browser[Hosted browser]\n  browser --> shot\n  cua --> done[Task done or user takeover]\n  done -.2025-08-31 shutdown.-> agent[ChatGPT agent successor]","members":[{"pattern":"computer-use","role":"first-class","note":"Operator was the first-party demonstration of OpenAI's Computer-Using Agent model.","evidence":[{"type":"doc","title":"Wikipedia — OpenAI Operator capability","url":"https://en.wikipedia.org/wiki/OpenAI_Operator","quote":"OpenAI Operator was an AI agent developed by OpenAI, capable of autonomously performing tasks through web browser interactions, including filling forms, placing online orders, scheduling appointments, and other repetitive browser-based tasks.","accessed":"2026-05-20"},{"type":"doc","title":"Wikipedia — ChatGPT, Operator launch","url":"https://en.wikipedia.org/wiki/ChatGPT","quote":"In January, Operator was released. It was capable of autonomously performing tasks through web browser interactions, including filling forms, placing online orders, scheduling appointments, and other browser-based tasks.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"browser-agent","role":"first-class","note":"Hosted browser is the action surface. OpenAI's launch and CUA pages describe Operator as autonomously performing tasks through web-browser interactions; CUA interprets screenshots and drives 'typical browser controls like a cursor and mouse'.","evidence":[{"type":"blog","title":"OpenAI — Introducing Operator (browser interactions)","url":"https://openai.com/index/introducing-operator/","quote":"Operator is a research preview of an agent that can use its own browser to perform tasks for you.","accessed":"2026-05-24"},{"type":"blog","title":"OpenAI — Computer-Using Agent (browser controls)","url":"https://openai.com/index/computer-using-agent/","quote":"Operator uses a model called Computer-Using Agent (CUA), built on OpenAI o3 to interpret screenshots and interact with sites using typical browser controls like a cursor and mouse.","accessed":"2026-05-24"},{"type":"doc","title":"Wikipedia — OpenAI Operator browser interactions","url":"https://en.wikipedia.org/wiki/OpenAI_Operator","quote":"autonomously performing tasks through web browser interactions","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"human-in-the-loop","role":"first-class","note":"Takeover mode and confirmation flows are first-party HITL primitives: Operator hands the browser back to the user for credentials/payment and asks the user 'Yes/No' to confirm task completion.","evidence":[{"type":"blog","title":"OpenAI — Introducing Operator (takeover mode)","url":"https://openai.com/index/introducing-operator/","quote":"Operator asks the user to take over when inputting sensitive information into the browser, such as login credentials or payment information.","accessed":"2026-05-24"},{"type":"blog","title":"OpenAI — Introducing Operator (no screenshot in takeover)","url":"https://openai.com/index/introducing-operator/","quote":"When in takeover mode, Operator does not collect or screenshot information entered by the user.","accessed":"2026-05-24"},{"type":"doc","title":"Wikipedia — ChatGPT agent successor","url":"https://en.wikipedia.org/wiki/ChatGPT","quote":"Like Operator, it controls a virtual computer.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"approval-queue","role":"first-class","note":"Operator is trained to ask for user approval before finalising any significant browser action (submitting orders, sending email) — OpenAI's launch post calls this out directly and the CUA model has a measured 92% recall for confirmation requests on a 607-task evaluation.","evidence":[{"type":"blog","title":"OpenAI — Introducing Operator (approval before significant action)","url":"https://openai.com/index/introducing-operator/","quote":"Before finalizing any significant action, such as submitting an order or sending an email, Operator should ask for approval.","accessed":"2026-05-24"},{"type":"blog","title":"OpenAI — Computer-Using Agent (sensitive-action confirmation)","url":"https://openai.com/index/computer-using-agent/","quote":"the CUA model is trained to ask for user confirmation before finalizing tasks with external side effects, for example before submitting an order, sending an email, etc., so that the user can double-check the model's work before it becomes permanent.","accessed":"2026-05-24"}],"evidence_status":"full"}],"alternatives":[{"composition":"anthropic-computer-use","relation":"competes-with","note":"First-party computer-use capability from Anthropic; still active as an API beta."},{"composition":"browser-use","relation":"competes-with","note":"Open-source alternative for browser automation."},{"composition":"stagehand","relation":"competes-with","note":"Browserbase SDK that exposes computer-use models against a hosted browser."},{"composition":"bytebot","relation":"competes-with","note":"Peer computer-use agent (browser-centric)."},{"composition":"mobile-agent","relation":"competes-with","note":"OpenAI Operator is browser-focused; Mobile-Agent spans mobile/desktop/browser."},{"composition":"multion","relation":"similar-shape","note":"Operator was the higher-profile hosted browser agent over the same period; also discontinued."}],"references":[{"type":"doc","title":"Wikipedia — OpenAI Operator","url":"https://en.wikipedia.org/wiki/OpenAI_Operator","quote":"Operator was deprecated after the release of ChatGPT agent, and shut down on August 31, 2025.","accessed":"2026-05-20"},{"type":"doc","title":"Wikipedia — ChatGPT, July 2025 ChatGPT agent","url":"https://en.wikipedia.org/wiki/ChatGPT","quote":"In July 2025, OpenAI released ChatGPT agent, an AI agent that can perform multi-step tasks. Like Operator, it controls a virtual computer.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"deprecated","tags":["discontinued","openai","browser-agent","computer-use","research-preview"],"successor":"openai-chatgpt-agent"},{"id":"stagehand","name":"Stagehand","kind":"framework","category":"browser-computer-use","build_surface":"full-code","vendor":"Browserbase, Inc.","language":"TypeScript, Python","license":"MIT","status":"active","first_released":"2024-03-24","url":"https://www.stagehand.dev","repo":"https://github.com/browserbase/stagehand","docs_url":"https://docs.stagehand.dev","intent":"Browserbase's open-source SDK for browser agents — a Playwright-based framework with three natural-language primitives (act, extract, observe) plus an agent() mode that supports computer-use models from OpenAI, Anthropic, Google and Microsoft.","description":"Stagehand layers AI primitives onto Playwright. The act() primitive runs single, self-healing actions described in plain English; extract() pulls structured data validated against Zod or JSON schemas; observe() returns the actionable elements on a page so a plan can be built before execution. The agent() primitive turns a high-level instruction into a fully autonomous browser workflow and supports both DOM-based reasoning with any LLM and coordinate-based computer-use models (e.g. Google gemini-2.5-computer-use-preview, OpenAI computer-use models). Stagehand is MIT-licensed and developed by Browserbase, against which it runs by default.","primary_use_cases":["AI-driven browser automation with natural-language actions","structured data extraction from web pages with schema validation","exploration and planning of multi-step browser workflows","computer-use agents driving Playwright via OpenAI / Anthropic / Google models"],"agent_loop_shape":"Playwright-based SDK with two surfaces. The atomic surface is act / extract / observe — each call is one LLM round-trip that either runs an action, pulls structured data, or returns candidate actions. The autonomous surface is agent(), which runs a multi-step loop until the task finishes; in CUA mode the agent grounds actions in screen coordinates, in DOM mode it grounds them in serialised DOM nodes, and in hybrid mode it can use both.","key_concepts":[{"name":"act()","summary":"Run a single, self-healing browser action from natural language.","maps_to_pattern":"browser-agent","url":"https://docs.stagehand.dev/basics/act"},{"name":"extract()","summary":"Pull schema-validated structured data from a page using Zod or JSON.","maps_to_pattern":"structured-output","url":"https://docs.stagehand.dev/basics/extract"},{"name":"observe()","summary":"Discover actionable elements on a page and return structured actions you can replay or validate before acting.","maps_to_pattern":"plan-and-execute","url":"https://docs.stagehand.dev/basics/observe"},{"name":"agent()","summary":"Multi-step autonomous workflows; supports CUA, DOM and hybrid grounding modes.","maps_to_pattern":"computer-use","url":"https://docs.stagehand.dev/basics/agent"},{"name":"Multi-provider LLMs","summary":"Pluggable LLM provider; CUA mode supports OpenAI, Anthropic, Google and Microsoft computer-use models.","maps_to_pattern":"multi-model-routing","url":"https://docs.stagehand.dev"}],"pattern_composition":"flowchart TD\n  dev[Developer] --> sh[Stagehand on Playwright]\n  sh --> atomic{Primitive}\n  atomic -->|act| act[act: one self-healing action]\n  atomic -->|extract| ex[extract: schema-validated data]\n  atomic -->|observe| obs[observe: candidate actions]\n  atomic -->|agent| agentmode{Agent mode}\n  agentmode -->|CUA| cua[Coordinate grounding]\n  agentmode -->|DOM| dom[DOM-node grounding]\n  agentmode -->|hybrid| hyb[Both]\n  cua --> step[Multi-step loop]\n  dom --> step\n  hyb --> step\n  step --> page[Real browser via Playwright]\n  page --> step\n  sh -.runs by default against.-> bb[Browserbase session]","members":[{"pattern":"browser-agent","role":"first-class","note":"Core framing of the SDK: natural-language browser automation.","evidence":[{"type":"repo","title":"Stagehand README","url":"https://github.com/browserbase/stagehand","quote":"Stagehand is a browser automation framework used to control web browsers with natural language and code.","accessed":"2026-05-20"},{"type":"doc","title":"Stagehand docs — act","url":"https://docs.stagehand.dev/basics/act","quote":"act enables Stagehand to perform individual actions on a web page.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"act / extract / observe / agent are the documented tool surface; agent() also supports MCP integrations and custom tools.","evidence":[{"type":"doc","title":"Stagehand docs — act self-healing","url":"https://docs.stagehand.dev/basics/act","quote":"Write automation in plain English. No selectors or complex syntax.","accessed":"2026-05-20"},{"type":"doc","title":"Stagehand docs — act self-healing 2","url":"https://docs.stagehand.dev/basics/act","quote":"Actions automatically adapt when websites change.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"structured-output","role":"first-class","note":"extract() validates against Zod (TypeScript) or JSON schemas.","evidence":[{"type":"doc","title":"Stagehand docs — extract schema","url":"https://docs.stagehand.dev/basics/extract","quote":"extract() grabs structured data from a webpage. You can define your schema with Zod (TypeScript) or JSON.","accessed":"2026-05-20"},{"type":"doc","title":"Stagehand docs — extract clean objects","url":"https://docs.stagehand.dev/basics/extract","quote":"Turn messy webpage data into clean objects that follow a schema.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"plan-and-execute","role":"first-class","note":"observe() returns structured actions you can run later; documented as a plan-then-execute pattern.","evidence":[{"type":"doc","title":"Stagehand docs — observe","url":"https://docs.stagehand.dev/basics/observe","quote":"discovers actionable elements on a page and returns structured actions you can execute or validate before acting.","accessed":"2026-05-20"},{"type":"doc","title":"Stagehand docs — observe plan-execute","url":"https://docs.stagehand.dev/basics/observe","quote":"discover all actions once, then execute without additional LLM calls.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"computer-use","role":"first-class","note":"agent() supports Computer-Using Agent models from Google, OpenAI, Anthropic and Microsoft, plus a hybrid coord+DOM mode.","evidence":[{"type":"doc","title":"Stagehand docs — agent CUA","url":"https://docs.stagehand.dev/basics/agent","quote":"See and understand web interfaces like humans do using computer vision.","accessed":"2026-05-20"},{"type":"doc","title":"Stagehand docs — hybrid grounding","url":"https://docs.stagehand.dev/basics/agent","quote":"access to both coordinate-based and DOM-based tools to better account for where each may fall short.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"agent() runs a multi-step observe-then-act loop until the high-level task is complete.","evidence":[{"type":"doc","title":"Stagehand docs — agent autonomous","url":"https://docs.stagehand.dev/basics/agent","quote":"agent turns high level tasks into fully autonomous browser workflows.","accessed":"2026-05-24"},{"type":"doc","title":"Stagehand docs — multi-step workflows","url":"https://docs.stagehand.dev/basics/agent","quote":"Execute complex sequences automatically.","accessed":"2026-05-24"},{"type":"doc","title":"Stagehand docs — MCP and custom tools","url":"https://docs.stagehand.dev/basics/agent","quote":"Agents can be enhanced with external tools and services through MCP (Model Context Protocol) integrations.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"multi-model-routing","role":"first-class","note":"Pluggable LLM provider; CUA mode spans OpenAI, Anthropic, Google and Microsoft models.","evidence":[{"type":"doc","title":"Stagehand docs — any model","url":"https://docs.stagehand.dev","quote":"Models: Use any LLM model with Stagehand","accessed":"2026-05-20"},{"type":"doc","title":"Stagehand docs — gemini computer use","url":"https://www.stagehand.dev","quote":"google/gemini-2.5-computer-use-preview-10-2025","accessed":"2026-05-20"}],"evidence_status":"full"}],"alternatives":[{"composition":"browser-use","relation":"competes-with","note":"Python peer with comparable scope (browser agent + multi-model)."},{"composition":"browserbase","relation":"complements","note":"Same vendor; Browserbase is the default execution substrate."},{"composition":"anthropic-computer-use","relation":"wraps","note":"Stagehand's agent() mode can run on top of Anthropic's computer use models."},{"composition":"openai-operator","relation":"competes-with","note":"Operator was the hosted alternative; Stagehand is the open SDK people use post-Operator."},{"composition":"skyvern","relation":"competes-with","note":"Peer Playwright-based AI browser automation."},{"composition":"mobile-agent","relation":"similar-shape","note":"Stagehand is a browser-control SDK; both work on the perception-action loop over GUIs."},{"composition":"multion","relation":"competes-with","note":"Browserbase-backed open-source browser agent SDK."}],"references":[{"type":"repo","title":"browserbase/stagehand","url":"https://github.com/browserbase/stagehand","accessed":"2026-05-20"},{"type":"doc","title":"Stagehand docs","url":"https://docs.stagehand.dev","accessed":"2026-05-20"},{"type":"doc","title":"Stagehand homepage","url":"https://www.stagehand.dev","quote":"Traditional frameworks like Playwright and Puppeteer force you to write brittle scripts that break with every UI change","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","typescript","python","playwright","browserbase","mit"]},{"id":"skyvern","name":"Skyvern","kind":"framework","category":"browser-computer-use","build_surface":"full-code","vendor":"Skyvern-AI","language":"Python, TypeScript","license":"AGPL-3.0","status":"active","url":"https://www.skyvern.com","repo":"https://github.com/Skyvern-AI/skyvern","docs_url":"https://docs.skyvern.com","intent":"Automate browser-based workflows with AI using vision LLMs instead of brittle selectors.","description":"Playwright-compatible browser automation that uses vision LLMs plus computer vision to interact with websites without per-site XPath/CSS selectors, adapting to layout changes and sites it has never seen. Ships a no-code workflow builder alongside the SDK; reports strong WebVoyager and form-filling results.","primary_use_cases":["automating browser workflows across unfamiliar or changing sites","form-filling and data-entry automation without per-site selectors","no-code browser automation for non-technical users"],"agent_loop_shape":"A vision-LLM observes the rendered page, decides the next interaction, and acts via a Playwright-compatible layer — looping perceive→decide→act without relying on code-defined selectors.","key_concepts":[{"name":"Vision-LLM interaction","summary":"The agent sees the rendered page and acts, instead of parsing the DOM.","maps_to_pattern":"computer-use","url":"https://docs.skyvern.com"},{"name":"Workflow builder","summary":"No-code composition of multi-step browser automations.","maps_to_pattern":"visual-workflow-graph"}],"pattern_composition":"flowchart TD\n  TASK[Task / workflow] --> PERCEIVE[Vision LLM perceives page]\n  PERCEIVE --> DECIDE[Decide next action]\n  DECIDE --> ACT[Act via Playwright-compatible layer]\n  ACT --> PERCEIVE\n  DECIDE -->|done| OUT[Result]\n  BUILDER[No-code workflow builder] -.defines.-> TASK","members":[{"pattern":"computer-use","role":"core","note":"Automate Browser-based workflows using LLMs and Computer Vision","evidence":[{"type":"repo","title":"Skyvern-AI/skyvern README","url":"https://github.com/Skyvern-AI/skyvern","quote":"Automate Browser-based workflows using LLMs and Computer Vision","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"browser-agent","role":"first-class","note":"It provides a Playwright-compatible SDK that adds AI functionality on top of playwright","evidence":[{"type":"repo","title":"Skyvern-AI/skyvern README","url":"https://github.com/Skyvern-AI/skyvern","quote":"It provides a Playwright-compatible SDK that adds AI functionality on top of playwright","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"visual-workflow-graph","role":"first-class","note":"a no-code workflow builder to help both technical and non-technical users automate manual workflows on any website","evidence":[{"type":"repo","title":"Skyvern-AI/skyvern README","url":"https://github.com/Skyvern-AI/skyvern","quote":"a no-code workflow builder to help both technical and non-technical users automate manual workflows on any website","accessed":"2026-06-04"}],"evidence_status":"limited"}],"instantiates":["browser-computer-use-stack"],"alternatives":[{"composition":"browser-use","relation":"competes-with","note":"Peer open-source browser agent."},{"composition":"stagehand","relation":"competes-with","note":"Peer Playwright-based AI browser automation."}],"references":[{"type":"repo","title":"Skyvern-AI/skyvern","url":"https://github.com/Skyvern-AI/skyvern","accessed":"2026-06-04"},{"type":"doc","title":"Skyvern documentation","url":"https://docs.skyvern.com","accessed":"2026-06-04"}],"last_analyzed":"2026-06-04","verification_status":"partial","last_updated":"2026-06-17","date_added":"2026-06-04","status_in_practice":"emerging","tags":["browser-automation","computer-vision","playwright","open-source","no-code"]},{"id":"bytebot","name":"Bytebot","kind":"framework","category":"browser-computer-use","build_surface":"full-code","vendor":"Tantl Labs","language":"TypeScript","license":"Apache-2.0","status":"active","url":"https://bytebot.ai","repo":"https://github.com/bytebot-ai/bytebot","docs_url":"https://docs.bytebot.ai","intent":"Give an AI its own containerized desktop so it can complete tasks across native apps, not just the browser.","description":"Self-hosted computer-use agent running inside a containerized Linux (Ubuntu) desktop. Full-desktop access lets the agent use native applications, a persistent filesystem, password-manager/2FA authentication, PDF/document handling and multi-application workflows — beyond what browser-only agents or API integrations can do.","primary_use_cases":["automating multi-application desktop workflows, not just web pages","document/PDF handling with a persistent filesystem","authenticated tasks via desktop password managers with 2FA"],"agent_loop_shape":"The agent receives a natural-language task and operates a full virtual desktop (mouse/keyboard/screen) inside a container, perceiving the screen and acting across native apps and the browser until the task is complete.","key_concepts":[{"name":"Virtual desktop","summary":"A containerized Ubuntu desktop the agent fully controls.","maps_to_pattern":"computer-use","url":"https://docs.bytebot.ai"},{"name":"Containerized session","summary":"Self-hosted isolation with a persistent filesystem.","maps_to_pattern":"sandbox-isolation"}],"pattern_composition":"flowchart TD\n  TASK[Natural-language task] --> AGENT[Computer-use agent]\n  AGENT --> SCREEN[Perceive virtual desktop]\n  SCREEN --> ACTION[Mouse / keyboard action]\n  ACTION --> APPS[Native apps / browser / files]\n  APPS --> SCREEN\n  AGENT -.confined to.-> CONTAINER[[Containerized Ubuntu desktop]]\n  ACTION -->|done| OUT[Result]","members":[{"pattern":"computer-use","role":"core","note":"move the mouse, type on the keyboard, and complete tasks just like a human would","evidence":[{"type":"repo","title":"bytebot-ai/bytebot README","url":"https://github.com/bytebot-ai/bytebot","quote":"move the mouse, type on the keyboard, and complete tasks just like a human would","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"sandbox-isolation","role":"first-class","note":"A complete Ubuntu Linux environment with pre-installed applications","evidence":[{"type":"repo","title":"bytebot-ai/bytebot README","url":"https://github.com/bytebot-ai/bytebot","quote":"A complete Ubuntu Linux environment with pre-installed applications","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"browser-agent","role":"supported","note":"Use any application (browsers, email clients, office tools, IDEs)","evidence":[{"type":"repo","title":"bytebot-ai/bytebot README","url":"https://github.com/bytebot-ai/bytebot","quote":"Use any application (browsers, email clients, office tools, IDEs)","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"full-desktop-computer-use","role":"core","note":"Bytebot runs the agent on a containerized Ubuntu (XFCE) desktop with native apps, a persistent filesystem, and password-manager extensions handling logins and 2FA, rather than a browser-only surface.","evidence":[{"type":"repo","title":"bytebot-ai/bytebot","url":"https://github.com/bytebot-ai/bytebot","accessed":"2026-06-04"}],"evidence_status":"full"}],"instantiates":["browser-computer-use-stack"],"alternatives":[{"composition":"anthropic-computer-use","relation":"similar-shape","note":"Peer computer-use approach driving a full desktop/GUI."},{"composition":"openai-operator","relation":"competes-with","note":"Peer computer-use agent (browser-centric)."}],"references":[{"type":"repo","title":"bytebot-ai/bytebot","url":"https://github.com/bytebot-ai/bytebot","accessed":"2026-06-04"},{"type":"doc","title":"Bytebot documentation","url":"https://docs.bytebot.ai","accessed":"2026-06-04"}],"last_analyzed":"2026-06-04","verification_status":"partial","last_updated":"2026-06-17","date_added":"2026-06-04","status_in_practice":"emerging","tags":["computer-use","desktop-agent","containerized","self-hosted","open-source"]},{"id":"notte","name":"Notte","kind":"framework","category":"browser-computer-use","build_surface":"full-code","vendor":"Notte Labs, Inc.","language":"Python","license":"SSPL-1.0","status":"active","url":"https://notte.cc","repo":"https://github.com/nottelabs/notte","docs_url":"https://docs.notte.cc","intent":"Build web agents and deploy serverless web-automation functions on managed browser infrastructure.","description":"Full-stack web-agent framework bundling three normally-separate layers: hosted browser infrastructure (stealth sessions, CAPTCHA solving, proxies), agent reasoning over web tasks, and a credential vault (passwords, MFA, digital personas). Mixes deterministic scripting with AI reasoning to cut LLM cost while improving reliability.","primary_use_cases":["deploying reliable web agents without stitching together browser, LLM and credential services","authenticated web automation using a managed credential vault","hybrid scripted + AI web workflows that minimize LLM cost"],"agent_loop_shape":"An agent reasons over web tasks on hosted, stealth browser sessions, automatically drawing credentials from a vault; deterministic scripted steps are interleaved with AI reasoning to reduce model calls.","key_concepts":[{"name":"Managed browser infra","summary":"Hosted stealth sessions with CAPTCHA solving and proxies.","maps_to_pattern":"browser-agent","url":"https://docs.notte.cc"},{"name":"Credential vault","summary":"Secure store of passwords, MFA and digital personas the agent draws on.","maps_to_pattern":"secrets-handling"}],"pattern_composition":"flowchart TD\n  TASK[Web task] --> AGENT[Web agent]\n  AGENT --> SCRIPT{Deterministic step?}\n  SCRIPT -->|yes| RUN[Scripted action]\n  SCRIPT -->|no| REASON[LLM reasoning]\n  RUN --> BROWSER[Managed stealth browser]\n  REASON --> BROWSER\n  VAULT[(Credential vault)] -.auth.-> BROWSER\n  BROWSER --> OUT[Result]","members":[{"pattern":"browser-agent","role":"core","note":"Stealth Browser Sessions → Browser instances with built-in CAPTCHA solving, proxies, and anti-detection","evidence":[{"type":"repo","title":"nottelabs/notte README","url":"https://github.com/nottelabs/notte","quote":"Stealth Browser Sessions → Browser instances with built-in CAPTCHA solving, proxies, and anti-detection","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"secrets-handling","role":"first-class","note":"Vaults are tools you can attach to your Agent instance to securely store and manage credentials.","evidence":[{"type":"repo","title":"nottelabs/notte README","url":"https://github.com/nottelabs/notte","quote":"Vaults are tools you can attach to your Agent instance to securely store and manage credentials.","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"deterministic-control-flow-not-prompt","role":"supported","note":"mix web automation primitives with agents for specific parts that require reasoning and adaptability","evidence":[{"type":"repo","title":"nottelabs/notte README","url":"https://github.com/nottelabs/notte","quote":"mix web automation primitives with agents for specific parts that require reasoning and adaptability","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"agent-credential-vault","role":"first-class","note":"Notte bundles an Agent Vault (per-site passwords), Agent Persona (email, phone, automated 2FA), and cookie/session upload the web agent draws on without credentials entering the prompt.","evidence":[{"type":"repo","title":"nottelabs/notte","url":"https://github.com/nottelabs/notte","accessed":"2026-06-04"}],"evidence_status":"full"}],"instantiates":["browser-computer-use-stack"],"alternatives":[{"composition":"browserbase","relation":"competes-with","note":"Peer managed browser infrastructure for agents."},{"composition":"browser-use","relation":"competes-with","note":"Peer web-agent framework."}],"references":[{"type":"repo","title":"nottelabs/notte","url":"https://github.com/nottelabs/notte","accessed":"2026-06-04"},{"type":"doc","title":"Notte documentation","url":"https://docs.notte.cc","accessed":"2026-06-04"}],"last_analyzed":"2026-06-04","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-06-04","status_in_practice":"emerging","tags":["browser-infra","web-agent","credential-vault","open-source"]},{"id":"appagent","name":"AppAgent","kind":"framework","category":"browser-computer-use","vendor":"Tencent (TencentQQGYLab)","language":"Python","license":"MIT","status":"active","first_released":"2023-12-21","url":"https://github.com/TencentQQGYLab/AppAgent","repo":"https://github.com/TencentQQGYLab/AppAgent","docs_url":"https://github.com/TencentQQGYLab/AppAgent","intent":"AppAgent is a multimodal agent framework that operates smartphone applications by observing the screen and performing human-like taps and swipes, after first exploring each app to build a per-element documentation base.","description":"AppAgent is an LLM-based multimodal agent framework designed to operate smartphone applications. It first runs an exploration phase, either exploring an app autonomously or learning from a human demonstration, and generates per-element documentation that it saves for later use. In the deployment phase it drives the app through a simplified action space that mimics human interactions such as tapping and swiping, using a vision-language model to read labeled screenshots. The approach bypasses the need for system back-end access, so it applies across diverse apps without per-app APIs.","primary_use_cases":["operating smartphone apps without backend APIs","exploration-driven documentation of opaque app UIs","GUI task automation on Android through taps and swipes","vision-language control of mobile interfaces"],"agent_loop_shape":"In the exploration phase the agent either explores an app on its own or learns from a human demonstration, generating documentation for the elements it interacts with and saving it. In the deployment phase it observes labeled screenshots with a vision-language model, selects an action from a simplified action space, and performs human-like interactions such as tapping and swiping, consulting the documentation base it built earlier.","build_surface":"full-code","members":[{"pattern":"app-exploration-phase","role":"core","note":"AppAgent runs an exploration phase (autonomous or learning from a human demo) that generates per-element documentation into a knowledge base, then a deployment phase that selects that documentation base to execute tasks.","evidence":[{"type":"repo","title":"AppAgent: Multimodal Agents as Smartphone Users","url":"https://github.com/TencentQQGYLab/AppAgent","quote":"The exploration phase starts with a task given by you, and you can choose to let the agent either explore the app on its own or learn from your demonstration. In both cases, the agent generates documentation for elements interacted during the exploration/demonstration and saves them for use in the deployment phase.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"mobile-ui-agent","role":"core","note":"In the deployment phase AppAgent drives the phone through a simplified, touch-native action space (tap, swipe) over labeled screenshots, bypassing back-end APIs so it generalizes across apps.","evidence":[{"type":"repo","title":"AppAgent: Multimodal Agents as Smartphone Users","url":"https://github.com/TencentQQGYLab/AppAgent","quote":"operate smartphone applications through a simplified action space, mimicking human-like interactions such as tapping and swiping","accessed":"2026-06-17"},{"type":"repo","title":"AppAgent: Multimodal Agents as Smartphone Users","url":"https://github.com/TencentQQGYLab/AppAgent","quote":"This novel approach bypasses the need for system back-end access, thereby broadening its applicability across diverse apps","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"computer-use","role":"supported","note":"AppAgent perceives the device the way a user does: each step captures a screenshot, labels every interactive element with a numeric tag, and a vision-language model picks the next action over that annotated image rather than calling any per-app API.","evidence":[{"type":"repo","title":"AppAgent: Multimodal Agents as Smartphone Users","url":"https://github.com/TencentQQGYLab/AppAgent","quote":"A screenshot of your phone will be captured and all interactive elements shown on the screen will be labeled with numeric tags.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"reflection","role":"supported","note":"During autonomous exploration AppAgent reflects on its previous action — checking that the action adhered to the given task — before writing per-element documentation, so each interaction is reviewed rather than blindly recorded.","evidence":[{"type":"repo","title":"AppAgent: Multimodal Agents as Smartphone Users","url":"https://github.com/TencentQQGYLab/AppAgent","quote":"Under this mode, AppAgent will reflect on its previous action making sure its action adheres to the given task and generate documentation for the elements explored.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"Exploration phase","summary":"The first stage in which the agent autonomously explores an app or learns from a human demonstration and writes documentation for each UI element it interacts with, building the knowledge base used later.","maps_to_pattern":"app-exploration-phase","url":"https://github.com/TencentQQGYLab/AppAgent"},{"name":"Deployment phase","summary":"The execution stage where the agent is given a task and a chosen documentation base, then carries out the task by reading labeled screenshots and acting on them.","maps_to_pattern":"mobile-ui-agent","url":"https://github.com/TencentQQGYLab/AppAgent"},{"name":"Numeric element labels","summary":"Numeric tags overlaid on every interactive element in each captured screenshot, giving the vision-language model a stable handle to reference the element it wants to tap or swipe.","maps_to_pattern":"computer-use","url":"https://github.com/TencentQQGYLab/AppAgent"},{"name":"Element documentation base","summary":"The saved per-element notes produced during exploration that the agent loads in the deployment phase, so prior experience with an app's controls carries into task execution.","maps_to_pattern":"procedural-memory","url":"https://github.com/TencentQQGYLab/AppAgent"}],"references":[{"type":"repo","title":"TencentQQGYLab/AppAgent (source repository and README)","url":"https://github.com/TencentQQGYLab/AppAgent","quote":"operate smartphone applications through a simplified action space, mimicking human-like interactions such as tapping and swiping","accessed":"2026-06-17"},{"type":"paper","title":"AppAgent: Multimodal Agents as Smartphone Users (arXiv:2312.13771)","url":"https://arxiv.org/abs/2312.13771","accessed":"2026-06-17"}],"alternatives":[{"composition":"mobile-agent","relation":"competes-with","note":"Both are vision-language mobile GUI agents that drive Android apps from screenshots; Mobile-Agent / GUI-Owl focuses on a perception-and-action pipeline, while AppAgent adds an up-front per-app exploration-and-documentation phase."},{"composition":"anthropic-computer-use","relation":"similar-shape","note":"Both control a UI through screenshots and human-like actions rather than APIs; Anthropic Computer Use targets a desktop screen whereas AppAgent targets smartphone apps with numeric-tagged elements."}],"verification_status":"partial","status_in_practice":"emerging","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"AppAgent\"]\n  fw --> p1[\"App Exploration Phase<br/>(core)\"]\n  fw --> p2[\"Mobile UI Agent<br/>(core)\"]\n  fw --> p3[\"Computer Use<br/>(supported)\"]\n  fw --> p4[\"Reflection<br/>(supported)\"]","tags":["browser-computer-use","tool-use-environment","verification-reflection","full-code"]},{"id":"openai-chatgpt-agent","name":"ChatGPT agent","kind":"framework","category":"browser-computer-use","vendor":"OpenAI","language":"proprietary","license":"proprietary","status":"active","first_released":"2025-07-17","url":"https://openai.com/index/introducing-chatgpt-agent/","docs_url":"https://help.openai.com/en/articles/11752874-chatgpt-agent","intent":"ChatGPT agent carries out multi-step tasks for a user by operating its own virtual computer with a browser, terminal, and connected tools, pausing for permission before consequential actions.","description":"ChatGPT agent is OpenAI's agentic mode inside ChatGPT that completes end-to-end tasks on a virtual computer rather than only answering questions. The agent plans steps, navigates a visual and text browser, runs code in a terminal, manipulates files, and reads from connectors such as Gmail and Google Drive. It is trained to request user permission before actions with real-world consequences such as purchases or sending email, and the user can interrupt the run or take over the browser at any point. It combines the earlier Operator browser agent, deep research web synthesis, and ChatGPT conversation in one system.","primary_use_cases":["autonomous web research and synthesis","multi-application task completion on a virtual computer","form filling and web navigation on the user's behalf","spreadsheet and presentation generation from gathered data"],"agent_loop_shape":"The agent runs inside an isolated virtual computer that keeps context across tools. It reasons about the goal, drives a visual or text browser and a terminal, reads files and connector data, and iterates step by step. Before any action with real-world consequences it pauses to ask the user for permission, and the user can interrupt, stop, or take over the browser at any point.","build_surface":"closed-application","members":[{"pattern":"computer-use","role":"core","note":"ChatGPT agent operates its own virtual computer end-to-end, driving a visual browser, a terminal, and file operations rather than calling bespoke per-application APIs.","evidence":[{"type":"tutorial","title":"ChatGPT agents explained","url":"https://www.techtarget.com/whatis/feature/ChatGPT-agents-explained","quote":"The agent operates within its own isolated environment that maintains context across all tools and tasks. Terminal access is provided to the agent as part of the virtual computer, which enables command-line capabilities for code execution, file manipulation, data analysis and running scripts to generate outputs such as spreadsheets and presentations.","accessed":"2026-06-17"},{"type":"tutorial","title":"ChatGPT agents explained","url":"https://www.techtarget.com/whatis/feature/ChatGPT-agents-explained","quote":"In those cases, users should enable the ChatGPT agent takeover mode, in which they take over and maintain control of inputting sensitive information via the visual browser, leaving the agent out of the loop.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"human-in-the-loop","role":"first-class","note":"The agent is trained to stop and request explicit user permission before taking consequential actions such as purchases or sending email, and the user can interrupt or take over at any point.","evidence":[{"type":"tutorial","title":"ChatGPT agents explained","url":"https://www.techtarget.com/whatis/feature/ChatGPT-agents-explained","quote":"For example, the system will request permission before taking consequential actions, such as purchases or sending emails.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"browser-agent","role":"first-class","note":"The agent navigates the web with both a text browser that scans page content and a visual browser that clicks, scrolls, fills forms, and navigates UI elements, so it completes tasks on sites that have no API.","evidence":[{"type":"tutorial","title":"ChatGPT agents explained","url":"https://www.techtarget.com/whatis/feature/ChatGPT-agents-explained","quote":"ChatGPT agents integrate both a text browser and a visual web browser. The text browser scans through text content on websites while the visual browser interacts with website user interfaces through clicking, scrolling, filling forms and navigating visual elements.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"code-execution","role":"supported","note":"A terminal in the virtual computer lets the agent run code, manipulate files, do data analysis, and run scripts to produce artefacts such as spreadsheets and presentations.","evidence":[{"type":"tutorial","title":"ChatGPT agents explained","url":"https://www.techtarget.com/whatis/feature/ChatGPT-agents-explained","quote":"Terminal access is provided to the agent as part of the virtual computer, which enables command-line capabilities for code execution, file manipulation, data analysis and running scripts to generate outputs such as spreadsheets and presentations.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"tool-use","role":"supported","note":"Through ChatGPT connectors the agent reads from external applications — Gmail, Google Drive, GitHub, Google Calendar, SharePoint — pulling task-relevant data from connected services rather than only the browser.","evidence":[{"type":"tutorial","title":"ChatGPT agents explained","url":"https://www.techtarget.com/whatis/feature/ChatGPT-agents-explained","quote":"The agent feature also benefits from ChatGPT connectors, which integrate with services such as Gmail, Google Drive, GitHub, Google Calendar and SharePoint.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"Virtual computer","summary":"An isolated environment with a browser, terminal, and file system whose state is shared across tools, so the agent can carry intermediate results from one step to the next within a single task.","maps_to_pattern":"computer-use","url":"https://www.techtarget.com/whatis/feature/ChatGPT-agents-explained"},{"name":"Takeover mode","summary":"A control in which the user takes over the visual browser to input sensitive information directly, leaving the agent out of the loop for that step.","maps_to_pattern":"human-in-the-loop","url":"https://www.techtarget.com/whatis/feature/ChatGPT-agents-explained"},{"name":"Connectors","summary":"Integrations that let the agent read from connected apps such as Gmail, Google Drive, and GitHub so it can use information from those services in a task.","maps_to_pattern":"tool-use","url":"https://www.techtarget.com/whatis/feature/ChatGPT-agents-explained"},{"name":"Deep research","summary":"The web-research capability, carried over from OpenAI's Deep Research, that conducts extensive multi-step searches and synthesises the findings into a report or structured output.","url":"https://www.techtarget.com/whatis/feature/ChatGPT-agents-explained"}],"references":[{"type":"blog","title":"Introducing ChatGPT agent: bridging research and action (OpenAI)","url":"https://openai.com/index/introducing-chatgpt-agent/","accessed":"2026-06-17"},{"type":"tutorial","title":"ChatGPT agents explained: Everything you need to know (TechTarget)","url":"https://www.techtarget.com/whatis/feature/ChatGPT-agents-explained","quote":"At the most basic level, ChatGPT agents include a combination of components initially developed for OpenAI's Deep Research and Operator technologies.","accessed":"2026-06-17"},{"type":"doc","title":"ChatGPT agent (OpenAI Help Center)","url":"https://help.openai.com/en/articles/11752874-chatgpt-agent","accessed":"2026-06-17"}],"alternatives":[{"composition":"openai-operator","relation":"predecessor","note":"Operator was OpenAI's standalone browser-operating agent; ChatGPT agent folds Operator's visual-browser capability into a broader virtual computer with a terminal and connectors."},{"composition":"anthropic-computer-use","relation":"competes-with","note":"Both let a model operate a computer through screenshots and human-like actions; ChatGPT agent ships as a closed end-user product, while Anthropic Computer Use is an API capability developers build on."},{"composition":"manus","relation":"similar-shape","note":"Both are general-purpose autonomous agents that run multi-step tasks on a sandboxed virtual machine with a browser and terminal, with the user able to intervene."}],"verification_status":"partial","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"ChatGPT agent\"]\n  fw --> p1[\"Computer Use<br/>(core)\"]\n  fw --> p2[\"Human-in-the-Loop<br/>(first-class)\"]\n  fw --> p3[\"Browser Agent<br/>(first-class)\"]\n  fw --> p4[\"Code Execution<br/>(supported)\"]\n  fw --> p5[\"Tool Use<br/>(supported)\"]","tags":["browser-computer-use","tool-use-environment","safety-control","closed-application"]},{"id":"aider","name":"Aider","kind":"framework","category":"coding-agent","build_surface":"full-code","vendor":"Paul Gauthier / Aider-AI community","language":"Python","license":"Apache-2.0","status":"active","first_released":"2023-05-09","url":"https://aider.chat","repo":"https://github.com/Aider-AI/aider","docs_url":"https://aider.chat/docs/","intent":"Open-source terminal-native AI pair-programmer that edits files in a git repo through diff-formatted edits, auto-commits each change, and works with almost any LLM provider.","description":"Aider runs in the terminal, takes natural-language requests, and applies them as diffs to files in your git repo. Every edit is auto-committed with a descriptive message so you can review via standard git tools and use /undo to roll back. A concise repo map of classes/functions is sent with each request so the model has whole-codebase awareness. Aider supports cloud and local models, prompt caching on supported providers, voice input, image input, automatic lint+test loops, and an 'architect' mode that splits planning from editing across two models.","primary_use_cases":["terminal-native AI pair-programming in a git repo","diff-formatted edits with auto-commit and /undo","architect+editor multi-model splits","auto-lint and auto-test feedback loops"],"agent_loop_shape":"Single-agent loop built around git as substrate. User types a request → Aider consults the repo-map → selects an edit format (whole-file, diff, udiff, search-replace) appropriate to the model → emits edits → applies them to files → optionally runs --lint-cmd and --test-cmd → tries to fix any non-zero exit → commits to git. /undo rewinds the last commit. Architect mode runs a planning model first, then hands the proposal to a faster editor model.","key_concepts":[{"name":"Edit formats","summary":"Model emits diffs (whole-file / diff / udiff / search-replace) Aider parses and applies.","url":"https://aider.chat/docs/usage.html"},{"name":"Repo-map","summary":"Graph-optimised concise map of the whole repo sent to the LLM.","maps_to_pattern":"agentic-rag","url":"https://aider.chat/docs/repomap.html"},{"name":"Git as memory","summary":"Every edit is a commit; /undo is git revert.","maps_to_pattern":"replay-time-travel","url":"https://aider.chat/docs/git.html"},{"name":"Modes (code/ask/architect/help)","summary":"Architect+editor split is the multi-model loop.","maps_to_pattern":"multi-model-routing","url":"https://aider.chat/docs/usage/modes.html"},{"name":"Prompt caching","summary":"Caches system prompt, read-only files, repo map, editable files.","maps_to_pattern":"prompt-caching","url":"https://aider.chat/docs/usage/caching.html"},{"name":"Auto lint + auto test","summary":"Automatic feedback loop after every edit.","maps_to_pattern":"code-execution","url":"https://aider.chat/docs/usage/lint-test.html"}],"pattern_composition":"flowchart TD\n  user[User request] --> repomap[Repo-map sent to LLM]\n  repomap --> mode{Mode}\n  mode -->|code| llm[LLM emits diff]\n  mode -->|architect| arch[Architect LLM plan]\n  arch --> edit[Editor LLM emits diff]\n  edit --> llm\n  llm --> apply[Apply diff to files]\n  apply --> lint[Auto-lint]\n  lint --> test[Auto-test]\n  test --> fix{Errors?}\n  fix -->|yes| llm\n  fix -->|no| commit[Git auto-commit]\n  commit --> done[Ready]\n  done -.->|/undo| revert[Git revert]","members":[{"pattern":"agent-computer-interface","role":"first-class","note":"Terminal-native ACI; also IDE-aware via comment-driven prompts.","evidence":[{"type":"repo","title":"Aider README — terminal","url":"https://github.com/Aider-AI/aider","quote":"aider is AI pair programming in your terminal","accessed":"2026-05-20"},{"type":"repo","title":"Aider README — IDE-aware","url":"https://github.com/Aider-AI/aider","quote":"Use aider from within your favorite IDE or editor. Ask for changes by adding comments to your code and aider will get to work.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"agentic-rag","role":"limited","note":"Repo-map is graph-based retrieval (not vector RAG); concise map of classes/functions sent with each request.","evidence":[{"type":"doc","title":"Aider docs — repo-map","url":"https://aider.chat/docs/repomap.html","quote":"Aider uses a concise map of your whole git repository that includes the most important classes and functions along with their types and call signatures.","accessed":"2026-05-20"},{"type":"doc","title":"Aider docs — graph-ranking retrieval","url":"https://aider.chat/docs/repomap.html","quote":"Aider solves this problem by sending just the most relevant portions of the repo map. It does this by analyzing the full repo map using a graph ranking algorithm, computed on a graph where each source file is a node and edges connect files which have dependencies.","accessed":"2026-05-24"},{"type":"doc","title":"Aider docs — repo-map sent with each request","url":"https://aider.chat/docs/repomap.html","quote":"Aider sends a repo map to the LLM along with each change request from the user. The repo map contains a list of the files in the repo, along with the key symbols which are defined in each file.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"code-as-action","role":"first-class","note":"Edits are the action; every edit auto-commits to git.","evidence":[{"type":"doc","title":"Aider docs — auto-commit","url":"https://aider.chat/docs/usage.html","quote":"Aider will git commit all of its changes, so they are easy to track and undo.","accessed":"2026-05-20"},{"type":"doc","title":"Aider docs — descriptive commits","url":"https://aider.chat/docs/git.html","quote":"Whenever aider edits a file, it commits those changes with a descriptive commit message.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"code-execution","role":"first-class","note":"Auto-lint + auto-test feedback loop; non-zero exits drive retry.","evidence":[{"type":"repo","title":"Aider README — lint/test","url":"https://github.com/Aider-AI/aider","quote":"Automatically lint and test your code every time aider makes changes. Aider can fix problems detected by your linters and test suites.","accessed":"2026-05-20"},{"type":"doc","title":"Aider docs — auto-fix on error","url":"https://aider.chat/docs/usage/lint-test.html","quote":"Aider will try and fix any errors if the command returns a non-zero exit code","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"multi-model-routing","role":"first-class","note":"/model switches in-chat; architect+editor mode is a two-model split.","evidence":[{"type":"doc","title":"Aider docs — /model","url":"https://aider.chat/docs/usage.html","quote":"During your chat you can switch models with the in-chat `/model` command.","accessed":"2026-05-20"},{"type":"doc","title":"Aider docs — architect mode","url":"https://aider.chat/docs/usage/modes.html","quote":"Like code mode, aider will change your files. An architect model will propose changes and an editor model will translate that proposal into specific file edits.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"plan-and-execute","role":"limited","note":"Architect+editor split is a one-step plan, not multi-step plan-and-execute.","evidence":[{"type":"doc","title":"Aider docs — architect mode","url":"https://aider.chat/docs/usage/modes.html","quote":"An architect model will propose changes and an editor model will translate that proposal into specific file edits.","accessed":"2026-05-20"},{"type":"doc","title":"Aider docs — split rationale","url":"https://aider.chat/docs/usage/modes.html","quote":"Certain LLMs aren't able to propose coding solutions and specify detailed file edits all in one go.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"prompt-caching","role":"first-class","note":"Caches system prompt, read-only files, repo map, editable files.","evidence":[{"type":"doc","title":"Aider docs — prompt caching","url":"https://aider.chat/docs/usage/caching.html","quote":"Aider supports prompt caching for cost savings and faster coding.","accessed":"2026-05-20"},{"type":"doc","title":"Aider docs — caching scope","url":"https://aider.chat/docs/usage/caching.html","quote":"Aider organizes the chat history to try and cache: The system prompt. Read only files added with `--read` or `/read-only`. The repository map. The editable files that have been added to the chat.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"limited","note":"Honest downgrade — Aider's loop is respond+apply+optionally retry on lint/test feedback, not a full ReAct trace; no upstream text names ReAct.","evidence":[{"type":"doc","title":"Aider docs — retry on error","url":"https://aider.chat/docs/usage/lint-test.html","quote":"Aider will try and fix any errors if the command returns a non-zero exit code","accessed":"2026-05-20"},{"type":"doc","title":"Aider docs — auto lint/test feedback loop","url":"https://aider.chat/docs/usage/lint-test.html","quote":"Aider can automatically lint and test your code every time it makes changes. This helps identify and repair any problems introduced by the AI edits.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"replay-time-travel","role":"first-class","note":"/undo rewinds the last commit; standard git tools work.","evidence":[{"type":"doc","title":"Aider docs — /undo","url":"https://aider.chat/docs/usage.html","quote":"You can always use the `/undo` command to undo AI changes that you don't like.","accessed":"2026-05-20"},{"type":"repo","title":"Aider README — git tools","url":"https://github.com/Aider-AI/aider","quote":"Use familiar git tools to easily diff, manage and undo AI changes.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"step-budget","role":"limited","note":"Soft token-budget knob (--max-chat-history-tokens) triggers summarization; --map-tokens caps repo-map size. No documented step-count cap, so honest 'limited'.","evidence":[{"type":"doc","title":"Aider docs — max-chat-history-tokens","url":"https://aider.chat/docs/config/options.html","quote":"Soft limit on tokens for chat history, after which summarization begins. If unspecified, defaults to the model's max_chat_history_tokens.","accessed":"2026-05-24"},{"type":"doc","title":"Aider docs — repo-map token budget","url":"https://aider.chat/docs/repomap.html","quote":"The token budget is influenced by the `--map-tokens` switch, which defaults to 1k tokens. Aider adjusts the size of the repo map dynamically based on the state of the chat.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"limited","note":"Aider predates modern tool-calling — its tools are slash commands the user/model invoke: /run, /test, /commit, /undo, /add — not LLM tool-calls.","evidence":[{"type":"doc","title":"Aider docs — /run shell tool","url":"https://aider.chat/docs/usage/commands.html","quote":"Run a shell command and optionally add the output to the chat (alias: !)","accessed":"2026-05-24"},{"type":"doc","title":"Aider docs — /test feedback tool","url":"https://aider.chat/docs/usage/commands.html","quote":"Run a shell command and add the output to the chat on non-zero exit code","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"repo-map-context","role":"first-class","note":"Aider's signature feature: a PageRank-ranked tree-sitter map of the repo sent with each change request.","evidence":[{"type":"doc","title":"Aider docs — repo-map","url":"https://aider.chat/docs/repomap.html","quote":"Aider uses a concise map of your whole git repository that includes the most important classes and functions along with their types and call signatures.","accessed":"2026-06-14"}],"evidence_status":"full"},{"pattern":"behavior-pinning-test-before-agent-edit","role":"supported","note":"Aider can run a configured test suite automatically after every edit and treat a non-zero exit as a failure to fix, supplying the regression-gate half of the pattern (the user still authors the pre-edit characterization tests).","evidence":[{"type":"doc","title":"Linting and testing | aider","url":"https://aider.chat/docs/usage/lint-test.html","quote":"You can configure aider to run your test suite after each time the AI edits your code using the `--test-cmd <test-command>` and `--auto-test` switch.","accessed":"2026-06-14"}],"evidence_status":"full"},{"pattern":"dry-run-harness","role":"opt-in","note":"Aider exposes a --dry-run flag that runs the edit loop without modifying files, letting the planned changes be inspected before any are committed.","evidence":[{"type":"doc","title":"Options reference | aider","url":"https://aider.chat/docs/config/options.html","quote":"Perform a dry run without modifying files (default: False)","accessed":"2026-06-17"}],"evidence_status":"limited"}],"instantiates":["modern-coding-agent","routing-and-fallback","reflection-and-self-correction"],"alternatives":[{"composition":"claude-code","relation":"competes-with","note":"Peer terminal agent with broader feature set."},{"composition":"codex-cli","relation":"competes-with","note":"Peer terminal agent from OpenAI."},{"composition":"cline","relation":"competes-with","note":"Peer IDE+CLI agent."},{"composition":"continue-dev","relation":"competes-with","note":"Peer IDE extension."},{"composition":"opencode","relation":"competes-with","note":"Peer open-source terminal coding agent."},{"composition":"codefuse","relation":"similar-shape","note":"Aider is a simpler single-agent coding loop; CodeFuse is multi-agent SDLC."},{"composition":"gpt-engineer","relation":"competes-with","note":"README directs users to aider for a maintained hackable CLI."},{"composition":"plandex","relation":"competes-with","note":"Peer terminal coding agent, different sandbox model (git auto-commit vs tentative-diff)."}],"references":[{"type":"repo","title":"Aider-AI/aider","url":"https://github.com/Aider-AI/aider","quote":"aider is AI pair programming in your terminal","accessed":"2026-05-20"},{"type":"doc","title":"Aider docs","url":"https://aider.chat/docs/","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","python","terminal","git-native","diff-edits"]},{"id":"boltnew","name":"bolt.new","kind":"framework","category":"coding-agent","build_surface":"no-code","vendor":"StackBlitz","language":"TypeScript (WebContainer)","license":"MIT","status":"active","first_released":"2024-10-03","url":"https://bolt.new/","repo":"https://github.com/stackblitz/bolt.new","docs_url":"https://github.com/stackblitz/bolt.new","intent":"Browser-hosted AI coding agent that prompts, runs, edits, and deploys full-stack web apps inside a StackBlitz WebContainer, giving the model complete control over filesystem, Node server, package manager, terminal, and browser console.","description":"bolt.new is StackBlitz's prompt-to-app product. It pairs a frontier LLM with a WebContainer — a Node.js runtime running entirely in the browser — so the agent can install npm packages, run dev servers, edit files, and deploy without provisioning a remote VM. The model is given complete control of the in-browser environment; the user iterates via chat, with diffs visible and a one-click deploy path. The repo hosts the open-source app shell behind bolt.new; the hosted product additionally bundles auth, database, hosting, and design-system imports.","primary_use_cases":["browser-based prompt-to-app for full-stack web projects","in-browser Node.js dev environment with no local setup","rapid prototyping with one-click deploy and shareable URLs","AI-driven npm install, dev server, and code editing inside the page"],"agent_loop_shape":"Single-agent chat-driven loop where the model is wired to a WebContainer instance. Each turn the agent emits file edits and/or shell actions; the WebContainer applies them, installs packages, starts the dev server, and streams output back to the agent. Errors and console output are fed back into the next turn. Deploy and share are explicit tool actions exposed in the same conversation surface.","key_concepts":[{"name":"WebContainer","summary":"In-browser Node.js runtime that hosts the project, package manager, and terminal.","maps_to_pattern":"code-execution","url":"https://github.com/stackblitz/bolt.new"},{"name":"Full-environment AI control","summary":"Model has complete control over filesystem, Node server, package manager, terminal, and browser console.","maps_to_pattern":"tool-use","url":"https://github.com/stackblitz/bolt.new"},{"name":"Prompt-to-app","summary":"Chat-driven generation of a working full-stack project from a natural-language prompt.","maps_to_pattern":"spec-driven-loop","url":"https://bolt.new/"},{"name":"Deploy from chat","summary":"One-click deploy and shareable URL from the same chat surface.","url":"https://github.com/stackblitz/bolt.new"}],"pattern_composition":"flowchart TD\n  user[User prompt] --> agent[LLM coding agent]\n  agent --> wc[WebContainer]\n  wc --> fs[Filesystem]\n  wc --> npm[npm / package manager]\n  wc --> term[Terminal]\n  wc --> node[Node server]\n  wc --> console[Browser console]\n  fs --> obs[Observed output]\n  term --> obs\n  node --> obs\n  console --> obs\n  obs --> agent\n  agent --> deploy[Deploy / share URL]","members":[{"pattern":"code-execution","role":"first-class","note":"WebContainer is an in-browser Node runtime; the agent runs npm tools, dev servers, and shells inside it.","evidence":[{"type":"repo","title":"bolt.new README — in-browser environment","url":"https://github.com/stackblitz/bolt.new","quote":"in-browser development environment powered by StackBlitz's WebContainers","accessed":"2026-05-20"},{"type":"repo","title":"bolt.new README — run npm / Node","url":"https://github.com/stackblitz/bolt.new","quote":"Install and run npm tools and libraries (like Vite, Next.js, and more)","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"spec-driven-loop","role":"first-class","note":"Whole product is a chat-to-app loop: prompt becomes a running full-stack project.","evidence":[{"type":"repo","title":"bolt.new README — prompt to deploy","url":"https://github.com/stackblitz/bolt.new","quote":"Prompt, run, edit, and deploy full-stack web applications","accessed":"2026-05-20"},{"type":"repo","title":"bolt.new README — full app lifecycle","url":"https://github.com/stackblitz/bolt.new","quote":"empowers AI agents to handle the entire app lifecycle—from creation to deployment","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Agent has explicit tools over filesystem, Node server, package manager, terminal, browser console.","evidence":[{"type":"repo","title":"bolt.new README — environment control","url":"https://github.com/stackblitz/bolt.new","quote":"AI models complete control over the entire environment including the filesystem, node server, package manager, terminal, and browser console.","accessed":"2026-05-20"},{"type":"repo","title":"bolt.new README — install npm tools","url":"https://github.com/stackblitz/bolt.new","quote":"Install and run npm tools and libraries (like Vite, Next.js, and more)","accessed":"2026-05-24"},{"type":"repo","title":"bolt.new README — interact with APIs","url":"https://github.com/stackblitz/bolt.new","quote":"Interact with third-party APIs","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"sandbox-isolation","role":"first-class","note":"WebContainer is a per-tab in-browser sandbox; the agent cannot reach the host OS.","evidence":[{"type":"repo","title":"bolt.new README — WebContainer sandbox","url":"https://github.com/stackblitz/bolt.new","quote":"in-browser development environment powered by StackBlitz's WebContainers","accessed":"2026-05-20"},{"type":"doc","title":"bolt.new support — WebContainer environment","url":"https://support.bolt.new/building/intro-bolt","quote":"Bolt uses StackBlitz's [WebContainers](https://webcontainers.io) to provide the development environment.","accessed":"2026-05-24"}],"evidence_status":"full"}],"instantiates":["modern-coding-agent"],"alternatives":[{"composition":"lovable","relation":"competes-with","note":"Peer browser-based prompt-to-app."},{"composition":"v0","relation":"competes-with","note":"Peer prompt-to-app from Vercel."},{"composition":"replit-agent","relation":"competes-with","note":"Peer hosted prompt-to-app on Replit."}],"references":[{"type":"repo","title":"stackblitz/bolt.new","url":"https://github.com/stackblitz/bolt.new","quote":"AI-powered web development agent that allows you to prompt, run, edit, and deploy full-stack applications directly from your browser","accessed":"2026-05-20"},{"type":"doc","title":"bolt.new homepage","url":"https://bolt.new/","quote":"Create stunning apps & websites by chatting with AI","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"mature","tags":["web-product","webcontainer","prompt-to-app","in-browser"]},{"id":"claude-code","name":"Claude Code","kind":"framework","category":"coding-agent","build_surface":"full-code","vendor":"Anthropic","language":"TypeScript, Shell","license":"proprietary (closed-source CLI; no LICENSE in repo)","status":"active","first_released":"2025-02-22","url":"https://code.claude.com","repo":"https://github.com/anthropics/claude-code","docs_url":"https://code.claude.com/docs/en/overview","intent":"Anthropic's first-party agentic coding tool — a single CLI/IDE/desktop/web surface that turns Claude into a tool-using engineer with persistent project memory, structured subagent delegation, hooks, skills, and scheduled cloud routines.","description":"Claude Code lives in the terminal, IDE, desktop app, and browser, sharing the same engine across surfaces. It reads the repository, edits files, runs commands, drives git, and exposes a documented agentic loop with explicit primitives for memory (CLAUDE.md, auto memory), skills, subagents, hooks, MCP, plan mode, todo lists, and scheduled routines. Anthropic positions it as both a developer tool and the reference implementation of an autonomous coding agent (the Claude Agent SDK is built on the same engine).","primary_use_cases":["agentic coding across terminal/IDE/desktop/web with shared engine","long-running autonomous coding sessions with checkpointed JSONL transcripts","scheduled or webhook-triggered Routines on managed cloud","subagent-delegated coding tasks in isolated contexts"],"agent_loop_shape":"ReAct-style think→act loop where Claude reasons, calls a tool (Read/Edit/Bash/etc. or MCP tool), observes the result, and continues until done. Plan mode runs a read-only exploration first; once accepted, execution runs the tool-use loop with checkpointing, todo tracking, and optional subagent forking. Hooks fire at SessionStart, UserPromptSubmit, PreToolUse, PostToolUse, Stop, SessionEnd. Sessions persist as JSONL under ~/.claude/projects/ and can be resumed, branched, or forked.","key_concepts":[{"name":"CLAUDE.md memory","summary":"Persistent project/user instructions, read at session start.","maps_to_pattern":"cross-session-memory","url":"https://code.claude.com/docs/en/memory"},{"name":"Skills","summary":"Packaged SKILL.md workflows loaded on demand, replacing custom commands.","maps_to_pattern":"agent-skills","url":"https://code.claude.com/docs/en/skills"},{"name":"Subagents","summary":"Specialised agents in isolated context windows, delegated to via description.","maps_to_pattern":"subagent-isolation","url":"https://code.claude.com/docs/en/sub-agents"},{"name":"Hooks","summary":"Shell/HTTP/LLM handlers that fire on lifecycle events.","maps_to_pattern":"event-driven-agent","url":"https://code.claude.com/docs/en/hooks"},{"name":"MCP","summary":"Connect external tools/data via Model Context Protocol.","maps_to_pattern":"mcp","url":"https://code.claude.com/docs/en/mcp"},{"name":"Routines","summary":"Cloud-scheduled or webhook-triggered Claude Code sessions on Anthropic infra.","maps_to_pattern":"scheduled-agent","url":"https://code.claude.com/docs/en/routines"}],"pattern_composition":"flowchart TD\n  user[User prompt] --> memory[CLAUDE.md + auto memory]\n  memory --> plan{Plan mode?}\n  plan -->|yes| explore[Read-only exploration]\n  explore --> accept[Plan accepted]\n  accept --> loop\n  plan -->|no| loop[ReAct tool-use loop]\n  loop --> tool[Tool call]\n  tool --> hookpre[PreToolUse hook]\n  hookpre --> exec[Execute: Read/Edit/Bash/MCP]\n  exec --> hookpost[PostToolUse hook]\n  hookpost --> obs[Observe result]\n  obs --> delegate{Delegate?}\n  delegate -->|yes| sub[Subagent isolated ctx]\n  sub --> obs\n  delegate -->|no| loop\n  loop --> done[Session JSONL]\n  done --> resume[claude --resume]\n  routines[Scheduled routines] -.fires.-> loop","members":[{"pattern":"agent-computer-interface","role":"first-class","note":"Single ACI across terminal, IDE, desktop, and browser surfaces; shared engine + CLAUDE.md + MCP across all.","evidence":[{"type":"doc","title":"Claude Code overview","url":"https://code.claude.com/docs/en/overview","quote":"Claude Code is an agentic coding tool that reads your codebase, edits files, runs commands, and integrates with your development tools. Available in your terminal, IDE, desktop app, and browser.","accessed":"2026-05-20"},{"type":"doc","title":"Claude Code overview — shared engine across surfaces","url":"https://code.claude.com/docs/en/overview","quote":"Each surface connects to the same underlying Claude Code engine, so your CLAUDE.md files, settings, and MCP servers work across all of them.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"agent-resumption","role":"first-class","note":"Sessions saved continuously to local JSONL transcripts; resumable via --continue or --resume.","evidence":[{"type":"doc","title":"Claude Code sessions","url":"https://code.claude.com/docs/en/sessions","quote":"Sessions are saved continuously to local transcript files as you work, so you can return to one after exiting or running `/clear`.","accessed":"2026-05-20"},{"type":"doc","title":"Claude Code --continue","url":"https://code.claude.com/docs/en/sessions","quote":"`claude --continue` Resumes the most recent session in the current directory","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"agent-skills","role":"first-class","note":"SKILL.md files extend Claude's toolkit; conforms to the Agent Skills open standard.","evidence":[{"type":"doc","title":"Claude Code Skills","url":"https://code.claude.com/docs/en/skills","quote":"Skills extend what Claude can do. Create a `SKILL.md` file with instructions, and Claude adds it to its toolkit. Claude uses skills when relevant, or you can invoke one directly with `/skill-name`.","accessed":"2026-05-20"},{"type":"doc","title":"Claude Code Skills — open standard","url":"https://code.claude.com/docs/en/skills","quote":"Claude Code skills follow the Agent Skills open standard, which works across multiple AI tools.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"approval-queue","role":"first-class","note":"Permission rules gate skill/tool execution; allowed-tools pre-approves; deny rules block; managed settings can enforce centrally.","evidence":[{"type":"doc","title":"Claude Code Skills — pre-approve tools","url":"https://code.claude.com/docs/en/skills","quote":"The `allowed-tools` field grants permission for the listed tools while the skill is active, so Claude can use them without prompting you for approval.","accessed":"2026-05-24"},{"type":"doc","title":"Claude Code Skills — deny rules","url":"https://code.claude.com/docs/en/skills","quote":"To block a skill from using certain tools, add deny rules in your permission settings instead.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"cross-session-memory","role":"first-class","note":"CLAUDE.md (user-authored) + auto memory (agent-authored) carry knowledge across sessions; both load at the start of every conversation.","evidence":[{"type":"doc","title":"Claude Code memory","url":"https://code.claude.com/docs/en/memory","quote":"Each Claude Code session begins with a fresh context window. Two mechanisms carry knowledge across sessions: CLAUDE.md files: instructions you write to give Claude persistent context. Auto memory: notes Claude writes itself based on your corrections and preferences.","accessed":"2026-05-20"},{"type":"doc","title":"Claude Code memory — auto memory storage","url":"https://code.claude.com/docs/en/memory","quote":"Each project gets its own memory directory at `~/.claude/projects/<project>/memory/`. The `<project>` path is derived from the git repository, so all worktrees and subdirectories within the same repo share one auto memory directory.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"event-driven-agent","role":"first-class","note":"Hooks fire on lifecycle events with JSON context passed to user-defined handlers.","evidence":[{"type":"doc","title":"Claude Code hooks","url":"https://code.claude.com/docs/en/hooks","quote":"Hooks are user-defined shell commands, HTTP endpoints, or LLM prompts that execute automatically at specific points in Claude Code's lifecycle.","accessed":"2026-05-20"},{"type":"doc","title":"Claude Code hooks — JSON context","url":"https://code.claude.com/docs/en/hooks","quote":"Hooks fire at specific points during a Claude Code session. When an event fires and a matcher matches, Claude Code passes JSON context about the event to your hook handler.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"extended-thinking","role":"first-class","note":"Extended thinking + `ultrathink` trigger phrase + effort levels.","evidence":[{"type":"doc","title":"Claude Code model-config — extended thinking","url":"https://code.claude.com/docs/en/model-config","quote":"Extended thinking is the reasoning Claude emits before responding.","accessed":"2026-05-20"},{"type":"doc","title":"Claude Code model-config — ultrathink","url":"https://code.claude.com/docs/en/model-config","quote":"Include `ultrathink` anywhere in your prompt to request deeper reasoning on that turn without changing your session effort setting.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"Full MCP client: stdio/SSE/HTTP transports, three scopes (local/project/user), OAuth, dynamic tool updates, automatic reconnect, tool search defaults on.","evidence":[{"type":"doc","title":"Claude Code MCP","url":"https://code.claude.com/docs/en/mcp","quote":"Claude Code can connect to hundreds of external tools and data sources through the Model Context Protocol (MCP), an open source standard for AI-tool integrations.","accessed":"2026-05-20"},{"type":"doc","title":"Claude Code MCP — dynamic tool updates","url":"https://code.claude.com/docs/en/mcp","quote":"Claude Code supports MCP `list_changed` notifications, allowing MCP servers to dynamically update their available tools, prompts, and resources without requiring you to disconnect and reconnect.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"multi-model-routing","role":"first-class","note":"opusplan alias splits Opus (plan) / Sonnet (execute); skill frontmatter `model` field overrides per skill; supports Anthropic/Bedrock/Vertex/Foundry providers.","evidence":[{"type":"doc","title":"Claude Code model-config — opusplan","url":"https://code.claude.com/docs/en/model-config","quote":"The `opusplan` model alias provides an automated hybrid approach: In plan mode - Uses `opus` for complex reasoning and architecture decisions; In execution mode - Automatically switches to `sonnet` for code generation and implementation.","accessed":"2026-05-20"},{"type":"doc","title":"Claude Code Skills — per-skill model override","url":"https://code.claude.com/docs/en/skills","quote":"Model to use when this skill is active. The override applies for the rest of the current turn and is not saved to settings; the session model resumes on your next prompt.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"plan-and-execute","role":"first-class","note":"Plan mode runs a read-only exploration; on accept, execution mode runs the tool loop.","evidence":[{"type":"doc","title":"Claude Code model-config — plan-mode context","url":"https://code.claude.com/docs/en/model-config","quote":"The plan-mode Opus phase runs with the standard 200K context window.","accessed":"2026-05-20"},{"type":"doc","title":"Claude Code sessions — plan accept","url":"https://code.claude.com/docs/en/sessions","quote":"Accepting a plan in plan mode names the session from the plan content unless you've already set one","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"ReAct shape is implicit throughout — Claude plans, writes, verifies; bug-fix loop traces issue → identifies root cause → implements fix.","evidence":[{"type":"doc","title":"Claude Code overview — plan/write/verify loop","url":"https://code.claude.com/docs/en/overview","quote":"Describe what you want in plain language. Claude Code plans the approach, writes the code across multiple files, and verifies it works.","accessed":"2026-05-20"},{"type":"doc","title":"Claude Code overview — debugging loop","url":"https://code.claude.com/docs/en/overview","quote":"For bugs, paste an error message or describe the symptom. Claude Code traces the issue through your codebase, identifies the root cause, and implements a fix.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"scheduled-agent","role":"first-class","note":"Routines run on Anthropic-managed infra; cron, API, or GitHub-event triggered.","evidence":[{"type":"doc","title":"Claude Code overview — routines","url":"https://code.claude.com/docs/en/overview","quote":"Routines run on Anthropic-managed infrastructure, so they keep running even when your computer is off. They can also trigger on API calls or GitHub events.","accessed":"2026-05-20"},{"type":"doc","title":"Claude Code routines","url":"https://code.claude.com/docs/en/routines","quote":"A routine is a saved Claude Code configuration: a prompt, one or more repositories, and a set of connectors, packaged once and run automatically.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"step-budget","role":"limited","note":"Effort-level scale (low/medium/high/xhigh/max) is a token/cost budget; MAX_MCP_OUTPUT_TOKENS and tool-search threshold are explicit context budgets; no documented step-count cap.","evidence":[{"type":"doc","title":"Claude Code model-config — effort","url":"https://code.claude.com/docs/en/model-config","quote":"Each level trades token spend against capability.","accessed":"2026-05-20"},{"type":"doc","title":"Claude Code MCP — MAX_MCP_OUTPUT_TOKENS","url":"https://code.claude.com/docs/en/mcp","quote":"Output warning threshold: Claude Code displays a warning when any MCP tool output exceeds 10,000 tokens","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"subagent-isolation","role":"first-class","note":"Each subagent runs in its own context window with custom prompt, tool access, and permissions.","evidence":[{"type":"doc","title":"Claude Code subagents — isolation","url":"https://code.claude.com/docs/en/sub-agents","quote":"Each subagent runs in its own context window with a custom system prompt, specific tool access, and independent permissions.","accessed":"2026-05-20"},{"type":"doc","title":"Claude Code subagents — purpose","url":"https://code.claude.com/docs/en/sub-agents","quote":"Subagents help you: Preserve context by keeping exploration and implementation out of your main conversation; Enforce constraints by limiting which tools a subagent can use","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"todo-list-driven-agent","role":"first-class","note":"Built-in TaskCreate/TaskGet/TaskList/TaskUpdate tools manage the session task checklist; TodoWrite is the prior primitive, still toggleable.","evidence":[{"type":"doc","title":"Claude Code tools-reference — TaskCreate","url":"https://code.claude.com/docs/en/tools-reference","quote":"TaskCreate Creates a new task in the task list","accessed":"2026-05-24"},{"type":"doc","title":"Claude Code tools-reference — TodoWrite","url":"https://code.claude.com/docs/en/tools-reference","quote":"TodoWrite Manages the session task checklist. Disabled by default as of v2.1.142 in favor of `TaskCreate`, `TaskGet`, `TaskList`, and `TaskUpdate`.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-result-caching","role":"limited","note":"Honest downgrade — Claude Code uses prompt caching (system prompt + tool defs + history) and WebFetch 15-min cache, not tool-result deduplication.","evidence":[{"type":"doc","title":"Claude Code model-config — prompt caching","url":"https://code.claude.com/docs/en/model-config","quote":"Claude Code automatically uses prompt caching to optimize performance and reduce costs.","accessed":"2026-05-20"},{"type":"doc","title":"Claude Code tools-reference — WebFetch 15-min cache","url":"https://code.claude.com/docs/en/tools-reference","quote":"Responses are cached for 15 minutes, so repeated fetches of the same URL return quickly.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Documented tool table: Read/Edit/Bash/Glob/Grep/WebSearch/WebFetch/Monitor/Agent/MCP tools etc., each with permission rules.","evidence":[{"type":"doc","title":"Claude Code tools-reference — built-in tools","url":"https://code.claude.com/docs/en/tools-reference","quote":"Claude Code has access to a set of built-in tools that help it understand and modify your codebase.","accessed":"2026-05-24"},{"type":"doc","title":"Claude Code tools-reference — Bash tool","url":"https://code.claude.com/docs/en/tools-reference","quote":"Bash Executes shell commands in your environment.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"filesystem-as-context","role":"first-class","note":"Maintains a todo list and reads and writes project files as durable working state across a long session rather than holding all of it in the live context window.","evidence":[{"type":"doc","title":"How Claude remembers your project - Claude Code","url":"https://code.claude.com/docs/en/memory","quote":"CLAUDE.md files are markdown files that give Claude persistent instructions for a project, your personal workflow, or your entire organization. You write these files in plain text; Claude reads them at the start of every session.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"guardrail-erosion-through-compaction","role":"first-class","note":"Auto-compaction summarises older conversation turns, and the corrective is exactly this pattern's: keep persistent rules in CLAUDE.md (pinned outside the compactable span and reloaded each session) rather than in the rollable history that compaction rewrites.","evidence":[{"type":"doc","title":"How Claude Code works - The context window","url":"https://code.claude.com/docs/en/how-claude-code-works","quote":"Claude compacts automatically, but instructions from early in the conversation can get lost. Put persistent rules in CLAUDE.md, and run /context to see what's using space.","accessed":"2026-06-14"},{"type":"doc","title":"How Claude Code works - When context fills up","url":"https://code.claude.com/docs/en/how-claude-code-works","quote":"Your requests and key code snippets are preserved; detailed instructions from early in the conversation may be lost. Put persistent rules in CLAUDE.md rather than relying on conversation history.","accessed":"2026-06-14"}],"evidence_status":"full"},{"pattern":"reversibility-aware-action-filter","role":"supported","note":"Classifies tools into reversibility tiers (read-only runs freely, file/Bash writes need approval) and routes destructive one-way commands such as rm -rf / and git push to an ask/circuit-breaker gate the model cannot relabel.","evidence":[{"type":"doc","title":"Configure permissions - Claude Code Docs","url":"https://code.claude.com/docs/en/permissions","quote":"Read-only | File reads, Grep | No ... Bash commands | Shell execution | Yes ... File modification | Edit/write files | Yes","accessed":"2026-06-14"},{"type":"doc","title":"Configure permissions - Claude Code Docs","url":"https://code.claude.com/docs/en/permissions","quote":"removals targeting the filesystem root or home directory, such as `rm -rf /` and `rm -rf ~`, still prompt as a circuit breaker against model error","accessed":"2026-06-14"}],"evidence_status":"full"},{"pattern":"stop-hook","role":"core","note":"Claude Code defines an explicit Stop hook event that fires when the agent finishes responding and can return a blocking decision to override termination and force the agent loop to continue.","evidence":[{"type":"doc","title":"Hooks reference - Claude Code Docs","url":"https://code.claude.com/docs/en/hooks","quote":"When Claude finishes responding","accessed":"2026-06-17"},{"type":"doc","title":"Hooks reference - Claude Code Docs","url":"https://code.claude.com/docs/en/hooks","quote":"Prevents Claude from stopping, continues the conversation","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"dry-run-harness","role":"first-class","note":"Plan mode lets the agent read files and propose a full plan of intended changes while making no edits to disk until the user approves, surfacing a reviewable proposal before any commit.","evidence":[{"type":"doc","title":"Common workflows - Plan before editing","url":"https://code.claude.com/docs/en/common-workflows","quote":"For changes you want to review before they touch disk, switch to plan mode. Claude reads files and proposes a plan but makes no edits until you approve.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"progressive-tool-access","role":"core","note":"Claude Code starts with a tiered least-privilege permission system that prompts on first use of each tool; users grant tools incrementally via allow rules, so the agent's tool access expands only as it is earned rather than being granted upfront.","evidence":[{"type":"doc","title":"Configure permissions - Claude Code Docs","url":"https://code.claude.com/docs/en/permissions","quote":"Claude Code uses a tiered permission system to balance power and safety:","accessed":"2026-06-17"},{"type":"doc","title":"Configure permissions - Claude Code Docs","url":"https://code.claude.com/docs/en/permissions","quote":"Standard behavior: prompts for permission on first use of each tool","accessed":"2026-06-17"}],"evidence_status":"full"}],"instantiates":["modern-coding-agent","planning-loops","long-running-autonomous-agent","multi-agent-coordination","memory-architecture"],"alternatives":[{"composition":"aider","relation":"competes-with","note":"Peer terminal coding agent — single-model loop, no subagents/skills."},{"composition":"codex-cli","relation":"competes-with","note":"Peer first-party (OpenAI) terminal coding agent."},{"composition":"cursor","relation":"competes-with","note":"Peer IDE-first coding agent."},{"composition":"cline","relation":"competes-with","note":"Peer open-source IDE+CLI agent, similar plan/act split."},{"composition":"continue-dev","relation":"competes-with","note":"Peer IDE extension."},{"composition":"claude-agent-sdk","relation":"wraps","note":"The SDK exposing the same engine as a programmable library."},{"composition":"opencode","relation":"competes-with","note":"Peer terminal coding agent (first-party, proprietary)."},{"composition":"google-antigravity","relation":"competes-with","note":"Peer first-party terminal+IDE coding agent."},{"composition":"amp","relation":"competes-with","note":"Peer CLI+IDE coding agent."},{"composition":"factory-droid","relation":"competes-with","note":"Peer terminal coding agent."},{"composition":"warp","relation":"competes-with","note":"Peer terminal coding agent."},{"composition":"devin","relation":"competes-with","note":"Anthropic's first-party agent in a different surface (terminal)."},{"composition":"github-copilot-coding-agent","relation":"complements","note":"Often pairs with foreground terminal agents."},{"composition":"goose","relation":"competes-with","note":"Peer first-party terminal agent."},{"composition":"junie","relation":"competes-with","note":"Peer terminal agent for Junie CLI."},{"composition":"open-interpreter","relation":"competes-with","note":"Peer terminal coding agent."},{"composition":"openhands","relation":"competes-with","note":"Peer agentic coding tool, closed source."},{"composition":"plandex","relation":"competes-with","note":"Peer terminal coding agent."},{"composition":"windsurf","relation":"competes-with","note":"Peer agentic coding tool."}],"references":[{"type":"doc","title":"Claude Code overview","url":"https://code.claude.com/docs/en/overview","accessed":"2026-05-20"},{"type":"repo","title":"anthropics/claude-code","url":"https://github.com/anthropics/claude-code","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-06-19","date_added":"2026-04-30","status_in_practice":"mature","tags":["proprietary","anthropic","cli","ide","subagents","skills","hooks","mcp"],"anti_patterns_avoided":[{"pattern":"self-exfiltration","note":"Sandboxed Bash enforces network isolation so a compromised agent cannot exfiltrate files; non-allowed domains are blocked rather than prompted.","evidence":[{"type":"doc","title":"Claude Code — sandboxing","url":"https://code.claude.com/docs/en/sandboxing","quote":"Effective sandboxing requires both filesystem and network isolation. Without network isolation, a compromised agent could exfiltrate sensitive files like SSH keys.","accessed":"2026-06-19"}]}]},{"id":"cline","name":"Cline","aliases":["Claude Dev"],"kind":"framework","category":"coding-agent","build_surface":"full-code","vendor":"Cline Bot Inc.","language":"TypeScript","license":"Apache-2.0","status":"active","first_released":"2024-07-06","url":"https://cline.bot/","repo":"https://github.com/cline/cline","docs_url":"https://docs.cline.bot","intent":"Open-source coding agent that delivers the same engine across CLI, VS Code, JetBrains, and a Kanban multi-agent board, with explicit human-in-the-loop approval, plan/act mode separation, and a programmable SDK.","description":"Cline is an autonomous coding agent that reads files, writes code, runs terminal commands, browses the web, and uses MCP tools, with every action defaulting to explicit user approval. It separates Plan mode (read-only exploration, no edits) from Act mode (executes the agreed plan). Cline ships as a VS Code extension, JetBrains plugin, terminal CLI (interactive and headless), a Node SDK, and the Kanban web UI for running many agents in parallel with worktrees. Supports most major model providers and ships first-class MCP, scheduled agents, multi-agent teams, and checkpoints backed by a shadow git repo.","primary_use_cases":["approval-gated IDE+CLI coding agent","multi-agent teams via Kanban board with worktrees","scheduled coding agents via cline schedule","headless CI/CD coding via cline CLI"],"agent_loop_shape":"Plan/Act split. Plan mode reads the codebase and discusses strategy — file edits and command execution are blocked. Once the plan is accepted, Act mode carries the same conversation context forward and executes: edits, terminal, browser, MCP tools, all gated by human-in-the-loop approval (or auto-approve). Every step takes a checkpoint into a shadow git repo so the user can restore. CLI supports headless mode and `cline schedule create` for cron-driven agents.","key_concepts":[{"name":"Plan vs Act","summary":"Read-only planning then approval-gated execution.","maps_to_pattern":"plan-and-execute","url":"https://docs.cline.bot/features/plan-and-act"},{"name":"Checkpoints","summary":"Shadow-git snapshots after every step; restorable.","maps_to_pattern":"agent-resumption","url":"https://docs.cline.bot/features/checkpoints"},{"name":".clinerules","summary":"Markdown rules files (also reads .cursorrules, .windsurfrules, AGENTS.md).","maps_to_pattern":"cross-session-memory","url":"https://docs.cline.bot/features/cline-rules"},{"name":"MCP","summary":"External tools via Model Context Protocol; works in VS Code + CLI with shared config.","maps_to_pattern":"mcp","url":"https://docs.cline.bot/mcp/mcp-overview"},{"name":"Multi-agent teams","summary":"Coordinator + specialist agents via `cline --team-name`."},{"name":"Scheduled agents","summary":"Cron-style recurring runs via `cline schedule`."}],"pattern_composition":"flowchart TD\n  user[User prompt] --> rules[.clinerules + AGENTS.md memory]\n  rules --> mode{Mode}\n  mode -->|Plan| plan[Read-only exploration]\n  plan --> agreed[Plan agreed]\n  agreed --> act\n  mode -->|Act| act[Tool-use loop]\n  act --> tool[File edit / terminal / browser / MCP]\n  tool --> approve{Approval?}\n  approve -->|yes| exec[Execute]\n  approve -->|auto| exec\n  exec --> ckpt[Checkpoint to shadow git]\n  ckpt --> obs[Observe]\n  obs --> done{Done?}\n  done -->|no| act\n  done -->|yes| ready[Ready]\n  team[Multi-agent team] -.coordinator delegates.-> act\n  sched[cline schedule cron] -.triggers.-> act","members":[{"pattern":"agent-computer-interface","role":"first-class","note":"Editor + terminal as shared ACI.","evidence":[{"type":"doc","title":"Cline — what is Cline","url":"https://docs.cline.bot/getting-started/what-is-cline","quote":"Cline is an AI coding agent that lives in your editor and your terminal. It can read and write files, run terminal commands, use a browser, and help you build features through natural conversation.","accessed":"2026-05-20"},{"type":"repo","title":"Cline README — CLI mode","url":"https://github.com/cline/cline","quote":"Run Cline in your terminal. Interactive chat or fully headless for CI/CD and scripting.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"agent-resumption","role":"first-class","note":"Checkpoints + team state both persist across sessions.","evidence":[{"type":"doc","title":"Cline — checkpoints persist","url":"https://docs.cline.bot/features/checkpoints","quote":"Checkpoints persist across editor sessions.","accessed":"2026-05-20"},{"type":"repo","title":"Cline README — team state","url":"https://github.com/cline/cline","quote":"Team state persists across sessions so you can pick up where you left off.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"agent-skills","role":"first-class","note":"Skills load specific rules when needed; .clinerules project rules guide Cline.","evidence":[{"type":"repo","title":"Cline README — skills","url":"https://github.com/cline/cline","quote":"Use skills to let the model load specific rules when needed.","accessed":"2026-05-20"},{"type":"repo","title":"Cline README — .clinerules","url":"https://github.com/cline/cline","quote":"Define project-specific rules in `.clinerules` files that guide how Cline works in your codebase: coding standards, architecture conventions, deployment procedures, testing requirements.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"approval-queue","role":"first-class","note":"Every action requires explicit approval; auto-approve is opt-in.","evidence":[{"type":"doc","title":"Cline — explicit approval","url":"https://docs.cline.bot/getting-started/what-is-cline","quote":"Every action requires your explicit approval. You're always in control.","accessed":"2026-05-20"},{"type":"repo","title":"Cline README — approval gating","url":"https://github.com/cline/cline","quote":"Every file edit and terminal command requires your approval, so you stay in control of what actually changes. Or toggle auto-approve and let Cline run autonomously.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"code-as-action","role":"first-class","note":"Edits + terminal commands are first-class agent actions; apply_patch + editor tools surface diffs; auto-approve gates which become silent.","evidence":[{"type":"repo","title":"Cline README — coordinated edits","url":"https://github.com/cline/cline","quote":"Cline reads your project structure, understands the relationships between files, and makes coordinated changes across your codebase.","accessed":"2026-05-20"},{"type":"doc","title":"Cline tools — apply_patch","url":"https://docs.cline.bot/exploring-clines-tools/cline-tools-guide","quote":"Cline tools are executable functions the model can call while working.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"code-execution","role":"first-class","note":"Commands run in user's terminal; long-running processes monitored in background.","evidence":[{"type":"repo","title":"Cline README — direct terminal execution","url":"https://github.com/cline/cline","quote":"Cline executes commands directly in your terminal and watches the output in real time.","accessed":"2026-05-20"},{"type":"repo","title":"Cline README — long-running processes","url":"https://github.com/cline/cline","quote":"For long-running processes like dev servers, Cline continues working in the background and reacts to new output as it appears, catching compile errors, test failures, and server crashes as they happen.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"computer-use","role":"first-class","note":"Browser + terminal + filesystem; auto-approve has explicit browser-actions gate alongside file/terminal/MCP toggles.","evidence":[{"type":"doc","title":"Cline — capabilities","url":"https://docs.cline.bot/getting-started/what-is-cline","quote":"It can read and write files, run terminal commands, use a browser, and help you build features through natural conversation.","accessed":"2026-05-20"},{"type":"doc","title":"Cline auto-approve — browser-actions gate","url":"https://docs.cline.bot/features/auto-approve","quote":"Check the box and Cline auto-approves everything: file changes, terminal commands, browser actions, MCP tools, and mode transitions.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"cross-session-memory","role":"first-class","note":"Markdown rules across .clinerules, .cursorrules, .windsurfrules, AGENTS.md.","evidence":[{"type":"doc","title":"Cline — persistent rules","url":"https://docs.cline.bot/features/cline-rules","quote":"Rules are markdown files that provide persistent instructions across all conversations.","accessed":"2026-05-20"},{"type":"doc","title":"Cline — rule source compatibility","url":"https://docs.cline.bot/features/cline-rules","quote":"Cline recognizes rules from multiple sources, including `.clinerules/`, `.cursorrules`, `.windsurfrules`, and `AGENTS.md`.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"MCP works in both VS Code and CLI with shared server config.","evidence":[{"type":"doc","title":"Cline MCP overview","url":"https://docs.cline.bot/mcp/mcp-overview","quote":"MCP (Model Context Protocol) lets Cline use external tools and data sources through MCP servers.","accessed":"2026-05-20"},{"type":"doc","title":"Cline MCP — CLI","url":"https://docs.cline.bot/mcp/mcp-overview","quote":"MCP also works in Cline CLI. Configure servers in CLI MCP settings and use the same server definitions.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"multi-model-routing","role":"first-class","note":"Provider-agnostic — Claude / GPT / Gemini / OpenRouter (200+) / Bedrock / Vertex / Cerebras / Groq / Ollama / OpenAI-compatible.","evidence":[{"type":"repo","title":"Cline README — provider choice","url":"https://github.com/cline/cline","quote":"Cline is not locked to a single AI provider. Use whichever model fits your workflow","accessed":"2026-05-20"},{"type":"repo","title":"Cline README — terminal execution","url":"https://github.com/cline/cline","quote":"Cline executes commands directly in your terminal and watches the output in real time. Install packages, run build scripts, execute tests, deploy applications, manage databases.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"plan-and-execute","role":"first-class","note":"Plan and Act modes are explicit and complementary.","evidence":[{"type":"doc","title":"Cline — Plan mode","url":"https://docs.cline.bot/features/plan-and-act","quote":"Plan mode is where you and Cline figure out what you're building and how.","accessed":"2026-05-20"},{"type":"doc","title":"Cline — Plan read-only","url":"https://docs.cline.bot/features/plan-and-act","quote":"In this mode, Cline can read your codebase, run searches, and discuss strategy, but cannot modify any files or execute commands.","accessed":"2026-05-20"},{"type":"doc","title":"Cline — Act mode","url":"https://docs.cline.bot/features/plan-and-act","quote":"Once you have a plan, switch to Act mode.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"Plan/Act loop in Act mode is the canonical ReAct shape (thinking separated from doing); iterate between modes when complexity grows.","evidence":[{"type":"doc","title":"Cline — Plan & Act separation","url":"https://docs.cline.bot/features/plan-and-act","quote":"Plan & Act modes separate thinking from doing.","accessed":"2026-05-20"},{"type":"doc","title":"Cline — Plan/Act iteration","url":"https://docs.cline.bot/features/plan-and-act","quote":"For complex projects, you may cycle between modes multiple times. Return to Plan mode when you hit unexpected complexity or need to rethink the approach, then switch back to Act mode to continue implementation.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"subagent-isolation","role":"first-class","note":"Coordinator + specialist agents, each with own tools and context; Plan/Act modes preserve context history when switching.","evidence":[{"type":"repo","title":"Cline README — multi-agent teams","url":"https://github.com/cline/cline","quote":"Coordinate multiple agents working together on complex tasks. A coordinator agent breaks the work into subtasks and delegates to specialist agents, each with their own tools and context.","accessed":"2026-05-20"},{"type":"doc","title":"Cline — context preservation across modes","url":"https://docs.cline.bot/features/plan-and-act","quote":"The conversation history carries over when you switch modes. Cline remembers everything you discussed in Plan mode, so you don't need to repeat yourself.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"todo-list-driven-agent","role":"limited","note":"Focus Chain is Cline's documented todo-list primitive: auto-generated markdown checklist that persists across context resets, periodically re-read.","evidence":[{"type":"doc","title":"Cline — Focus Chain feature","url":"https://docs.cline.bot/features/focus-chain","quote":"Focus Chain is automatic todo list management with real-time progress tracking.","accessed":"2026-05-24"},{"type":"doc","title":"Cline — Plan & Act mode (todo list)","url":"https://docs.cline.bot/core-workflows/plan-and-act","quote":"For large tasks, ask Cline to create a todo list during planning that you can track in Act mode","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Built-in tools (read/write/edit/Bash/browser) plus MCP-server-extensible tools.","evidence":[{"type":"doc","title":"Cline MCP — custom tools","url":"https://docs.cline.bot/mcp/mcp-overview","quote":"Add custom tools beyond built-in Cline tools","accessed":"2026-05-20"},{"type":"doc","title":"Cline MCP — external APIs","url":"https://docs.cline.bot/mcp/mcp-overview","quote":"Connect Cline to external APIs and services","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"shadow-workspace-vcs","role":"first-class","note":"Cline keeps an isolated git repo per workspace and offers diff-based rollback of agent edits.","evidence":[{"type":"paper","title":"Inside the Scaffold: A Source-Code Taxonomy of Coding Agent Architectures","url":"https://arxiv.org/abs/2604.03515","quote":"Shadow git checkpoints (Cline: isolated git repo per workspace, diff-based rollback)","accessed":"2026-06-14"}],"evidence_status":"full"},{"pattern":"repo-map-context","role":"first-class","note":"Cline parses source files with tree-sitter into an AST and runs tag queries to extract only definitions, producing a structural overview of the codebase before reading files.","evidence":[{"type":"doc","title":"cline/src/services/tree-sitter/index.ts","url":"https://github.com/cline/cline/blob/8fbccff8538cde243ceb08d644a15dbf2256b544/src/services/tree-sitter/index.ts","quote":"Our custom tag queries are based on tree-sitter's default tag queries, but modified to only capture definitions.","accessed":"2026-06-14"}],"evidence_status":"full"}],"instantiates":["modern-coding-agent","planning-loops","safety-hardening","long-running-autonomous-agent","multi-agent-coordination"],"alternatives":[{"composition":"claude-code","relation":"competes-with","note":"Peer first-party CLI agent."},{"composition":"roo-code","relation":"fork-of","note":"Direct fork of Cline."},{"composition":"cursor","relation":"competes-with","note":"Peer IDE-based agent."},{"composition":"continue-dev","relation":"competes-with","note":"Peer IDE extension."},{"composition":"aider","relation":"competes-with","note":"Peer terminal agent."},{"composition":"codex-cli","relation":"competes-with","note":"Peer open-source IDE+CLI agent."},{"composition":"pochi","relation":"competes-with","note":"Peer open-source VS Code coding agent."}],"references":[{"type":"repo","title":"cline/cline","url":"https://github.com/cline/cline","accessed":"2026-05-20"},{"type":"doc","title":"Cline docs","url":"https://docs.cline.bot","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-14","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","ide","cli","approval-gated","plan-act","kanban","multi-agent"]},{"id":"codebuddy","name":"CodeBuddy","kind":"framework","category":"coding-agent","build_surface":"full-code","vendor":"Tencent Cloud (腾讯云代码助手)","language":"TypeScript (extension)","license":"proprietary (free + enterprise)","status":"active","url":"https://copilot.tencent.com/","docs_url":"https://copilot.tencent.com/docs/","intent":"Tencent Cloud's AI coding assistant on the Hunyuan (混元) model family, providing completion, diagnostics, technical Q&A, and performance optimization across mainstream programming languages.","description":"CodeBuddy (腾讯云代码助手) is Tencent Cloud's coding assistant, powered by the Hunyuan (Yuanbao Code) large model. It ships VS Code and JetBrains extensions and offers code completion, error diagnostics, technical Q&A, and performance-optimisation hints. Tencent positions it for enterprise customers (90% productivity gain, 35% error-rate reduction in marketing material) and integrates it with the broader Tencent Cloud product surface. The standalone CodeBuddy IDE (codebuddy.ai) extends to a full AI code editor with agent capabilities.","primary_use_cases":["enterprise IDE-integrated coding assistance for teams on Tencent Cloud","code error diagnosis and performance suggestion","Chinese-language technical Q&A inside the IDE","standalone CodeBuddy IDE for agentic multi-file editing"],"agent_loop_shape":"Extension or standalone IDE calling hosted Hunyuan models. Completion is per-cursor; chat is tool-augmented; standalone IDE adds plan-and-execute agent mode similar to Cursor / Windsurf.","key_concepts":[{"name":"Hunyuan / Yuanbao Code backing","summary":"Tencent's in-house coding model family."},{"name":"CodeBuddy IDE","summary":"Standalone AI code editor branched from the extension.","maps_to_pattern":"agent-computer-interface"},{"name":"Agent mode (standalone IDE)","summary":"Plan-and-execute multi-file editing in the standalone CodeBuddy IDE branch.","maps_to_pattern":"plan-and-execute"}],"pattern_composition":"flowchart TD\n  USER[Developer] --> SURFACE{Surface}\n  SURFACE -->|Plugin| EXT[VSCode / JetBrains]\n  SURFACE -->|Standalone| BUDDY[CodeBuddy IDE]\n  EXT --> HUNYUAN[Hunyuan / Yuanbao Code]\n  BUDDY --> HUNYUAN\n  HUNYUAN --> MODE{Mode}\n  MODE -->|completion| RESP[Reply]\n  MODE -->|chat| TOOLS[Tool loop]\n  MODE -->|agent IDE| PLAN[Plan and execute<br/>multi-file edit]\n  TOOLS --> RESP\n  PLAN --> RESP\n  PLAN -.step budget.-> RESP","members":[{"pattern":"tool-use","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"code-as-action","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"react","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"plan-and-execute","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"agent-computer-interface","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"step-budget","role":"supported","evidence":[],"evidence_status":"none"}],"alternatives":[{"composition":"tongyi-lingma","relation":"competes-with"},{"composition":"marscode","relation":"competes-with"},{"composition":"codegeex","relation":"competes-with"},{"composition":"comate","relation":"competes-with"},{"composition":"cursor","relation":"similar-shape","note":"The standalone CodeBuddy IDE branches into Cursor-like agent territory."}],"last_analyzed":"2026-05-22","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-05-22","status_in_practice":"mature","tags":["coding-agent","tool-use-environment","planning-control-flow","safety-control","full-code"],"references":[{"type":"doc","title":"CodeBuddy — Documentation","url":"https://copilot.tencent.com/docs/"},{"type":"doc","title":"CodeBuddy — Homepage","url":"https://copilot.tencent.com/"}]},{"id":"codefuse","name":"CodeFuse","kind":"framework","category":"coding-agent","build_surface":"full-code","vendor":"Ant Group (蚂蚁集团)","language":"Python","license":"open-source (MIT for components)","status":"active","url":"https://codefuse.ai/","repo":"https://github.com/codefuse-ai/codefuse","intent":"Ant Group's open-source code LLM family covering the full software development lifecycle (design, requirements, coding, testing, deployment, operations) with both pre-trained models and downstream agent products.","description":"CodeFuse is Ant Group's umbrella project for Code LLMs and coding agents spanning the full SDLC. The codefuse-ai GitHub org hosts pre-trained code models (CodeFuse-13B, -CodeLlama-34B, CodeFuse-MFT-VLM for multimodal), evaluation harnesses (CodeFuseEval), and downstream agent components (Muagent for multi-agent orchestration, ModelCache for cache, ChatBot UI). Distinct from a pure code-assistant by treating the SDLC as a coordinated multi-stage workflow rather than a single completion loop.","primary_use_cases":["open-source code LLMs for fine-tuning and research","multi-agent SDLC orchestration (Muagent) for full lifecycle automation","code-eval benchmark (CodeFuseEval) for in-house model evaluation","self-hosted enterprise coding agent for Ant Group's internal products"],"agent_loop_shape":"Multi-component stack. The base layer is a coding LLM (CodeFuse-CodeLlama-34B, etc.). Muagent layers a multi-agent orchestration (planner, coder, reviewer, executor roles) on top, with ModelCache to avoid duplicate LLM calls. Each agent runs a tool-augmented ReAct loop scoped to its SDLC stage.","key_concepts":[{"name":"Muagent","summary":"CodeFuse's multi-agent framework for SDLC orchestration.","url":"https://github.com/codefuse-ai/codefuse-muagent","maps_to_pattern":"orchestrator-workers"},{"name":"CodeFuseEval","summary":"Code-eval harness shipped alongside the models.","maps_to_pattern":"eval-harness"},{"name":"ModelCache","summary":"Semantic cache layer for LLM calls to reduce duplicate inference.","maps_to_pattern":"prompt-caching"}],"pattern_composition":"flowchart TD\n  REQ[SDLC request] --> MUAGENT[Muagent orchestrator]\n  MUAGENT --> PLANNER[Planner role]\n  PLANNER --> CODER[Coder role]\n  CODER --> REVIEWER[Reviewer role]\n  REVIEWER --> EXEC[Executor role]\n  EXEC --> ART[Code / tests / docs]\n  REVIEWER -->|reject| CODER\n  CODER --> MODEL[CodeFuse Code LLMs<br/>13B / 34B / VLM]\n  CACHE[(ModelCache<br/>semantic cache)] -.dedup calls.-> MODEL\n  EVAL[(CodeFuseEval)] -.evaluates.-> MODEL","members":[{"pattern":"react","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"tool-use","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"code-as-action","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"supervisor","role":"supported","note":"Muagent multi-role orchestration.","evidence":[],"evidence_status":"none"},{"pattern":"orchestrator-workers","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"role-assignment","role":"supported","note":"Planner/coder/reviewer/executor split.","evidence":[],"evidence_status":"none"},{"pattern":"prompt-caching","role":"first-class","note":"ModelCache is a dedicated semantic cache.","evidence":[],"evidence_status":"none"},{"pattern":"eval-harness","role":"first-class","note":"CodeFuseEval.","evidence":[],"evidence_status":"none"}],"alternatives":[{"composition":"chatdev","relation":"similar-shape","note":"Both target multi-agent SDLC; ChatDev is academic, CodeFuse is industrial (Ant Group)."},{"composition":"openhands","relation":"similar-shape","note":"Both are open-source coding-agent stacks; OpenHands is Western, CodeFuse is Chinese."},{"composition":"aider","relation":"similar-shape","note":"Aider is a simpler single-agent coding loop; CodeFuse is multi-agent SDLC."}],"last_analyzed":"2026-05-22","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-05-22","status_in_practice":"mature","tags":["coding-agent","planning-control-flow","tool-use-environment","multi-agent","governance-observability","full-code"],"references":[{"type":"doc","title":"CodeFuse — Homepage","url":"https://codefuse.ai/"},{"type":"repo","title":"CodeFuse — Source repository","url":"https://github.com/codefuse-ai/codefuse"}]},{"id":"codegeex","name":"CodeGeeX","kind":"framework","category":"coding-agent","build_surface":"full-code","vendor":"Z.ai (formerly Zhipu AI) / Tsinghua KEG","language":"Python (model), TypeScript (extensions)","license":"Apache-2.0 (extensions + open-source models); proprietary hosted tier","status":"active","url":"https://codegeex.cn/","repo":"https://github.com/zai-org/CodeGeeX","docs_url":"https://codegeex.cn/","intent":"Open-source multilingual code generation extension from Tsinghua KEG / Zhipu, with completion, translation between languages, and an agent mode on top of the CodeGeeX4 (GLM-4-9B) model.","description":"CodeGeeX is an open-source coding assistant family launched in 2022 from Tsinghua University's Knowledge Engineering Group, now commercialised by Z.ai (formerly Zhipu AI). It ships VS Code, JetBrains, and Visual Studio extensions and exposes its models openly under Apache-2.0 (including CodeGeeX4-ALL-9B, a 128K-context coding model on GLM-4-9B that scored 82.3% on HumanEval). Modes include code completion, function-level generation, cross-language translation, code explanation, and an agent mode for multi-step coding tasks. Distinct from most coding assistants by virtue of being genuinely open-source end-to-end (model + extension + paper).","primary_use_cases":["open-source self-hostable code completion (Apache-2.0 model and extension)","cross-language code translation across 300+ programming languages","free-tier coding assistant for individual developers and students","academic research on coding model evaluation (HumanEval-X benchmark ships with it)"],"agent_loop_shape":"IDE extension that calls a hosted or local CodeGeeX model per user action. Completion is single-shot; agent mode runs a short ReAct loop over IDE tools (read, write, run). Repository indexing uses repository-level retrieval-augmented generation when context exceeds the 128K window.","key_concepts":[{"name":"CodeGeeX4-ALL-9B","summary":"128K-context coding model on GLM-4-9B, open-sourced under Apache-2.0.","url":"https://github.com/zai-org/CodeGeeX"},{"name":"HumanEval-X","summary":"Multilingual code-eval benchmark released alongside CodeGeeX (KDD 2023).","maps_to_pattern":"eval-harness"},{"name":"Open-source end-to-end stack","summary":"Model weights + extension + paper all under Apache-2.0, distinct from most coding assistants."},{"name":"Repository-level RAG","summary":"Retrieval-augmented prompting kicks in when context exceeds 128K window.","maps_to_pattern":"naive-rag"}],"pattern_composition":"flowchart TD\n  USER[Developer] --> EXT[CodeGeeX extension<br/>VSCode / JetBrains / VS]\n  EXT --> MODE{Mode}\n  MODE -->|Completion| MODEL[CodeGeeX4-ALL-9B<br/>128K context]\n  MODE -->|Chat| MODEL\n  MODE -->|Agent| LOOP[ReAct loop]\n  LOOP --> TOOLS[IDE tools<br/>read / write / run]\n  TOOLS --> LOOP\n  EXT -.context > 128K.-> RAG[Repo-level RAG]\n  RAG --> MODEL\n  MODEL --> EXT\n  EVAL[(HumanEval-X<br/>shipped with project)] -.evaluates.-> MODEL","members":[{"pattern":"react","role":"supported","note":"Agent mode is a ReAct loop.","evidence":[],"evidence_status":"none"},{"pattern":"tool-use","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"code-as-action","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"naive-rag","role":"supported","note":"Repository-level RAG for context beyond 128K.","evidence":[],"evidence_status":"none"},{"pattern":"agent-computer-interface","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"eval-harness","role":"first-class","note":"HumanEval-X ships as part of the project.","evidence":[],"evidence_status":"none"}],"alternatives":[{"composition":"tongyi-lingma","relation":"competes-with","note":"Closed-source Chinese alternative on Qwen; CodeGeeX differentiates on open-source."},{"composition":"marscode","relation":"competes-with","note":"Closed-source Chinese alternative; both target individual developers."},{"composition":"comate","relation":"competes-with"},{"composition":"codebuddy","relation":"competes-with"},{"composition":"continue-dev","relation":"similar-shape","note":"Both are open-source coding-assistant extensions, but Continue is a Western project with no proprietary model attached."}],"last_analyzed":"2026-05-22","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-05-22","status_in_practice":"mature","tags":["coding-agent","planning-control-flow","tool-use-environment","retrieval","governance-observability","full-code"],"references":[{"type":"doc","title":"CodeGeeX — Documentation","url":"https://codegeex.cn/"},{"type":"repo","title":"CodeGeeX — Source repository","url":"https://github.com/zai-org/CodeGeeX"}]},{"id":"codex-cli","name":"Codex CLI","kind":"framework","category":"coding-agent","build_surface":"full-code","vendor":"OpenAI","language":"Rust, TypeScript","license":"Apache-2.0","status":"active","first_released":"2025-04-13","url":"https://developers.openai.com/codex","repo":"https://github.com/openai/codex","docs_url":"https://developers.openai.com/codex","intent":"OpenAI's first-party terminal coding agent: a lightweight Rust CLI that runs models from OpenAI inside a sandbox with a configurable approval policy, AGENTS.md project memory, MCP, and an apply_patch primitive for safe diff application.","description":"Codex CLI is the terminal version of OpenAI's Codex agent. It runs locally, signs in via ChatGPT subscription or API key, and operates inside a layered sandbox where the user picks both a sandbox mode (read-only / workspace-write / danger-full-access) and an approval policy (untrusted / on-request / never). The agent reads AGENTS.md files for project conventions, calls MCP servers in both the CLI and IDE extension, and applies edits via the apply_patch tool. A /plan slash command proposes an execution plan before implementation, and /resume continues prior sessions.","primary_use_cases":["terminal-native coding agent with sandboxed execution","approval-gated automation across sandbox modes","MCP-tool integration in the CLI and IDE extension","shared AGENTS.md project conventions across editors","plan-then-execute and session resume workflows"],"agent_loop_shape":"ReAct-style tool-use loop running entirely inside a sandbox. Each turn: model proposes an action; the sandbox+approval policy decides whether to execute, ask, or refuse; the action runs; output goes back to the model; repeat. AGENTS.md files are read at startup to assemble a layered instruction chain. apply_patch is the diff-editing primitive; /plan switches into a planning mode; /resume picks up a saved conversation.","key_concepts":[{"name":"Sandbox modes","summary":"read-only / workspace-write / danger-full-access selectors gate filesystem and network reach.","maps_to_pattern":"sandbox-isolation","url":"https://developers.openai.com/codex/concepts/sandboxing"},{"name":"Approval policies","summary":"untrusted / on-request / never decide when the user is asked.","maps_to_pattern":"approval-queue","url":"https://developers.openai.com/codex/concepts/sandboxing"},{"name":"AGENTS.md","summary":"Layered project instructions read at startup.","maps_to_pattern":"cross-session-memory"},{"name":"MCP","summary":"First-class MCP client; shared config between CLI and IDE.","maps_to_pattern":"mcp","url":"https://developers.openai.com/codex/mcp"},{"name":"apply_patch","summary":"Structured-diff tool for create/update/delete file operations.","maps_to_pattern":"code-as-action","url":"https://developers.openai.com/api/docs/guides/tools-apply-patch"},{"name":"Slash commands","summary":"/permissions, /mcp, /plan, /resume in the TUI."}],"pattern_composition":"flowchart TD\n  user[User prompt] --> agents[AGENTS.md instruction chain]\n  agents --> loop[ReAct loop]\n  loop --> propose[Model proposes tool call]\n  propose --> sandbox{Sandbox mode}\n  sandbox -->|read-only| ro[Read only]\n  sandbox -->|workspace-write| ww[Workspace write]\n  sandbox -->|danger-full-access| dfa[Full access]\n  ro --> approval{Approval policy}\n  ww --> approval\n  dfa --> approval\n  approval -->|untrusted| ask[Ask user]\n  approval -->|on-request| auto[Auto if in sandbox]\n  approval -->|never| run[Run]\n  ask --> run\n  auto --> run\n  run --> mcp[MCP tools]\n  run --> patch[apply_patch]\n  run --> bash[Bash]\n  mcp --> obs[Observe]\n  patch --> obs\n  bash --> obs\n  obs --> loop\n  loop --> done[Done]","members":[{"pattern":"agent-computer-interface","role":"first-class","note":"Terminal-native ACI.","evidence":[{"type":"repo","title":"openai/codex repo description","url":"https://github.com/openai/codex","quote":"Lightweight coding agent that runs in your terminal","accessed":"2026-05-20"},{"type":"doc","title":"Codex CLI — local execution","url":"https://developers.openai.com/codex/cli","quote":"OpenAI's coding agent that you can run locally from your terminal. It can read, change, and run code on your machine","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"agent-resumption","role":"first-class","note":"/resume continues prior CLI sessions.","evidence":[{"type":"doc","title":"Codex CLI — /resume","url":"https://developers.openai.com/codex/cli/slash-commands","quote":"Resume a saved conversation from your session list.","accessed":"2026-05-20"},{"type":"doc","title":"Codex CLI — /resume purpose","url":"https://developers.openai.com/codex/cli/slash-commands","quote":"Continue work from a previous CLI session without starting over.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"approval-queue","role":"first-class","note":"Three approval policies — untrusted / on-request / never — switchable mid-session via /permissions.","evidence":[{"type":"doc","title":"Codex sandboxing — untrusted","url":"https://developers.openai.com/codex/concepts/sandboxing","quote":"untrusted: Codex asks before running commands that aren't in its trusted set.","accessed":"2026-05-20"},{"type":"doc","title":"Codex sandboxing — on-request","url":"https://developers.openai.com/codex/concepts/sandboxing","quote":"on-request: Codex works inside the sandbox by default and asks when it needs to go beyond that boundary.","accessed":"2026-05-20"},{"type":"doc","title":"Codex CLI — /permissions","url":"https://developers.openai.com/codex/cli/slash-commands","quote":"Set what Codex can do without asking first.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"code-as-action","role":"first-class","note":"apply_patch is the diff-application primitive; model emits structured create/update/delete operations.","evidence":[{"type":"doc","title":"OpenAI — apply_patch tool","url":"https://developers.openai.com/api/docs/guides/tools-apply-patch","quote":"The `apply_patch` tool lets GPT-5.1 create, update, and delete files in your codebase using structured diffs.","accessed":"2026-05-20"},{"type":"doc","title":"OpenAI — patch operations","url":"https://developers.openai.com/api/docs/guides/tools-apply-patch","quote":"Instead of just suggesting edits, the model emits patch operations that your application applies and then reports back on, enabling iterative, multi-step code editing workflows.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"code-execution","role":"first-class","note":"workspace-write mode lets Codex edit files and run local commands inside the sandbox boundary.","evidence":[{"type":"doc","title":"Codex sandboxing — workspace-write","url":"https://developers.openai.com/codex/concepts/sandboxing","quote":"workspace-write: Codex can read files, edit within the workspace, and run routine local commands inside that boundary.","accessed":"2026-05-24"},{"type":"doc","title":"Codex CLI — runs code on your machine","url":"https://developers.openai.com/codex/cli","quote":"Codex CLI is OpenAI's coding agent that you can run locally from your terminal. It can read, change, and run code on your machine in the selected directory.","accessed":"2026-05-24"},{"type":"doc","title":"Codex sandboxing — task progress without approval","url":"https://developers.openai.com/codex/concepts/sandboxing","quote":"When a task stays inside those boundaries, Codex can keep moving without stopping for confirmation. When it needs to go beyond them, Codex falls back to the approval flow.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"computer-use","role":"limited","note":"Computer Use is documented on the Codex Desktop app surface (macOS), not the CLI itself: GUI control via screenshots, browser, app interaction. CLI surface is file+shell only.","evidence":[{"type":"doc","title":"OpenAI Codex — Computer Use","url":"https://developers.openai.com/codex/app/computer-use","quote":"Codex can see and operate graphical user interfaces on macOS.","accessed":"2026-05-24"},{"type":"doc","title":"OpenAI Codex — Computer Use (capabilities)","url":"https://developers.openai.com/codex/app/computer-use","quote":"With computer use, Codex can view screen content, take screenshots, and interact with windows, menus, keyboard input, and clipboard state in the target app.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"First-class MCP client across CLI and IDE extension.","evidence":[{"type":"doc","title":"Codex MCP — CLI + IDE","url":"https://developers.openai.com/codex/mcp","quote":"Codex supports MCP servers in both the CLI and the IDE extension.","accessed":"2026-05-20"},{"type":"doc","title":"Codex CLI — /mcp","url":"https://developers.openai.com/codex/cli/slash-commands","quote":"List configured Model Context Protocol (MCP) tools.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"plan-and-execute","role":"first-class","note":"/plan switches into a planning mode that proposes execution plans before implementation.","evidence":[{"type":"doc","title":"Codex CLI — /plan","url":"https://developers.openai.com/codex/cli/slash-commands","quote":"Switch to plan mode and optionally send a prompt.","accessed":"2026-05-20"},{"type":"doc","title":"Codex CLI — plan purpose","url":"https://developers.openai.com/codex/cli/slash-commands","quote":"Ask Codex to propose an execution plan before implementation work starts.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"sandbox-isolation","role":"first-class","note":"Three sandbox modes form an explicit isolation matrix.","evidence":[{"type":"doc","title":"Codex sandboxing — read-only","url":"https://developers.openai.com/codex/concepts/sandboxing","quote":"read-only: Codex can inspect files, but it can't edit files or run commands without approval.","accessed":"2026-05-20"},{"type":"doc","title":"Codex sandboxing — danger-full-access","url":"https://developers.openai.com/codex/concepts/sandboxing","quote":"danger-full-access: Codex runs without sandbox restrictions.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"cross-session-memory","role":"first-class","note":"AGENTS.md project conventions are read at startup and concatenated into a layered instruction chain.","evidence":[{"type":"doc","title":"Codex AGENTS.md — read before work","url":"https://developers.openai.com/codex/guides/agents-md","quote":"Codex reads `AGENTS.md` files before doing any work. By layering global guidance with project-specific overrides, you can start each task with consistent expectations.","accessed":"2026-05-24"},{"type":"doc","title":"Codex AGENTS.md — instruction chain","url":"https://developers.openai.com/codex/guides/agents-md","quote":"Codex builds an instruction chain when it starts (once per run; in the TUI this usually means once per launched session).","accessed":"2026-05-24"},{"type":"doc","title":"Codex AGENTS.md — layered concatenation","url":"https://developers.openai.com/codex/guides/agents-md","quote":"Codex concatenates files from the root down, joining them with blank lines. Files closer to your current directory override earlier guidance because they appear later in the combined prompt.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Native tools (Bash, apply_patch, file edit) + MCP-server-backed tools.","evidence":[{"type":"doc","title":"Codex MCP — tool surface","url":"https://developers.openai.com/codex/mcp","quote":"Codex supports MCP servers in both the CLI and the IDE extension.","accessed":"2026-05-20"},{"type":"doc","title":"Codex CLI — MCP integration","url":"https://developers.openai.com/codex/cli","quote":"Model Context Protocol (MCP) integration for third-party tools","accessed":"2026-05-20"}],"evidence_status":"full"}],"instantiates":["modern-coding-agent","safety-hardening","routing-and-fallback"],"alternatives":[{"composition":"claude-code","relation":"competes-with","note":"Peer first-party terminal agent from Anthropic."},{"composition":"aider","relation":"competes-with","note":"Peer open-source terminal agent."},{"composition":"cline","relation":"competes-with","note":"Peer open-source IDE+CLI agent."},{"composition":"cursor","relation":"competes-with","note":"Peer IDE-based agent."},{"composition":"opencode","relation":"competes-with","note":"Peer terminal coding agent (OpenAI)."},{"composition":"google-antigravity","relation":"competes-with","note":"Peer first-party terminal coding agent (OpenAI)."},{"composition":"goose","relation":"competes-with","note":"Peer first-party terminal agent from OpenAI."},{"composition":"junie","relation":"competes-with","note":"Peer terminal agent for Junie CLI."},{"composition":"open-interpreter","relation":"competes-with","note":"Peer terminal coding agent."},{"composition":"openhands","relation":"competes-with","note":"Peer first-party terminal agent."},{"composition":"plandex","relation":"competes-with","note":"Peer terminal coding agent."}],"references":[{"type":"repo","title":"openai/codex","url":"https://github.com/openai/codex","accessed":"2026-05-20"},{"type":"doc","title":"Codex docs","url":"https://developers.openai.com/codex","accessed":"2026-05-20"},{"type":"doc","title":"Codex CLI slash commands","url":"https://developers.openai.com/codex/cli/slash-commands","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","rust","terminal","openai","sandbox-modes","agents-md"]},{"id":"comate","name":"Comate (Wenxin Kuaima)","kind":"framework","category":"coding-agent","build_surface":"full-code","vendor":"Baidu (文心快码)","language":"TypeScript (extension)","license":"proprietary (free individual + enterprise)","status":"active","url":"https://comate.baidu.com/","docs_url":"https://comate.baidu.com/zh/help","intent":"Baidu's coding assistant on Wenxin (Ernie) models, providing IDE-integrated code completion, generation, and chat across Baidu's own programming corpus and external open-source data.","description":"Comate (文心快码, formerly Baidu Comate) is Baidu's enterprise-grade coding assistant powered by Wenxin (Ernie) large models. It ships as a VS Code / JetBrains extension and offers completion, function generation, code explanation, test generation, and a chat mode with repository context. It positions itself for Chinese-language software teams and ships an enterprise edition with private deployment, fine-tuning on the customer's codebase, and integration with Baidu's broader Qianfan platform.","primary_use_cases":["code completion and chat in VS Code / JetBrains for Chinese software teams","enterprise private deployment with code-base fine-tuning","integration with Baidu Qianfan platform for end-to-end MLOps","Chinese-language natural-language → code generation"],"agent_loop_shape":"IDE extension calling hosted or in-VPC Wenxin models. Completion is single-shot per cursor event; chat runs short tool-loop with file context. Enterprise edition supports private RAG over the company's codebase.","key_concepts":[{"name":"Wenxin (Ernie) backing","summary":"Coding-tuned variant of Baidu's Ernie model family powers the assistant."},{"name":"Enterprise private deployment","summary":"In-VPC inference with optional fine-tuning on the customer's repository."},{"name":"Codebase RAG (enterprise)","summary":"In-VPC retrieval over the customer's codebase for repository-aware completion.","maps_to_pattern":"naive-rag"},{"name":"Qianfan platform integration","summary":"Plugs into Baidu's broader MLOps stack rather than standing alone."}],"pattern_composition":"flowchart TD\n  USER[Developer] --> EXT[Comate extension<br/>VSCode / JetBrains]\n  EXT --> DEPLOY{Deployment}\n  DEPLOY -->|public| WENXIN[Wenxin/Ernie<br/>hosted]\n  DEPLOY -->|enterprise| PRIV[In-VPC Wenxin<br/>+ private RAG]\n  PRIV --> REPO[(Company codebase<br/>vector index)]\n  REPO -->|context| PRIV\n  WENXIN --> EXT\n  PRIV --> EXT\n  EXT --> USER","members":[{"pattern":"tool-use","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"code-as-action","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"naive-rag","role":"supported","note":"Enterprise repo RAG for code-context retrieval.","evidence":[],"evidence_status":"none"},{"pattern":"agent-computer-interface","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"session-isolation","role":"supported","note":"Enterprise edition isolates per-user sessions.","evidence":[],"evidence_status":"none"}],"alternatives":[{"composition":"tongyi-lingma","relation":"competes-with"},{"composition":"marscode","relation":"competes-with"},{"composition":"codegeex","relation":"competes-with"},{"composition":"codebuddy","relation":"competes-with"},{"composition":"appbuilder","relation":"same-vendor","note":"Both are Baidu Qianfan products; Comate targets coding, AppBuilder targets low-code agent building."}],"last_analyzed":"2026-05-22","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-05-22","status_in_practice":"mature","tags":["coding-agent","tool-use-environment","retrieval","memory","full-code"],"references":[{"type":"doc","title":"Comate (Wenxin Kuaima) — Documentation","url":"https://comate.baidu.com/zh/help"},{"type":"doc","title":"Comate (Wenxin Kuaima) — Homepage","url":"https://comate.baidu.com/"}]},{"id":"continue-dev","name":"Continue","kind":"framework","category":"coding-agent","build_surface":"full-code","vendor":"Continue Dev, Inc.","language":"TypeScript","license":"Apache-2.0","status":"active","first_released":"2023-05-24","url":"https://www.continue.dev/","repo":"https://github.com/continuedev/continue","docs_url":"https://docs.continue.dev","intent":"Open-source AI dev tooling that started as an in-IDE coding assistant (Chat/Edit/Agent/Autocomplete in VS Code and JetBrains) and has pivoted to a CI-enforceable 'AI checks on every PR' framing via the `cn` CLI; the legacy IDE agent remains shipped.","description":"Continue today positions itself around the `cn` CLI that runs AI checks on pull requests. Each check is a markdown file in `.continue/checks/` whose result surfaces as a GitHub status check (green/red, with suggested fix). Underneath, the same engine still powers the long-standing VS Code and JetBrains extensions that provide Agent mode (autonomous task completion with tools and MCP), Chat (Q&A), Edit (inline edit), and Autocomplete (predictive completions). Custom assistants/blocks can be configured through Hub configs and via `mcpServers` for MCP integration.","primary_use_cases":["in-IDE agent + chat + edit + autocomplete (VS Code, JetBrains)","PR-check enforcement via `cn` CLI + GitHub status checks","MCP-tool extensibility inside Agent mode","Hub-shared assistant / block / MCP configurations"],"agent_loop_shape":"Two distinct loops live in the project: (1) PR-check loop — on every PR, each check's markdown prompt runs against the diff and results post as GitHub status checks; failures suggest a diff users can accept from GitHub. (2) IDE agent loop — a ReAct tool-use loop where the model uses chat-equipped tools (file edit, terminal, MCP, codebase context), asking permission per-tool by default; tool outputs feed back into the model as context. Agent mode requires MCP tools; MCP is only available in agent mode.","key_concepts":[{"name":"AI checks (`cn` CLI)","summary":"Markdown checks in `.continue/checks/` enforced in CI via GitHub status checks.","url":"https://docs.continue.dev/"},{"name":"Agent mode (IDE)","summary":"Autonomous tool-using agent inside VS Code/JetBrains.","maps_to_pattern":"react","url":"https://docs.continue.dev/ide-extensions/agent/quick-start"},{"name":"MCP","summary":"Only usable in agent mode; configured via hub mcpServers.","maps_to_pattern":"mcp","url":"https://docs.continue.dev/customize/deep-dives/mcp"},{"name":"Hub","summary":"Host for assistant / block / MCP-server configs."},{"name":"Modes (Chat / Edit / Agent / Autocomplete)","summary":"Four interaction modes in the IDE extension."}],"pattern_composition":"flowchart TD\n  subgraph IDE[IDE extension - VS Code/JetBrains]\n    user[User prompt] --> chatmode{Mode}\n    chatmode -->|Chat| qa[Q&A only]\n    chatmode -->|Edit| inline[Inline edit one file]\n    chatmode -->|Agent| agent[Agent tool-use loop]\n    chatmode -->|Autocomplete| tab[Predictive completions]\n    agent --> tool[File edit / terminal / MCP]\n    tool --> approve{Permission?}\n    approve -->|yes| exec[Execute]\n    exec --> obs[Tool output -> context]\n    obs --> agent\n  end\n  subgraph CI[cn CLI in GitHub PR check]\n    pr[PR opened] --> checks[Run .continue/checks/*.md]\n    checks --> status[GitHub status check]\n    status --> fail{Failed?}\n    fail -->|yes| suggest[Suggest diff]\n    fail -->|no| green[Green]\n  end","members":[{"pattern":"agent-computer-interface","role":"first-class","note":"IDE-resident Agent mode + standalone `cn` CLI for PR checks.","evidence":[{"type":"repo","title":"Continue vscode README — agent","url":"https://github.com/continuedev/continue","quote":"Agent to work on development tasks together with AI","accessed":"2026-05-20"},{"type":"doc","title":"Continue — Agent quickstart","url":"https://docs.continue.dev/ide-extensions/agent/quick-start","quote":"Agent mode equips the Chat model with the tools needed to handle a wide range of coding tasks","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"agent-skills","role":"limited","note":"Continue's rules are the closest primitive to Anthropic Agent Skills: system-message instructions that guide Agent/Chat/Edit behaviour. No verbatim 'skills' equivalent surfaced.","evidence":[{"type":"doc","title":"Continue — Rules (rules deep-dive)","url":"https://docs.continue.dev/customize/deep-dives/rules","quote":"Rules are used to provide system message instructions to the model for Agent mode, Chat mode, and Edit mode requests","accessed":"2026-05-24"},{"type":"doc","title":"Continue — Customize Rules","url":"https://docs.continue.dev/customize/rules","quote":"Rules allow you to provide specific instructions that guide how the AI agent behaves when working with your code.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"agentic-rag","role":"first-class","note":"Continue indexes the codebase using embeddings + keyword search; @Codebase / @Folder triggers similarity retrieval. Custom code-RAG guide documents the indexing pipeline.","evidence":[{"type":"doc","title":"Continue — custom code RAG","url":"https://docs.continue.dev/guides/custom-code-rag","quote":"Build a custom retrieval-augmented generation (RAG) system for faster and more cost-efficient code search across large codebases.","accessed":"2026-05-24"},{"type":"doc","title":"Continue — vector similarity search","url":"https://docs.continue.dev/guides/custom-code-rag","quote":"Search the codebase using vector similarity.","accessed":"2026-05-24"},{"type":"doc","title":"Continue — indexing pipeline","url":"https://docs.continue.dev/guides/custom-code-rag","quote":"your script should iterate over all of the files that you wish to index, chunk them, generate embeddings for each chunk, and then insert all of the chunks into your vector database","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"MCP is the canonical custom-tool surface; only available in agent mode.","evidence":[{"type":"doc","title":"Continue — MCP only in agent mode","url":"https://docs.continue.dev/customize/deep-dives/mcp","quote":"MCP can only be used in the agent mode.","accessed":"2026-05-20"},{"type":"doc","title":"Continue — MCP custom tools","url":"https://docs.continue.dev/customize/deep-dives/mcp","quote":"Currently custom tools can be configured using the Model Context Protocol standard to unify prompts, context, and tool use.","accessed":"2026-05-20"},{"type":"doc","title":"Continue — MCP via hub configs","url":"https://docs.continue.dev/customize/deep-dives/mcp","quote":"MCP Servers can be added to hub configs using `mcpServers`.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"Agent mode tool-use loop feeds tool outputs back into the model as context.","evidence":[{"type":"doc","title":"Continue — Agent autonomy","url":"https://docs.continue.dev/ide-extensions/agent/quick-start","quote":"allowing the model to make decisions and save you the work of manually finding context and performing actions","accessed":"2026-05-20"},{"type":"doc","title":"Continue — tool output as context","url":"https://docs.continue.dev/ide-extensions/agent/quick-start","quote":"Any data returned from a tool call is automatically fed back into the model as a context item","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Built-in tools + MCP-extensible tools with per-tool permission prompts.","evidence":[{"type":"doc","title":"Continue — built-in tools","url":"https://docs.continue.dev/ide-extensions/agent/quick-start","quote":"automatically implement code changes, fix bugs, and run commands using AI-powered tools","accessed":"2026-05-20"},{"type":"doc","title":"Continue — permission prompts","url":"https://docs.continue.dev/ide-extensions/agent/quick-start","quote":"Agent mode will ask permission when it wants to use a tool","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"repo-map-context","role":"first-class","note":"Continue ships a Repository Map context provider that emits a list of files plus the call signatures of top-level classes, functions, and methods so the model orients on structure.","evidence":[{"type":"doc","title":"Context Providers - Continue.dev (Repository Map provider)","url":"https://docs.continue.dev/customize/deep-dives/custom-providers","quote":"Provides a list of files and the call signatures of top-level classes, functions, and methods in those files.","accessed":"2026-06-14"}],"evidence_status":"full"}],"instantiates":["modern-coding-agent","safety-hardening"],"alternatives":[{"composition":"cursor","relation":"competes-with","note":"Peer IDE-based agent."},{"composition":"cline","relation":"competes-with","note":"Peer open-source IDE+CLI agent."},{"composition":"claude-code","relation":"competes-with","note":"Peer with broader surface area."},{"composition":"aider","relation":"competes-with","note":"Peer terminal-based agent."},{"composition":"codegeex","relation":"similar-shape","note":"Both are open-source coding-assistant extensions, but Continue is a Western project with no proprietary model attached."},{"composition":"pochi","relation":"competes-with","note":"Peer open-source VS Code coding assistant."},{"composition":"roo-code","relation":"competes-with","note":"Peer open-source VS Code coding assistant."},{"composition":"sourcegraph-cody","relation":"competes-with","note":"Peer IDE assistant; both target VS Code and JetBrains."},{"composition":"zed-ai","relation":"competes-with","note":"Peer open-source IDE assistant."}],"references":[{"type":"repo","title":"continuedev/continue","url":"https://github.com/continuedev/continue","accessed":"2026-05-20"},{"type":"doc","title":"Continue docs","url":"https://docs.continue.dev","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","ide","vs-code","jetbrains","pr-checks","cn-cli","pivot"]},{"id":"cursor","name":"Cursor","kind":"framework","category":"coding-agent","build_surface":"full-code","vendor":"Anysphere","language":"TypeScript (Electron IDE)","license":"proprietary (closed-source)","status":"active","first_released":"2023","url":"https://cursor.com/","docs_url":"https://cursor.com/docs","intent":"A proprietary VS Code fork built around an integrated AI agent ('Agent') and tab-completion model ('Tab') that turns the IDE into the surface for tool-using coding agents.","description":"Cursor is the IDE; Agent is the embedded coding agent inside it. Agent runs autonomously with no fixed tool-call limit, edits files, runs terminal commands, searches the codebase via a semantic index, calls MCP tools, and creates checkpoints before significant changes. The IDE provides inline diffs, chat, apply-edit, and autocomplete (Tab). The codebase is indexed locally into encrypted vector embeddings that are kept in sync, and the index can be shared across team members. Cursor advertises multi-mode interaction: Ask, Agent, and Auto modes.","primary_use_cases":["IDE-embedded autonomous coding agent","semantic-vector codebase search","predictive autocomplete via Tab model","MCP-tool integration with approval gating"],"agent_loop_shape":"Agent runs an unbounded tool-use loop inside the IDE. Each tool call (read, search semantic-index, edit-file, run-terminal, MCP) is reasoned about and executed; results are observed and fed back. Checkpoints are taken automatically before significant changes so users can roll back. The Tab model is a separate, much-tighter loop: predict-as-you-type completions independent of Agent. Agent asks for approval before MCP tools by default; users can enable auto-run.","key_concepts":[{"name":"Agent","summary":"Autonomous IDE agent that edits, runs commands, and calls tools.","url":"https://cursor.com/docs/agent"},{"name":"Tab","summary":"Predictive autocomplete model.","url":"https://cursor.com/"},{"name":"Codebase indexing","summary":"Encrypted semantic vector index synced every 5 minutes.","maps_to_pattern":"agentic-rag","url":"https://cursor.com/docs/context/codebase-indexing"},{"name":"MCP client","summary":"First-class MCP client supporting stdio/SSE/HTTP.","maps_to_pattern":"mcp","url":"https://cursor.com/docs/context/mcp"},{"name":"Checkpoints","summary":"Automatic codebase snapshots during Agent sessions."},{"name":"Rules","summary":".cursorrules and modern rules files steering Agent."}],"pattern_composition":"flowchart TD\n  user[User prompt in chat] --> mode{Mode}\n  mode -->|Ask| answer[Read-only Q&A]\n  mode -->|Agent| agent[Agent loop]\n  agent --> index[Semantic codebase index]\n  index --> tool[Tool call: edit/terminal/MCP/web]\n  tool --> approve{Needs approval?}\n  approve -->|yes| user2[Approve in IDE]\n  approve -->|no| exec[Execute]\n  user2 --> exec\n  exec --> ckpt[Auto-checkpoint]\n  ckpt --> obs[Observe result]\n  obs --> done{Task done?}\n  done -->|no| agent\n  done -->|yes| diff[Inline diff for review]\n  diff --> apply[Apply edits]\n  tab[Tab model] -.parallel.-> user","members":[{"pattern":"agent-computer-interface","role":"first-class","note":"IDE-embedded ACI accessed via Cmd+I sidepane; agent edits, runs terminal commands, calls tools, takes browser screenshots, generates images.","evidence":[{"type":"doc","title":"Cursor docs — Agent","url":"https://cursor.com/docs/agent","quote":"Agent is Cursor's assistant that can complete complex coding tasks independently, run terminal commands, and edit code.","accessed":"2026-05-20"},{"type":"doc","title":"Cursor docs — Agent components","url":"https://cursor.com/docs/agent","quote":"Instructions: The system prompt and rules that guide agent behavior; Tools: File editing, codebase search, terminal execution, and more; Model: The agent model you pick for the task.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"agentic-rag","role":"first-class","note":"Local encrypted semantic vector index, synced every 5 minutes; chunked by function/class/logical block.","evidence":[{"type":"doc","title":"Cursor docs — codebase indexing chunks","url":"https://cursor.com/docs/context/codebase-indexing","quote":"Cursor breaks your code into meaningful chunks (functions, classes, logical blocks)","accessed":"2026-05-20"},{"type":"doc","title":"Cursor docs — index sync interval","url":"https://cursor.com/docs/context/codebase-indexing","quote":"The index stays current through automatic sync every 5 minutes, processing only changed files","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"approval-queue","role":"first-class","note":"Agent prompts for approval before using MCP tools by default; auto-run is opt-in.","evidence":[{"type":"doc","title":"Cursor docs — MCP approval default","url":"https://cursor.com/docs/context/mcp","quote":"Agent asks for approval before using MCP tools by default","accessed":"2026-05-20"},{"type":"doc","title":"Cursor docs — MCP auto-run opt-in","url":"https://cursor.com/docs/context/mcp","quote":"Enable auto-run for Agent to use MCP tools without asking","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"First-class MCP client.","evidence":[{"type":"doc","title":"Cursor docs — MCP support","url":"https://cursor.com/docs/context/mcp","quote":"Cursor supports these MCP protocol capabilities and extensions","accessed":"2026-05-20"},{"type":"doc","title":"Cursor docs — automatic MCP tool use","url":"https://cursor.com/docs/context/mcp","quote":"Agent automatically uses MCP tools listed under Available Tools when relevant","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"Unbounded tool-use loop; Agent reasons, calls a tool, observes, repeats.","evidence":[{"type":"doc","title":"Cursor docs — unbounded tool calls","url":"https://cursor.com/docs/agent","quote":"There is no limit on the number of tool calls Agent can make during a task.","accessed":"2026-05-20"},{"type":"doc","title":"Cursor docs — tools as building blocks","url":"https://cursor.com/docs/agent","quote":"Tools are the building blocks of Agent. They are used to search your codebase and the web to find relevant information, make edits to your files, run terminal commands, and more.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"subagent-isolation","role":"limited","note":"Background/Cloud Agents run in their own VMs; parallel without local machine. Still 'limited' — VM isolation not the same as named per-task subagent primitive.","evidence":[{"type":"doc","title":"Cursor docs — cloud agents in isolated VMs","url":"https://cursor.com/docs/background-agent","quote":"Because they have access to their own virtual machine, cloud agents can build, test, and interact with the changed software.","accessed":"2026-05-24"},{"type":"doc","title":"Cursor docs — parallel cloud agents","url":"https://cursor.com/docs/background-agent","quote":"You can run as many agents as you want in parallel, and they do not require your local machine to be connected to the internet.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Built-in tools (search/edit/terminal/web/MCP).","evidence":[{"type":"doc","title":"Cursor docs — tools building blocks","url":"https://cursor.com/docs/agent","quote":"Tools are the building blocks of Agent. They are used to search your codebase and the web to find relevant information, make edits to your files, run terminal commands, and more.","accessed":"2026-05-20"},{"type":"doc","title":"Cursor docs — terminal tool","url":"https://cursor.com/docs/agent","quote":"Execute terminal commands and monitor output. By default, Cursor uses the first terminal profile available.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"local-to-cloud-handoff","role":"first-class","note":"Cursor CLI Cloud Handoff pushes the local conversation to a Cloud Agent that keeps running while the developer is away, resumable later on web or mobile.","evidence":[{"type":"changelog","title":"CLI Agent Modes and Cloud Handoff - Cursor Changelog (Jan 16, 2026)","url":"https://cursor.com/changelog/cli-jan-16-2026","quote":"Push your local conversation to a Cloud Agent and let it keep running while you're away.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"shadow-workspace-vcs","role":"first-class","note":"The Agent auto-creates checkpoints before significant changes and lets the user restore any prior file state, stored locally and separate from git history.","evidence":[{"type":"doc","title":"Cursor - Checkpoints","url":"https://docs.cursor.com/agent/chat/checkpoints","quote":"Checkpoints save snapshots of your codebase during an Agent session, and Agent automatically creates them before making significant changes, capturing the state of all modified files.","accessed":"2026-06-14"}],"evidence_status":"full"},{"pattern":"context-driven-architecture-drift","role":"supported","note":"Cursor's project rules let a team encode architecture decisions and codebase conventions as persistent context the agent reads on every edit, mitigating the local-sample inference that causes drift.","evidence":[{"type":"doc","title":"Rules - Cursor Docs","url":"https://cursor.com/docs/context/rules","quote":"Rules provide persistent, reusable context at the prompt level","accessed":"2026-06-14"}],"evidence_status":"full"},{"pattern":"scaffold-ablation-on-model-upgrade","role":"supported","note":"Knocked down early guardrails and moved to dynamic context as models improved.","evidence":[{"type":"doc","url":"https://cursor.com/blog/continually-improving-agent-harness","quote":"Knocked down early guardrails and moved to dynamic context as models improved.","accessed":"2026-06-19"}],"evidence_status":"full"}],"instantiates":["modern-coding-agent","production-rag","planning-loops"],"alternatives":[{"composition":"claude-code","relation":"competes-with","note":"Peer with terminal+IDE surfaces."},{"composition":"cline","relation":"competes-with","note":"Peer open-source IDE agent."},{"composition":"continue-dev","relation":"competes-with","note":"Peer IDE extension."},{"composition":"windsurf","relation":"competes-with","note":"Direct competitor — Codeium's VS Code fork."},{"composition":"kiro","relation":"competes-with","note":"Peer agentic IDE."},{"composition":"codebuddy","relation":"similar-shape","note":"The standalone CodeBuddy IDE branches into Cursor-like agent territory."},{"composition":"codex-cli","relation":"competes-with","note":"Peer IDE-based agent."},{"composition":"junie","relation":"competes-with","note":"Peer IDE-based agent (Cursor is its own IDE, Junie lives inside JetBrains IDEs)."},{"composition":"sourcegraph-cody","relation":"competes-with","note":"Peer IDE-based coding assistant."},{"composition":"tongyi-lingma","relation":"competes-with","note":"Both are agent-mode IDE coding assistants; Lingma is the canonical 'Chinese Cursor alternative'."},{"composition":"zed-ai","relation":"competes-with","note":"Peer agentic IDE; closed source."}],"references":[{"type":"doc","title":"Cursor docs — Agent","url":"https://cursor.com/docs/agent","accessed":"2026-05-20"},{"type":"doc","title":"Cursor docs — Codebase indexing","url":"https://cursor.com/docs/context/codebase-indexing","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-14","date_added":"2026-04-30","status_in_practice":"mature","tags":["proprietary","ide","vs-code-fork","semantic-index","mcp"]},{"id":"devin","name":"Devin","kind":"framework","category":"coding-agent","build_surface":"closed-application","vendor":"Cognition","language":"N/A","license":"proprietary","status":"active","first_released":"2024-03-12","url":"https://devin.ai/","docs_url":"https://docs.devin.ai","intent":"Cognition's hosted autonomous AI software engineer that takes engineering tasks end-to-end inside its own Workspace — shell, IDE, and browser — charged in Agent Compute Units (ACUs).","description":"Devin is a proprietary cloud agent positioned as an autonomous AI software engineer. Each session runs on a virtual machine where Devin can write, run, and test code, with a Workspace UI exposing three primary developer tools: a shell, an embedded IDE, and a browser. Cognition meters usage in Agent Compute Units (ACUs) covering planning, context gathering, task execution, browser actions, and code execution. Devin pulls cross-session context from a Knowledge store auto-populated from repo READMEs and from .rules / AGENTS.md / CLAUDE.md style files; Knowledge is retrieved per session by a Trigger.","primary_use_cases":["autonomous end-to-end engineering tasks in a hosted VM","shipping pull requests via Slack/Linear/GitHub handoff","incident-response and CI-failure triage as agent tasks","long-running multi-hour tasks scoped at ~3-hour granularity"],"agent_loop_shape":"Cloud-hosted single-agent session inside a Devin VM. Devin opens a Workspace with shell, IDE, and browser, plans the task, gathers context (including org Knowledge), and iterates: write code, run, test, browse docs, repeat. The session can sleep and resume when messaged. Usage accumulates in ACUs per action class (planning, context, execution, browser, code execution).","key_concepts":[{"name":"Devin VM / Workspace","summary":"Each session runs on a virtual machine; the Workspace exposes shell, IDE, and browser.","maps_to_pattern":"sandbox-isolation","url":"https://docs.devin.ai/get-started/devin-intro"},{"name":"Shell, IDE, Browser","summary":"Three first-class tools Devin uses to complete tasks.","maps_to_pattern":"computer-use","url":"https://docs.devin.ai/get-started/devin-intro"},{"name":"Knowledge","summary":"Cross-session instructions and advice, auto-generated from READMEs and rules files.","maps_to_pattern":"cross-session-memory","url":"https://docs.devin.ai/onboard-devin/knowledge-onboarding"},{"name":"ACUs","summary":"Agent Compute Units meter planning, context gathering, execution, browser, and code execution.","maps_to_pattern":"step-budget","url":"https://docs.devin.ai/admin/billing/usage"},{"name":"Sessions","summary":"Long-running tasks that can sleep and wake on new messages."}],"pattern_composition":"flowchart TD\n  user[User task] --> session[Devin session]\n  session --> vm[Devin VM]\n  vm --> shell[Shell]\n  vm --> ide[IDE]\n  vm --> browser[Browser]\n  session --> know[Knowledge retrieval]\n  know -.pinned/triggered.-> session\n  shell --> obs[Observe]\n  ide --> obs\n  browser --> obs\n  obs --> session\n  session --> acu[ACU consumption]\n  session --> pr[PR / handoff]","members":[{"pattern":"agent-computer-interface","role":"first-class","note":"Workspace exposes shell, embedded IDE, and browser as the developer tool surface.","evidence":[{"type":"doc","title":"Devin — Workspace tools","url":"https://docs.devin.ai/get-started/devin-intro","quote":"In Devin's Workspace, you'll find developer tools that Devin will use to complete your task.","accessed":"2026-05-20"},{"type":"doc","title":"Devin — Shell","url":"https://docs.devin.ai/get-started/devin-intro","quote":"Devin's terminal, where you can watch commands being executed and view output logs.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"agent-resumption","role":"first-class","note":"Sessions sleep when idle and resume on new messages; documented sleep/wake mechanic with ACU-metered idle floor.","evidence":[{"type":"doc","title":"Devin — session wake-up","url":"https://docs.devin.ai/admin/billing/usage","quote":"You can wake the session up at any time by sending another message.","accessed":"2026-05-24"},{"type":"doc","title":"Devin — idle session sleeps","url":"https://docs.devin.ai/admin/billing/usage","quote":"When a session is idle, Devin goes to sleep. While sleeping, Devin does not consume usage.","accessed":"2026-05-24"},{"type":"doc","title":"Devin — auto-sleep threshold","url":"https://docs.devin.ai/admin/billing/usage","quote":"Devin typically sleeps automatically after roughly 0.1 ACUs (or the equivalent quota / on-demand credit) of inactivity.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"code-execution","role":"first-class","note":"Devin writes, runs, and tests code inside its VM.","evidence":[{"type":"doc","title":"Devin — autonomous engineer","url":"https://docs.devin.ai/get-started/devin-intro","quote":"Devin is an autonomous AI software engineer that can write, run and test code.","accessed":"2026-05-20"},{"type":"doc","title":"Devin — code execution as ACU","url":"https://docs.devin.ai/admin/billing/usage","quote":"Number and complexity of actions Devin takes (planning, context gathering, task execution, browser actions, code execution, and so on)","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"computer-use","role":"first-class","note":"Devin actually drives a browser to read docs and test web apps.","evidence":[{"type":"doc","title":"Devin — browser tool","url":"https://docs.devin.ai/get-started/devin-intro","quote":"Watch Devin browse through documentation, test web applications it builds, download/upload information, and more.","accessed":"2026-05-20"},{"type":"blog","title":"Devin.ai — Visual QA","url":"https://devin.ai/","quote":"Visual QA with full browser and desktop use","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"Per-action loop: plan, gather context, execute, observe — each action class meters into ACU. Usage doc enumerates the loop's metered action classes.","evidence":[{"type":"doc","title":"Devin — action loop classes","url":"https://docs.devin.ai/admin/billing/usage","quote":"Usage accrues based on the work Devin actually performs in a session, including: Number and complexity of actions Devin takes (planning, context gathering, task execution, browser actions, code execution, and so on)","accessed":"2026-05-24"},{"type":"doc","title":"Devin — write, run, test loop","url":"https://docs.devin.ai/get-started/devin-intro","quote":"Devin is an autonomous AI software engineer that can write, run and test code.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"step-budget","role":"first-class","note":"ACU consumption is the documented cost/step ceiling — per-action metering against a session budget.","evidence":[{"type":"doc","title":"Devin — ACU consumption","url":"https://docs.devin.ai/admin/billing/usage","quote":"Enterprise customers consume Agent Compute Units (ACUs) against the volume in their order form.","accessed":"2026-05-20"},{"type":"doc","title":"Devin — idle VM still meters","url":"https://docs.devin.ai/admin/billing/usage","quote":"Aside from the few units required to keep the Devin VM running, Devin will not consume usage when:","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"subagent-isolation","role":"first-class","note":"Managed Devins: each subagent runs in its own isolated VM with its own terminal, browser, dev environment; the main session coordinates.","evidence":[{"type":"blog","title":"Cognition — managed Devins parallel","url":"https://cognition.ai/blog/devin-can-now-manage-devins","quote":"Devin can break down large tasks and delegate them to a team of managed Devins that work in parallel.","accessed":"2026-05-24"},{"type":"blog","title":"Cognition — managed Devin isolation","url":"https://cognition.ai/blog/devin-can-now-manage-devins","quote":"Each managed Devin is a full Devin, running in its own isolated virtual machine with its own terminal, browser, and development environment.","accessed":"2026-05-24"},{"type":"blog","title":"Cognition — main Devin as coordinator","url":"https://cognition.ai/blog/devin-can-now-manage-devins","quote":"The main Devin session acts as a coordinator: it scopes the work, assigns each piece to a managed Devin, monitors progress, resolves any conflicts, and compiles the results.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Shell, IDE, browser are tool surfaces; agent also operates against Slack/Linear/GitHub.","evidence":[{"type":"doc","title":"Devin — IDE tool","url":"https://docs.devin.ai/get-started/devin-intro","quote":"Devin's embedded code editor equipped with all the IDE tools and shortcuts you're familiar with.","accessed":"2026-05-20"},{"type":"blog","title":"Devin.ai — ships PRs","url":"https://devin.ai/","quote":"Devin ships PRs the way your team does — picking up review feedback and CI results","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"cross-session-memory","role":"first-class","note":"Knowledge is the cross-session memory; auto-generated from READMEs and rules files.","evidence":[{"type":"doc","title":"Devin — Knowledge across sessions","url":"https://docs.devin.ai/onboard-devin/knowledge-onboarding","quote":"Knowledge is a collection of instructions and advice that Devin can reference in all sessions.","accessed":"2026-05-20"},{"type":"doc","title":"Devin — Knowledge auto-pull","url":"https://docs.devin.ai/onboard-devin/knowledge-onboarding","quote":"Devin will automatically pull and update Knowledge based on specialized files in your codebase including `.rules`, `.mdc`, `.cursorrules`, `.windsurf`, `CLAUDE.md`, and `AGENTS.md`.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"sync-execution-plan-confirmation","role":"first-class","note":"Devin emits its plan synchronously for user confirmation before it starts working, and records the session into a replay/video timeline that can be reviewed after the fact.","evidence":[{"type":"doc","title":"Testing and Video Recordings - Devin Docs","url":"https://docs.devin.ai/work-with-devin/testing-and-recordings","quote":"Devin sends you the plan as a short message before executing, so you can course-correct if needed.","accessed":"2026-06-17"},{"type":"doc","title":"Testing and Video Recordings - Devin Docs","url":"https://docs.devin.ai/work-with-devin/testing-and-recordings","quote":"Raw recordings are processed to highlight important actions and compress idle time","accessed":"2026-06-17"}],"evidence_status":"full"}],"instantiates":["modern-coding-agent","safety-hardening"],"alternatives":[{"composition":"github-copilot-coding-agent","relation":"competes-with","note":"Peer hosted async coding agent that ships PRs."},{"composition":"openhands","relation":"competes-with","note":"Open-source competitor (formerly OpenDevin)."},{"composition":"claude-code","relation":"competes-with","note":"Anthropic's first-party agent in a different surface (terminal)."},{"composition":"factory-droid","relation":"competes-with","note":"Peer autonomous software engineering agent."},{"composition":"google-jules","relation":"competes-with","note":"Peer autonomous async software engineering agent."},{"composition":"windsurf","relation":"competes-with","note":"Same parent (Cognition) post-acquisition; Devin is the autonomous cloud agent now embedded inside Windsurf."},{"composition":"manus","relation":"competes-with","note":"Cognition's autonomous AI engineer with a sandbox + browser + plan loop; closest US competitor pre-Manus."}],"references":[{"type":"doc","title":"Devin docs","url":"https://docs.devin.ai","accessed":"2026-05-20"},{"type":"doc","title":"Devin — intro","url":"https://docs.devin.ai/get-started/devin-intro","accessed":"2026-05-20"},{"type":"doc","title":"Devin — Knowledge","url":"https://docs.devin.ai/onboard-devin/knowledge-onboarding","accessed":"2026-05-20"},{"type":"doc","title":"Devin — usage / ACUs","url":"https://docs.devin.ai/admin/billing/usage","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["hosted","autonomous","vm-sandbox","acu","proprietary"]},{"id":"github-copilot-coding-agent","name":"GitHub Copilot Coding Agent","kind":"framework","category":"coding-agent","build_surface":"closed-application","vendor":"GitHub","language":"N/A","license":"proprietary","status":"active","first_released":"2025-05-19","url":"https://docs.github.com/en/copilot/concepts/agents/coding-agent/about-coding-agent","docs_url":"https://docs.github.com/en/copilot/concepts/agents/coding-agent","intent":"Asynchronous GitHub-native coding agent that researches a repository, drafts an implementation plan, and opens a pull request on a branch, running entirely inside an ephemeral GitHub Actions development environment.","description":"GitHub Copilot coding agent runs against an issue or task and works on a branch in the background. Its execution environment is an ephemeral dev container powered by GitHub Actions, customisable via a copilot-setup-steps job. The agent can be extended with MCP servers and given project-specific behaviour through custom instructions; access is repo-scoped by default. The handoff is a pull request that the human reviews, iterates on, and merges — the agent uses standard GitHub branch / PR semantics rather than a foreground IDE loop.","primary_use_cases":["asynchronous issue-to-PR coding tasks","background research and implementation on a branch","customisable agent environment via GitHub Actions setup steps","MCP-augmented automation inside GitHub-native workflows"],"agent_loop_shape":"Background async agent. A task (typically an issue or assignment) kicks off an ephemeral GitHub Actions dev container; the agent researches the repo, drafts a plan, makes commits on a branch, and surfaces results as a pull request. Iteration happens via review feedback on the PR, not an interactive console. MCP servers extend the tool surface; custom instructions shape behaviour.","key_concepts":[{"name":"Ephemeral GitHub Actions dev container","summary":"Per-task throwaway environment running on GitHub-hosted runners.","maps_to_pattern":"sandbox-isolation","url":"https://docs.github.com/en/copilot/how-tos/use-copilot-agents/coding-agent/customize-the-agent-environment"},{"name":"Branch + PR handoff","summary":"Agent writes to a branch; human review happens on the resulting pull request.","maps_to_pattern":"approval-queue","url":"https://docs.github.com/en/copilot/concepts/agents/coding-agent/about-coding-agent"},{"name":"Custom instructions","summary":"Per-project instructions for build/test/validation behaviour.","maps_to_pattern":"cross-session-memory","url":"https://docs.github.com/en/copilot/concepts/agents/coding-agent/about-coding-agent"},{"name":"MCP servers","summary":"Extra data sources and tools wired in via Model Context Protocol.","maps_to_pattern":"mcp","url":"https://docs.github.com/en/copilot/concepts/agents/coding-agent/about-coding-agent"},{"name":"copilot-setup-steps","summary":"GitHub Actions job that runs before the agent starts — the seam for installing dependencies or switching runners."}],"pattern_composition":"flowchart TD\n  task[Issue or task] --> kickoff[Copilot agent kickoff]\n  kickoff --> setup[copilot-setup-steps]\n  setup --> env[Ephemeral GitHub Actions dev container]\n  env --> plan[Research + implementation plan]\n  plan --> code[Edits on branch]\n  code --> tests[Run tests / linters]\n  tests --> code\n  env --> mcp[MCP servers]\n  code --> pr[Pull request]\n  pr --> review[Human review]\n  review -->|feedback| code\n  review -->|merge| done[Merged]","members":[{"pattern":"approval-queue","role":"first-class","note":"Approval happens on the pull request — humans review the diff, iterate, and a second peer reviewer must approve before merge.","evidence":[{"type":"doc","title":"Copilot coding agent — PR review","url":"https://docs.github.com/en/copilot/concepts/agents/coding-agent/about-coding-agent","quote":"Copilot can research a repository, create an implementation plan, and make code changes on a branch. You can review the diff, iterate, and create a pull request when you're ready.","accessed":"2026-05-24"},{"type":"doc","title":"Copilot — review thoroughly before merge","url":"https://docs.github.com/copilot/how-tos/agents/copilot-coding-agent/reviewing-a-pull-request-created-by-copilot","quote":"When Copilot finishes a coding task and requests your review, check the pull request thoroughly before merging.","accessed":"2026-05-24"},{"type":"doc","title":"Copilot — peer approval required","url":"https://docs.github.com/copilot/how-tos/agents/copilot-coding-agent/reviewing-a-pull-request-created-by-copilot","quote":"If your repository requires pull request approvals, your approval of a Copilot pull request won't count toward the required number.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"spec-driven-loop","role":"first-class","note":"Workflow is issue-to-plan-to-PR: agent researches, plans, implements before opening a PR.","evidence":[{"type":"doc","title":"Copilot coding agent — plan-then-implement","url":"https://docs.github.com/en/copilot/concepts/agents/coding-agent/about-coding-agent","quote":"all coding and iterating happens on GitHub. You can ask Copilot to research a repository, create a plan, and make code changes on a branch—all before opening a pull request.","accessed":"2026-05-24"},{"type":"doc","title":"Copilot coding agent — implementation plans","url":"https://docs.github.com/en/copilot/concepts/agents/coding-agent/about-coding-agent","quote":"Copilot can research a repository, create an implementation plan, and make code changes on a branch.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"sandbox-isolation","role":"first-class","note":"Ephemeral GitHub Actions dev container per task; access is repo-scoped by default.","evidence":[{"type":"doc","title":"Copilot coding agent — ephemeral env","url":"https://docs.github.com/en/copilot/concepts/agents/coding-agent/about-coding-agent","quote":"its own ephemeral development environment, powered by GitHub Actions, where it can explore your code, make changes, execute automated tests and linters and more.","accessed":"2026-05-20"},{"type":"doc","title":"Copilot coding agent — repo scope","url":"https://docs.github.com/en/copilot/concepts/agents/coding-agent/about-coding-agent","quote":"Copilot can only access context in the repository specified when you start a task.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"code-execution","role":"first-class","note":"The agent executes automated tests and linters inside the ephemeral GitHub Actions environment.","evidence":[{"type":"doc","title":"Copilot coding agent — tests and linters","url":"https://docs.github.com/en/copilot/concepts/agents/coding-agent/about-coding-agent","quote":"its own ephemeral development environment, powered by GitHub Actions, where it can explore your code, make changes, execute automated tests and linters and more.","accessed":"2026-05-24"},{"type":"doc","title":"Copilot coding agent — preinstall tools","url":"https://docs.github.com/en/copilot/how-tos/use-copilot-agents/coding-agent/customize-the-agent-environment","quote":"You can use a Copilot setup steps file to deterministically install tools or dependencies before Copilot starts work.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"MCP servers extend the tool surface; once configured, the agent uses them autonomously without per-tool approval.","evidence":[{"type":"doc","title":"Copilot coding agent — MCP","url":"https://docs.github.com/en/copilot/concepts/agents/coding-agent/about-coding-agent","quote":"MCP servers allow you to give Copilot access to different data sources and tools.","accessed":"2026-05-24"},{"type":"doc","title":"Copilot MCP — autonomous use","url":"https://docs.github.com/en/copilot/how-tos/use-copilot-agents/coding-agent/extend-coding-agent-with-mcp","quote":"Once you've configured an MCP server, Copilot will be able to use the tools provided by the server autonomously, and will not ask for your approval before using them.","accessed":"2026-05-24"},{"type":"doc","title":"Copilot MCP — tools-only","url":"https://docs.github.com/en/copilot/how-tos/use-copilot-agents/coding-agent/extend-coding-agent-with-mcp","quote":"Copilot cloud agent only supports tools provided by MCP servers. It does not support resources or prompts.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"subagent-isolation","role":"first-class","note":"Each task spins up its own ephemeral dev container, isolated from other tasks and from the user's machine.","evidence":[{"type":"doc","title":"Copilot coding agent — isolated runners","url":"https://docs.github.com/en/copilot/how-tos/use-copilot-agents/coding-agent/customize-the-agent-environment","quote":"Copilot has access to its own ephemeral development environment, powered by GitHub Actions","accessed":"2026-05-20"},{"type":"doc","title":"Copilot coding agent — GitHub-hosted runners","url":"https://docs.github.com/en/copilot/how-tos/use-copilot-agents/coding-agent/customize-the-agent-environment","quote":"standard GitHub-hosted GitHub Actions runners","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Tool surface = shell+git inside the dev container, GitHub APIs, and any MCP servers attached.","evidence":[{"type":"doc","title":"Copilot coding agent — MCP tools","url":"https://docs.github.com/en/copilot/concepts/agents/coding-agent/about-coding-agent","quote":"MCP servers allow you to give Copilot access to different data sources and tools.","accessed":"2026-05-20"},{"type":"doc","title":"Copilot coding agent — environment customisation","url":"https://docs.github.com/en/copilot/how-tos/use-copilot-agents/coding-agent/customize-the-agent-environment","quote":"Preinstall tools or dependencies in Copilot's environment","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"cross-session-memory","role":"first-class","note":"Custom instructions + preinstalled setup steps both carry per-repo context across the agent's tasks.","evidence":[{"type":"doc","title":"Copilot coding agent — custom instructions","url":"https://docs.github.com/en/copilot/concepts/agents/coding-agent/about-coding-agent","quote":"Custom instructions allow you to give Copilot additional context on your project and how to build, test and validate its changes.","accessed":"2026-05-24"},{"type":"doc","title":"Copilot coding agent — setup steps file","url":"https://docs.github.com/en/copilot/how-tos/use-copilot-agents/coding-agent/customize-the-agent-environment","quote":"You can customize Copilot's environment by creating a special GitHub Actions workflow file, located at `.github/workflows/copilot-setup-steps.yml`","accessed":"2026-05-24"}],"evidence_status":"full"}],"instantiates":["modern-coding-agent","safety-hardening"],"alternatives":[{"composition":"devin","relation":"competes-with","note":"Peer hosted async coding agent that ships PRs."},{"composition":"openhands","relation":"competes-with","note":"Open-source self-hostable async agent."},{"composition":"claude-code","relation":"complements","note":"Often pairs with foreground terminal agents."},{"composition":"google-jules","relation":"competes-with","note":"Peer async cloud coding agent tied to a repo host."},{"composition":"marscode","relation":"competes-with","note":"MarsCode positions free-of-charge against paid Copilot."},{"composition":"sweep","relation":"competes-with","note":"GitHub's own issue-to-PR coding agent superseded the original Sweep use case."},{"composition":"tongyi-lingma","relation":"competes-with","note":"GitHub Copilot is the global leader; Lingma is Alibaba's domestic counter."}],"references":[{"type":"doc","title":"About GitHub Copilot coding agent","url":"https://docs.github.com/en/copilot/concepts/agents/coding-agent/about-coding-agent","accessed":"2026-05-20"},{"type":"doc","title":"Customize the coding agent environment","url":"https://docs.github.com/en/copilot/how-tos/use-copilot-agents/coding-agent/customize-the-agent-environment","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["github-native","async","ephemeral-runner","issue-to-pr","mcp"]},{"id":"goose","name":"Goose","kind":"framework","category":"coding-agent","build_surface":"full-code","vendor":"Block","language":"Rust (CLI), TypeScript (desktop)","license":"Apache-2.0","status":"active","first_released":"2025-01-28","url":"https://block.github.io/goose/","repo":"https://github.com/block/goose","docs_url":"https://goose-docs.ai","intent":"Block's open-source on-device general-purpose AI agent for code and workflows, with first-class MCP extensions, portable YAML recipes, parallel subagents, and a broad provider matrix.","description":"Goose is a Rust-based desktop app + CLI + API agent that runs on the user's machine and connects to 15+ LLM providers and 70+ MCP extensions. Workflows are captured as portable YAML recipes that bundle instructions, extensions, parameters, and subrecipes — shareable across teams and runnable in CI. Subagents fan out independent tasks in parallel against isolated file scopes. Computer-use is via MCP extensions (browsers, screens) rather than a native screen-grab loop.","primary_use_cases":["on-device general-purpose AI agent for code and workflows","MCP-extension-driven tool integration across 70+ tools","portable YAML recipes for shareable, CI-runnable workflows","parallel subagents for fan-out tasks"],"agent_loop_shape":"Single-process agent (desktop app or CLI) with a ReAct tool-use loop over MCP extensions. The active recipe (if any) seeds instructions, parameters, and extensions; the agent then iterates: tool call → observation → next call, until done. Subagents can be spawned to run independent tasks in parallel against isolated file scopes; subrecipes encapsulate reusable sub-workflows.","key_concepts":[{"name":"MCP extensions","summary":"70+ MCP servers wire databases, APIs, browsers, GitHub, and more.","maps_to_pattern":"mcp","url":"https://block.github.io/goose/"},{"name":"Recipes","summary":"Portable YAML workflows packaging instructions, extensions, parameters, and subrecipes.","maps_to_pattern":"skill-library","url":"https://goose-docs.ai/docs/guides/recipes/"},{"name":"Subagents","summary":"Independent agents spawned to handle tasks in parallel.","maps_to_pattern":"subagent-isolation"},{"name":"Provider matrix","summary":"15+ LLM providers (Anthropic, OpenAI, Google, Ollama, OpenRouter, Azure, Bedrock).","url":"https://block.github.io/goose/"},{"name":"Desktop + CLI + API","summary":"Three surfaces over the same core agent."}],"pattern_composition":"flowchart TD\n  user[User prompt] --> goose[Goose agent]\n  recipe[Recipe YAML] -.seeds.-> goose\n  goose --> loop[ReAct tool loop]\n  loop --> ext[MCP extensions]\n  ext --> obs[Observe]\n  obs --> loop\n  goose --> sub[Spawn subagent]\n  sub --> subloop[Subagent loop]\n  subloop --> subobs[Subagent observe]\n  subobs --> goose\n  loop --> done[Done]","members":[{"pattern":"code-execution","role":"first-class","note":"Goose can install, execute, edit, and test inside the user's machine.","evidence":[{"type":"repo","title":"block/goose — capabilities","url":"https://github.com/block/goose","quote":"install, execute, edit, and test with any LLM","accessed":"2026-05-20"},{"type":"repo","title":"block/goose — on machine","url":"https://github.com/block/goose","quote":"goose is a general-purpose AI agent that runs on your machine.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"computer-use","role":"via-integration","note":"Computer-use is wired in via MCP extensions (browsers and similar tools), not a native screen-grab loop.","evidence":[{"type":"doc","title":"Goose — extensions reach browsers","url":"https://block.github.io/goose/","quote":"Connect to 70+ extensions — databases, APIs, browsers, GitHub, Google Drive, and more — via the Model Context Protocol open standard.","accessed":"2026-05-24"},{"type":"doc","title":"Goose — extensions definition","url":"https://goose-docs.ai/","quote":"Extensions are add-ons that provide a way to extend the functionality of goose by connecting with applications and tools you already use.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"70+ MCP extensions are the documented tool surface.","evidence":[{"type":"doc","title":"Goose — MCP standard","url":"https://block.github.io/goose/","quote":"Connect to 70+ extensions — databases, APIs, browsers, GitHub, Google Drive, and more — via the Model Context Protocol open standard.","accessed":"2026-05-20"},{"type":"repo","title":"block/goose — 70+ MCP","url":"https://github.com/block/goose","quote":"Connect to 70+ extensions via the Model Context Protocol open standard.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"ReAct-style tool-use loop: model uses extension-provided tools and tool calls drive iteration until task completes.","evidence":[{"type":"repo","title":"Goose — install/execute/edit/test","url":"https://github.com/block/goose","quote":"install, execute, edit, and test with any LLM","accessed":"2026-05-24"},{"type":"doc","title":"Goose — tools are extension functions","url":"https://goose-docs.ai/docs/guides/managing-tools","quote":"Tools are specific functions within extensions that give goose its capabilities.","accessed":"2026-05-24"},{"type":"doc","title":"Goose — autonomy over tool use","url":"https://goose-docs.ai/docs/guides/managing-tools","quote":"Choose how much autonomy goose has when using tools, editing files, and taking action","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"step-budget","role":"first-class","note":"GOOSE_MAX_TURNS environment variable + recipe `max_turns` setting + per-subagent budget (GOOSE_SUBAGENT_MAX_TURNS) define a documented turn-limit step-budget knob.","evidence":[{"type":"doc","title":"Goose — GOOSE_MAX_TURNS","url":"https://goose-docs.ai/docs/guides/environment-variables/","quote":"`GOOSE_MAX_TURNS` - Maximum number of turns allowed without user input","accessed":"2026-05-24"},{"type":"doc","title":"Goose — GOOSE_SUBAGENT_MAX_TURNS","url":"https://goose-docs.ai/docs/guides/environment-variables/","quote":"Sets the maximum turns allowed for a subagent to complete before timeout","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Tools are functions within extensions; 70+ extensions provide them; autonomy/permission per tool is configurable.","evidence":[{"type":"doc","title":"Goose — tools as extension functions","url":"https://goose-docs.ai/docs/guides/managing-tools","quote":"Tools are specific functions within extensions that give goose its capabilities.","accessed":"2026-05-24"},{"type":"doc","title":"Goose — fine-grained tool permissions","url":"https://goose-docs.ai/docs/guides/managing-tools","quote":"Configure fine-grained permissions to control which tools goose can use and when","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"subagent-isolation","role":"first-class","note":"Subagents handle tasks in parallel with isolated file scopes per agent.","evidence":[{"type":"doc","title":"Goose — subagents","url":"https://block.github.io/goose/","quote":"Spawn independent subagents to handle tasks in parallel — code review, research, file processing — keeping the main conversation clean.","accessed":"2026-05-20"},{"type":"doc","title":"Goose tutorial — parallel subagents","url":"https://goose-docs.ai/docs/tutorials/subagents","quote":"Use parallel execution to run two subagents: a 🛠️ Backend Developer [and] a 💻 Frontend Developer. Both should work simultaneously — not sequentially","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"skill-library","role":"first-class","note":"Recipes (YAML) are portable, shareable workflows with subrecipes — a skill library by another name.","evidence":[{"type":"doc","title":"Goose — recipes","url":"https://block.github.io/goose/","quote":"Capture workflows as portable YAML configs. Share with your team, run in CI, include instructions, extensions, parameters, and subrecipes.","accessed":"2026-05-20"},{"type":"doc","title":"Goose recipes — reusable workflows","url":"https://goose-docs.ai/docs/guides/recipes/","quote":"Recipes are reusable workflows that package extensions, prompts, and settings together.","accessed":"2026-05-20"}],"evidence_status":"full"}],"instantiates":["modern-coding-agent","routing-and-fallback"],"alternatives":[{"composition":"claude-code","relation":"competes-with","note":"Peer first-party terminal agent."},{"composition":"codex-cli","relation":"competes-with","note":"Peer first-party terminal agent from OpenAI."},{"composition":"open-interpreter","relation":"similar-shape","note":"Both run as on-device general-purpose agents."}],"references":[{"type":"repo","title":"block/goose","url":"https://github.com/block/goose","quote":"your native open source AI agent — desktop app, CLI, and API — for code, workflows, and everything in between","accessed":"2026-05-20"},{"type":"doc","title":"Goose homepage","url":"https://block.github.io/goose/","accessed":"2026-05-20"},{"type":"doc","title":"Goose docs","url":"https://goose-docs.ai/","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"emerging","tags":["open-source","rust","mcp","desktop","cli","recipes"]},{"id":"gpt-engineer","name":"GPT Engineer","kind":"framework","category":"coding-agent","build_surface":"full-code","vendor":"Anton Osika / community","language":"Python","license":"MIT","status":"abandoned","first_released":"2023-04-30","url":"https://gptengineer.app/","repo":"https://github.com/AntonOsika/gpt-engineer","intent":"Early spec-first code-generation CLI: the user writes a natural-language prompt file describing the software, gpte generates the project end-to-end, and -i mode iterates on improvements.","description":"GPT Engineer was one of the original LLM coding agents — a Python CLI that turned a `prompt` file into a generated codebase, with an improvement loop via `gpte <project> -i`. The repo was archived by the owner on 2026-04-22 and explicitly redirects users to aider or to its commercial successor Lovable. Historically significant as the prototypical spec-first single-shot generator: drop in a prompt, watch the AI write and execute the project, then iterate.","primary_use_cases":["single-shot project generation from a prompt file","iterative improvement of a generated project via -i flag","experimenting with pre-prompts and custom agents via the benchmark harness"],"agent_loop_shape":"Spec-first one-shot generation followed by optional improvement passes. Phase 1: read the prompt file, generate the project, execute. Phase 2 (-i): take an existing codebase, ask the model for improvements, apply, repeat. Minimal tool-use surface compared to modern agents — the loop is generate → write files → execute, not a multi-tool ReAct loop.","key_concepts":[{"name":"Spec file (prompt)","summary":"Plain-text natural-language specification of the desired software.","maps_to_pattern":"spec-first-agent","url":"https://github.com/AntonOsika/gpt-engineer"},{"name":"Improve mode (-i)","summary":"Ask the AI to improve an existing project.","maps_to_pattern":"spec-driven-loop"},{"name":"Pre Prompts","summary":"Customise agent behaviour via --use-custom-preprompts."},{"name":"bench","summary":"Benchmark harness for evaluating custom agents."}],"pattern_composition":"flowchart TD\n  prompt[Prompt file] --> gpte[gpte CLI]\n  gpte --> gen[LLM generates project]\n  gen --> write[Write files]\n  write --> exec[Execute project]\n  exec --> done[Done]\n  done -.->|-i flag| improve[Ask AI to improve]\n  improve --> gen","members":[{"pattern":"code-as-action","role":"first-class","note":"Output of each call is whole files / patches written to disk — the model's emitted code is the action. AI writes and executes the code.","evidence":[{"type":"doc","title":"gpt-engineer — README","url":"https://github.com/AntonOsika/gpt-engineer","quote":"Sit back and watch as an AI writes and executes the code","accessed":"2026-05-24"},{"type":"doc","title":"gpt-engineer — README (spec)","url":"https://github.com/AntonOsika/gpt-engineer","quote":"Specify software in natural language","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"react","role":"limited","note":"Honest downgrade — the loop is generate/execute/improve, not a tool-call ReAct loop. No verbatim names ReAct.","evidence":[],"evidence_status":"none"},{"pattern":"spec-driven-loop","role":"first-class","note":"Improve mode (gpte -i) iterates on the spec/code until the user is satisfied; the natural-language specification drives the build loop.","evidence":[{"type":"doc","title":"gpt-engineer — README (specification)","url":"https://github.com/AntonOsika/gpt-engineer","quote":"Specify software in natural language","accessed":"2026-05-24"},{"type":"doc","title":"gpt-engineer — README (improve mode)","url":"https://github.com/AntonOsika/gpt-engineer","quote":"Ask the AI to implement improvements","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"spec-first-agent","role":"first-class","note":"The whole product is built around a prompt file describing the software.","evidence":[{"type":"repo","title":"gpt-engineer README — spec in natural language","url":"https://github.com/AntonOsika/gpt-engineer","quote":"Specify software in natural language","accessed":"2026-05-20"},{"type":"repo","title":"gpt-engineer README — OG codegen platform","url":"https://github.com/AntonOsika/gpt-engineer","quote":"The OG code genereation experimentation platform!","accessed":"2026-05-20"}],"evidence_status":"full"}],"instantiates":[],"alternatives":[{"composition":"lovable","relation":"successor","note":"Author explicitly names Lovable as the precursor's successor."},{"composition":"aider","relation":"competes-with","note":"README directs users to aider for a maintained hackable CLI."}],"references":[{"type":"repo","title":"AntonOsika/gpt-engineer","url":"https://github.com/AntonOsika/gpt-engineer","quote":"This repository was archived by the owner on Apr 22, 2026. It is now read-only.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"deprecated","tags":["python","archived","spec-first","historical"]},{"id":"junie","name":"JetBrains Junie","kind":"framework","category":"coding-agent","build_surface":"full-code","vendor":"JetBrains","language":"Kotlin (IDE plugin), TypeScript (CLI)","license":"proprietary","status":"active","first_released":"2025-04-16","url":"https://www.jetbrains.com/junie/","docs_url":"https://www.jetbrains.com/help/junie/","intent":"JetBrains' coding agent that lives inside the JetBrains IDE AI Chat and as a separate Junie CLI, with multi-step planning, Guidelines-as-memory, approval-gated execution, and an opt-in Brave Mode.","description":"Junie is JetBrains' first-party coding agent. It is integrated into the AI Chat across JetBrains IDEs (IntelliJ IDEA, PyCharm, WebStorm, GoLand, PhpStorm, RubyMine, RustRover, CLion, Rider, Android Studio) and also ships as Junie CLI, an LLM-agnostic terminal agent. Junie breaks tasks into multistep plans and reports back as it executes them. Sensitive actions — terminal commands, code execution, MCP tool calls — require explicit user approval by default; Brave Mode authorises Junie to run them without prompting. Guidelines provide persistent reusable context to every task.","primary_use_cases":["IDE-resident coding agent in the JetBrains AI Chat","terminal coding agent via Junie CLI (LLM-agnostic)","multistep plan-driven execution with approval gates","headless runs in CI/CD, GitHub, GitLab pipelines"],"agent_loop_shape":"Plan-then-execute loop with explicit approval. Junie ingests project Guidelines, breaks the task into a multistep plan, and executes the plan step by step while streaming progress. Sensitive steps (terminal, code execution, MCP tool calls) prompt the user unless Brave Mode is on. The same agent powers both the IDE AI Chat surface and the Junie CLI; in CLI/headless mode it runs programmatically.","key_concepts":[{"name":"Multistep plan","summary":"Junie breaks the task into a plan and reports progress as it executes.","maps_to_pattern":"plan-and-execute","url":"https://junie.jetbrains.com/docs/junie-ide-plugin.html"},{"name":"Guidelines","summary":"Persistent, reusable per-project context added to every task.","maps_to_pattern":"cross-session-memory","url":"https://junie.jetbrains.com/docs/junie-ide-plugin.html"},{"name":"Brave Mode","summary":"Opt-in mode that authorises sensitive actions without approval.","maps_to_pattern":"approval-queue","url":"https://junie.jetbrains.com/docs/junie-ide-plugin.html"},{"name":"Junie CLI","summary":"Interactive terminal agent; also runs headless in CI/CD.","url":"https://junie.jetbrains.com/docs/junie-cli.html"},{"name":"Unified AI Chat","summary":"Junie is now part of the single JetBrains AI Chat."}],"pattern_composition":"flowchart TD\n  user[User task] --> guidelines[Guidelines]\n  guidelines --> plan[Multistep plan]\n  plan --> step[Plan step]\n  step --> sensitive{Sensitive action?}\n  sensitive -->|yes / not brave| approve[Approval prompt]\n  sensitive -->|no| run[Execute]\n  sensitive -->|brave mode| run\n  approve -->|approved| run\n  approve -->|denied| step\n  run --> obs[Observe + report]\n  obs --> step\n  step --> done[Done]\n  cli[Junie CLI] -.same agent.- plan","members":[{"pattern":"agent-computer-interface","role":"first-class","note":"JetBrains IDE AI Chat is the primary ACI; Junie CLI is an interactive terminal alternative.","evidence":[{"type":"doc","title":"Junie — AI Chat integration","url":"https://junie.jetbrains.com/docs/junie-ide-plugin.html","quote":"The recommended way is to use Junie in the single AI Chat in your JetBrains IDE.","accessed":"2026-05-20"},{"type":"doc","title":"Junie CLI — terminal interface","url":"https://junie.jetbrains.com/docs/junie-cli.html","quote":"agentic coding tool by JetBrains that provides an interactive terminal interface for developers to review, write, and modify code","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"approval-queue","role":"first-class","note":"Sensitive actions need explicit approval by default; Brave Mode opt-in disables prompts.","evidence":[{"type":"doc","title":"Junie — sensitive actions","url":"https://junie.jetbrains.com/docs/junie-ide-plugin.html","quote":"Most terminal commands, code execution, and execution of MCP tools are considered to be sensitive actions, and Junie by default requires explicit approval from the user.","accessed":"2026-05-20"},{"type":"doc","title":"Junie — Brave Mode","url":"https://junie.jetbrains.com/docs/junie-ide-plugin.html","quote":"You can authorize Junie to execute all potentially sensitive actions without user approval by selecting Brave Mode.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"Iterative tool-use loop wrapped in plan/execute reporting; autonomously runs commands, creates files, edits, runs tests, verifies.","evidence":[{"type":"doc","title":"Junie — plan + execute + report","url":"https://junie.jetbrains.com/docs/junie-ide-plugin.html","quote":"Junie breaks the task into a multistep plan and executes the proposed plan while reporting back to the user on the progress.","accessed":"2026-05-20"},{"type":"doc","title":"Junie — autonomous run/edit/test/verify loop","url":"https://junie.jetbrains.com/docs/junie-ide-plugin.html","quote":"it can autonomously run terminal commands, create new files, write or edit code, run tests, and verify the changes.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Tool surface includes terminal commands, code execution, and MCP tool calls.","evidence":[{"type":"doc","title":"Junie — tool surface","url":"https://junie.jetbrains.com/docs/junie-ide-plugin.html","quote":"Most terminal commands, code execution, and execution of MCP tools are considered to be sensitive actions","accessed":"2026-05-20"},{"type":"doc","title":"Junie CLI — sensitive actions list","url":"https://junie.jetbrains.com/docs/junie-cli-usage.html","quote":"For running potentially sensitive actions, such as executing most of the terminal commands, editing files outside the project, or invoking MCP tools, Junie will ask for approval from the user.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"plan-and-execute","role":"first-class","note":"Multistep plan is generated then executed step by step with progress reporting; CLI has dedicated plan mode.","evidence":[{"type":"doc","title":"Junie — multistep plan","url":"https://junie.jetbrains.com/docs/junie-ide-plugin.html","quote":"Junie breaks the task into a multistep plan and executes the proposed plan while reporting back to the user on the progress.","accessed":"2026-05-20"},{"type":"doc","title":"Junie CLI — plan mode","url":"https://junie.jetbrains.com/docs/junie-cli-usage.html","quote":"In plan mode, Junie CLI analyzes the codebase with read-only operations and produces a design document for the task before any code is written.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"cross-session-memory","role":"first-class","note":"Guidelines provide persistent reusable context added to every task; stored in AGENTS.md / .junie/AGENTS.md.","evidence":[{"type":"doc","title":"Junie — Guidelines","url":"https://junie.jetbrains.com/docs/junie-ide-plugin.html","quote":"Guidelines allow you to provide persistent, reusable context to the agent. Junie adds this context to every task it works on.","accessed":"2026-05-24"},{"type":"doc","title":"Junie — guidelines and memory","url":"https://junie.jetbrains.com/docs/guidelines-and-memory.html","quote":"Guidelines allow you to provide persistent, reusable context to the agent.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"Junie can call MCP tools (treated as sensitive actions); MCP server connection is a documented seam.","evidence":[{"type":"doc","title":"Junie — MCP server connection","url":"https://junie.jetbrains.com/docs/junie-ide-plugin.html","quote":"You can connect Junie to Model Context Protocol (MCP) servers. This will provide Junie with executable functionality for working with data sources and tools, such as file systems, productivity tools, or databases.","accessed":"2026-05-24"},{"type":"doc","title":"Junie — MCP tool execution","url":"https://junie.jetbrains.com/docs/junie-ide-plugin.html","quote":"Most terminal commands, code execution, and execution of MCP tools are considered to be sensitive actions","accessed":"2026-05-20"}],"evidence_status":"full"}],"instantiates":["modern-coding-agent","safety-hardening"],"alternatives":[{"composition":"cursor","relation":"competes-with","note":"Peer IDE-based agent (Cursor is its own IDE, Junie lives inside JetBrains IDEs)."},{"composition":"claude-code","relation":"competes-with","note":"Peer terminal agent for Junie CLI."},{"composition":"codex-cli","relation":"competes-with","note":"Peer terminal agent for Junie CLI."},{"composition":"sweep","relation":"competes-with","note":"JetBrains' own first-party agent competes directly with the current Sweep JetBrains plugin."}],"references":[{"type":"doc","title":"Junie homepage","url":"https://www.jetbrains.com/junie/","quote":"Junie, the AI coding agent by JetBrains","accessed":"2026-05-20"},{"type":"doc","title":"Junie docs","url":"https://junie.jetbrains.com/docs/","accessed":"2026-05-20"},{"type":"doc","title":"Junie IDE plugin docs","url":"https://junie.jetbrains.com/docs/junie-ide-plugin.html","accessed":"2026-05-20"},{"type":"doc","title":"Junie CLI docs","url":"https://junie.jetbrains.com/docs/junie-cli.html","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"emerging","tags":["jetbrains","ide","cli","plan-and-execute","brave-mode"]},{"id":"lovable","name":"Lovable","kind":"framework","category":"coding-agent","build_surface":"no-code","vendor":"Lovable","language":"N/A","license":"proprietary","status":"active","first_released":"2024-06-10","url":"https://lovable.dev/","docs_url":"https://docs.lovable.dev/","intent":"Chat-driven full-stack app builder — the model generates a working frontend, backend, database, auth, and integrations, with editable code, Plan/Agent mode separation, GitHub sync, and Lovable Cloud (Supabase-backed) as the default backend.","description":"Lovable is a hosted prompt-to-app product: users describe what they want, Lovable generates an editable codebase with frontend, backend, database, authentication, and integrations. Plan mode brainstorms and decides before code is written; Agent mode is the autonomous execution mode that applies changes directly, inspects logs and runtime, and iterates until issues are resolved. Projects sync bidirectionally with GitHub. Lovable Cloud is the built-in backend (database, auth, storage, edge functions) running on a Supabase open-source foundation. Lovable is positioned as the commercial successor to gpt-engineer.","primary_use_cases":["chat-driven full-stack app generation with editable code","Plan-then-Agent execution loop with autonomous iteration","GitHub-synced projects for downstream engineering workflows","Lovable Cloud backend (Supabase) with database, auth, storage, edge functions"],"agent_loop_shape":"Two-mode loop. Plan mode is a read-only brainstorming/decision surface — ask questions, debug, compare approaches — with no code changes. Once a plan is approved, Agent mode takes ownership end-to-end: applies file diffs, runs the app, inspects logs/runtime/network, iterates until resolved or clarification is needed. All changes flow through an editable code tree that can be synced to GitHub.","key_concepts":[{"name":"Agent mode","summary":"Autonomous execution mode that applies changes and iterates.","maps_to_pattern":"spec-driven-loop","url":"https://docs.lovable.dev/features/agent-mode"},{"name":"Plan mode","summary":"Read-only brainstorming and decision before any code is written.","maps_to_pattern":"plan-and-execute","url":"https://docs.lovable.dev/features/plan-mode"},{"name":"Lovable Cloud","summary":"Built-in Supabase-based backend (database, auth, storage, edge functions).","url":"https://docs.lovable.dev/integrations/cloud"},{"name":"GitHub sync","summary":"Bidirectional sync between Lovable and a GitHub repository.","url":"https://docs.lovable.dev/integrations/github"},{"name":"Editable code","summary":"Generated codebase is editable and synced to GitHub for downstream workflows."}],"pattern_composition":"flowchart TD\n  user[User prompt] --> mode{Mode}\n  mode -->|Plan| plan[Brainstorm + decide]\n  plan --> approve[Plan approved]\n  approve --> agent\n  mode -->|Agent| agent[Agent mode loop]\n  agent --> apply[Apply diff to codebase]\n  apply --> run[Run app]\n  run --> obs[Inspect logs / runtime / network]\n  obs --> agent\n  apply --> gh[GitHub sync]\n  agent --> cloud[Lovable Cloud / Supabase]","members":[{"pattern":"code-as-action","role":"first-class","note":"Agent mode applies file diffs directly to the project; diffs and summaries are visible.","evidence":[{"type":"doc","title":"Lovable — Agent applies diffs","url":"https://docs.lovable.dev/features/agent-mode","quote":"Lovable applies changes directly to your project when using Agent mode. All modifications are visible through file diffs and summaries.","accessed":"2026-05-20"},{"type":"doc","title":"Lovable — generates working app","url":"https://docs.lovable.dev/introduction","quote":"You describe what you want to build, and Lovable generates a working application that includes frontend, backend, database, authentication, and integrations, all backed by editable code.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"spec-first-agent","role":"first-class","note":"User describes the desired app; Lovable spins up the project from that natural-language spec.","evidence":[{"type":"doc","title":"Lovable — chat-to-app","url":"https://lovable.dev/","quote":"Create apps and websites by chatting with AI","accessed":"2026-05-20"},{"type":"doc","title":"Lovable — idea to software","url":"https://docs.lovable.dev/introduction","quote":"Move from idea to working software by building through natural language instead of starting from a blank codebase.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Agent inspects logs/runtime/network and uses verification tools (browser tests, frontend tests, edge function checks) plus integrations (GitHub, Supabase-backed Cloud, MCP servers).","evidence":[{"type":"doc","title":"Lovable — inspects runtime","url":"https://docs.lovable.dev/features/agent-mode","quote":"Lovable can inspect logs, runtime output, and network activity and iterate on fixes until the issue is resolved or clarified.","accessed":"2026-05-20"},{"type":"doc","title":"Lovable — verification tools","url":"https://docs.lovable.dev/features/agent-mode","quote":"Lovable has access to verification tools such as browser testing, frontend tests, and edge function verification","accessed":"2026-05-24"},{"type":"doc","title":"Lovable Cloud — Supabase foundation","url":"https://docs.lovable.dev/integrations/cloud","quote":"Lovable Cloud is a full-stack hosting platform with built-in database, authentication, storage, edge functions, and AI, no infrastructure setup required.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"plan-and-execute","role":"first-class","note":"Plan mode is for decision-making; Agent mode is for execution — documented as paired modes.","evidence":[{"type":"doc","title":"Lovable — Plan vs Agent","url":"https://docs.lovable.dev/features/plan-mode","quote":"Plan mode is for decision-making. Agent mode is for execution.","accessed":"2026-05-20"},{"type":"doc","title":"Lovable — editable plans","url":"https://docs.lovable.dev/features/plan-mode","quote":"Plans are fully editable before approval","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"spec-driven-loop","role":"first-class","note":"Agent mode is the end-to-end execution loop: take ownership, apply changes, iterate until resolved.","evidence":[{"type":"doc","title":"Lovable — end-to-end execution","url":"https://docs.lovable.dev/features/agent-mode","quote":"When you give Lovable a task, it takes ownership of execution end to end.","accessed":"2026-05-20"},{"type":"doc","title":"Lovable — iterate on fixes","url":"https://docs.lovable.dev/features/agent-mode","quote":"Lovable can inspect logs, runtime output, and network activity and iterate on fixes until the issue is resolved or clarified.","accessed":"2026-05-20"}],"evidence_status":"full"}],"instantiates":["modern-coding-agent"],"alternatives":[{"composition":"boltnew","relation":"competes-with","note":"Peer browser-based prompt-to-app from StackBlitz."},{"composition":"v0","relation":"competes-with","note":"Peer prompt-to-app from Vercel."},{"composition":"replit-agent","relation":"competes-with","note":"Peer hosted prompt-to-app on Replit."},{"composition":"gpt-engineer","relation":"predecessor","note":"gpt-engineer README names Lovable as its successor."}],"references":[{"type":"doc","title":"Lovable homepage","url":"https://lovable.dev/","accessed":"2026-05-20"},{"type":"doc","title":"Lovable docs","url":"https://docs.lovable.dev/","accessed":"2026-05-20"},{"type":"doc","title":"Lovable — Agent mode","url":"https://docs.lovable.dev/features/agent-mode","accessed":"2026-05-20"},{"type":"doc","title":"Lovable — Plan mode","url":"https://docs.lovable.dev/features/plan-mode","accessed":"2026-05-20"},{"type":"doc","title":"Lovable Cloud","url":"https://docs.lovable.dev/integrations/cloud","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["web-product","prompt-to-app","supabase","github-sync","plan-and-agent"]},{"id":"marscode","name":"MarsCode","kind":"framework","category":"coding-agent","build_surface":"full-code","vendor":"ByteDance (Doubao — 豆包 MarsCode)","language":"TypeScript (extension), Go (cloud IDE backend)","license":"proprietary (free)","status":"active","url":"https://www.marscode.com/","docs_url":"https://docs.marscode.com/","intent":"ByteDance's free coding assistant ecosystem comprising a cloud IDE and VS Code/JetBrains extensions, powered by Doubao models, with completion, generation, explanation, and bug-fix agent capabilities.","description":"MarsCode is ByteDance's coding assistant family under the Doubao brand. It is uniquely positioned as fully free (including the cloud IDE tier), targeting individual developers and learners with VS Code/JetBrains extensions plus a browser-based cloud IDE pre-configured per language. Beyond completion and chat, it offers an agent mode for multi-step bug fixing and feature implementation. Powered by Doubao large models, it competes domestically with Tongyi Lingma, CodeBuddy, and Comate, distinguishing on price (free) and the integrated cloud-IDE story.","primary_use_cases":["free code completion and chat in VS Code and JetBrains","browser-based cloud IDE for learning and prototyping","bug-fix agent (analyse → propose fix → apply) for individual developers","education and student onboarding to AI-assisted coding"],"agent_loop_shape":"Extension or cloud IDE that calls hosted Doubao models. Completion is single-shot per cursor event; chat runs a tool-augmented loop with file-read / file-write / shell tools; bug-fix agent runs a short plan-and-execute loop scoped to a file or workspace.","key_concepts":[{"name":"MarsCode Cloud IDE","summary":"Browser-based development environment pre-configured per language and pre-paired with the assistant.","maps_to_pattern":"agent-computer-interface"},{"name":"Doubao Code","summary":"ByteDance's coding-tuned Doubao model family that backs the assistant."},{"name":"Free-tier positioning","summary":"Distinguishing market position vs. paid Cursor / Copilot tier — free including the cloud-IDE surface."},{"name":"Bug-fix agent mode","summary":"Plan-and-execute loop scoped to a single bug report or workspace.","maps_to_pattern":"plan-and-execute"}],"pattern_composition":"flowchart TD\n  USER[Developer] --> ENTRY{Surface}\n  ENTRY -->|Extension| EXT[VSCode / JetBrains]\n  ENTRY -->|Cloud| IDE[MarsCode Cloud IDE<br/>browser]\n  EXT --> DOUBAO[Doubao Code model]\n  IDE --> DOUBAO\n  DOUBAO --> MODE{Mode}\n  MODE -->|completion| RET[Return completion]\n  MODE -->|chat| LOOP[Tool-augmented loop<br/>file / shell tools]\n  MODE -->|bug-fix agent| PLAN[Plan and execute<br/>scoped to file/workspace]\n  LOOP --> RET\n  PLAN --> RET\n  RET --> ENTRY\n  PLAN -.confirm edits.-> HITL[Human approves]","members":[{"pattern":"react","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"tool-use","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"code-as-action","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"agent-computer-interface","role":"first-class","note":"Cloud IDE + IDE extension.","evidence":[],"evidence_status":"none"},{"pattern":"plan-and-execute","role":"supported","note":"Bug-fix and multi-step agent flows.","evidence":[],"evidence_status":"none"},{"pattern":"step-budget","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"human-in-the-loop","role":"first-class","evidence":[],"evidence_status":"none"}],"alternatives":[{"composition":"tongyi-lingma","relation":"competes-with"},{"composition":"codegeex","relation":"competes-with"},{"composition":"comate","relation":"competes-with"},{"composition":"codebuddy","relation":"competes-with"},{"composition":"trae","relation":"same-vendor","note":"Both are ByteDance coding products on Doubao; Trae is the standalone IDE, MarsCode the extension + cloud IDE."},{"composition":"github-copilot-coding-agent","relation":"competes-with","note":"MarsCode positions free-of-charge against paid Copilot."}],"last_analyzed":"2026-05-22","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-05-22","status_in_practice":"mature","tags":["coding-agent","planning-control-flow","tool-use-environment","safety-control","full-code"],"references":[{"type":"doc","title":"MarsCode — Documentation","url":"https://docs.marscode.com/"},{"type":"doc","title":"MarsCode — Homepage","url":"https://www.marscode.com/"}]},{"id":"open-interpreter","name":"Open Interpreter","kind":"framework","category":"coding-agent","build_surface":"full-code","vendor":"Open Interpreter","language":"Python","license":"AGPL-3.0","status":"active","first_released":"2023-07-14","url":"https://www.openinterpreter.com/","repo":"https://github.com/OpenInterpreter/open-interpreter","docs_url":"https://docs.openinterpreter.com/","intent":"Open-source local code-execution agent: equips a function-calling LLM with an exec() function so it can run Python, JavaScript, Shell, and more on the user's machine, with default human approval and an experimental OS mode for visual desktop control.","description":"Open Interpreter runs as a terminal chat (`interpreter`) or as a Python library. Under the hood it gives a function-calling model an exec() function and lets it execute generated code locally across Python, JavaScript, Shell, and more. By default each block of code requires user approval before it runs; `auto_run` / `interpreter -y` bypasses that. OS mode (--os) is an experimental computer-use mode where the model captures screenshots and drives the mouse/keyboard. Safe mode is an experimental code-scanning layer with off/ask/auto settings.","primary_use_cases":["natural-language interface to the user's computer via local code execution","ChatGPT Code Interpreter-style workflows running locally","ad-hoc data, file, and OS automation across multiple languages","experimental visual desktop control via OS mode"],"agent_loop_shape":"ReAct-style local loop. Function-calling LLM emits a code block in a chosen language; the runtime asks the user to approve, then executes via exec() and feeds stdout/stderr/return values back. With auto_run on, approval is skipped. OS mode swaps the textual exec() loop for a screenshot/mouse/keyboard loop driven by a multimodal model.","key_concepts":[{"name":"exec() over multiple languages","summary":"Local function-calling LLM with an exec() function for Python, JavaScript, Shell, etc.","maps_to_pattern":"code-execution","url":"https://github.com/OpenInterpreter/open-interpreter"},{"name":"Approval per code block","summary":"Default behaviour: user approves each block before it runs.","maps_to_pattern":"approval-queue","url":"https://github.com/OpenInterpreter/open-interpreter"},{"name":"auto_run","summary":"Bypass approval (`interpreter -y` or `interpreter.auto_run = True`).","url":"https://docs.openinterpreter.com/settings/all-settings"},{"name":"OS mode","summary":"Experimental visual desktop control via screenshots, mouse, and keyboard.","maps_to_pattern":"computer-use","url":"https://docs.openinterpreter.com/guides/os-mode"},{"name":"Safe mode","summary":"Experimental code-scanning with off/ask/auto settings."}],"pattern_composition":"flowchart TD\n  user[User chat] --> llm[Function-calling LLM]\n  llm --> codeblock[Code block in language]\n  codeblock --> approve{auto_run?}\n  approve -->|no| ask[Ask user]\n  approve -->|yes| exec\n  ask -->|approved| exec[exec()]\n  ask -->|denied| llm\n  exec --> obs[stdout / stderr / value]\n  obs --> llm\n  llm --> done[Done]\n  osmode[OS mode --os] -.replaces exec() loop.-> screen[Screenshots + mouse/keyboard]","members":[{"pattern":"approval-queue","role":"first-class","note":"By default the user approves each code block before execution; auto_run bypasses.","evidence":[{"type":"repo","title":"Open Interpreter README — approval before run","url":"https://github.com/OpenInterpreter/open-interpreter","quote":"You'll be asked to approve code before it's run","accessed":"2026-05-20"},{"type":"repo","title":"Open Interpreter README — -y bypass","url":"https://github.com/OpenInterpreter/open-interpreter","quote":"You can run `interpreter -y` or set `interpreter.auto_run = True` to bypass confirmation.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"code-as-action","role":"first-class","note":"Action is always a code block emitted by the model and executed via exec(); Open Interpreter equips a function-calling model with an exec() function.","evidence":[{"type":"doc","title":"Open Interpreter — README","url":"https://github.com/OpenInterpreter/open-interpreter","quote":"Open Interpreter lets LLMs run code (Python, Javascript, Shell, and more) locally.","accessed":"2026-05-24"},{"type":"doc","title":"Open Interpreter — README (exec)","url":"https://github.com/OpenInterpreter/open-interpreter","quote":"Open Interpreter equips a [function-calling language model](https://platform.openai.com/docs/guides/function-calling) with an `exec()` function, which accepts a `language` (like \"Python\" or \"JavaScript\") and `code` to run.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"code-execution","role":"first-class","note":"Runs Python, JavaScript, Shell, and more locally.","evidence":[{"type":"repo","title":"Open Interpreter README — local code execution","url":"https://github.com/OpenInterpreter/open-interpreter","quote":"Open Interpreter lets LLMs run code (Python, Javascript, Shell, and more) locally.","accessed":"2026-05-20"},{"type":"doc","title":"Open Interpreter docs — run code","url":"https://docs.openinterpreter.com/","quote":"Open Interpreter lets language models run code.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"computer-use","role":"supported","note":"OS mode is documented as an experimental visual computer-use mode with screenshot + mouse/keyboard control.","evidence":[{"type":"doc","title":"Open Interpreter — OS mode","url":"https://docs.openinterpreter.com/guides/os-mode","quote":"OS mode is a highly experimental mode that allows Open Interpreter to control the operating system visually through the mouse and keyboard.","accessed":"2026-05-20"},{"type":"doc","title":"Open Interpreter — multimodal screenshots","url":"https://docs.openinterpreter.com/guides/os-mode","quote":"It provides a multimodal LLM like GPT-4V with the necessary tools to capture screenshots of the display and interact with on-screen elements such as text and icons.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Function-calling model is the substrate; exec() is the universal tool, with the computer module exposing additional language-specific runners.","evidence":[{"type":"doc","title":"Open Interpreter — README (function-calling)","url":"https://github.com/OpenInterpreter/open-interpreter","quote":"Open Interpreter equips a [function-calling language model](https://platform.openai.com/docs/guides/function-calling) with an `exec()` function, which accepts a `language` (like \"Python\" or \"JavaScript\") and `code` to run.","accessed":"2026-05-24"},{"type":"doc","title":"Open Interpreter — Code Execution (computer module)","url":"https://docs.openinterpreter.com/code-execution/usage","quote":"You also have control over the `computer`'s languages (like Python, Javascript, and Shell), and can easily append custom languages","accessed":"2026-05-24"}],"evidence_status":"full"}],"instantiates":["modern-coding-agent"],"alternatives":[{"composition":"goose","relation":"similar-shape","note":"Both are on-device general-purpose agents."},{"composition":"claude-code","relation":"competes-with","note":"Peer terminal coding agent."},{"composition":"codex-cli","relation":"competes-with","note":"Peer terminal coding agent."}],"references":[{"type":"repo","title":"OpenInterpreter/open-interpreter","url":"https://github.com/OpenInterpreter/open-interpreter","accessed":"2026-05-20"},{"type":"doc","title":"Open Interpreter docs","url":"https://docs.openinterpreter.com/","accessed":"2026-05-20"},{"type":"doc","title":"Open Interpreter — OS mode","url":"https://docs.openinterpreter.com/guides/os-mode","accessed":"2026-05-20"},{"type":"doc","title":"Open Interpreter — settings","url":"https://docs.openinterpreter.com/settings/all-settings","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","python","terminal","local-exec","multilingual-code"]},{"id":"openhands","name":"OpenHands","aliases":["OpenDevin"],"kind":"framework","category":"coding-agent","build_surface":"full-code","vendor":"All Hands AI","language":"Python","license":"MIT (enterprise/ excluded)","status":"active","first_released":"2024-03-13","url":"https://github.com/All-Hands-AI/OpenHands","repo":"https://github.com/All-Hands-AI/OpenHands","docs_url":"https://docs.openhands.dev/","intent":"Open-source AI software-development platform: a composable Python SDK plus CLI, local GUI, and cloud GUI that runs tool-using agents inside an isolated sandbox (Docker / process / remote) with MCP tool extension and AGENTS.md-based skills.","description":"OpenHands (formerly OpenDevin) is a community project for AI-driven development. The core is a composable Python SDK that runs an agent loop: an LLM-driven agent executes bash, edits files, browses the web, and reaches external tools via MCP. Execution happens inside a configurable sandbox (Docker by default, process or remote alternatives), which gives strong host isolation. Repository-wide instructions live in AGENTS.md (or CLAUDE.md / GEMINI.md); on-demand skills live under .agents/skills/ or the legacy .openhands/microagents/ directory. OpenHands ships as CLI, Local GUI, and OpenHands Cloud on the same SDK.","primary_use_cases":["sandboxed autonomous software-development agent","bash + file-edit + browser tool use inside Docker","MCP-extended toolchain for code agents","AGENTS.md-driven project skills","self-hosted alternative to closed cloud coding agents"],"agent_loop_shape":"Tool-using agent loop: the agent proposes a tool call (bash, file edit, browser, or MCP-provided tool); the call executes inside the configured sandbox; the resulting observation is returned to the model; the loop repeats until the task is done. AGENTS.md content is injected at conversation start; on-demand skills under .agents/skills/ are summarised first and read fully only when needed.","key_concepts":[{"name":"Sandbox","summary":"Execution environment where OpenHands runs commands, edits files, and starts servers. Docker / process / remote.","maps_to_pattern":"sandbox-isolation","url":"https://docs.openhands.dev/usage/runtimes/overview"},{"name":"Agent / Tools / Workspace / Conversation","summary":"Core SDK abstractions: an LLM-powered Agent uses Tools inside a Workspace via a Conversation.","url":"https://docs.openhands.dev/sdk/getting-started"},{"name":"MCP","summary":"Open standard for external tool servers; OpenHands routes tool calls to configured MCP servers.","maps_to_pattern":"mcp","url":"https://docs.openhands.dev/overview/model-context-protocol"},{"name":"AGENTS.md and skills","summary":"Always-on AGENTS.md for repo-wide instructions; on-demand skills under .agents/skills/ or legacy .openhands/microagents/.","maps_to_pattern":"agent-skills","url":"https://docs.openhands.dev/usage/prompting/microagents-overview"},{"name":"Multi-interface SDK","summary":"Same agent runs as CLI, Local GUI, or OpenHands Cloud on a shared SDK.","url":"https://github.com/All-Hands-AI/OpenHands"}],"pattern_composition":"flowchart TD\n  user[User task] --> ctx[AGENTS.md + skills context]\n  ctx --> loop[Agent loop]\n  loop --> propose[LLM proposes tool call]\n  propose --> sandbox{Sandbox}\n  sandbox -->|docker| dock[Docker isolation]\n  sandbox -->|process| proc[Local process]\n  sandbox -->|remote| rem[Remote sandbox]\n  dock --> exec[Execute action]\n  proc --> exec\n  rem --> exec\n  exec --> bash[Bash]\n  exec --> edit[File edit]\n  exec --> browser[Browser]\n  exec --> mcp[MCP tool]\n  bash --> obs[Observation]\n  edit --> obs\n  browser --> obs\n  mcp --> obs\n  obs --> loop\n  loop --> done[Done]","members":[{"pattern":"agent-skills","role":"first-class","note":"AGENTS.md is always-on context; on-demand skills under .agents/skills/ are summarised then loaded on demand.","evidence":[{"type":"doc","title":"OpenHands docs - skills","url":"https://docs.openhands.dev/usage/prompting/microagents-overview","quote":"Skills are specialized prompts that enhance OpenHands with domain-specific knowledge, expert guidance, and automated task handling.","accessed":"2026-05-20"},{"type":"doc","title":"OpenHands docs - on-demand skills","url":"https://docs.openhands.dev/usage/prompting/microagents-overview","quote":"On-demand skills help keep the system prompt smaller because the agent sees a summary first and reads the full content only when needed.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"code-as-action","role":"first-class","note":"Tools include bash + file edit + browser; tools are the agent's capability surface for acting on code.","evidence":[{"type":"doc","title":"OpenHands SDK - actions","url":"https://docs.openhands.dev/sdk/getting-started","quote":"execute bash commands, edit files, browse the web, and more","accessed":"2026-05-20"},{"type":"doc","title":"OpenHands SDK — tools as capabilities","url":"https://docs.openhands.dev/sdk/getting-started","quote":"Capabilities like executing bash commands, editing files, or browsing the web.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"code-execution","role":"first-class","note":"Bash and code execution run inside Docker sandbox; process and remote alternatives also documented.","evidence":[{"type":"doc","title":"OpenHands sandbox","url":"https://docs.openhands.dev/usage/runtimes/overview","quote":"A sandbox is the environment where OpenHands runs commands, edits files, and starts servers while working on your task.","accessed":"2026-05-20"},{"type":"doc","title":"OpenHands sandbox — Docker isolation","url":"https://docs.openhands.dev/usage/runtimes/overview","quote":"Runs the agent server inside a Docker container.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"computer-use","role":"supported","note":"BrowserToolSet enables web navigation, clicking, extraction; honest downgrade from first-class because this is web browsing, not OS mouse/screen control.","evidence":[{"type":"doc","title":"OpenHands BrowserToolSet","url":"https://docs.openhands.dev/sdk/guides/agent-browser-use","quote":"The BrowserToolSet integration enables your agent to interact with web pages through automated browser control.","accessed":"2026-05-24"},{"type":"doc","title":"OpenHands browser — interactions","url":"https://docs.openhands.dev/sdk/guides/agent-browser-use","quote":"Interact with web page elements (clicking, scrolling, etc.)","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"First-class MCP client across CLI, SDK, Local GUI, and Cloud; supports SSE/SHTTP/stdio transports.","evidence":[{"type":"doc","title":"OpenHands MCP overview","url":"https://docs.openhands.dev/overview/model-context-protocol","quote":"Model Context Protocol (MCP) is an open standard that allows OpenHands to communicate with external tool servers, extending the agent's capabilities with custom tools","accessed":"2026-05-20"},{"type":"doc","title":"OpenHands MCP — full capability scope","url":"https://docs.openhands.dev/overview/model-context-protocol","quote":"extending the agent's capabilities with custom tools, specialized data processing, external API access, and more.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"Reason-then-tool-call loop with observations fed back; Workspace is where actions execute and observations come from.","evidence":[{"type":"doc","title":"OpenHands SDK - agent definition","url":"https://docs.openhands.dev/sdk/getting-started","quote":"An AI-powered entity that can reason, plan, and execute actions using tools","accessed":"2026-05-20"},{"type":"doc","title":"OpenHands SDK — workspace","url":"https://docs.openhands.dev/sdk/getting-started","quote":"The execution environment where agents operate (local, Docker, or remote).","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"sandbox-isolation","role":"first-class","note":"Docker sandbox is the recommended isolation boundary; process and remote sandboxes exist as alternatives.","evidence":[{"type":"doc","title":"OpenHands docker sandbox","url":"https://docs.openhands.dev/usage/runtimes/overview","quote":"Runs the agent server inside a Docker container","accessed":"2026-05-20"},{"type":"doc","title":"OpenHands sandbox definition","url":"https://docs.openhands.dev/usage/runtimes/overview","quote":"A sandbox is the environment where OpenHands runs commands, edits files, and starts servers while working on your task.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Modular tool system; native bash/edit/browser tools plus MCP-discovered tools.","evidence":[{"type":"doc","title":"OpenHands SDK - tool-based system","url":"https://docs.openhands.dev/sdk/getting-started","quote":"execute bash commands, edit files, browse the web, and more","accessed":"2026-05-20"},{"type":"doc","title":"OpenHands MCP - external tools","url":"https://docs.openhands.dev/overview/model-context-protocol","quote":"extending the agent's capabilities with custom tools, specialized data processing, external API access","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"context-window-packing","role":"limited","note":"Summary-first on-demand skill loading is the explicit context-packing tactic; AGENTS.md is always-on context injected at start.","evidence":[{"type":"doc","title":"OpenHands docs - summary-first skill loading","url":"https://docs.openhands.dev/usage/prompting/microagents-overview","quote":"the agent sees a summary first and reads the full content only when needed","accessed":"2026-05-20"},{"type":"doc","title":"OpenHands docs — skills as targeted prompts","url":"https://docs.openhands.dev/usage/prompting/microagents-overview","quote":"Skills are specialized prompts that enhance OpenHands with domain-specific knowledge, expert guidance, and automated task handling.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"subagent-isolation","role":"first-class","note":"Sub-Agent Delegation is a documented SDK primitive — parallel work with independent contexts; consolidated results returned to main agent.","evidence":[{"type":"repo","title":"OpenHands README - scale","url":"https://github.com/All-Hands-AI/OpenHands","quote":"Define agents in code, then run them locally, or scale to 1000s of agents in the cloud.","accessed":"2026-05-20"},{"type":"doc","title":"OpenHands SDK — sub-agent delegation","url":"https://docs.openhands.dev/sdk/guides/agent-delegation","quote":"Enable parallel task execution by delegating work to multiple sub-agents that run independently and return consolidated results.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"agent-resumption","role":"first-class","note":"CLI supports `openhands --resume <id>` and `--last`; SDK persists conversation state to disk for multi-session workflows.","evidence":[{"type":"doc","title":"OpenHands CLI — --resume","url":"https://docs.openhands.dev/openhands/usage/cli/resume","quote":"To resume a specific conversation, use the `--resume` flag with the conversation ID: `openhands --resume <conversation-id>`","accessed":"2026-05-24"},{"type":"doc","title":"OpenHands SDK — persistence","url":"https://docs.openhands.dev/sdk/guides/convo-persistence","quote":"Save conversation state to disk and restore it later for long-running or multi-session workflows.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"step-budget","role":"limited","note":"Iterative-refinement guide documents MAX_ITERATIONS + quality-threshold as caps on agent loops; not a default-on session-wide cap, so 'limited'.","evidence":[{"type":"doc","title":"OpenHands SDK — iterative refinement loop","url":"https://docs.openhands.dev/sdk/guides/iterative-refinement","quote":"Iterative refinement is a powerful pattern where multiple agents work together in a feedback loop","accessed":"2026-05-24"},{"type":"doc","title":"OpenHands SDK — explicit iteration cap","url":"https://docs.openhands.dev/sdk/guides/iterative-refinement","quote":"while current_score < QUALITY_THRESHOLD and iteration < MAX_ITERATIONS","accessed":"2026-05-24"}],"evidence_status":"full"}],"instantiates":["modern-coding-agent","safety-hardening"],"alternatives":[{"composition":"claude-code","relation":"competes-with","note":"Peer agentic coding tool, closed source."},{"composition":"codex-cli","relation":"competes-with","note":"Peer first-party terminal agent."},{"composition":"devin","relation":"competes-with","note":"OpenDevin was the original name; Devin is the proprietary peer."},{"composition":"codefuse","relation":"similar-shape","note":"Both are open-source coding-agent stacks; OpenHands is Western, CodeFuse is Chinese."},{"composition":"github-copilot-coding-agent","relation":"competes-with","note":"Open-source self-hostable async agent."}],"references":[{"type":"repo","title":"All-Hands-AI/OpenHands","url":"https://github.com/All-Hands-AI/OpenHands","accessed":"2026-05-20"},{"type":"doc","title":"OpenHands docs","url":"https://docs.openhands.dev/","accessed":"2026-05-20"},{"type":"doc","title":"OpenHands sandbox","url":"https://docs.openhands.dev/usage/runtimes/overview","accessed":"2026-05-20"},{"type":"doc","title":"OpenHands MCP","url":"https://docs.openhands.dev/overview/model-context-protocol","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","python","mit","sandbox","mcp","agents-md"]},{"id":"plandex","name":"Plandex","kind":"framework","category":"coding-agent","build_surface":"full-code","vendor":"Plandex AI","language":"Go","license":"AGPL-3.0 / MIT (mixed; see repo)","status":"active","url":"https://github.com/plandex-ai/plandex","repo":"https://github.com/plandex-ai/plandex","docs_url":"https://github.com/plandex-ai/plandex/tree/main/docs","intent":"Open-source terminal AI coding agent for large projects: stages edits and command execution in a cumulative diff-review sandbox, supports configurable autonomy from full-auto to step-by-step, and version-controls every plan with branches.","description":"Plandex is a Go terminal agent aimed at multi-file, large-scope coding tasks. Its central idea is a sandbox of tentative changes: AI-generated diffs sit outside the project files until reviewed and applied. Plans are first-class objects with their own version history and branches, so multiple paths or model comparisons can run side by side. Autonomy is a slider - None, Basic, Plus, Semi-Auto, Full-Auto - controlling what the agent may do without confirmation. A REPL with fuzzy completion and tree-sitter-driven project maps round out the loop.","primary_use_cases":["long-running multi-file refactors in a terminal","tentative-diff review before files mutate","configurable autonomy from step-by-step to full auto","branching plans for exploring alternatives"],"agent_loop_shape":"Plan-and-execute over a tentative-diff sandbox. The user gives a task; Plandex builds a project map, drafts a plan, generates diffs and commands inside the sandbox, optionally runs commands with rollback, and only applies edits to project files when the user (or the configured autonomy level) accepts them. Plans are versioned; branches let multiple plans coexist.","key_concepts":[{"name":"Cumulative diff-review sandbox","summary":"AI-generated changes stay separate from project files until applied.","maps_to_pattern":"approval-queue","url":"https://github.com/plandex-ai/plandex"},{"name":"Autonomy levels","summary":"None / Basic / Plus / Semi-Auto / Full-Auto control auto-context, auto-apply, auto-exec, auto-debug.","url":"https://github.com/plandex-ai/plandex"},{"name":"Versioned plans and branches","summary":"Every plan update is versioned; branches explore multiple paths or models.","maps_to_pattern":"replay-time-travel","url":"https://github.com/plandex-ai/plandex"},{"name":"Project map (tree-sitter)","summary":"Fast project map generation and syntax validation across 30+ languages.","maps_to_pattern":"agentic-rag","url":"https://github.com/plandex-ai/plandex"},{"name":"REPL","summary":"REPL with fuzzy auto-complete for commands and file loading.","url":"https://github.com/plandex-ai/plandex"}],"pattern_composition":"flowchart TD\n  user[User task] --> map[Project map tree-sitter]\n  map --> plan[Plan draft]\n  plan --> branch{Branch}\n  branch --> sandbox[Tentative diff sandbox]\n  sandbox --> auton{Autonomy level}\n  auton -->|none/basic| ask[Ask user]\n  auton -->|plus/semi| auto[Auto-context auto-commit]\n  auton -->|full-auto| run[Auto-apply auto-exec auto-debug]\n  ask --> apply[Apply diff]\n  auto --> apply\n  run --> apply\n  apply --> exec[Optional command exec]\n  exec --> rollback{Errors?}\n  rollback -->|yes| plan\n  rollback -->|no| commit[Version plan]\n  commit --> done[Done]","members":[{"pattern":"approval-queue","role":"first-class","note":"Tentative-diff review is the central UX; nothing touches project files until accepted.","evidence":[{"type":"repo","title":"Plandex README - diff sandbox","url":"https://github.com/plandex-ai/plandex","quote":"A cumulative diff review sandbox keeps AI-generated changes separate from your project files until they are ready to go.","accessed":"2026-05-20"},{"type":"repo","title":"Plandex README - controlled execution","url":"https://github.com/plandex-ai/plandex","quote":"Command execution is controlled so you can easily roll back and debug.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"code-as-action","role":"first-class","note":"Diffs and shell commands are the primitive actions, staged before application.","evidence":[{"type":"repo","title":"Plandex README - diff sandbox","url":"https://github.com/plandex-ai/plandex","quote":"A cumulative diff review sandbox keeps AI-generated changes separate from your project files until they are ready to go.","accessed":"2026-05-20"},{"type":"repo","title":"Plandex README - automated debugging","url":"https://github.com/plandex-ai/plandex","quote":"Automated debugging of terminal commands (like builds, linters, tests, deployments, and scripts).","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"plan-and-execute","role":"first-class","note":"Plan is the central first-class object - generated, branched, versioned, then executed.","evidence":[{"type":"doc","title":"Plandex homepage","url":"https://github.com/plandex-ai/plandex","quote":"An AI coding agent designed for large tasks and real world projects","accessed":"2026-05-20"},{"type":"repo","title":"Plandex README - version control","url":"https://github.com/plandex-ai/plandex","quote":"Full-fledged version control for every update to the plan, including branches for exploring multiple paths.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"replay-time-travel","role":"first-class","note":"Plan-level version history plus branches for exploring multiple paths or models.","evidence":[{"type":"repo","title":"Plandex README - version control","url":"https://github.com/plandex-ai/plandex","quote":"Full-fledged version control for every update to the plan, including branches for exploring multiple paths or comparing different models.","accessed":"2026-05-24"},{"type":"repo","title":"Plandex README - controlled execution / rollback","url":"https://github.com/plandex-ai/plandex","quote":"Command execution is controlled so you can easily roll back and debug.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"code-execution","role":"first-class","note":"Plandex runs commands and auto-debugs in full-auto; controlled rollback is integral.","evidence":[{"type":"repo","title":"Plandex README - command execution","url":"https://github.com/plandex-ai/plandex","quote":"Command execution is controlled so you can easily roll back and debug.","accessed":"2026-05-20"},{"type":"repo","title":"Plandex README - automated debugging","url":"https://github.com/plandex-ai/plandex","quote":"Automated debugging of terminal commands (like builds, linters, tests, deployments, and scripts).","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"agentic-rag","role":"limited","note":"Tree-sitter project map is a graph-based retrieval substrate covering 30+ languages, not vector RAG.","evidence":[{"type":"repo","title":"Plandex README - project map","url":"https://github.com/plandex-ai/plandex","quote":"Fast project map generation and syntax validation with tree-sitter. Supports 30+ languages.","accessed":"2026-05-24"},{"type":"repo","title":"Plandex README - tree-sitter mention","url":"https://github.com/plandex-ai/plandex","quote":"fast project map generation and syntax validation with tree-sitter","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"multi-model-routing","role":"first-class","note":"First-class multi-provider support with curated model packs trading off capability/cost/speed.","evidence":[{"type":"repo","title":"Plandex README - multi-provider + packs","url":"https://github.com/plandex-ai/plandex","quote":"Easily try + combine models from multiple providers. Curated model packs offer different tradeoffs of capability, cost, and speed.","accessed":"2026-05-24"},{"type":"repo","title":"Plandex README - branches for model comparison","url":"https://github.com/plandex-ai/plandex","quote":"Full-fledged version control for every update to the plan, including branches for exploring multiple paths or comparing different models.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"todo-list-driven-agent","role":"limited","note":"Plans function as checklists in spirit, but no explicit verbatim 'todo list' UI surfaced on the README/homepage; honest none.","evidence":[],"evidence_status":"none"}],"instantiates":["modern-coding-agent","safety-hardening"],"alternatives":[{"composition":"aider","relation":"competes-with","note":"Peer terminal coding agent, different sandbox model (git auto-commit vs tentative-diff)."},{"composition":"claude-code","relation":"competes-with","note":"Peer terminal coding agent."},{"composition":"codex-cli","relation":"competes-with","note":"Peer terminal coding agent."}],"references":[{"type":"repo","title":"plandex-ai/plandex","url":"https://github.com/plandex-ai/plandex","accessed":"2026-05-20"},{"type":"doc","title":"Plandex homepage","url":"https://github.com/plandex-ai/plandex","accessed":"2026-05-20"}],"last_analyzed":"2026-05-20","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"emerging","tags":["open-source","go","terminal","tentative-diff","plan-branching"]},{"id":"pochi","name":"Pochi","kind":"framework","category":"coding-agent","build_surface":"full-code","vendor":"TabbyML","language":"TypeScript","license":"Apache-2.0","status":"active","url":"https://www.getpochi.com/","repo":"https://github.com/TabbyML/pochi","docs_url":"https://docs.getpochi.com/","intent":"Open-source AI coding agent shipped primarily as a VS Code extension by TabbyML: reads/writes files, runs commands, supports MCP, isolates concurrent tasks in git worktrees, and accepts custom/self-hosted LLMs.","description":"Pochi is TabbyML's coding agent. It lives in VS Code (Marketplace and OpenVSX), with global config at ~/.pochi/config.jsonc and per-project overrides at .pochi/config.jsonc. The agent uses file-read/write/edit/exec tools to handle multi-step development tasks. MCP support lets it integrate APIs, databases, and internal services. Parallel Agents isolate concurrent tasks in their own git worktrees, so multiple tasks stay live without stashing. Pochi accepts any LLM provider via API key and explicitly targets self-hosted/fine-tuned models.","primary_use_cases":["VS Code coding agent backed by any LLM provider","self-hosted / fine-tuned model deployments","parallel git-worktree-isolated tasks","MCP-extended internal workflows","GitHub-issue-to-PR pipelines from the sidebar"],"agent_loop_shape":"Tool-using agent loop inside VS Code: the agent reads project files, executes shell commands, applies edits, and reaches external systems via MCP. Each top-level task can be run as a Parallel Agent in its own git worktree, so independent tasks have isolated workspaces and do not collide on file state.","key_concepts":[{"name":"Tool use","summary":"File reads/writes, command execution, codebase search.","maps_to_pattern":"tool-use","url":"https://github.com/TabbyML/pochi"},{"name":"Custom / self-hosted models","summary":"Any LLM provider via API key; self-hosted and fine-tuned models supported.","maps_to_pattern":"multi-model-routing","url":"https://github.com/TabbyML/pochi"},{"name":"Parallel Agents","summary":"Each task runs in its own git worktree; multiple tasks active simultaneously.","maps_to_pattern":"subagent-isolation","url":"https://github.com/TabbyML/pochi"},{"name":"MCP","summary":"Model Context Protocol integrates APIs, databases, internal tools.","maps_to_pattern":"mcp","url":"https://docs.getpochi.com/tutorials/supabase-mcp-server/"},{"name":"Layered config","summary":"Global ~/.pochi/config.jsonc plus per-project .pochi/config.jsonc.","url":"https://docs.getpochi.com/"}],"pattern_composition":"flowchart TD\n  user[User task in VS Code] --> config[Layered config global plus project]\n  config --> loop[Agent loop]\n  loop --> propose[LLM proposes tool]\n  propose --> tool{Tool}\n  tool --> read[Read files]\n  tool --> write[Write files]\n  tool --> exec[Execute command]\n  tool --> mcp[MCP server]\n  read --> obs[Observation]\n  write --> obs\n  exec --> obs\n  mcp --> obs\n  obs --> loop\n  loop --> parallel{Parallel Agent?}\n  parallel -->|yes| worktree[Isolated git worktree]\n  parallel -->|no| ws[Workspace]\n  worktree --> done[Done]\n  ws --> done","members":[{"pattern":"agent-computer-interface","role":"first-class","note":"VS Code extension is the ACI surface; right-sidebar placement recommended.","evidence":[{"type":"doc","title":"Pochi docs - VS Code extension","url":"https://docs.getpochi.com/","quote":"Pochi is an open-source AI coding agent built as a VS Code extension.","accessed":"2026-05-20"},{"type":"repo","title":"Pochi README - IDE operation","url":"https://github.com/TabbyML/pochi","quote":"It operates within your IDE, using a toolkit of commands to execute complex tasks, from code generation to project-wide refactoring.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"code-as-action","role":"first-class","note":"Edits and command execution are the agent's primitive actions; custom agents define readFile/writeToFile/executeCommand seams.","evidence":[{"type":"repo","title":"Pochi README - tool usage","url":"https://github.com/TabbyML/pochi","quote":"Pochi has access to a set of tools that allow it to interact with your development environment, such as reading and writing files, executing commands, and searching your codebase.","accessed":"2026-05-20"},{"type":"doc","title":"Pochi docs - custom agent file glob tools","url":"https://docs.getpochi.com/custom-agent/","quote":"Path globs, e.g. `readFile(src/**)`, `writeToFile(.pochi/**)`","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Read/write/exec/search are first-class tools; custom agents can scope their own tool list.","evidence":[{"type":"repo","title":"Pochi README - tool usage","url":"https://github.com/TabbyML/pochi","quote":"Pochi has access to a set of tools that allow it to interact with your development environment, such as reading and writing files, executing commands, and searching your codebase.","accessed":"2026-05-20"},{"type":"doc","title":"Pochi docs - custom agent tools","url":"https://docs.getpochi.com/custom-agent/","quote":"A list of tools that the agent is allowed to use. This can be a comma-separated string or a YAML array. If omitted, the agent inherits all available tools.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"MCP is the documented external-tool extension mechanism; tutorial pages exist for Supabase MCP and Tabby server.","evidence":[{"type":"doc","title":"Pochi docs - MCP definition","url":"https://docs.getpochi.com/tutorials/supabase-mcp-server/","quote":"MCP (Model Context Protocol) is a lightweight, flexible interface that allows large language models (LLMs) like those used in Pochi to connect and interact with external tools, APIs, or databases.","accessed":"2026-05-24"},{"type":"doc","title":"Pochi docs - MCP server subprocess","url":"https://docs.getpochi.com/tutorials/supabase-mcp-server/","quote":"MCP lets you build custom servers that connect Pochi to your unique internal tools and workflows.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"subagent-isolation","role":"first-class","note":"Parallel Agents pattern: each task runs in its own git worktree, isolated state.","evidence":[{"type":"repo","title":"Pochi README - Parallel Agents","url":"https://github.com/TabbyML/pochi","quote":"Keep tasks fully isolated by running each one in its own Git worktree. You can keep multiple tasks active at once and switch between them without stashing or losing context.","accessed":"2026-05-24"},{"type":"doc","title":"Pochi docs - Parallel Agents isolation","url":"https://docs.getpochi.com/parallel-agents/","quote":"Each agent can explore the code base, develop different PRs, or test different ideas in parallel without interfering with each other.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"multi-model-routing","role":"first-class","note":"Any LLM provider via API key; explicit support for self-hosted/fine-tuned models; OpenAI-compatible providers.","evidence":[{"type":"repo","title":"Pochi README - custom models","url":"https://github.com/TabbyML/pochi","quote":"Pochi supports custom AI models, including your own fine-tuned or self-hosted models. Fully control the model, the data, and the compute costs.","accessed":"2026-05-20"},{"type":"doc","title":"Pochi docs - OpenAI-compatible providers","url":"https://docs.getpochi.com/models/","quote":"Pochi allows you to configure any LLM provider that offers an OpenAI-compatible API by setting up the appropriate API keys.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"code-execution","role":"first-class","note":"Command execution is a documented tool; custom agents can restrict shell commands via executeCommand specifiers.","evidence":[{"type":"repo","title":"Pochi README - executing commands","url":"https://github.com/TabbyML/pochi","quote":"Pochi has access to a set of tools that allow it to interact with your development environment, such as reading and writing files, executing commands, and searching your codebase.","accessed":"2026-05-20"},{"type":"doc","title":"Pochi docs - custom agent executeCommand","url":"https://docs.getpochi.com/custom-agent/","quote":"Restrict which shell commands the agent can run via `executeCommand`. The specifier is matched against each individual command segment (split on `;`, `&&`, `||`, `|`).","accessed":"2026-05-24"}],"evidence_status":"full"}],"instantiates":["modern-coding-agent"],"alternatives":[{"composition":"cline","relation":"competes-with","note":"Peer open-source VS Code coding agent."},{"composition":"continue-dev","relation":"competes-with","note":"Peer open-source VS Code coding assistant."},{"composition":"roo-code","relation":"competes-with","note":"Peer VS Code agent (Roo Code now archived)."}],"references":[{"type":"repo","title":"TabbyML/pochi","url":"https://github.com/TabbyML/pochi","accessed":"2026-05-20"},{"type":"doc","title":"Pochi docs","url":"https://docs.getpochi.com/","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"emerging","tags":["open-source","typescript","apache-2.0","vscode-extension","mcp","git-worktree"]},{"id":"replit-agent","name":"Replit Agent","kind":"framework","category":"coding-agent","build_surface":"no-code","vendor":"Replit","language":"Web product","status":"active","url":"https://replit.com/agent","docs_url":"https://docs.replit.com/core-concepts/agent","intent":"Replit's cloud-native AI development partner that plans, builds, tests, and deploys applications from natural-language descriptions inside the Replit workspace, with snapshot-based checkpoints and one-click deployment.","description":"Replit Agent is a cloud-only coding agent built into Replit's workspace. It reads/writes files, runs commands in the Replit project sandbox, sets up infrastructure, and deploys via Replit Deployments. It exposes three modes (Lite / Economy / Power) trading off cost vs model capability, with a Pro-only Turbo accelerator. A Plan Mode breaks tasks into ordered lists before any code is touched. Snapshot-style checkpoints record meaningful progress; users can roll back to any prior state through git-backed metadata. Built-in PostgreSQL provisioning and Stripe/GitHub/Slack connectors plus custom MCP servers extend reach.","primary_use_cases":["natural-language web app and prototype building","checkpointed iterative development with rollback","one-click cloud deployment from Agent","Plan Mode for breakdowns before editing","non-engineers building working software"],"agent_loop_shape":"Spec-driven plan-and-execute loop inside Replit's hosted environment. User describes intent; Agent (optionally in Plan Mode) drafts an ordered task list, then sets up the project, writes code, runs and tests it in the sandboxed workspace, and creates a checkpoint at meaningful progress. Errors trigger self-fix attempts; rollback to any checkpoint restores via git. Deployment is a single tool call that compiles, packages, hosts, and TLS-terminates.","key_concepts":[{"name":"Plan Mode","summary":"Break down complex projects into ordered task lists before edits.","maps_to_pattern":"plan-and-execute","url":"https://docs.replit.com/core-concepts/agent"},{"name":"Modes (Lite / Economy / Power / Turbo)","summary":"Performance tiers controlling model capability vs cost.","maps_to_pattern":"multi-model-routing","url":"https://docs.replit.com/core-concepts/agent"},{"name":"Checkpoints","summary":"Snapshot-style git-backed checkpoints; rollback to any prior state.","maps_to_pattern":"replay-time-travel","url":"https://docs.replit.com/replitai/agent"},{"name":"Replit workspace sandbox","summary":"Cloud project as the agent's sandboxed execution environment.","maps_to_pattern":"sandbox-isolation","url":"https://docs.replit.com/core-concepts/agent"},{"name":"Deployment tool","summary":"One agent tool call drives compile/package/host/TLS via Replit Deployments.","url":"https://docs.replit.com/core-concepts/agent"}],"pattern_composition":"flowchart TD\n  user[User describes app] --> mode{Mode Lite Economy Power}\n  mode --> plan{Plan Mode}\n  plan -->|yes| tasks[Ordered task list]\n  plan -->|no| direct[Direct execution]\n  tasks --> loop[Agent loop]\n  direct --> loop\n  loop --> write[Write code]\n  loop --> test[Run and test]\n  test --> err{Errors?}\n  err -->|yes| loop\n  err -->|no| ckpt[Create checkpoint]\n  ckpt --> next{More work?}\n  next -->|yes| loop\n  next -->|no| deploy[Deploy tool]\n  deploy --> live[Hosted app]\n  ckpt -.->|rollback| restore[Restore prior state]","members":[{"pattern":"code-as-action","role":"first-class","note":"Agent writes code, sets up project, and shows live preview; file edits and command runs are the action surface.","evidence":[{"type":"doc","title":"Replit docs - Agent writes code","url":"https://docs.replit.com/core-concepts/workspace","quote":"From here, you can describe what you want to build, and Agent takes it from there, writing code, setting up your project, and showing you a live preview in real time.","accessed":"2026-05-24"},{"type":"doc","title":"Replit docs - Agent infrastructure","url":"https://docs.replit.com/replitai/agent","quote":"Agent writes code, sets up infrastructure, and tests the result.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"code-execution","role":"first-class","note":"Cloud workspace runs the code Agent writes; Agent tests its own work.","evidence":[{"type":"doc","title":"Replit docs - Agent tests","url":"https://docs.replit.com/replitai/agent","quote":"Agent tests its own work on a regular basis.","accessed":"2026-05-20"},{"type":"doc","title":"Replit docs - Agent infrastructure","url":"https://docs.replit.com/replitai/agent","quote":"Agent writes code, sets up infrastructure, and tests the result.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"plan-and-execute","role":"first-class","note":"Plan Mode is a documented first-class mode: brainstorm + ordered task list, approve, then build.","evidence":[{"type":"doc","title":"Replit docs - Plan Mode (core-concepts)","url":"https://docs.replit.com/core-concepts/agent","quote":"Enable Plan mode to brainstorm, ask questions, and map out your project before Agent changes any code or data.","accessed":"2026-05-24"},{"type":"doc","title":"Replit docs - Plan Mode task list","url":"https://docs.replit.com/references/agent/plan-mode","quote":"an ordered task list of the development tasks necessary to complete your request","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"replay-time-travel","role":"first-class","note":"Agent creates checkpoints during work; users roll back to any prior state.","evidence":[{"type":"doc","title":"Replit docs - checkpoints","url":"https://docs.replit.com/replitai/agent","quote":"Agent also creates checkpoints as it works, so you can roll back to any previous state.","accessed":"2026-05-20"},{"type":"doc","title":"Replit docs - History panel checkpoints","url":"https://docs.replit.com/learn/build-with-agent","quote":"Checkpoints help you recover when a change goes in the wrong direction.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"spec-driven-loop","role":"first-class","note":"Agent takes a natural-language spec and drives end-to-end build, planning to deployment.","evidence":[{"type":"doc","title":"Replit Agent - core capability","url":"https://docs.replit.com/replitai/agent","quote":"Describe what you want in everyday language. No code or technical knowledge required.","accessed":"2026-05-20"},{"type":"doc","title":"Replit Agent - end-to-end","url":"https://docs.replit.com/replitai/agent","quote":"Agent handles the rest, from planning to deployment.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"multi-model-routing","role":"first-class","note":"Lite / Economy / Power modes (plus Turbo) explicitly route across different model capabilities.","evidence":[{"type":"doc","title":"Replit docs - Economy mode","url":"https://docs.replit.com/references/agent/agent-modes","quote":"uses fewer credits per task and is the best default when you want strong results without paying for the most capable models","accessed":"2026-05-24"},{"type":"doc","title":"Replit docs - Power mode","url":"https://docs.replit.com/references/agent/agent-modes","quote":"uses more capable models for complex tasks, larger codebases, and harder problems","accessed":"2026-05-24"},{"type":"doc","title":"Replit docs - Turbo mode","url":"https://docs.replit.com/references/agent/agent-modes","quote":"2.5x faster responses using the fastest models","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"sandbox-isolation","role":"first-class","note":"Each Replit project is a cloud-hosted workspace where the agent runs; isolated per-project.","evidence":[{"type":"doc","title":"Replit docs - Project Editor is cloud env","url":"https://docs.replit.com/core-concepts/workspace","quote":"The Project Editor is your home base on Replit: the environment where you start building, manage your projects, and see your apps come to life.","accessed":"2026-05-24"},{"type":"doc","title":"Replit docs - cloud persistence","url":"https://docs.replit.com/core-concepts/workspace","quote":"Your project is saved in the cloud. Come back any time and pick up right where you left off: your files, history, and running app are all preserved.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Agent tools include code execution, deployment, named connectors (GitHub, Stripe, Slack), and custom MCP servers.","evidence":[{"type":"doc","title":"Replit Agent docs - data integrations","url":"https://docs.replit.com/core-concepts/agent","quote":"pull data from BigQuery, Linear, Slack, Notion, and more directly from chat","accessed":"2026-05-24"},{"type":"doc","title":"Replit docs - MCP tools","url":"https://docs.replit.com/build/connect-via-mcp","quote":"Replit Agent can connect to hundreds of external tools and data sources through the Model Context Protocol (MCP)","accessed":"2026-05-24"},{"type":"doc","title":"Replit docs - GitHub connector","url":"https://docs.replit.com/references/integrations/overview","quote":"GitHub: Access repositories, users, and organizations","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"agent-resumption","role":"limited","note":"Checkpoints + Rollback here button provide resume-from-state behaviour; the agent's own conversation can be picked up again after rollback.","evidence":[{"type":"doc","title":"Replit docs - checkpoints rollback","url":"https://docs.replit.com/replitai/agent","quote":"Agent also creates checkpoints as it works, so you can roll back to any previous state.","accessed":"2026-05-20"},{"type":"doc","title":"Replit docs - Rollback here confirmation","url":"https://docs.replit.com/learn/build-with-agent","quote":"When you select **Rollback here**, Agent confirms what will be restored before applying the rollback.","accessed":"2026-05-24"}],"evidence_status":"full"}],"instantiates":["modern-coding-agent"],"alternatives":[{"composition":"boltnew","relation":"competes-with","note":"Peer hosted natural-language-to-app builder."},{"composition":"lovable","relation":"competes-with","note":"Peer hosted prompt-to-app product."},{"composition":"v0","relation":"competes-with","note":"Peer hosted prompt-to-app product on Vercel."}],"references":[{"type":"doc","title":"Replit Agent docs (core concepts)","url":"https://docs.replit.com/core-concepts/agent","accessed":"2026-05-20"},{"type":"doc","title":"Replit Agent docs","url":"https://docs.replit.com/replitai/agent","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["closed-source","cloud","checkpoints","plan-mode","deployment"]},{"id":"roo-code","name":"Roo Code","aliases":["Roo Cline"],"kind":"framework","category":"coding-agent","build_surface":"full-code","vendor":"Roo Code Inc.","language":"TypeScript","status":"discontinued","discontinued_date":"2026-05-15","url":"https://roomote.dev/","repo":"https://github.com/RooCodeInc/Roo-Code","docs_url":"https://roocodeinc.github.io/Roo-Code/","intent":"Open-source VS Code AI coding agent (originally a fork of Cline) with role-based modes - Architect / Code / Ask / Debug / Orchestrator - plus MCP integration and custom-mode authoring. Extension archived 2026-05-15; vendor pivoted to Roomote (cloud agent on Slack).","description":"Roo Code was a VS Code extension that exposed a multi-role agent: Code for everyday edits, Architect for planning, Ask for explanations, Debug for tracing issues, and Orchestrator for delegating to other modes. Custom modes let teams add specialised roles. MCP servers extended the toolchain. The extension was shut down on 2026-05-15; the vendor (Roo Code Inc.) now ships Roomote, a Slack-integrated cloud engineering agent. Documentation directs former users to ZooCode or Cline as forks/alternatives.","primary_use_cases":["role-typed VS Code agent (Architect / Code / Ask / Debug / Orchestrator)","Orchestrator-mode delegation to other modes","custom modes for team-specific workflows","MCP-extended toolchain inside VS Code"],"agent_loop_shape":"Mode-switched tool-using loop inside VS Code. User picks (or Orchestrator picks) a mode; that mode's prompt+tool budget runs against the LLM; tool calls (file edits, exec, MCP) execute with optional approval; results return; Orchestrator may delegate further subtasks back to other modes.","key_concepts":[{"name":"Modes (Architect / Code / Ask / Debug / Orchestrator)","summary":"Role-typed prompts keep models focused on a single task.","maps_to_pattern":"role-typed-subagents","url":"https://github.com/RooCodeInc/Roo-Code"},{"name":"Orchestrator mode","summary":"Delegates to other modes; recommended for complex multi-step tasks.","maps_to_pattern":"supervisor","url":"https://roocodeinc.github.io/Roo-Code/"},{"name":"Custom modes","summary":"Author specialised modes for team or workflow.","url":"https://github.com/RooCodeInc/Roo-Code"},{"name":"MCP","summary":"MCP server integration for external tools.","maps_to_pattern":"mcp","url":"https://github.com/RooCodeInc/Roo-Code"}],"pattern_composition":"flowchart TD\n  user[User task] --> mode{Mode}\n  mode -->|code| code[Code mode]\n  mode -->|architect| arch[Architect mode]\n  mode -->|ask| ask[Ask mode]\n  mode -->|debug| dbg[Debug mode]\n  mode -->|orchestrator| orch[Orchestrator mode]\n  orch --> delegate[Delegate to other mode]\n  delegate --> code\n  delegate --> arch\n  delegate --> dbg\n  code --> tools[Tool call]\n  arch --> tools\n  ask --> tools\n  dbg --> tools\n  tools --> approval{Auto-approve?}\n  approval -->|no| ask_user[Ask user]\n  approval -->|yes| run[Run tool]\n  ask_user --> run\n  run --> mcp[MCP server]\n  run --> edit[Edit / Exec]\n  mcp --> obs[Observe]\n  edit --> obs\n  obs --> mode\n  mode --> done[Done]","members":[{"pattern":"role-typed-subagents","role":"first-class","note":"Architect / Code / Ask / Debug / Orchestrator are role-typed prompts.","evidence":[{"type":"repo","title":"Roo Code README - modes","url":"https://github.com/RooCodeInc/Roo-Code","quote":"Code Mode: everyday coding, edits, and file ops","accessed":"2026-05-20"},{"type":"doc","title":"Roo Code docs - modes","url":"https://roocodeinc.github.io/Roo-Code/","quote":"The extension included customizable roles (Architect, Code, Ask, Debug, Orchestrator) to keep models focused on specific tasks","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"supervisor","role":"first-class","note":"Orchestrator (aka Boomerang) mode is the canonical supervisor primitive — breaks down complex tasks and delegates subtasks to specialised modes.","evidence":[{"type":"doc","title":"Roo Code — Using Modes (Orchestrator)","url":"https://docs.roocode.com/basic-usage/using-modes","quote":"A strategic workflow orchestrator (aka Boomerang Mode) that breaks down complex tasks and delegates them to specialized modes.","accessed":"2026-05-24"},{"type":"doc","title":"Roo Code — Using Modes (new_task delegation)","url":"https://docs.roocode.com/basic-usage/using-modes","quote":"Uses the `new_task` tool to delegate subtasks to other modes.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"subagent-isolation","role":"limited","note":"Mode-as-subagent gives prompt+tool isolation: each mode (Code, Architect, Debug, Ask) has its own prompt and toolset; no full process-level isolation.","evidence":[{"type":"doc","title":"Roo Code — Modes (Code/Architect/Debug/Ask)","url":"https://docs.roocode.com/basic-usage/using-modes","quote":"A skilled software engineer with expertise in programming languages, design patterns, and best practices","accessed":"2026-05-24"},{"type":"doc","title":"Roo Code — Modes (Debug specialty)","url":"https://docs.roocode.com/basic-usage/using-modes","quote":"An expert problem solver specializing in systematic troubleshooting and diagnostics","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"MCP server support is listed as a core capability; per-tool 'Always allow' approval is built into the MCP integration.","evidence":[{"type":"doc","title":"Roo Code — README (MCP)","url":"https://roocodeinc.github.io/Roo-Code/","quote":"Utilize MCP Servers","accessed":"2026-05-24"},{"type":"doc","title":"Roo Code — Auto-Approving Actions (MCP)","url":"https://docs.roocode.com/features/auto-approving-actions","quote":"Enable auto-approval of individual MCP tools...requires both this setting and the tool's individual 'Always allow' checkbox","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"approval-queue","role":"first-class","note":"Per-tool approval flow with optional auto-approval; documented in Auto-Approving Actions with permission tiles per action category.","evidence":[{"type":"doc","title":"Roo Code — Auto-Approving Actions","url":"https://docs.roocode.com/features/auto-approving-actions","quote":"Auto-approve settings bypass confirmation prompts, giving Roo direct access to your system. This can result in data loss, file corruption, or worse.","accessed":"2026-05-24"},{"type":"doc","title":"Roo Code — README (auto-approve)","url":"https://roocodeinc.github.io/Roo-Code/","quote":"the extension will ask for your permission to do everything, but as you become more comfortable and enable Auto-Approve, you'll see it work for long periods on its own","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"code-as-action","role":"first-class","note":"File edits and command execution are the primitive actions in Code/Debug modes; documented as native file system access and terminal control.","evidence":[{"type":"doc","title":"Roo Code — README (file system, terminal)","url":"https://roocodeinc.github.io/Roo-Code/","quote":"actual file system access, terminal control, and multi-step workflows","accessed":"2026-05-24"},{"type":"doc","title":"Roo Code — Using Modes (Code mode)","url":"https://docs.roocode.com/basic-usage/using-modes","quote":"A skilled software engineer with expertise in programming languages, design patterns, and best practices","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Mode prompts plus MCP tools form the tool surface; auto-approval permissions are configured per tool category (read, write, command, MCP).","evidence":[{"type":"doc","title":"Roo Code — README (tools)","url":"https://roocodeinc.github.io/Roo-Code/","quote":"actual file system access, terminal control, and multi-step workflows","accessed":"2026-05-24"},{"type":"doc","title":"Roo Code — Auto-Approving Actions (command execution)","url":"https://docs.roocode.com/features/auto-approving-actions","quote":"Runs whitelisted terminal commands automatically","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"repo-map-context","role":"first-class","note":"Roo Code's list_code_definition_names uses tree-sitter to extract top-level classes, functions, and methods, giving the agent a structural map of the project at the start of a task.","evidence":[{"type":"doc","title":"list_code_definition_names | Roo Code Documentation","url":"https://docs.roocode.com/advanced-usage/available-tools/list-code-definition-names","quote":"The list_code_definition_names tool provides a structural overview of your codebase by listing code definitions from source files at the top level.","accessed":"2026-06-14"}],"evidence_status":"full"},{"pattern":"shadow-workspace-vcs","role":"core","note":"Snapshots the workspace into a shadow Git repository before every file modification, giving diff-based restore of agent edits without touching the user's own VCS.","evidence":[{"type":"doc","title":"Checkpoints | Roo Code Documentation","url":"https://docs.roocode.com/features/checkpoints","quote":"Roo Code captures snapshots of your project's state using a shadow Git repository, separate from your main version control system.","accessed":"2026-06-14"}],"evidence_status":"full"}],"instantiates":["modern-coding-agent"],"alternatives":[{"composition":"cline","relation":"competes-with","note":"Roo Code originated as a fork of Cline; Cline is the recommended successor per Roo Code's own deprecation notice."},{"composition":"continue-dev","relation":"competes-with","note":"Peer open-source VS Code coding assistant."},{"composition":"pochi","relation":"competes-with","note":"Peer VS Code agent (Roo Code now archived)."}],"references":[{"type":"repo","title":"RooCodeInc/Roo-Code","url":"https://github.com/RooCodeInc/Roo-Code","accessed":"2026-05-20"},{"type":"doc","title":"Roo Code docs (archived)","url":"https://roocodeinc.github.io/Roo-Code/","accessed":"2026-05-20"},{"type":"blog","title":"Roomote (vendor's successor product)","url":"https://roomote.dev/","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"deprecated","tags":["open-source","typescript","vscode-extension","discontinued","cline-fork","orchestrator-mode"]},{"id":"sourcegraph-cody","name":"Sourcegraph Cody","kind":"framework","category":"coding-agent","build_surface":"full-code","vendor":"Sourcegraph","language":"TypeScript (IDE extensions)","status":"active","url":"https://sourcegraph.com/cody","repo":"https://github.com/sourcegraph/cody-public-snapshot","docs_url":"https://sourcegraph.com/docs/cody","intent":"Sourcegraph's enterprise AI coding assistant: chat, completions, and edits backed by Sourcegraph's search-API context fetching across local and remote codebases, with agentic context fetching and local-MCP tool integration.","description":"Cody is an AI coding assistant available across VS Code, JetBrains, Visual Studio, the Sourcegraph web app, and a CLI. Its differentiator is context: Cody pulls from local and remote codebases via Sourcegraph's Search API. Agentic Context Fetching iteratively gathers context using Code Search, Codebase File, Terminal, Web Browser, and MCP tools, refining what it knows before responding. Cody supports local MCP servers (Tools capability only). As of 2025-07-23 Sourcegraph deprecated Cody Free/Pro and Enterprise Starter; only Cody Enterprise remains, with Amp as a separate consumer product.","primary_use_cases":["enterprise codebase chat with remote+local context","code completions and customizable prompts inside the IDE","agentic context fetching for accurate retrieval","MCP-extended internal tooling (local servers)","multi-repo question answering via Sourcegraph search"],"agent_loop_shape":"Retrieval-augmented agent loop: user asks; Cody runs agentic context fetching that combines Sourcegraph Search, codebase files, terminal output, web browsing, and MCP tools, iteratively refining context until comprehensive; the LLM then produces a chat answer or an edit. Tool calls (especially terminal) require explicit user consent and a trusted workspace.","key_concepts":[{"name":"Search-API context","summary":"Pulls context from local and remote codebases via Sourcegraph's Search API.","maps_to_pattern":"agentic-rag","url":"https://sourcegraph.com/docs/cody"},{"name":"Agentic context fetching","summary":"Proactively gathers and refines context with multiple tools before responding.","maps_to_pattern":"agentic-rag","url":"https://sourcegraph.com/docs/cody/capabilities/agentic-chat"},{"name":"Tools (Code Search, Codebase File, Terminal, Web Browser, MCP)","summary":"Tool surface used during agentic context fetching.","maps_to_pattern":"tool-use","url":"https://sourcegraph.com/docs/cody/capabilities/agentic-chat"},{"name":"MCP (local servers only)","summary":"Cody supports only local MCP servers and only the Tools capability.","maps_to_pattern":"mcp","url":"https://sourcegraph.com/docs/cody/capabilities/agentic-chat"}],"pattern_composition":"flowchart TD\n  user[User question or edit] --> fetch[Agentic context fetching]\n  fetch --> search[Sourcegraph search]\n  fetch --> files[Codebase files]\n  fetch --> term[Terminal output]\n  fetch --> web[Web browser]\n  fetch --> mcp[Local MCP tools]\n  search --> ctx[Refined context]\n  files --> ctx\n  term --> ctx\n  web --> ctx\n  mcp --> ctx\n  ctx --> enough{Comprehensive?}\n  enough -->|no| fetch\n  enough -->|yes| llm[LLM answer or edit]\n  llm --> deliver[Chat or apply edit]","members":[{"pattern":"agentic-rag","role":"first-class","note":"Agentic context fetching iteratively gathers and refines context across multiple sources.","evidence":[{"type":"doc","title":"Cody docs - agentic context","url":"https://sourcegraph.com/docs/cody/capabilities/agentic-chat","quote":"proactively understanding your coding environment and gathering relevant information based on your requests","accessed":"2026-05-20"},{"type":"doc","title":"Cody docs - search API context","url":"https://sourcegraph.com/docs/cody","quote":"uses the powerful Sourcegraph's advanced Search API to pull context from both local and remote codebases","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Documented tools: Code Search, Codebase File, Terminal, Web Browser, MCP — each with a per-tool description.","evidence":[{"type":"doc","title":"Cody docs - tools list","url":"https://sourcegraph.com/docs/cody/capabilities/agentic-chat","quote":"Code Search, Codebase File, Terminal, Web Browser, and MCP","accessed":"2026-05-20"},{"type":"doc","title":"Cody docs - Code Search tool","url":"https://sourcegraph.com/docs/cody/capabilities/agentic-chat","quote":"Code Search: Performs code searches","accessed":"2026-05-24"},{"type":"doc","title":"Cody docs - Terminal tool","url":"https://sourcegraph.com/docs/cody/capabilities/agentic-chat","quote":"Terminal: Executes shell commands in your terminal","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"mcp","role":"limited","note":"Local MCP servers only; only Tools capability supported. Honest downgrade from first-class.","evidence":[{"type":"doc","title":"Cody docs - local MCP only","url":"https://sourcegraph.com/docs/cody/capabilities/agentic-chat","quote":"Cody currently supports only local MCP servers","accessed":"2026-05-20"},{"type":"doc","title":"Cody docs - tools-only MCP","url":"https://sourcegraph.com/docs/cody/capabilities/agentic-chat","quote":"only Tools among the three MCP capabilities","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"agent-computer-interface","role":"first-class","note":"Cody ships as VS Code, JetBrains, Visual Studio, web app, and CLI surfaces.","evidence":[{"type":"doc","title":"Cody docs - clients overview","url":"https://sourcegraph.com/docs/cody","quote":"Available on VS Code, JetBrains, Visual Studio, and the Web app.","accessed":"2026-05-24"},{"type":"doc","title":"Cody docs - VS Code extension","url":"https://sourcegraph.com/docs/cody/clients","quote":"Install Cody's extension for VS Code.","accessed":"2026-05-24"},{"type":"doc","title":"Cody docs - JetBrains extension","url":"https://sourcegraph.com/docs/cody/clients","quote":"Install Cody's extension for JetBrains.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"human-in-the-loop","role":"first-class","note":"Terminal tool requires explicit consent each time and only works in trusted workspaces.","evidence":[{"type":"doc","title":"Cody docs - pause and ask permission","url":"https://sourcegraph.com/docs/cody/capabilities/agentic-chat","quote":"Agentic context fetching will pause and ask for permission each time before executing any shell command.","accessed":"2026-05-24"},{"type":"doc","title":"Cody docs - trusted workspace restriction","url":"https://sourcegraph.com/docs/cody/capabilities/agentic-chat","quote":"Commands can only be executed within trusted workspaces with a valid shell","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"code-as-action","role":"limited","note":"Cody focuses more on chat + completions + customizable prompts than autonomous edits; SmartApply/Execute and code-edits exist in VS Code.","evidence":[{"type":"doc","title":"Cody docs - completions and edits","url":"https://sourcegraph.com/docs/cody","quote":"Code completions, code edits, and customizable prompts.","accessed":"2026-05-20"},{"type":"doc","title":"Cody docs - cursor-driven edits","url":"https://sourcegraph.com/docs/cody","quote":"Suggests code changes by analyzing cursor movements and typing.","accessed":"2026-05-24"}],"evidence_status":"full"}],"instantiates":["modern-coding-agent","production-rag"],"alternatives":[{"composition":"continue-dev","relation":"competes-with","note":"Peer IDE assistant; both target VS Code and JetBrains."},{"composition":"cursor","relation":"competes-with","note":"Peer IDE-based coding assistant."},{"composition":"amp","relation":"same-vendor","note":"Sourcegraph's earlier code-AI product; Amp is the newer agent."}],"references":[{"type":"doc","title":"Cody overview","url":"https://sourcegraph.com/docs/cody","accessed":"2026-05-20"},{"type":"doc","title":"Cody agentic chat","url":"https://sourcegraph.com/docs/cody/capabilities/agentic-chat","accessed":"2026-05-20"},{"type":"doc","title":"Cody clients","url":"https://sourcegraph.com/docs/cody/clients","accessed":"2026-05-24"},{"type":"repo","title":"sourcegraph/cody","url":"https://github.com/sourcegraph/cody-public-snapshot","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["closed-source","enterprise","rag","agentic-context","mcp"]},{"id":"sweep","name":"Sweep","kind":"framework","category":"coding-agent","build_surface":"closed-application","vendor":"Sweep AI","language":"Python (legacy) / JetBrains plugin (current)","status":"rebranded","url":"https://sweep.dev/","repo":"https://github.com/sweepai/sweep","intent":"Originally a GitHub-issue-to-PR open-source agent; the team has since pivoted entirely to a JetBrains AI coding assistant. The legacy issue-to-PR product is no longer the active focus.","description":"Sweep launched as a GitHub bot that turned issues into pull requests using an LLM-driven spec-to-PR loop. The team has since shifted: the public README now points readers to a JetBrains AI coding assistant plugin, and the legacy GitHub repository (7.7k stars) is no longer the active product. The new JetBrains product offers tab completion, AI Code Review across branches, and is marketed as enterprise/SOC-2 ready. This composition captures both phases because the legacy product is what the open-source community references as Sweep.","primary_use_cases":["(legacy) GitHub-issue-to-PR automation","(current) JetBrains IDE AI coding assistant plugin","(current) AI code review across branches"],"agent_loop_shape":"Legacy: spec-driven loop. Sweep took a GitHub issue as a spec, planned changes, edited files, and opened a PR. Current: an IDE-side completion and edit assistant inside JetBrains IDEs with a custom Tab model and a separate code-review feature.","key_concepts":[{"name":"Issue-to-PR (legacy)","summary":"GitHub issue triggers planning, edit, and PR creation by the bot.","maps_to_pattern":"spec-driven-loop","url":"https://github.com/sweepai/sweep"},{"name":"JetBrains plugin (current product)","summary":"Tab completion plus AI Code Review marketed via sweep.dev.","url":"https://sweep.dev/"},{"name":"Pivot","summary":"Team transitioned from GitHub bot to JetBrains IDE plugin.","url":"https://github.com/sweepai/sweep"}],"pattern_composition":"flowchart TD\n  legacy[Legacy GitHub bot] --> issue[GitHub issue spec]\n  issue --> plan[LLM planning]\n  plan --> edits[File edits]\n  edits --> pr[Open PR]\n  pr --> review[Human review]\n  current[Current JetBrains plugin] --> tab[Tab completion]\n  current --> aicr[AI code review]\n  tab --> edit[Inline edit]\n  aicr --> diff[Branch diff review]","members":[{"pattern":"spec-driven-loop","role":"first-class","note":"Legacy product: GitHub issue text drives the entire build-and-PR loop.","evidence":[{"type":"repo","title":"Sweep README - JetBrains pivot","url":"https://github.com/sweepai/sweep","quote":"Thank you for all of the support on Sweep. We're now building an AI coding assistant for JetBrains","accessed":"2026-05-20"}],"evidence_status":"limited"},{"pattern":"code-as-action","role":"limited","note":"Legacy: edits applied to a branch and pushed as PR. Current JetBrains product: tab edits.","evidence":[{"type":"doc","title":"Sweep homepage - tab edits","url":"https://sweep.dev/","quote":"custom Tab model suggests precise code changes in milliseconds","accessed":"2026-05-20"}],"evidence_status":"limited"},{"pattern":"tool-use","role":"limited","note":"Tooling existed in legacy product (git, code search) but no first-class verbatim list on the surfaces reviewed.","evidence":[],"evidence_status":"none"},{"pattern":"react","role":"limited","note":"Legacy used iterative plan+edit cycles per issue, not a canonical ReAct loop in the modern sense.","evidence":[],"evidence_status":"none"}],"instantiates":["modern-coding-agent"],"alternatives":[{"composition":"github-copilot-coding-agent","relation":"competes-with","note":"GitHub's own issue-to-PR coding agent superseded the original Sweep use case."},{"composition":"junie","relation":"competes-with","note":"JetBrains' own first-party agent competes directly with the current Sweep JetBrains plugin."}],"references":[{"type":"repo","title":"sweepai/sweep","url":"https://github.com/sweepai/sweep","accessed":"2026-05-20"},{"type":"doc","title":"Sweep homepage","url":"https://sweep.dev/","accessed":"2026-05-20"}],"last_analyzed":"2026-05-20","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"deprecated","tags":["rebranded","github-bot-legacy","jetbrains-plugin-current"]},{"id":"tongyi-lingma","name":"Tongyi Lingma","kind":"framework","category":"coding-agent","build_surface":"full-code","vendor":"Alibaba Cloud (通义灵码 / Lingma / Qoder CN)","language":"TypeScript (extension), Java (server)","license":"proprietary (free individual tier + enterprise editions)","status":"active","url":"https://www.alibabacloud.com/product/lingma","docs_url":"https://www.alibabacloud.com/help/en/lingma","intent":"Alibaba's IDE-native coding assistant on Qwen models with an Agent mode that autonomously breaks tasks into to-dos, edits across multiple files, and reflects on intermediate steps.","description":"Tongyi Lingma (通义灵码) is Alibaba Cloud's IDE coding assistant, powered by Qwen. It ships VS Code and JetBrains extensions and offers two operating modes: an Ask mode for instant Q&A and code completion, and an Agent mode that takes an objective, decomposes it into to-dos, performs multi-file editing, and dynamically adjusts the plan as it executes. The product also ships as an enterprise edition (in-VPC deployment) and as a separately-branded research agent called LingmaAgent (described in the FSE 2025 industry paper) for autonomous issue resolution against a code repository.","primary_use_cases":["line- and function-level code completion in VS Code and JetBrains","agent-mode autonomous multi-file edits scoped to a workspace","natural-language code generation, refactoring, unit-test generation","issue triage and resolution via LingmaAgent (FSE 2025) for enterprise repos"],"agent_loop_shape":"Two-mode IDE plugin. Ask mode runs a single Qwen call per user turn with the current file and cursor context. Agent mode runs an iterative planner-executor loop: the planner emits a to-do list from the user objective, the executor performs file edits and tool calls (read/write/grep/run-tests) per to-do, and the planner reflects after each tool batch to update the to-do list. The Qoder/Lingma editor is the host UI; tool calls go through the IDE's filesystem and process APIs.","key_concepts":[{"name":"Ask mode vs Agent mode","summary":"Single-shot Q&A vs. autonomous multi-step task execution.","url":"https://www.alibabacloud.com/help/en/lingma/product-overview/introduction-of-lingma"},{"name":"LingmaAgent","summary":"Research agent for autonomous issue resolution via comprehensive repository exploration (FSE 2025 industry paper).","url":"https://conf.researchr.org/details/fse-2025/fse-2025-industry-papers/21/Alibaba-LingmaAgent-Improving-Automated-Issue-Resolution-via-Comprehensive-Repositor"},{"name":"To-do plan refresh","summary":"Agent mode dynamically adjusts the active to-do list as each tool batch completes.","maps_to_pattern":"plan-and-execute"},{"name":"Repository exploration","summary":"LingmaAgent's distinguishing behaviour: comprehensive multi-file exploration before proposing changes (FSE 2025).","maps_to_pattern":"agentic-rag"}],"pattern_composition":"flowchart TD\n  USER[Developer in IDE] --> EXT[Lingma extension<br/>VSCode / JetBrains]\n  EXT --> MODE{Mode}\n  MODE -->|Ask| QWEN[Qwen Coder model]\n  MODE -->|Agent| PLAN[Planner<br/>emit to-do list]\n  PLAN --> EXEC[Executor<br/>file read / write / grep / run-tests]\n  EXEC --> TOOLS[(IDE filesystem<br/>+ process API)]\n  EXEC -->|each batch| REFLECT[Reflect on results]\n  REFLECT -->|update to-dos| PLAN\n  EXEC -.confirm edits.-> HITL[Human-in-the-loop<br/>diff review]\n  QWEN --> EXT\n  EXEC --> EXT","members":[{"pattern":"react","role":"first-class","note":"Agent mode is a ReAct loop over IDE tools.","evidence":[],"evidence_status":"none"},{"pattern":"plan-and-execute","role":"first-class","note":"Agent mode emits a to-do plan, then executes step by step.","evidence":[],"evidence_status":"none"},{"pattern":"tool-use","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"code-as-action","role":"first-class","note":"Agent mode authors and applies code edits directly.","evidence":[],"evidence_status":"none"},{"pattern":"agent-computer-interface","role":"first-class","note":"IDE-mediated filesystem/process API.","evidence":[],"evidence_status":"none"},{"pattern":"step-budget","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"agent-resumption","role":"supported","note":"Agent mode plans persist across IDE restarts within a workspace.","evidence":[],"evidence_status":"none"},{"pattern":"human-in-the-loop","role":"first-class","note":"User confirms file edits before they apply (default).","evidence":[],"evidence_status":"none"}],"alternatives":[{"composition":"cursor","relation":"competes-with","note":"Both are agent-mode IDE coding assistants; Lingma is the canonical 'Chinese Cursor alternative'."},{"composition":"github-copilot-coding-agent","relation":"competes-with","note":"GitHub Copilot is the global leader; Lingma is Alibaba's domestic counter."},{"composition":"marscode","relation":"competes-with","note":"Domestic peer from ByteDance with cloud-IDE positioning."},{"composition":"codegeex","relation":"competes-with","note":"Open-source Chinese peer from Tsinghua/Zhipu."},{"composition":"comate","relation":"competes-with","note":"Baidu's domestic peer on Wenxin."},{"composition":"codebuddy","relation":"competes-with","note":"Tencent's domestic peer on Hunyuan."}],"last_analyzed":"2026-05-22","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-05-22","status_in_practice":"mature","tags":["coding-agent","planning-control-flow","tool-use-environment","safety-control","governance-observability","full-code"],"references":[{"type":"doc","title":"Tongyi Lingma — Documentation","url":"https://www.alibabacloud.com/help/en/lingma"},{"type":"doc","title":"Tongyi Lingma — Homepage","url":"https://www.alibabacloud.com/product/lingma"}]},{"id":"v0","name":"v0","kind":"framework","category":"coding-agent","build_surface":"no-code","vendor":"Vercel","language":"Web product","status":"active","url":"https://v0.app/","docs_url":"https://v0.app/docs/introduction","intent":"Vercel's hosted AI agent for generating full-stack Next.js apps and live UIs from natural-language prompts, with one-click Vercel deployment, design-mode visual editing, and autonomous web search / error fixing.","description":"v0 is Vercel's hosted prompt-to-app product. It generates Next.js + Tailwind + shadcn/ui UI and backend code from natural-language prompts (or wireframes/mockups), shows a real-time live preview, and deploys to Vercel with one click. The agent plans, creates tasks, and connects to databases as it builds; it can autonomously web-search, inspect sites, fix errors, and call external tools. A design mode lets users iterate visually on generated output. Users can deploy directly or open a PR for review.","primary_use_cases":["prompt-to-UI generation in Next.js + Tailwind + shadcn/ui","full-stack app prototyping with real-time preview","one-click deployment to Vercel from the agent","visual design-mode iteration on generated code","autonomous web search and error fixing during generation"],"agent_loop_shape":"Spec-driven generation loop. User prompts (text or image); v0 plans tasks, generates code (Next.js, Tailwind, shadcn/ui), previews live, optionally inspects sites and web-searches, fixes errors automatically, and either deploys to Vercel or opens a PR. Design mode lets the user nudge the generated output through visual controls rather than text.","key_concepts":[{"name":"Prompt-to-app","summary":"Build agents, apps, and websites with AI from a single prompt.","maps_to_pattern":"spec-first-agent","url":"https://v0.app/"},{"name":"Next.js + Tailwind + shadcn/ui","summary":"Generated stack is opinionated around modern React tooling.","url":"https://v0.app/docs/introduction"},{"name":"Design mode","summary":"Visual controls and live preview fine-tune generated output.","url":"https://v0.app/"},{"name":"Autonomous capabilities","summary":"Web search, site inspection, error fixing, external tool integration.","maps_to_pattern":"tool-use","url":"https://v0.app/docs/introduction"},{"name":"One-click deploy to Vercel","summary":"Publish as live websites in seconds or open a PR.","url":"https://v0.app/"}],"pattern_composition":"flowchart TD\n  user[Prompt / wireframe] --> plan[v0 plans tasks]\n  plan --> gen[Generate Next.js code]\n  gen --> preview[Live preview]\n  gen --> tools{Need tools?}\n  tools -->|yes| web[Web search]\n  tools -->|yes| inspect[Site inspection]\n  tools -->|yes| db[Connect database]\n  tools -->|yes| err[Auto error fix]\n  web --> gen\n  inspect --> gen\n  db --> gen\n  err --> gen\n  preview --> design{Edit?}\n  design -->|design mode| visual[Visual controls]\n  design -->|prompt| gen\n  visual --> preview\n  preview --> ship{Ship}\n  ship -->|deploy| live[Vercel deploy]\n  ship -->|pr| pr[Open pull request]","members":[{"pattern":"spec-first-agent","role":"first-class","note":"Natural-language prompt is the spec; agent generates the whole project.","evidence":[{"type":"doc","title":"v0 docs - intro","url":"https://v0.app/docs/introduction","quote":"v0 is an AI agent that helps anyone create real code and full-stack apps and agents.","accessed":"2026-05-20"},{"type":"doc","title":"v0 homepage - prompt to app","url":"https://v0.app/","quote":"Generate working applications in minutes with AI. Publish as live websites in seconds.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"code-as-action","role":"first-class","note":"Generated code IS the deliverable; deploy / PR are the action options.","evidence":[{"type":"doc","title":"v0 docs - end-to-end","url":"https://v0.app/docs/introduction","quote":"End-to-end Development: Build both UI and backend logic, not just mockups","accessed":"2026-05-20"},{"type":"doc","title":"v0 docs - deploy or PR","url":"https://v0.app/docs/introduction","quote":"Deploy to production immediately, or open a pull request for review.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Agent uses web search, browser use, terminal commands, error fixing, external APIs and marketplace tools.","evidence":[{"type":"doc","title":"v0 docs - autonomous capabilities","url":"https://v0.app/docs/introduction","quote":"Autonomous capabilities including web search, site inspection, error fixing, and external tool integration","accessed":"2026-05-20"},{"type":"doc","title":"v0 agentic features - tool list","url":"https://v0.app/docs/agentic-features","quote":"It combines web search, browser use, error fixing, terminal commands, and external tool integration.","accessed":"2026-05-24"},{"type":"doc","title":"v0 agentic features - marketplace tools","url":"https://v0.app/docs/agentic-features","quote":"Marketplace integrations can also expose tools that v0 calls during generation to query your data.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"plan-and-execute","role":"first-class","note":"v0 coordinates multi-step workflows and maintains context across actions.","evidence":[{"type":"doc","title":"v0 agentic features - coordinates tasks","url":"https://v0.app/docs/agentic-features","quote":"Coordinates multiple tasks: Handles complex workflows involving multiple steps","accessed":"2026-05-24"},{"type":"doc","title":"v0 agentic features - maintains context","url":"https://v0.app/docs/agentic-features","quote":"Maintains context: Remembers previous actions and results throughout the conversation","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"spec-driven-loop","role":"first-class","note":"Loop runs prompt to code to preview to error-fix to deploy/PR.","evidence":[{"type":"doc","title":"v0 docs - combined platform loop","url":"https://v0.app/docs/introduction","quote":"Combines code generation, web browsing, debugging, and external API interactions in one platform","accessed":"2026-05-20"},{"type":"doc","title":"v0 docs - prompt to deploy","url":"https://v0.app/docs/introduction","quote":"Ship features, refine designs, update copy, and create live prototypes, all with a prompt. Deploy to production immediately, or open a pull request for review.","accessed":"2026-05-24"},{"type":"doc","title":"v0 docs - autonomous error fix","url":"https://v0.app/docs/agentic-features","quote":"When v0 detects errors in your project, it can diagnose and fix them automatically.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"structured-output","role":"limited","note":"Generated code follows an opinionated Next.js / Tailwind / shadcn/ui stack; no first-class JSON-mode primitive — the structure is the React/Next.js scaffold itself.","evidence":[{"type":"doc","title":"v0 — Docs (stack)","url":"https://v0.app/docs","quote":"Use modern tools like Next.js, Tailwind, shadcn/ui, and more.","accessed":"2026-05-24"},{"type":"doc","title":"v0 — Docs (scaffolding)","url":"https://v0.app/docs","quote":"scaffold full-stack apps or components following best practices and modern standards.","accessed":"2026-05-24"}],"evidence_status":"full"}],"instantiates":["modern-coding-agent"],"alternatives":[{"composition":"boltnew","relation":"competes-with","note":"Peer prompt-to-app hosted product."},{"composition":"lovable","relation":"competes-with","note":"Peer prompt-to-app hosted product."},{"composition":"replit-agent","relation":"competes-with","note":"Peer hosted natural-language app builder."}],"references":[{"type":"doc","title":"v0 homepage","url":"https://v0.app/","accessed":"2026-05-20"},{"type":"doc","title":"v0 docs - introduction","url":"https://v0.app/docs/introduction","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["closed-source","vercel","nextjs","tailwind","shadcn","deploy-from-agent"]},{"id":"windsurf","name":"Windsurf","aliases":["Codeium (legacy)"],"kind":"framework","category":"coding-agent","build_surface":"full-code","vendor":"Cognition (acquired Windsurf 2025-07)","language":"TypeScript (VS Code fork)","status":"active","url":"https://windsurf.com/","docs_url":"https://docs.windsurf.com/","intent":"Cognition's agentic IDE (a VS Code fork formerly from Codeium): the Cascade agent runs Code/Chat modes with tool-calling, MCP, terminal, web search, named checkpoints, real-time awareness, and a planning agent.","description":"Windsurf is an IDE built around Cascade, an agentic assistant with Code and Chat modes. Cascade calls tools (Search, Analyze, Web Search, MCP, terminal), creates an in-conversation Todo list for complex tasks, supports named project checkpoints, integrates the linter, and reads real-time user actions without explicit prompts. A specialised planning agent refines long-term strategy while the primary model executes. MCP is natively supported with stdio, Streamable HTTP, and SSE transports, OAuth, and a 100-tool ceiling per session. Cascade is capped at 20 tool calls per prompt before continue is required. Cognition acquired Windsurf in July 2025.","primary_use_cases":["agentic coding in a VS Code-like IDE","Code mode for autonomous edits, Chat mode for Q&A","MCP-extended toolchain inside the editor","named-checkpoint state navigation","planning-agent + execution-agent split"],"agent_loop_shape":"Cascade's mode-switched tool-using loop. User picks Code or Chat; Cascade plans (optionally building an in-thread Todo list), then runs tool calls (search, analyze, web search, MCP, terminal) up to a documented 20-tool-call budget per prompt. Real-time awareness reads user activity. Named checkpoints let the user revert state. A separate planning agent refines long-term strategy.","key_concepts":[{"name":"Cascade","summary":"The agentic assistant - Code/Chat modes, tool-calling, voice input, checkpoints, real-time awareness, linter integration.","url":"https://docs.windsurf.com/windsurf/cascade"},{"name":"MCP","summary":"Native MCP with stdio / Streamable HTTP / SSE + OAuth; 100-tool ceiling.","maps_to_pattern":"mcp","url":"https://docs.windsurf.com/windsurf/cascade/mcp"},{"name":"Checkpoints","summary":"Named project snapshots to navigate and revert state.","maps_to_pattern":"replay-time-travel","url":"https://docs.windsurf.com/windsurf/cascade"},{"name":"In-thread Todo list","summary":"Cascade builds a Todo list within the conversation for complex tasks.","maps_to_pattern":"todo-list-driven-agent","url":"https://docs.windsurf.com/windsurf/cascade"},{"name":"Planning agent + primary model","summary":"Planner refines long-term strategy while primary model handles immediate actions.","maps_to_pattern":"plan-and-execute","url":"https://docs.windsurf.com/windsurf/cascade"},{"name":"Tool-call cap","summary":"Cascade is limited to 20 tool calls per prompt; continue resumes.","maps_to_pattern":"step-budget","url":"https://docs.windsurf.com/windsurf/cascade"}],"pattern_composition":"flowchart TD\n  user[User prompt] --> mode{Code or Chat}\n  mode --> plan[Planning agent]\n  plan --> todo[Cascade Todo list]\n  todo --> loop[Cascade loop]\n  loop --> tool{Tool}\n  tool --> search[Search]\n  tool --> analyze[Analyze]\n  tool --> web[Web Search]\n  tool --> mcp[MCP tool]\n  tool --> term[Terminal]\n  search --> obs[Observation]\n  analyze --> obs\n  web --> obs\n  mcp --> obs\n  term --> obs\n  obs --> budget{20-tool cap reached?}\n  budget -->|no| loop\n  budget -->|yes| cont[Press continue]\n  cont --> loop\n  obs --> ckpt[Named checkpoint]\n  ckpt --> done[Done]","members":[{"pattern":"agent-computer-interface","role":"first-class","note":"Windsurf is a full VS Code-derived IDE; Cascade is its embedded agentic panel with real-time editor awareness.","evidence":[{"type":"doc","title":"Windsurf docs - Cascade","url":"https://docs.windsurf.com/windsurf/cascade","quote":"Cascade is Windsurf's agentic AI assistant with Code/Chat modes, tool calling, voice input, checkpoints, real-time awareness, and linter integration.","accessed":"2026-05-20"},{"type":"doc","title":"Windsurf docs - real-time awareness","url":"https://docs.windsurf.com/windsurf/cascade","quote":"A unique capability of Windsurf and Cascade is that it is aware of your real-time actions, removing the need to prompt with context on your prior actions.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"Native MCP integration with three transports plus OAuth and a 100-tool ceiling.","evidence":[{"type":"doc","title":"Windsurf MCP - protocol","url":"https://docs.windsurf.com/windsurf/cascade/mcp","quote":"MCP (Model Context Protocol) is a protocol that enables LLMs to access custom tools and services.","accessed":"2026-05-20"},{"type":"doc","title":"Windsurf MCP - tool cap","url":"https://docs.windsurf.com/windsurf/cascade/mcp","quote":"Cascade has a limit of 100 total tools that it has access to at any given time","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Documented tool surface: Search, Analyze, Web Search, MCP, terminal; Code mode performs codebase modifications.","evidence":[{"type":"doc","title":"Windsurf docs - tool list","url":"https://docs.windsurf.com/windsurf/cascade","quote":"Cascade has a variety of tools at its disposal, such as Search, Analyze, Web Search, MCP, and the terminal.","accessed":"2026-05-20"},{"type":"doc","title":"Windsurf docs - Code mode","url":"https://docs.windsurf.com/windsurf/cascade","quote":"Code mode allows Cascade to create and make modifications to your codebase, while Chat mode is optimized for questions around your codebase or general coding principles.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"step-budget","role":"first-class","note":"20 tool calls per prompt; continue button resumes, costing a new prompt credit.","evidence":[{"type":"doc","title":"Windsurf docs - 20-tool cap","url":"https://docs.windsurf.com/windsurf/cascade","quote":"Cascade can make up to 20 tool calls per prompt. If the trajectory stops, simply press the `continue` button.","accessed":"2026-05-24"},{"type":"doc","title":"Windsurf docs - continue costs new credit","url":"https://docs.windsurf.com/windsurf/cascade","quote":"Each `continue` will count as a new prompt credit due to tool calling costs.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"todo-list-driven-agent","role":"first-class","note":"Cascade builds an in-conversation Todo list for complex tasks; user and Cascade can both edit it.","evidence":[{"type":"doc","title":"Windsurf docs - Todo list","url":"https://docs.windsurf.com/windsurf/cascade","quote":"Cascade will create a Todo list within the conversation to track progress on complex tasks.","accessed":"2026-05-20"},{"type":"doc","title":"Windsurf docs - Todo list updates","url":"https://docs.windsurf.com/windsurf/cascade","quote":"To make changes to the plan, simply ask Cascade to make updates to the Todo list.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"plan-and-execute","role":"first-class","note":"Separate planning agent continuously refines a long-term plan; the user-selected model handles short-term actions.","evidence":[{"type":"doc","title":"Windsurf docs - planning agent","url":"https://docs.windsurf.com/windsurf/cascade","quote":"a specialized planning agent continuously refines the long-term plan while your selected model focuses on taking short-term actions","accessed":"2026-05-24"},{"type":"doc","title":"Windsurf docs - Todo list as plan","url":"https://docs.windsurf.com/windsurf/cascade","quote":"Cascade will create a Todo list within the conversation to track progress on complex tasks.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"replay-time-travel","role":"first-class","note":"Named snapshots/checkpoints navigate and revert project state from within the conversation.","evidence":[{"type":"doc","title":"Windsurf docs - checkpoints","url":"https://docs.windsurf.com/windsurf/cascade","quote":"You can also create a named snapshot/checkpoint of the current state of your project from within the conversation, which you can easily navigate to and revert at any time.","accessed":"2026-05-24"},{"type":"doc","title":"Windsurf docs - Cascade checkpoints feature","url":"https://docs.windsurf.com/windsurf/cascade","quote":"Cascade is Windsurf's agentic AI assistant with Code/Chat modes, tool calling, voice input, checkpoints, real-time awareness, and linter integration.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"approval-queue","role":"limited","note":"No documented per-tool approval queue. The 20-tool-call cap, named checkpoints, and per-MCP-tool toggle act as soft gates instead of an explicit approval prompt per action.","evidence":[{"type":"doc","title":"Windsurf — Cascade (tool call cap)","url":"https://docs.windsurf.com/windsurf/cascade/cascade","quote":"Cascade can make up to 20 tool calls per prompt.","accessed":"2026-05-24"},{"type":"doc","title":"Windsurf — Cascade (checkpoints)","url":"https://docs.windsurf.com/windsurf/cascade/cascade","quote":"You can also create a named snapshot/checkpoint of the current state of your project from within the conversation, which you can easily navigate to and revert at any time.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"Reason-then-tool-call loop with observations fed back; canonical agentic loop with a 20-tool budget.","evidence":[{"type":"doc","title":"Windsurf docs - tool-calling loop","url":"https://docs.windsurf.com/windsurf/cascade","quote":"Cascade is Windsurf's agentic AI assistant with Code/Chat modes, tool calling, voice input, checkpoints, real-time awareness, and linter integration.","accessed":"2026-05-24"},{"type":"doc","title":"Windsurf docs - tool-call budget per prompt","url":"https://docs.windsurf.com/windsurf/cascade","quote":"Cascade can make up to 20 tool calls per prompt. If the trajectory stops, simply press the `continue` button.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"agentic-rag","role":"limited","note":"Cascade auto-triggers web search and chunks/reads pages; @web and @docs explicit retrieval triggers exist; no documented vector RAG primitive on pages reviewed.","evidence":[{"type":"doc","title":"Windsurf docs - search tool","url":"https://docs.windsurf.com/windsurf/cascade","quote":"Cascade has a variety of tools at its disposal, such as Search, Analyze, Web Search, MCP, and the terminal.","accessed":"2026-05-20"},{"type":"doc","title":"Windsurf docs - web search trigger","url":"https://docs.windsurf.com/windsurf/cascade/web-search","quote":"Cascade can deduce that certain prompts from the user may require a real-time web search to provide the optimal response.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"shadow-workspace-vcs","role":"first-class","note":"Cascade lets the user revert all agent code changes back to a previous step and create named checkpoint snapshots to return to.","evidence":[{"type":"doc","title":"Cascade - Windsurf Docs","url":"https://docs.windsurf.com/windsurf/cascade/cascade","quote":"This will revert all code changes back to the state of your codebase at the desired step.","accessed":"2026-06-14"}],"evidence_status":"full"}],"instantiates":["modern-coding-agent","safety-hardening"],"alternatives":[{"composition":"cursor","relation":"competes-with","note":"Peer VS Code-derived agentic IDE."},{"composition":"claude-code","relation":"competes-with","note":"Peer agentic coding tool."},{"composition":"devin","relation":"competes-with","note":"Same parent (Cognition) post-acquisition; Devin is the autonomous cloud agent now embedded inside Windsurf."},{"composition":"kiro","relation":"competes-with","note":"Peer agentic IDE."},{"composition":"zed-ai","relation":"competes-with","note":"Peer agentic IDE."}],"references":[{"type":"doc","title":"Windsurf editor","url":"https://windsurf.com/editor","accessed":"2026-05-20"},{"type":"doc","title":"Cascade docs","url":"https://docs.windsurf.com/windsurf/cascade","accessed":"2026-05-20"},{"type":"doc","title":"Cascade MCP docs","url":"https://docs.windsurf.com/windsurf/cascade/mcp","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["closed-source","cognition","vscode-fork","ide-agent","mcp","checkpoints"]},{"id":"zed-ai","name":"Zed AI","kind":"framework","category":"coding-agent","build_surface":"full-code","vendor":"Zed Industries","language":"Rust","license":"AGPL-3.0 / Apache-2.0 / GPL-3.0 (mixed; see repo)","status":"active","url":"https://zed.dev/","repo":"https://github.com/zed-industries/zed","docs_url":"https://zed.dev/docs/ai/overview","intent":"The AI surface inside Zed - an open-source Rust code editor. Built around an Agent Panel with Ask/Write/Minimal profiles, configurable tool permissions, MCP server tools, Inline Assistant, and the Agent Client Protocol for embedding external agents (Claude Agent, Codex, Gemini CLI).","description":"Zed AI is the AI feature surface in the Zed editor: an Agent Panel hosting threads, Inline Assistant for in-place rewrites, and Edit Prediction for keystroke-level completion. Three built-in profiles ship - Ask (read-only tools), Write (file edits + terminal), and Minimal - with custom profiles available. Tool permissions default to confirm but can be set to allow or deny. MCP servers extend the toolchain (Tools and Prompts supported). External agents like Claude Agent, Codex CLI, and Gemini CLI run inside Zed through the Agent Client Protocol, with billing handled by the external provider.","primary_use_cases":["Agent Panel inside Zed for thread-based coding","Inline Assistant for in-place rewrites","Edit Prediction for keystroke completions","MCP server tool extension","embedding external CLI agents via ACP"],"agent_loop_shape":"Profile-scoped tool-using loop. User opens a thread in the Agent Panel; the active profile (Ask / Write / Minimal / custom) determines which tools the model may call. Per-tool permission defaults to confirm, prompting before any tool action; allow and deny modes exist. MCP servers contribute additional tools. External agents follow their own loops, surfacing through Zed's UI via the Agent Client Protocol.","key_concepts":[{"name":"Agent Panel","summary":"Conversation view for the active thread; the primary UI for agent interaction.","maps_to_pattern":"agent-computer-interface","url":"https://zed.dev/docs/ai/agent-panel"},{"name":"Profiles (Ask / Write / Minimal)","summary":"Built-in tool sets; Ask is read-only, Write has file edits and terminal, Minimal is reduced.","maps_to_pattern":"role-typed-subagents","url":"https://zed.dev/docs/ai/agent-panel"},{"name":"Tool permissions","summary":"agent.tool_permissions.default - confirm (default) / allow / deny.","maps_to_pattern":"approval-queue","url":"https://zed.dev/docs/ai/agent-panel"},{"name":"MCP servers","summary":"Tools and Prompts capabilities; runtime tool-list changes auto-reload.","maps_to_pattern":"mcp","url":"https://zed.dev/docs/ai/mcp"},{"name":"Agent Client Protocol (ACP)","summary":"Open protocol Zed uses to embed Claude Agent, Codex, Gemini CLI, Copilot, and custom agents.","url":"https://zed.dev/docs/ai/external-agents"},{"name":"Inline Assistant and Edit Prediction","summary":"In-place rewrites and keystroke completions inside the editor.","url":"https://zed.dev/docs/ai/overview"}],"pattern_composition":"flowchart TD\n  user[User in Zed] --> ap[Agent Panel thread]\n  ap --> profile{Profile}\n  profile -->|Ask| ro[Read-only tools]\n  profile -->|Write| rw[File edits + terminal]\n  profile -->|Minimal| min[Minimal tools]\n  profile -->|Custom| cust[Custom profile]\n  ro --> perm{Tool permissions}\n  rw --> perm\n  min --> perm\n  cust --> perm\n  perm -->|confirm| ask[Prompt for approval]\n  perm -->|allow| auto[Run]\n  perm -->|deny| block[Block]\n  ask --> run[Tool call]\n  auto --> run\n  run --> mcp[MCP tool]\n  run --> builtin[Built-in tool]\n  mcp --> obs[Observe]\n  builtin --> obs\n  obs --> ap\n  ap --> ext{External agent via ACP?}\n  ext -->|yes| acp[Claude Agent / Codex / Gemini CLI]\n  ext -->|no| done[Done]\n  acp --> done","members":[{"pattern":"agent-computer-interface","role":"first-class","note":"Agent Panel is the documented thread UI; Inline Assistant adds in-place editing; Edit Prediction adds keystroke completions.","evidence":[{"type":"doc","title":"Zed docs - Agent Panel as conversation view","url":"https://zed.dev/docs/ai/overview","quote":"The [Agent Panel](./agent-panel.html) is the conversation view for the active thread.","accessed":"2026-05-24"},{"type":"doc","title":"Zed docs - Inline Assistant","url":"https://zed.dev/docs/ai/overview","quote":"The [Inline Assistant](./inline-assistant.html) works differently: select code or a terminal command, describe what you want, and the model rewrites the selection in place.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"approval-queue","role":"first-class","note":"agent.tool_permissions.default = confirm / allow / deny is a canonical approval gate.","evidence":[{"type":"doc","title":"Zed docs - confirm default","url":"https://zed.dev/docs/ai/agent-panel","quote":"Prompts for approval before running any tool action","accessed":"2026-05-20"},{"type":"doc","title":"Zed docs - allow / deny","url":"https://zed.dev/docs/ai/agent-panel","quote":"Auto-approves tool actions without prompting","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"Tools and Prompts capabilities; runtime tool-list auto-reload via notifications/tools/list_changed.","evidence":[{"type":"doc","title":"Zed docs - MCP definition","url":"https://zed.dev/docs/ai/mcp","quote":"The Model Context Protocol (MCP) is an open protocol for connecting LLM applications to external tools and data sources through a standard interface.","accessed":"2026-05-20"},{"type":"doc","title":"Zed docs - MCP Tools and Prompts capabilities","url":"https://zed.dev/docs/ai/mcp","quote":"Zed currently supports MCP's Tools and Prompts features.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Built-in tools per profile plus MCP-server tools; agentic editing comes through tool calling.","evidence":[{"type":"doc","title":"Zed docs - Write profile","url":"https://zed.dev/docs/ai/agent-panel","quote":"Write: A profile with tools to allow the LLM to write to your files and run terminal commands","accessed":"2026-05-24"},{"type":"doc","title":"Zed docs - tool calling enables agentic editing","url":"https://zed.dev/docs/ai/agent-panel","quote":"The Agent Panel supports tool calling, which enables agentic editing. Zed includes several built-in tools for searching your codebase, editing files","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"role-typed-subagents","role":"first-class","note":"Ask / Write / Minimal are role-typed tool envelopes; custom profiles too.","evidence":[{"type":"doc","title":"Zed docs - Ask profile","url":"https://zed.dev/docs/ai/agent-panel","quote":"A profile with read-only tools. Best for asking questions about your code base","accessed":"2026-05-20"},{"type":"doc","title":"Zed docs - Write profile","url":"https://zed.dev/docs/ai/agent-panel","quote":"A profile with tools to allow the LLM to write to your files and run terminal commands","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"multi-model-routing","role":"first-class","note":"Hosted models or BYOK across Anthropic / OpenAI / Google / Ollama and 8+ others.","evidence":[{"type":"doc","title":"Zed docs - models / BYOK","url":"https://zed.dev/docs/ai/overview","quote":"Use Zed's hosted models or [bring your own API keys](./llm-providers.html) from Anthropic, OpenAI, Google, Ollama, and 8+ other providers.","accessed":"2026-05-24"},{"type":"doc","title":"Zed docs - external agents BYOA via ACP","url":"https://zed.dev/docs/ai/overview","quote":"Run Claude Agent, Gemini CLI, Codex, and other CLI-based agents directly in Zed through the [Agent Client Protocol](https://zed.dev/acp).","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"handoff","role":"first-class","note":"Agent Client Protocol lets Zed embed external agents (Claude Agent, Codex, Gemini CLI, Copilot) as separate processes. UI hands off to the external provider; billing is direct.","evidence":[{"type":"doc","title":"Zed docs - external agents via ACP","url":"https://zed.dev/docs/ai/external-agents","quote":"Note that Zed's interaction with external agents is strictly UI-based; the billing, legal, and terms arrangement is directly between you and the agent provider.","accessed":"2026-05-24"},{"type":"doc","title":"Zed docs - external agents as separate processes","url":"https://zed.dev/docs/ai/external-agents","quote":"External agents run as separate processes that communicate with Zed via the [Agent Client Protocol (ACP)](https://agentclientprotocol.com).","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"code-as-action","role":"first-class","note":"Write profile and Inline Assistant edit files; terminal commands are tool calls; restore-checkpoint button surfaces after each edit.","evidence":[{"type":"doc","title":"Zed docs - Write profile edits files and runs terminal","url":"https://zed.dev/docs/ai/agent-panel","quote":"Write: A profile with tools to allow the LLM to write to your files and run terminal commands","accessed":"2026-05-24"},{"type":"doc","title":"Zed docs - restore checkpoint after edit","url":"https://zed.dev/docs/ai/agent-panel","quote":"Every time the model performs an edit, you should see a 'Restore Checkpoint' button at the top of your message","accessed":"2026-05-24"}],"evidence_status":"full"}],"instantiates":["modern-coding-agent"],"alternatives":[{"composition":"cursor","relation":"competes-with","note":"Peer agentic IDE; closed source."},{"composition":"continue-dev","relation":"competes-with","note":"Peer open-source IDE assistant."},{"composition":"windsurf","relation":"competes-with","note":"Peer agentic IDE."}],"references":[{"type":"doc","title":"Zed AI overview","url":"https://zed.dev/docs/ai/overview","accessed":"2026-05-20"},{"type":"doc","title":"Zed AI agent panel","url":"https://zed.dev/docs/ai/agent-panel","accessed":"2026-05-20"},{"type":"doc","title":"Zed AI MCP","url":"https://zed.dev/docs/ai/mcp","accessed":"2026-05-20"},{"type":"doc","title":"Zed external agents via ACP","url":"https://zed.dev/docs/ai/external-agents","accessed":"2026-05-20"},{"type":"repo","title":"zed-industries/zed","url":"https://github.com/zed-industries/zed","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","rust","ide-agent","mcp","acp","profiles"]},{"id":"spec-kit","name":"GitHub Spec Kit","kind":"framework","category":"coding-agent","build_surface":"full-code","vendor":"GitHub","language":"Python (CLI), Markdown templates","license":"MIT","status":"active","url":"https://github.com/github/spec-kit","repo":"https://github.com/github/spec-kit","intent":"Spec-Driven Development toolkit from GitHub: a CLI plus template suite that forces an explicit Specify / Plan / Tasks authoring phase before any agent implementation step runs.","description":"Spec Kit is GitHub's open-source toolkit for Spec-Driven Development with coding agents. The `specify` CLI scaffolds a project structure (constitution, spec, plan, and tasks templates) and walks the user through four phases — Specify (high-level description), Plan (tech stack and architecture), Tasks (executable task list), Implement (the coding agent tackles tasks one by one). The toolkit is editor- and agent-agnostic: it works with 30+ coding agents including Claude Code, GitHub Copilot, Gemini CLI, Cursor CLI, Codex CLI, Qwen CLI, opencode, and Goose. The spec, plan, and tasks files become the authoritative inputs the implementation loop reads on every turn, anchored by a constitution.md that captures the project's foundational guidelines.","primary_use_cases":["Spec-Driven Development with coding agents","structured pre-flight authoring before implementation","agent-agnostic spec / plan / tasks scaffolding","reducing re-asking in long agent loops by persisting a reviewed spec","enforcing project-wide architectural principles via constitution.md"],"agent_loop_shape":"CLI-driven phased workflow. `specify init <project_name>` scaffolds the templates; `/speckit.specify`, `/speckit.plan`, and `/speckit.tasks` walk the author through pre-flight authoring inside their coding-agent of choice; only `/speckit.implement` then executes the tasks against the plan. Agent execution itself happens inside the host coding agent (Claude Code, Copilot, Cursor, etc.) reading the spec / plan / tasks markdown produced by Spec Kit.","key_concepts":[{"name":"Four-phase workflow","summary":"Specify -> Plan -> Tasks -> Implement, with clear checkpoints between phases.","maps_to_pattern":"spec-driven-loop","url":"https://github.blog/ai-and-ml/generative-ai/spec-driven-development-with-ai-get-started-with-a-new-open-source-toolkit/"},{"name":"specify CLI","summary":"`specify init <project_name>` bootstraps the project with required Spec Kit artefacts and detects which coding agent runners are installed.","url":"https://github.com/github/spec-kit"},{"name":"Slash commands","summary":"/speckit.specify, /speckit.plan, /speckit.tasks, /speckit.implement drive the phased workflow inside the host coding agent.","maps_to_pattern":"pre-flight-spec-authoring","url":"https://github.com/github/spec-kit"},{"name":"constitution.md","summary":"Acts as the architectural DNA of the system, capturing foundational project guidelines so every generated implementation maintains consistency, simplicity, and quality.","maps_to_pattern":"spec-first-agent","url":"https://github.com/github/spec-kit/blob/main/spec-driven.md"},{"name":"Feature artefacts","summary":"specs/NNN-<feature>/plan.md and tasks.md are the persistent inputs the coding agent reads on each implementation turn.","maps_to_pattern":"spec-first-agent","url":"https://github.com/github/spec-kit"},{"name":"Agent-agnostic runner support","summary":"Spec Kit works with 30+ AI coding agents including Claude Code, GitHub Copilot, Gemini CLI, Cursor CLI, Codex CLI, Qwen CLI, opencode, and Goose.","url":"https://github.com/github/spec-kit"},{"name":"Executable specifications","summary":"Specifications become executable, directly generating working implementations rather than just guiding them.","url":"https://github.com/github/spec-kit/blob/main/spec-driven.md"}],"members":[{"pattern":"pre-flight-spec-authoring","role":"first-class","note":"Spec Kit's whole point: enforce an explicit Specify phase that authors and critiques the spec before code generation is allowed to start.","evidence":[{"type":"repo","title":"github/spec-kit README — /speckit.specify","url":"https://github.com/github/spec-kit","quote":"Use the `/speckit.specify` command to describe what you want to build.","accessed":"2026-05-24"},{"type":"blog","title":"GitHub Blog — Spec-Driven Development with AI","url":"https://github.blog/ai-and-ml/generative-ai/spec-driven-development-with-ai-get-started-with-a-new-open-source-toolkit/","quote":"It works in four phases with clear checkpoints... Specify: You provide a high-level description... Plan: Now you get technical... Tasks: The coding agent takes the spec and the plan and breaks them down... Implement: Your coding agent tackles the tasks one by one.","accessed":"2026-05-24"},{"type":"doc","title":"spec-kit/spec-driven.md — specification primacy","url":"https://github.com/github/spec-kit/blob/main/spec-driven.md","quote":"The specification becomes the primary artifact. Code becomes its expression in a particular language and framework.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"spec-first-agent","role":"first-class","note":"The implementation loop reads spec.md / plan.md / tasks.md as authoritative inputs on every turn — the spec-first agent shape — and is anchored by an immutable constitution.md.","evidence":[{"type":"doc","title":"spec-kit/spec-driven.md — constitution as architectural DNA","url":"https://github.com/github/spec-kit/blob/main/spec-driven.md","quote":"The constitution (`memory/constitution.md`) acts as the architectural DNA of the system, ensuring that every generated implementation maintains consistency, simplicity, and quality.","accessed":"2026-05-24"},{"type":"repo","title":"github/spec-kit README — constitution file","url":"https://github.com/github/spec-kit","quote":"This step creates or updates the `.specify/memory/constitution.md` file with your project's foundational guidelines.","accessed":"2026-05-24"},{"type":"blog","title":"GitHub Blog — spec/plan/tasks as authoritative inputs","url":"https://github.blog/ai-and-ml/generative-ai/spec-driven-development-with-ai-get-started-with-a-new-open-source-toolkit/","quote":"The specification captures the intent clearly, the plan translates it into technical decisions, the tasks break it into implementable pieces, and your AI coding agent handles the actual coding.","accessed":"2026-05-24"},{"type":"repo","title":"github/spec-kit README — agent-runner support","url":"https://github.com/github/spec-kit","quote":"Spec Kit works with 30+ AI coding agents — both CLI tools and IDE-based assistants.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"spec-driven-loop","role":"supported","note":"The phased workflow (Specify → Plan → Tasks → Implement) is a concrete instance of the spec-driven loop with explicit phase boundaries; specify/plan/tasks files persist between turns.","evidence":[{"type":"repo","title":"github/spec-kit README — phase commands","url":"https://github.com/github/spec-kit","quote":"Use `/speckit.tasks` to create an actionable task list from your implementation plan.","accessed":"2026-05-24"},{"type":"repo","title":"github/spec-kit README — implement phase","url":"https://github.com/github/spec-kit","quote":"Use `/speckit.implement` to execute all tasks and build your feature according to the plan.","accessed":"2026-05-24"},{"type":"doc","title":"spec-kit/spec-driven.md — three powerful commands","url":"https://github.com/github/spec-kit/blob/main/spec-driven.md","quote":"The SDD methodology is significantly enhanced through three powerful commands that automate the specification → planning → tasking workflow","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"context-driven-architecture-drift","role":"core","note":"Spec Kit's constitution captures non-negotiable architectural principles as a machine-readable artifact that governs every agent-generated diff, the encoded spec whose absence drives this drift.","evidence":[{"type":"doc","title":"spec-kit/spec-driven.md at main","url":"https://github.com/github/spec-kit/blob/main/spec-driven.md","quote":"At the heart of SDD lies a constitution—a set of immutable principles that govern how specifications become code.","accessed":"2026-06-14"}],"evidence_status":"full"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-05-24","status_in_practice":"emerging","pattern_composition":"flowchart TD\n  fw[\"GitHub Spec Kit\"]\n  fw --> p1[\"Context-Driven Architecture Drift<br/>(core)\"]\n  fw --> p2[\"Pre-Flight Spec Authoring<br/>(first-class)\"]\n  fw --> p3[\"Spec-First Agent<br/>(first-class)\"]\n  fw --> p4[\"Spec-Driven Loop<br/>(supported)\"]","tags":["coding-agent","planning-control-flow","anti-patterns","full-code"],"references":[{"type":"doc","title":"GitHub Spec Kit — Homepage","url":"https://github.com/github/spec-kit"}]},{"id":"opencode","name":"OpenCode","kind":"framework","category":"coding-agent","build_surface":"full-code","vendor":"anomalyco / open-source community","language":"TypeScript","license":"MIT","status":"active","url":"https://opencode.ai","repo":"https://github.com/sst/opencode","docs_url":"https://opencode.ai/docs/","intent":"The open source AI coding agent for the terminal, model- and provider-agnostic.","description":"Terminal-native open-source coding agent with a dual-agent model: a build agent with full development access and a plan agent for read-only exploration, switchable via Tab, plus a general subagent for complex search and multistep tasks. Broad model-provider support, LSP integration and a beta desktop app. One of the most-starred open-source coding agents.","primary_use_cases":["terminal-native agentic coding across many model providers","read-only planning then full-access execution","delegating complex search/multistep work to a subagent"],"agent_loop_shape":"ReAct-style tool loop with a mode switch: the plan agent explores read-only; on switch the build agent edits files and runs commands. A general subagent handles isolated multistep search tasks.","key_concepts":[{"name":"Build / plan agents","summary":"Full-access build agent and read-only plan agent, switchable via Tab.","maps_to_pattern":"plan-and-execute","url":"https://opencode.ai/docs/"},{"name":"General subagent","summary":"Isolated agent for complex search and multistep tasks.","maps_to_pattern":"subagent-isolation"}],"pattern_composition":"flowchart TD\n  USER[Prompt] --> MODE{Plan or build?}\n  MODE -->|plan| EXPLORE[Read-only exploration]\n  EXPLORE --> MODE\n  MODE -->|build| LOOP[Tool loop: read/edit/run]\n  LOOP --> SUB[General subagent<br/>isolated search]\n  SUB --> LOOP\n  LOOP --> OUT[Changes]\n  PROVIDERS[(Many model providers)] -.power.-> LOOP","members":[{"pattern":"plan-and-execute","role":"first-class","note":"plan - Read-only agent for analysis and code exploration","evidence":[{"type":"repo","title":"sst/opencode README","url":"https://github.com/sst/opencode","quote":"plan - Read-only agent for analysis and code exploration","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"subagent-isolation","role":"first-class","note":"a general subagent for complex searches and multistep tasks","evidence":[{"type":"repo","title":"sst/opencode README","url":"https://github.com/sst/opencode","quote":"a general subagent for complex searches and multistep tasks","accessed":"2026-06-04"}],"evidence_status":"limited"}],"instantiates":["modern-coding-agent"],"alternatives":[{"composition":"claude-code","relation":"competes-with","note":"Peer terminal coding agent (first-party, proprietary)."},{"composition":"aider","relation":"competes-with","note":"Peer open-source terminal coding agent."},{"composition":"codex-cli","relation":"competes-with","note":"Peer terminal coding agent (OpenAI)."}],"references":[{"type":"repo","title":"sst/opencode","url":"https://github.com/sst/opencode","accessed":"2026-06-04"},{"type":"doc","title":"OpenCode documentation","url":"https://opencode.ai/docs/","accessed":"2026-06-04"}],"last_analyzed":"2026-06-04","verification_status":"partial","last_updated":"2026-06-17","date_added":"2026-06-04","status_in_practice":"mature","tags":["coding-agent","cli","open-source","typescript","multi-provider"]},{"id":"google-antigravity","name":"Google Antigravity","kind":"framework","category":"coding-agent","build_surface":"closed-application","vendor":"Google","language":"Go (CLI)","license":"proprietary","status":"active","first_released":"2026","url":"https://antigravity.google/","docs_url":"https://antigravity.google/docs","intent":"Google's agent-first development platform (IDE + Go CLI) with multi-agent orchestration; successor to Gemini CLI.","description":"Agentic IDE plus a Go-based Antigravity CLI announced at Google I/O 2026. It carries forward Gemini CLI's Agent Skills, Hooks, Subagents and Extensions (now plugins) and adds multi-agent orchestration. Google is deprecating Gemini CLI for free/individual users on 2026-06-18 and steering them to Antigravity.","primary_use_cases":["agent-first coding in an IDE and terminal on Google models","skills/hooks/subagents-based extensibility migrated from Gemini CLI","multi-agent orchestration of coding tasks"],"agent_loop_shape":"A tool-using coding agent with lifecycle hooks and skill loading; subagents handle delegated work, and a multi-agent orchestration layer coordinates them across the IDE and CLI surfaces.","key_concepts":[{"name":"Agent Skills","summary":"On-demand loaded skill workflows (migrated from Gemini CLI).","maps_to_pattern":"agent-skills","url":"https://antigravity.google/"},{"name":"Hooks","summary":"Lifecycle event handlers.","maps_to_pattern":"event-driven-agent"},{"name":"Subagents","summary":"Delegated agents in isolated contexts.","maps_to_pattern":"subagent-isolation"}],"pattern_composition":"flowchart TD\n  USER[Prompt] --> SKILLS[Agent Skills loaded on demand]\n  SKILLS --> LOOP[Tool-use coding loop]\n  LOOP --> HOOK[Lifecycle hooks]\n  HOOK --> LOOP\n  LOOP --> SUB[Subagents]\n  SUB --> LOOP\n  LOOP --> ORCH[Multi-agent orchestration]\n  ORCH --> OUT[Result]","members":[{"pattern":"agent-skills","role":"first-class","note":"Agent Skills, Hooks, Subagents, and Extensions (now as Antigravity plugins)","evidence":[{"type":"blog","title":"Transitioning Gemini CLI to Antigravity CLI","url":"https://developers.googleblog.com/an-important-update-transitioning-gemini-cli-to-antigravity-cli/","quote":"Agent Skills, Hooks, Subagents, and Extensions (now as Antigravity plugins)","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"event-driven-agent","role":"first-class","note":"Agent Skills, Hooks, Subagents, and Extensions (now as Antigravity plugins)","evidence":[{"type":"blog","title":"Transitioning Gemini CLI to Antigravity CLI","url":"https://developers.googleblog.com/an-important-update-transitioning-gemini-cli-to-antigravity-cli/","quote":"Agent Skills, Hooks, Subagents, and Extensions (now as Antigravity plugins)","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"subagent-isolation","role":"first-class","note":"Agent Skills, Hooks, Subagents, and Extensions (now as Antigravity plugins)","evidence":[{"type":"blog","title":"Transitioning Gemini CLI to Antigravity CLI","url":"https://developers.googleblog.com/an-important-update-transitioning-gemini-cli-to-antigravity-cli/","quote":"Agent Skills, Hooks, Subagents, and Extensions (now as Antigravity plugins)","accessed":"2026-06-04"}],"evidence_status":"limited"}],"instantiates":["modern-coding-agent"],"alternatives":[{"composition":"claude-code","relation":"competes-with","note":"Peer first-party terminal+IDE coding agent."},{"composition":"codex-cli","relation":"competes-with","note":"Peer first-party terminal coding agent (OpenAI)."}],"references":[{"type":"blog","title":"An important update: Transitioning Gemini CLI to Antigravity CLI","url":"https://developers.googleblog.com/an-important-update-transitioning-gemini-cli-to-antigravity-cli/","accessed":"2026-06-04"},{"type":"blog","title":"Bye-bye, Gemini CLI; Google nudges devs toward Antigravity (The Register)","url":"https://www.theregister.com/ai-ml/2026/05/20/bye-bye-gemini-cli-google-nudges-devs-toward-antigravity/","accessed":"2026-06-04"},{"type":"doc","title":"Google Antigravity","url":"https://antigravity.google/","accessed":"2026-06-04"}],"last_analyzed":"2026-06-04","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-06-04","status_in_practice":"emerging","tags":["google","coding-agent","cli","ide","proprietary","successor-to-gemini-cli"]},{"id":"amp","name":"Amp","kind":"framework","category":"coding-agent","build_surface":"closed-application","vendor":"Sourcegraph","language":"proprietary","license":"proprietary","status":"active","url":"https://ampcode.com/","docs_url":"https://ampcode.com/manual","intent":"Sourcegraph's frontier coding agent across CLI and IDE, with an autonomous deep-research mode.","description":"Sourcegraph's coding agent available as both a CLI and IDE integration. Headline feature is a 'Deep mode' autonomous research/problem-solving mode using extended reasoning; also ships a code-review agent, an image tool, a walkthrough skill and plugin extensibility. Pay-as-you-go pricing.","primary_use_cases":["agentic coding across CLI and IDE","autonomous deep research/problem-solving on complex tasks","automated code review within the agent"],"agent_loop_shape":"A tool-using coding loop with subagents; Deep mode switches into an extended-reasoning autonomous research/solve loop for hard tasks before returning to implementation.","key_concepts":[{"name":"Deep mode","summary":"Autonomous research/problem-solving with extended reasoning.","maps_to_pattern":"extended-thinking","url":"https://ampcode.com/"},{"name":"Code-review agent","summary":"A separate agent reviews generated code.","maps_to_pattern":"agent-as-judge"}],"pattern_composition":"flowchart TD\n  USER[Prompt] --> LOOP[Reason-act coding loop]\n  LOOP --> DEEP{Hard task?}\n  DEEP -->|yes| DEEPMODE[Deep mode<br/>extended reasoning]\n  DEEPMODE --> LOOP\n  LOOP --> SUB[Subagents]\n  SUB --> LOOP\n  LOOP --> REVIEW[Code-review agent]\n  REVIEW --> OUT[Result]","members":[{"pattern":"react","role":"core","note":"Amp is the frontier coding agent for your terminal and editor.","evidence":[{"type":"doc","title":"Amp manual","url":"https://ampcode.com/manual","quote":"Amp is the frontier coding agent for your terminal and editor.","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"subagent-isolation","role":"first-class","note":"Amp will sometimes spawn subagents for complex tasks that benefit from independent execution.","evidence":[{"type":"doc","title":"Amp manual","url":"https://ampcode.com/manual","quote":"Amp will sometimes spawn subagents for complex tasks that benefit from independent execution.","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"extended-thinking","role":"first-class","note":"deep: Deep reasoning with GPT-5.5 for extended thinking on complex problems.","evidence":[{"type":"doc","title":"Amp manual","url":"https://ampcode.com/manual","quote":"deep: Deep reasoning with GPT-5.5 for extended thinking on complex problems.","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"agent-as-judge","role":"supported","note":"Amp can review your code for bugs, security issues, performance problems, and style violations.","evidence":[{"type":"doc","title":"Amp manual","url":"https://ampcode.com/manual","quote":"Amp can review your code for bugs, security issues, performance problems, and style violations.","accessed":"2026-06-04"}],"evidence_status":"limited"}],"instantiates":["modern-coding-agent"],"alternatives":[{"composition":"sourcegraph-cody","relation":"same-vendor","note":"Sourcegraph's earlier code-AI product; Amp is the newer agent."},{"composition":"claude-code","relation":"competes-with","note":"Peer CLI+IDE coding agent."}],"references":[{"type":"doc","title":"Amp","url":"https://ampcode.com/","accessed":"2026-06-04"},{"type":"doc","title":"Amp — Sourcegraph","url":"https://sourcegraph.com/amp","accessed":"2026-06-04"},{"type":"doc","title":"Amp manual","url":"https://ampcode.com/manual","accessed":"2026-06-04"}],"last_analyzed":"2026-06-04","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-06-04","status_in_practice":"emerging","tags":["sourcegraph","coding-agent","cli","ide","proprietary"]},{"id":"kiro","name":"Kiro","kind":"framework","category":"coding-agent","build_surface":"closed-application","vendor":"Amazon Web Services","language":"based on Code OSS","license":"proprietary","status":"active","url":"https://kiro.dev/","docs_url":"https://kiro.dev/docs/","intent":"Bring engineering rigor to agentic development through spec-driven workflows.","description":"AWS agentic IDE (with CLI and web interfaces) built on Code OSS, centered on spec-driven development: a natural-language prompt is first turned into requirements and acceptance criteria in EARS notation, then architecture and tasks, before any code is generated. Imports VS Code settings/themes and Open VSX plugins.","primary_use_cases":["spec-driven feature development with explicit requirements before code","long-running task completion with correctness validation","VS Code-compatible agentic IDE workflows"],"agent_loop_shape":"Spec-first loop: prompt → EARS requirements + acceptance criteria → architecture/design → task breakdown → implementation, with validation against the spec; runs as an IDE/CLI agent.","key_concepts":[{"name":"Spec-driven development","summary":"Prompts become EARS requirements and acceptance criteria before code.","maps_to_pattern":"spec-driven-loop","url":"https://kiro.dev/"}],"pattern_composition":"flowchart TD\n  PROMPT[Natural-language prompt] --> REQ[Requirements + acceptance criteria<br/>EARS notation]\n  REQ --> DESIGN[Architecture / design]\n  DESIGN --> TASKS[Task breakdown]\n  TASKS --> IMPL[Implementation]\n  IMPL --> VALIDATE{Meets spec?}\n  VALIDATE -->|no| IMPL\n  VALIDATE -->|yes| OUT[Validated code]","members":[{"pattern":"spec-driven-loop","role":"core","note":"Specs or specifications are structured artifacts that formalize the development process for features and bug fixes in your application.","evidence":[{"type":"doc","title":"Kiro — Specs","url":"https://kiro.dev/docs/specs/","quote":"Specs or specifications are structured artifacts that formalize the development process for features and bug fixes in your application.","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"pre-flight-spec-authoring","role":"first-class","note":"Feature Specs: User stories and acceptance criteria in requirements.md","evidence":[{"type":"doc","title":"Kiro — Specs","url":"https://kiro.dev/docs/specs/","quote":"Feature Specs: User stories and acceptance criteria in requirements.md","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"plan-and-execute","role":"first-class","note":"Tasks — Generate discrete, executable implementation tasks in tasks.md","evidence":[{"type":"doc","title":"Kiro — Specs","url":"https://kiro.dev/docs/specs/","quote":"Tasks — Generate discrete, executable implementation tasks in tasks.md","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"agent-computer-interface","role":"supported","note":"Kiro is based on Code OSS, so you can import your VS Code settings, themes, and Open VSX compatible plugins","evidence":[{"type":"doc","title":"Kiro","url":"https://kiro.dev/","quote":"Kiro is based on Code OSS, so you can import your VS Code settings, themes, and Open VSX compatible plugins","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"context-driven-architecture-drift","role":"core","note":"Kiro's steering files encode the team's tacit conventions and architecture decisions as machine-readable markdown the agent must follow, the exact spec this anti-pattern says is missing.","evidence":[{"type":"doc","title":"Steering - IDE - Docs - Kiro","url":"https://kiro.dev/docs/steering/","quote":"Steering gives Kiro persistent knowledge about your workspace through markdown files. Instead of explaining your conventions in every chat, steering files ensure Kiro consistently follows your established patterns, libraries, and standards.","accessed":"2026-06-14"}],"evidence_status":"full"}],"instantiates":["modern-coding-agent"],"alternatives":[{"composition":"cursor","relation":"competes-with","note":"Peer agentic IDE."},{"composition":"windsurf","relation":"competes-with","note":"Peer agentic IDE."}],"references":[{"type":"doc","title":"Kiro","url":"https://kiro.dev/","accessed":"2026-06-04"},{"type":"doc","title":"Kiro documentation","url":"https://kiro.dev/docs/","accessed":"2026-06-04"}],"last_analyzed":"2026-06-04","verification_status":"partial","last_updated":"2026-06-17","date_added":"2026-06-04","status_in_practice":"emerging","tags":["aws","coding-agent","ide","spec-driven","code-oss","proprietary"]},{"id":"factory-droid","name":"Factory (Droid)","kind":"framework","category":"coding-agent","build_surface":"closed-application","vendor":"Factory.ai","language":"proprietary","license":"proprietary","status":"active","url":"https://factory.ai/","docs_url":"https://docs.factory.ai/","intent":"Agent-native software development driven by specialized terminal 'Droid' sub-agents.","description":"Factory's terminal-based coding agent platform built around specialized sub-agents — Droids — such as a Code Droid for implementation and a Knowledge Droid for research and documentation, with further reliability/product variants. Positioned for enterprise agent-native development.","primary_use_cases":["enterprise agent-native software development from the terminal","role-specialized delegation across implementation, research and reliability","spec/backlog work driven by dedicated droids"],"agent_loop_shape":"A terminal agent orchestrates role-specialized Droid sub-agents; each Droid runs its own tool-using loop for implementation, research or reliability work, coordinated toward the overall task.","key_concepts":[{"name":"Droids","summary":"Role-specialized sub-agents (Code, Knowledge, ...).","maps_to_pattern":"role-assignment","url":"https://docs.factory.ai/"}],"pattern_composition":"flowchart TD\n  TASK[Task] --> ORCH[Terminal agent / orchestrator]\n  ORCH --> CODE[Code Droid<br/>implementation]\n  ORCH --> KNOW[Knowledge Droid<br/>research / docs]\n  ORCH --> REL[Reliability Droid]\n  CODE --> MERGE[Coordinate results]\n  KNOW --> MERGE\n  REL --> MERGE\n  MERGE --> OUT[Result]","members":[{"pattern":"role-assignment","role":"core","note":"Create specialized subagents with their own prompts, tools, and models.","evidence":[{"type":"doc","title":"Factory docs — Custom Droids","url":"https://docs.factory.ai/","quote":"Create specialized subagents with their own prompts, tools, and models.","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"subagent-isolation","role":"first-class","note":"Create specialized subagents with their own prompts, tools, and models.","evidence":[{"type":"doc","title":"Factory docs — Custom Droids","url":"https://docs.factory.ai/","quote":"Create specialized subagents with their own prompts, tools, and models.","accessed":"2026-06-04"}],"evidence_status":"limited"}],"instantiates":["modern-coding-agent","multi-agent-coordination"],"alternatives":[{"composition":"devin","relation":"competes-with","note":"Peer autonomous software engineering agent."},{"composition":"claude-code","relation":"competes-with","note":"Peer terminal coding agent."}],"references":[{"type":"doc","title":"Factory","url":"https://factory.ai/","accessed":"2026-06-04"},{"type":"doc","title":"Factory documentation","url":"https://docs.factory.ai/","accessed":"2026-06-04"}],"last_analyzed":"2026-06-04","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-06-04","status_in_practice":"emerging","tags":["factory","coding-agent","cli","multi-agent","enterprise","proprietary"]},{"id":"warp","name":"Warp","kind":"framework","category":"coding-agent","build_surface":"closed-application","vendor":"Warp","language":"Rust","license":"proprietary","status":"active","url":"https://www.warp.dev/","docs_url":"https://docs.warp.dev/","intent":"Agentic terminal that understands tasks, runs commands, edits files and orchestrates multi-step workflows.","description":"Modern terminal with a built-in AI agent mode that plans and executes multi-step developer workflows directly in the shell — running commands, editing files and coordinating steps toward a stated task.","primary_use_cases":["running multi-step dev workflows from an agentic terminal","natural-language command execution and file edits in the shell"],"agent_loop_shape":"Agent mode interprets a natural-language task and runs a plan→command→observe loop in the terminal, editing files and chaining steps until done.","key_concepts":[{"name":"Agent mode","summary":"Built-in terminal agent that plans and runs multi-step workflows.","url":"https://docs.warp.dev/"}],"pattern_composition":"flowchart TD\n  USER[Natural-language task] --> AGENT[Agent mode]\n  AGENT --> PLAN[Plan steps]\n  PLAN --> CMD[Run command / edit file]\n  CMD --> OBS[Observe output]\n  OBS --> AGENT\n  AGENT -->|done| OUT[Result]","members":[{"pattern":"multi-model-routing","role":"first-class","note":"Run coding agents locally and in the cloud, across any model, any harness, repo, or tool.","evidence":[{"type":"doc","title":"Warp","url":"https://www.warp.dev/","quote":"Run coding agents locally and in the cloud, across any model, any harness, repo, or tool.","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"orchestrator-workers","role":"first-class","note":"Delegate to a state of the art agent harness with multi-agent orchestration, access to all the best models, codebase indexing, and granular permission contro...","evidence":[{"type":"doc","title":"Warp","url":"https://www.warp.dev/","quote":"Delegate to a state of the art agent harness with multi-agent orchestration, access to all the best models, codebase indexing, and granular permission controls.","accessed":"2026-06-04"}],"evidence_status":"limited"}],"instantiates":["modern-coding-agent"],"alternatives":[{"composition":"claude-code","relation":"competes-with","note":"Peer terminal coding agent."}],"references":[{"type":"doc","title":"Warp","url":"https://www.warp.dev/","accessed":"2026-06-04"},{"type":"doc","title":"Warp documentation","url":"https://docs.warp.dev/","accessed":"2026-06-04"}],"last_analyzed":"2026-06-04","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-06-04","status_in_practice":"emerging","tags":["warp","terminal","coding-agent","proprietary"]},{"id":"google-jules","name":"Google Jules","kind":"framework","category":"coding-agent","build_surface":"closed-application","vendor":"Google","language":"proprietary (cloud service)","license":"proprietary","status":"active","url":"https://jules.google/","docs_url":"https://jules.google/docs","intent":"Asynchronous cloud coding agent that fixes bugs, adds docs and builds features autonomously.","description":"Google's experimental asynchronous coding agent. It clones a repository into a cloud VM, installs dependencies, generates a plan for human approval, then autonomously modifies files while the developer works on other things. GitHub-integrated with browser notifications when tasks finish.","primary_use_cases":["asynchronous, autonomous bug fixes and feature work in the cloud","plan-then-approve coding tasks against a GitHub repo","offloading background coding while working elsewhere"],"agent_loop_shape":"Async: clone repo into a cloud VM → install deps → generate a plan → human approves → autonomously edit files → notify on completion. The approval gate sits between planning and execution.","key_concepts":[{"name":"Cloud VM session","summary":"Clones the repo and works in an isolated cloud VM.","maps_to_pattern":"sandbox-isolation","url":"https://jules.google/docs"},{"name":"Plan approval","summary":"Generates a plan the developer must approve before changes.","maps_to_pattern":"human-in-the-loop"}],"pattern_composition":"flowchart TD\n  REPO[GitHub repo] --> CLONE[Clone into cloud VM]\n  CLONE --> DEPS[Install dependencies]\n  DEPS --> PLAN[Generate plan]\n  PLAN --> APPROVE{Human approves?}\n  APPROVE -->|no| PLAN\n  APPROVE -->|yes| EXEC[Autonomously edit files]\n  EXEC --> NOTIFY[Notify on completion]\n  NOTIFY --> OUT[PR / changes]","members":[{"pattern":"plan-and-execute","role":"first-class","note":"Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.","evidence":[{"type":"doc","title":"Jules docs","url":"https://jules.google/docs","quote":"Once you submit a task, Jules will generate a plan. You can review and approve it before any code changes are made.","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"human-in-the-loop","role":"first-class","note":"You can review and approve it before any code changes are made.","evidence":[{"type":"doc","title":"Jules docs","url":"https://jules.google/docs","quote":"You can review and approve it before any code changes are made.","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"sandbox-isolation","role":"first-class","note":"Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.","evidence":[{"type":"doc","title":"Jules docs","url":"https://jules.google/docs","quote":"Jules runs in a virtual machine where it clones your code, installs dependencies, and modifies files.","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"local-to-cloud-handoff","role":"core","note":"Jules runs the approved plan autonomously in a cloud VM that clones the repo and notifies on completion, so the developer can disconnect while it finishes the task.","evidence":[{"type":"doc","title":"Jules docs","url":"https://jules.google/docs","accessed":"2026-06-04"}],"evidence_status":"full"}],"instantiates":["long-running-autonomous-agent","modern-coding-agent"],"alternatives":[{"composition":"devin","relation":"competes-with","note":"Peer autonomous async software engineering agent."},{"composition":"github-copilot-coding-agent","relation":"competes-with","note":"Peer async cloud coding agent tied to a repo host."}],"references":[{"type":"doc","title":"Jules docs","url":"https://jules.google/docs","accessed":"2026-06-04"}],"last_analyzed":"2026-06-04","verification_status":"partial","last_updated":"2026-06-17","date_added":"2026-06-04","status_in_practice":"emerging","tags":["google","coding-agent","async","cloud","github","proprietary"]},{"id":"botpress","name":"Botpress","kind":"framework","category":"conversational-bot","build_surface":"low-code","vendor":"Botpress","language":"TypeScript","license":"MIT (legacy v12 OSS); proprietary cloud platform","status":"active","url":"https://botpress.com","repo":"https://github.com/botpress/botpress","docs_url":"https://botpress.com/docs","intent":"Visual + code AI agent platform (Studio drag-and-drop, ADK TypeScript library, Desk for human handoff, Webchat, Hub integrations) for building LLM-powered chatbots with autonomous nodes, knowledge bases, and event triggers.","description":"Botpress is a complete AI agent platform combining a visual Studio (drag-and-drop), an ADK (TypeScript library for code-first agents), Desk (customer support workspace for human-AI teams), Webchat, and a Hub of integrations. Agent behaviour is built from Workflows containing Nodes, including Autonomous Nodes that delegate decisions to an LLM with tools. Eight built-in Agents (Summary, Personality, Policy, Translator, Knowledge, Vision, Analytics, HITL) run in a fixed sequence each turn. The legacy v12 OSS engine is MIT-licensed; the current cloud Studio/Desk product is proprietary.","primary_use_cases":["customer-support chatbots with human handoff to live agents via Desk or HITL plugin","visual no-code agent design with LLM-backed Autonomous Nodes","code-first agents built with the Botpress ADK (TypeScript)","multi-channel conversational agents across Webchat, Slack, WhatsApp, Teams, and other Hub integrations","knowledge-base-grounded Q&A agents over uploaded documents and websites"],"agent_loop_shape":"Per-turn execution runs eight built-in Agents in a fixed order (Summary, Personality, Policy, Translator, Knowledge, Vision, Router, Analytics, HITL). The bot's logic itself is a graph of Workflows of Nodes; a standard Node executes its Cards top to bottom, while an Autonomous Node hands control to an LLM that decides which tools to call and which transition to take. Triggers (e.g. Conversation Started, Custom Triggers from Webchat or integrations) enter Workflows at well-defined entry points.","key_concepts":[{"name":"Autonomous Node","summary":"LLM-driven node that picks tools, writes replies, and chooses transitions from context.","maps_to_pattern":"tool-use","url":"https://botpress.com/docs/studio/concepts/nodes/autonomous-node"},{"name":"Agents (Summary/Personality/Policy/Translator/Knowledge/Vision/Analytics/HITL)","summary":"Eight built-in Agents that extend a bot's capabilities; they execute in a fixed sequence each turn.","url":"https://botpress.com/docs"},{"name":"Knowledge Base","summary":"Documents, websites, and files the bot can query for grounded answers.","maps_to_pattern":"agentic-rag","url":"https://botpress.com/docs/adk/data/knowledge"},{"name":"Triggers","summary":"Nodes that only execute when a named event fires (conversation start, custom Webchat events, integration events).","maps_to_pattern":"event-driven-agent","url":"https://botpress.com/docs/studio/concepts/triggers"},{"name":"Desk / HITL plugin","summary":"Escalates conversations from the bot to a human agent in Botpress Desk or a third-party platform.","maps_to_pattern":"conversation-handoff","url":"https://botpress.com/docs/adk/advanced/desk-hitl"},{"name":"Conversation variables","summary":"Per-conversation scoped state cleared when the conversation ends.","maps_to_pattern":"session-isolation","url":"https://botpress.com/docs/studio/concepts/variables/scopes/conversation"}],"pattern_composition":"flowchart TD\n  user[User message] --> trigger{Trigger?}\n  trigger -->|Conversation Started / Custom| entry[Workflow entry]\n  entry --> agents[Built-in Agents pipeline]\n  agents --> summary[Summary] --> personality[Personality] --> policy[Policy] --> translator[Translator]\n  translator --> knowledge[Knowledge KB lookup]\n  knowledge --> node{Node type}\n  node -->|Standard| cards[Execute Cards sequentially]\n  node -->|Autonomous| llm[LLM picks tools and transitions]\n  llm --> tools[Tools / integrations]\n  tools --> respond[Send messages]\n  cards --> respond\n  respond --> hitl{Escalate?}\n  hitl -->|yes| desk[Desk / HITL plugin to human agent]\n  hitl -->|no| done[End turn]\n  desk --> done","members":[{"pattern":"agent-skills","role":"limited","note":"Built-in Agents extend a bot's capabilities, but they are a fixed catalogue (Summary, Personality, Policy, Translator, Knowledge, Vision, Analytics, HITL) running in a fixed order, not user-authored skills.","evidence":[{"type":"doc","title":"Botpress agents introduction","url":"https://botpress.com/docs/studio/concepts/agents/introduction","quote":"Agents extend your bot's capabilities beyond its basic functions.","accessed":"2026-05-20"},{"type":"doc","title":"Botpress agents introduction - specialized components","url":"https://botpress.com/docs/studio/concepts/agents/introduction","quote":"Specialized components that extend the capabilities of a bot.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"agentic-rag","role":"first-class","note":"Knowledge Bases are a documented first-class resource for grounding bot answers in documents/websites/files; the Knowledge Agent retrieves from them.","evidence":[{"type":"doc","title":"Botpress docs index — Knowledge bases","url":"https://botpress.com/docs/adk/data/knowledge","quote":"Give your agent access to documents, websites, and files.","accessed":"2026-05-20"},{"type":"doc","title":"Botpress llms.txt — knowledge bases","url":"https://botpress.com/docs/llms.txt","quote":"Knowledge Bases: Resource system enabling bots to access documents, websites, and files for information retrieval.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"conversation-handoff","role":"first-class","note":"Desk + HITL plugin escalate to human agents; Start HITL Card placed in a Workflow triggers the handoff and the bot resumes if the user cancels or queue times out.","evidence":[{"type":"doc","title":"Botpress llms.txt — Desk HITL","url":"https://botpress.com/docs/adk/advanced/desk-hitl","quote":"Escalate conversations to support agents in Botpress Desk.","accessed":"2026-05-20"},{"type":"doc","title":"Botpress HITL plugin","url":"https://botpress.com/docs/integrations/integration-guides/hitl/introduction","quote":"The Human-in-the-Loop (HITL) integration allows members of your Workspace to ... Add the Start HITL Card at the point where you want to hand off conversations to a live agent","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"cross-session-memory","role":"limited","note":"Conversation variables persist within one conversation and are cleared at end; ADK 'Manage state' exposes user-scoped and bot-scoped state, but the documented default scope is the single conversation.","evidence":[{"type":"doc","title":"Botpress conversation variables","url":"https://botpress.com/docs/studio/concepts/variables/scopes/conversation","quote":"Information stored in conversation variables is deleted when the current conversation is over.","accessed":"2026-05-20"},{"type":"doc","title":"Botpress ADK — manage state","url":"https://botpress.com/docs/adk/conversations/state","quote":"Store and reuse data across conversations, users, and the entire bot.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"event-driven-agent","role":"first-class","note":"Triggers are first-class nodes that only execute when a named event (Conversation Started, Custom Trigger from Webchat, integration event) fires.","evidence":[{"type":"doc","title":"Botpress triggers","url":"https://botpress.com/docs/studio/concepts/triggers","quote":"Triggers are a special kind of Node that only execute when a certain event occurs.","accessed":"2026-05-20"},{"type":"doc","title":"Botpress triggers — custom","url":"https://botpress.com/docs/studio/concepts/triggers","quote":"You can use Custom Triggers to make Webchat react to custom events from your website.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"multilingual-voice-agent","role":"limited","note":"Botpress ships a Translator Agent in its 8-agent pipeline; voice-specific multilingual handling is not a documented headline feature.","evidence":[{"type":"doc","title":"Botpress llms.txt — translator agent","url":"https://botpress.com/docs/studio/concepts/agents/translator-agent","quote":"Enables your bot to communicate in multiple languages","accessed":"2026-05-20"},{"type":"doc","title":"Botpress agents introduction — Translator role","url":"https://botpress.com/docs/studio/concepts/agents/introduction","quote":"Translator Agent: Enables multi-language communication","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"session-isolation","role":"first-class","note":"Conversation-scoped variables auto-isolate per conversation and are deleted at end; bot/user/conversation scopes are explicit.","evidence":[{"type":"doc","title":"Botpress conversation variables — scope","url":"https://botpress.com/docs/studio/concepts/variables/scopes/conversation","quote":"Conversation variables let you store and reuse information within the scope of a single conversation.","accessed":"2026-05-20"},{"type":"doc","title":"Botpress conversation variables — cleanup","url":"https://botpress.com/docs/studio/concepts/variables/scopes/conversation","quote":"Information stored in conversation variables is deleted when the current conversation is over.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Autonomous Node calls user-defined tools; ADK 'Define tools' API exposes them to the LLM.","evidence":[{"type":"doc","title":"Botpress Autonomous Node — tools","url":"https://botpress.com/docs/studio/concepts/nodes/autonomous-node","quote":"Unlike a standard Node, which executes its Cards one by one, the Autonomous Node uses a Large Language Model to decide when to execute tools.","accessed":"2026-05-20"},{"type":"doc","title":"Botpress ADK — define tools","url":"https://botpress.com/docs/adk/conversations/tools","quote":"Give the AI model functions it can call during execution.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"visual-workflow-graph","role":"first-class","note":"Studio is a visual, drag-and-drop interface for building agents; Workflows are graphs of Nodes connected by transitions.","evidence":[{"type":"doc","title":"Botpress Studio description","url":"https://botpress.com/docs","quote":"Visual, drag-and-drop interface for building AI agents","accessed":"2026-05-20"},{"type":"doc","title":"Botpress Autonomous Node — transitions","url":"https://botpress.com/docs/studio/concepts/nodes/autonomous-node","quote":"Transitions to a sub-Workflow ... Transitions to a Node that's connected to the Autonomous Node.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"scheduled-agent","role":"limited","note":"Honest 'none' for scheduled-agent — the docs index page (llms.txt) describes Workflows as 'long-running background processes' in its index entry, but the actual Workflows concept page (fetched 2026-05-24) describes Workflows only as sequences of steps with no cron / schedule / periodic execution primitive surfaced.","evidence":[{"type":"doc","title":"Botpress llms.txt — workflows index entry","url":"https://botpress.com/docs/llms.txt","quote":"Documentation for building long-running background processes that operate independently of conversations.","accessed":"2026-05-20"}],"evidence_status":"limited"},{"pattern":"approval-queue","role":"first-class","note":"HITL integration explicitly supports approval of AI-generated outputs before delivery; the Start HITL Card is placed in a Workflow to hand off conversations, and the HITL Agent's Queue Timeout governs wait behaviour.","evidence":[{"type":"doc","title":"Botpress HITL integration — approve AI outputs","url":"https://botpress.com/docs/integrations/integration-guides/hitl/introduction","quote":"Approve AI-generated outputs before they're sent","accessed":"2026-05-24"},{"type":"doc","title":"Botpress HITL Agent — Queue Timeout","url":"https://botpress.com/docs/studio/concepts/agents/hitl-agent","quote":"This is the maximum wait time the bot will wait for an agent to handle the user's query.","accessed":"2026-05-20"},{"type":"doc","title":"Botpress HITL integration — Start HITL Card","url":"https://botpress.com/docs/integrations/integration-guides/hitl/introduction","quote":"Add the **Start HITL** Card at the point where you want to hand off conversations to a live agent.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"code-execution","role":"first-class","note":"Tools in the ADK are functions the AI model can call during execute(); the model decides which tools to call based on the conversation, and TypeScript handler functions hold the custom logic. Workflows can also be converted to tools via .asTool().","evidence":[{"type":"doc","title":"Botpress ADK tools — model-callable functions","url":"https://botpress.com/docs/adk/conversations/tools","quote":"Tools are functions the AI model can call during `execute()`.","accessed":"2026-05-24"},{"type":"doc","title":"Botpress ADK tools — model decides which to call","url":"https://botpress.com/docs/adk/conversations/tools","quote":"The model decides which tools to call based on the conversation.","accessed":"2026-05-24"},{"type":"repo","title":"Botpress v12 README — TypeScript SDK","url":"https://github.com/botpress/botpress","quote":"Developer Tools: CLI, SDK (@botpress/sdk), and API client (@botpress/client)","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"channel-decoupled-agent-core","role":"supported","evidence_status":"none","note":"Botpress runs one bot logic across web, messaging, and voice channels via channel integrations; needs a primary channels-doc URL to confirm and upgrade.","evidence":[]}],"references":[{"type":"doc","title":"Botpress homepage","url":"https://botpress.com","quote":"The complete AI agent platform","accessed":"2026-05-20"},{"type":"doc","title":"Botpress docs index (llms.txt)","url":"https://botpress.com/docs/llms.txt","accessed":"2026-05-20"},{"type":"repo","title":"botpress/botpress repo","url":"https://github.com/botpress/botpress","quote":"the ultimate platform for building next-generation chatbots and assistants powered by OpenAI","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"partial","last_updated":"2026-06-14","date_added":"2026-04-30","status_in_practice":"mature","tags":["chatbot","low-code","visual-builder","knowledge-base","hitl","typescript"]},{"id":"ms-bot-framework","name":"Microsoft Bot Framework","kind":"framework","category":"conversational-bot","build_surface":"full-code","vendor":"Microsoft","language":"C#, JavaScript, TypeScript, Python, Java","license":"MIT","status":"deprecated","first_released":"2016-03-30","url":"https://github.com/microsoft/botframework-sdk","repo":"https://github.com/microsoft/botframework-sdk","docs_url":"https://learn.microsoft.com/en-us/azure/bot-service/","intent":"Microsoft's pre-LLM SDK for building turn-based conversational bots in C#/JS/Python/Java with dialog stacks, channel adapters, scoped state, and a documented handoff-to-human protocol; archived on GitHub with support ending December 2025.","description":"Microsoft Bot Framework SDK v4 is a modular SDK and set of services (Azure Bot Service, Bot Connector) for building bots that converse across channels (Teams, Slack, Telegram, Facebook, web chat, SMS via Twilio, and others). Bots are activity-driven (each user message is a turn), use the Dialog library (component, waterfall, prompt, adaptive dialogs) to manage multi-turn flows, persist state in three scopes (user / conversation / private-conversation) over Memory / Azure Blob / Cosmos storage, and support a standardised handoff protocol with `handoff.initiate` and `handoff.status` events. The SDK and Bot Framework Emulator have been archived; Microsoft recommends migrating to the Microsoft 365 Agents SDK.","primary_use_cases":["multi-channel enterprise chatbots (Teams, Slack, Facebook Messenger, web chat, SMS)","dialog-stack-driven multi-turn conversation flows with waterfall and prompts","QnA Maker / Custom Question Answering knowledge bots","bot-as-proxy or bot-as-agent handoff to human-agent platforms"],"agent_loop_shape":"Activity-driven turn loop: the Bot Connector relays an activity from a channel to the bot, the bot's turn handler reads/writes state through state property accessors, the active Dialog on the dialog stack continues from where it left off (multi-turn persistence via dialog state), the bot replies, and state is saved. There is no LLM agent loop in the SDK itself — the bot is imperative; LUIS/QnA Maker provide NLU.","key_concepts":[{"name":"Activity / Turn","summary":"A bot processes activities one turn at a time; conversation state must be re-read each turn.","url":"https://learn.microsoft.com/en-us/azure/bot-service/bot-service-overview"},{"name":"Dialogs","summary":"Container, component, waterfall, prompt, adaptive, skill, and QnA Maker dialogs manage multi-turn flows via a dialog stack.","maps_to_pattern":"plan-and-execute","url":"https://learn.microsoft.com/en-us/azure/bot-service/bot-builder-concept-dialog"},{"name":"State buckets (user / conversation / private-conversation)","summary":"Three scoped state buckets persisted via state property accessors over Memory / Blob / Cosmos.","maps_to_pattern":"session-isolation","url":"https://learn.microsoft.com/en-us/azure/bot-service/bot-builder-concept-state"},{"name":"Channels / Bot Connector","summary":"Connect a single bot to Teams, Slack, Facebook, Telegram, SMS, web chat with a normalised activity stream.","url":"https://learn.microsoft.com/en-us/azure/bot-service/bot-service-overview"},{"name":"Handoff protocol","summary":"Standard `handoff.initiate` and `handoff.status` events for bot-to-human escalation; bot-as-agent and bot-as-proxy integration models.","maps_to_pattern":"conversation-handoff","url":"https://learn.microsoft.com/en-us/azure/bot-service/bot-service-design-pattern-handoff-human"}],"pattern_composition":"flowchart TD\n  user[User on channel] --> conn[Bot Connector]\n  conn --> turn[Turn handler]\n  turn --> state[Read state buckets: user/conversation/private]\n  state --> stack[Active Dialog from dialog stack]\n  stack --> dlg{Dialog type}\n  dlg -->|Waterfall| step[Sequential step]\n  dlg -->|Prompt| ask[Ask user]\n  dlg -->|QnA Maker| kb[Knowledge base]\n  dlg -->|Skill| skill[Delegate to skill bot]\n  step --> reply[Send activity]\n  ask --> reply\n  kb --> reply\n  skill --> reply\n  reply --> handoff{Escalate?}\n  handoff -->|yes| ev[handoff.initiate event]\n  ev --> hub[Agent hub]\n  hub --> stat[handoff.status: accepted / failed / completed]\n  handoff -->|no| save[Save state]\n  stat --> save","members":[{"pattern":"conversation-handoff","role":"first-class","note":"Standardised handoff protocol with `handoff.initiate` and `handoff.status` events, plus two named integration models (bot-as-agent and bot-as-proxy) and a Handoff Library sample.","evidence":[{"type":"doc","title":"Bot Framework — handoff initiation event","url":"https://learn.microsoft.com/en-us/azure/bot-service/bot-service-design-pattern-handoff-human","quote":"When a bot detects the need to hand the conversation off to an agent, it signals its intent by sending a handoff initiation event.","accessed":"2026-05-20"},{"type":"doc","title":"Bot Framework — handoff status event","url":"https://learn.microsoft.com/en-us/azure/bot-service/bot-service-design-pattern-handoff-human","quote":"A handoff status event is sent to the bot by the agent hub. The event informs the bot about the status of the initiated handoff operation.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"session-isolation","role":"first-class","note":"Three predefined scoped state buckets (user / conversation / private-conversation), each keyed by channel-id plus user/conversation id.","evidence":[{"type":"doc","title":"Bot Framework — state buckets","url":"https://learn.microsoft.com/en-us/azure/bot-service/bot-builder-concept-state","quote":"The SDK includes three of these 'buckets': User state, Conversation state, Private conversation state","accessed":"2026-05-20"},{"type":"doc","title":"Bot Framework — keying","url":"https://learn.microsoft.com/en-us/azure/bot-service/bot-builder-concept-state","quote":"Both user and conversation state are scoped by channel. The same person using different channels to access your bot appears as different users","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"fallback-chain","role":"first-class","note":"Bot Framework Orchestrator and orchestration workflow are documented intent-only recognition engines used to determine which LUIS model or QnA Maker knowledge base can best handle a given message; QnA Maker dialog follow-up prompts add multi-turn fallback within a knowledge base.","evidence":[{"type":"doc","title":"Bot Framework — Orchestrator routes between services","url":"https://learn.microsoft.com/en-us/azure/bot-service/bot-builder-concept-luis","quote":"Bot Framework Orchestrator is an intent-only recognition engine. The Bot Framework CLI includes tools to generate a language model for Orchestrator from a collection of QnA Maker knowledge bases and LUIS language models. Your bot can then use Orchestrator to determine which service can best respond to the user's input.","accessed":"2026-05-24"},{"type":"doc","title":"Bot Framework — Orchestrator tutorial","url":"https://learn.microsoft.com/en-us/azure/bot-service/bot-builder-tutorial-orchestrator","quote":"If a bot uses multiple Language Understanding (LUIS) models and QnA Maker knowledge bases, you can use Bot Framework Orchestrator to determine which LUIS model or QnA Maker knowledge base best matches the user input.","accessed":"2026-05-24"},{"type":"doc","title":"Bot Framework — QnA Maker follow-up prompts","url":"https://learn.microsoft.com/en-us/azure/bot-service/bot-builder-concept-dialog","quote":"Follow-up prompts, also known as multi-turn prompts, allow a knowledge base to ask the user for more information before answering their question.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"limited","note":"Bots can call APIs and external services from turn-handler code, but the SDK predates LLM tool-calling — no native tool-call primitive. Custom logic patterns suggest calling all relevant services per message or in preferred order.","evidence":[{"type":"doc","title":"Bot Framework overview — bots as web apps","url":"https://learn.microsoft.com/en-us/azure/bot-service/bot-service-overview","quote":"Bots can do the same things other types of software can do—read from and write to files, use databases and APIs","accessed":"2026-05-20"},{"type":"doc","title":"Bot Framework — custom call multiple services","url":"https://learn.microsoft.com/en-us/azure/bot-service/bot-builder-concept-luis","quote":"For each message, call all relevant services that your bot supports. Use the results from the service that has the best confidence score.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"plan-and-execute","role":"first-class","note":"Dialog stack is the SDK's explicit multi-turn plan; component dialogs encapsulate sub-plans, waterfall dialogs sequence steps, prompt dialogs gather inputs.","evidence":[{"type":"doc","title":"Bot Framework — dialog stack","url":"https://learn.microsoft.com/en-us/azure/bot-service/bot-builder-concept-dialog","quote":"A dialog context contains information about all active dialogs and includes a dialog stack, which acts as a call stack for all the active dialogs.","accessed":"2026-05-20"},{"type":"doc","title":"Bot Framework — waterfall dialog","url":"https://learn.microsoft.com/en-us/azure/bot-service/bot-builder-concept-dialog","quote":"waterfall dialog | Defines a sequence of steps, allowing your bot to guide a user through a linear process.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"cross-session-memory","role":"first-class","note":"User state is keyed by channel-id+from-id and persists across conversations; can be backed by Azure Blob or Cosmos partitioned storage.","evidence":[{"type":"doc","title":"Bot Framework — user state","url":"https://learn.microsoft.com/en-us/azure/bot-service/bot-builder-concept-state","quote":"User state is available in any turn that the bot is conversing with that user on that channel, regardless of the conversation","accessed":"2026-05-20"},{"type":"doc","title":"Bot Framework — storage options","url":"https://learn.microsoft.com/en-us/azure/bot-service/bot-builder-concept-state","quote":"Azure Blob Storage connects to an Azure Blob Storage object database. Azure Cosmos DB partitioned storage connects to a partitioned Cosmos DB NoSQL database.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"channel-decoupled-agent-core","role":"first-class","evidence_status":"full","note":"The channel model is Bot Framework's core abstraction: one bot built against the activity schema is connected to many channels (Teams, Web Chat, telephony, email, Slack) by the Azure Bot Service without code changes.","evidence":[{"type":"doc","title":"Configure a bot to run on one or more channels — Azure Bot Service","url":"https://learn.microsoft.com/en-us/azure/bot-service/bot-service-manage-channels","quote":"The Bot Framework allows you to develop a bot in a channel-agnostic way by normalizing messages that the bot sends to a channel.","accessed":"2026-06-19"}]}],"anti_patterns_avoided":[],"references":[{"type":"doc","title":"What is the Bot Framework SDK?","url":"https://learn.microsoft.com/en-us/azure/bot-service/bot-service-overview","quote":"Microsoft Bot Framework and Azure AI Bot Service are a collection of libraries, tools, and services that let you build, test, deploy, and manage intelligent bots.","accessed":"2026-05-20"},{"type":"repo","title":"microsoft/botframework-sdk","url":"https://github.com/microsoft/botframework-sdk","quote":"Bot Framework provides the most comprehensive experience for building conversation applications.","accessed":"2026-05-20"},{"type":"doc","title":"Bot Framework SDK retirement notice","url":"https://learn.microsoft.com/en-us/azure/bot-service/bot-service-overview","quote":"The Bot Framework SDK and Bot Framework Emulator have been archived on GitHub. The project is no longer updated or maintained. Support tickets for the Bot Framework SDK will no longer be serviced as of December 31, 2025.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-14","date_added":"2026-04-30","status_in_practice":"deprecated","tags":["microsoft","azure","pre-llm","dialog-stack","channels","archived"],"successor":"ms-365-agents-sdk"},{"id":"rasa","name":"Rasa","kind":"framework","category":"conversational-bot","build_surface":"full-code","vendor":"Rasa Technologies","language":"Python","license":"Apache-2.0","status":"active","first_released":"2016-10-15","url":"https://rasa.com","repo":"https://github.com/RasaHQ/rasa","docs_url":"https://rasa.com/docs/","intent":"Developer platform for enterprise text and voice AI assistants combining LLM-based dialogue understanding (CALM) with deterministic business logic encoded as Flows, custom actions for tool calls, slots for collected data, and built-in patterns for human handoff and fallback.","description":"Rasa is an open-source (legacy Open Source) and commercial (Rasa Platform / Pro) framework for building text and voice conversational assistants. The current dialogue engine is CALM (Conversational AI with Language Models): an LLM interprets each user message and emits internal commands (start flow, set slot, correct slot, cancel flow, clarify flows, chitchat, knowledge answer, human handoff), while a deterministic dialogue manager executes those commands against business logic written as Flows. Flows are step-by-step processes built from collect (slot filling), set_slots, action (custom-action server call), and branching steps; the tracker persists slot values across turns.","primary_use_cases":["enterprise customer-support and CX assistants over text and voice","LLM-driven dialogue understanding with deterministic flows enforcing business logic","custom-action-driven tool use against internal APIs","human handoff via the built-in human_handoff command","knowledge-grounded answers via Enterprise RAG and the knowledge_answer command"],"agent_loop_shape":"Each turn the user message is fed to a CALM dialogue-understanding LLM with a structured prompt that emits a set of internal commands (start flow, set slot, correct slot, cancel flow, clarify flows, chitchat, knowledge answer, human handoff). The dialogue manager interprets those commands against the currently active Flow on the tracker, executing collect / set_slots / action / branching steps. Custom actions run in an external action server; the tracker carries slot state across turns. Conversation patterns provide reusable structured sub-flows for things like clarification, correction, and handoff.","key_concepts":[{"name":"CALM","summary":"Conversational AI with Language Models — the dialogue system pairing an LLM-driven dialogue understanding layer with a deterministic dialogue manager.","url":"https://rasa.com/docs/learn/concepts/calm/"},{"name":"Flows","summary":"Step-by-step descriptions of business processes (collect slots, run actions, branch on data); the LLM can dynamically route between flows.","maps_to_pattern":"plan-and-execute","url":"https://rasa.com/docs/learn/concepts/calm/"},{"name":"Dialogue understanding / commands","summary":"LLM emits internal commands (start flow, set slot, correct slot, cancel flow, clarify flows, chitchat, knowledge answer, human handoff) for the dialogue manager to execute.","maps_to_pattern":"structured-output","url":"https://rasa.com/docs/learn/concepts/dialogue-understanding/"},{"name":"Slots & tracker","summary":"Slots are typed values collected through flow steps; the tracker is per-conversation persistent state.","maps_to_pattern":"cross-session-memory","url":"https://rasa.com/docs/reference/primitives/flows/"},{"name":"Custom actions","summary":"External action server endpoints called from action steps for tool/API integration.","maps_to_pattern":"tool-use"},{"name":"Human handoff","summary":"Built-in command and conversation pattern for escalating to a live agent.","maps_to_pattern":"conversation-handoff","url":"https://rasa.com/docs/learn/concepts/dialogue-understanding/"}],"pattern_composition":"flowchart TD\n  user[User message] --> du[Dialogue understanding LLM]\n  du --> cmds[Commands: start flow / set slot / cancel / chitchat / knowledge answer / human handoff]\n  cmds --> dm[Dialogue manager]\n  dm --> flow{Active Flow step}\n  flow -->|collect| ask[Ask user, set slot]\n  flow -->|set_slots| upd[Update tracker]\n  flow -->|action| act[Custom action server call]\n  flow -->|branch| br[Branch on slot value]\n  ask --> tracker[Tracker slots]\n  upd --> tracker\n  act --> tracker\n  br --> tracker\n  tracker --> respond[Bot response]\n  cmds -->|human_handoff| handoff[Live-agent handoff pattern]\n  cmds -->|knowledge_answer| rag[Enterprise RAG]\n  rag --> respond\n  handoff --> respond","members":[{"pattern":"agentic-rag","role":"first-class","note":"Enterprise RAG is a named Rasa product; the `knowledge_answer` command is a built-in CALM command that routes to grounded answers.","evidence":[{"type":"doc","title":"Rasa Enterprise RAG","url":"https://www.rasa.com/enterprise-rag","quote":"Retrieve information in real time, so every answer is fresh, verifiable, and aligned.","accessed":"2026-05-20"},{"type":"doc","title":"Rasa dialogue understanding — commands","url":"https://rasa.com/docs/learn/concepts/dialogue-understanding/","quote":"start flow, cancel flow, clarify flows, set slot, correct slot, chitchat, knowledge answer, human handoff","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"conversation-handoff","role":"first-class","note":"`human handoff` is a built-in CALM command and `pattern_human_handoff` is a named conversation pattern documented in Rasa's reference primitives.","evidence":[{"type":"doc","title":"Rasa dialogue understanding — human handoff command","url":"https://rasa.com/docs/learn/concepts/dialogue-understanding/","quote":"start flow, cancel flow, clarify flows, set slot, correct slot, chitchat, knowledge answer, human handoff","accessed":"2026-05-20"},{"type":"doc","title":"Rasa patterns — pattern_human_handoff default behaviour","url":"https://rasa.com/docs/reference/primitives/patterns/","quote":"Currently, the default behaviour for a human handoff is to inform the user that the assistant cannot help with the request.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"fallback-chain","role":"first-class","note":"Rasa positions itself explicitly around 'built-in recovery patterns'; clarify_flows, correct_slot, cancel_flow act as the LLM-side fallback chain.","evidence":[{"type":"doc","title":"Rasa platform positioning","url":"https://www.rasa.com/platform","quote":"Rasa extends LLMs with structured flows, deterministic logic, and built-in recovery patterns.","accessed":"2026-05-20"},{"type":"doc","title":"Rasa dialogue understanding — recovery commands","url":"https://rasa.com/docs/learn/concepts/dialogue-understanding/","quote":"These next steps are communicated as a set of 'commands' for the dialogue manager to follow and might include instructions to continue a flow, start a new flow, or activate conversation pattern flows.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"session-isolation","role":"first-class","note":"Slots set inside a flow are reset when the flow ends — documented per-flow isolation; persistence requires opt-in via `persisted_slots`.","evidence":[{"type":"doc","title":"Rasa flows — slot lifecycle","url":"https://rasa.com/docs/reference/primitives/flows/","quote":"By default, slots set in a `collect` and `set_slot` step are reset when the flow ends.","accessed":"2026-05-20"},{"type":"doc","title":"Rasa flows — persisted_slots opt-in","url":"https://rasa.com/docs/reference/primitives/flows/","quote":"To change this behavior, you can add these slots to the `persisted_slots` field.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"structured-output","role":"first-class","note":"CALM's whole architecture is built on the LLM emitting a constrained, typed command set rather than free text — the canonical structured-output use.","evidence":[{"type":"doc","title":"Rasa CALM — LLM constrained to commands","url":"https://rasa.com/docs/learn/concepts/dialogue-understanding/","quote":"CALM assistants use an LLM for dialogue understanding within a controlled framework. A structured prompt ensures reliable and predictable interpretation.","accessed":"2026-05-20"},{"type":"doc","title":"Rasa CALM — separation of concerns","url":"https://rasa.com/docs/learn/concepts/calm/","quote":"In CALM assistants, LLMs keep the conversation fluent but don't guess your business logic.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Action steps in flows trigger custom actions served by an external action server, used for API and tool calls.","evidence":[{"type":"doc","title":"Rasa flows — action steps","url":"https://rasa.com/docs/reference/primitives/flows/","quote":"It describes the information you need from the user, data you need to retrieve from an API or a database, and branching logic based on the information collected.","accessed":"2026-05-20"},{"type":"doc","title":"Rasa flows — step definition","url":"https://rasa.com/docs/reference/primitives/flows/","quote":"Each flow describes the logical steps your AI assistant uses to complete a task.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"plan-and-execute","role":"first-class","note":"Flows are predefined plans; the LLM picks which flow to start and the dialogue manager executes it deterministically.","evidence":[{"type":"doc","title":"Rasa CALM — flows are business plans","url":"https://rasa.com/docs/learn/concepts/calm/","quote":"Business logic, step-by-step outlines of how to solve a problem, is defined in Flows. The LLM can dynamically route from flow to flow.","accessed":"2026-05-20"},{"type":"doc","title":"Rasa CALM — structured workflows","url":"https://rasa.com/docs/learn/concepts/calm/","quote":"Follows structured workflows for reliable, debuggable interactions.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"channel-decoupled-agent-core","role":"first-class","evidence_status":"limited","note":"Rasa connects one assistant to a surface through channel connectors; the web chat-widget connector is one such input/output channel, alongside REST, Slack, Telegram, and voice.","evidence":[{"type":"doc","title":"Rasa channel connectors — your own website","url":"https://rasa.com/docs/rasa/connectors/your-own-website/","quote":"If you already have an existing website and want to add a Rasa assistant to it, you can use the Rasa Chat Widget","accessed":"2026-06-19"}]}],"references":[{"type":"repo","title":"RasaHQ/rasa","url":"https://github.com/RasaHQ/rasa","quote":"An open source machine learning framework for automating text and voice-based conversations.","accessed":"2026-05-20"},{"type":"doc","title":"Rasa platform overview","url":"https://www.rasa.com/platform","quote":"Rasa extends LLMs with structured flows, deterministic logic, and built-in recovery patterns.","accessed":"2026-05-20"},{"type":"doc","title":"Rasa CALM","url":"https://rasa.com/docs/learn/concepts/calm/","quote":"CALM (Conversational AI with Language Models) is the dialogue system that runs Rasa text and voice assistants.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-14","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","python","calm","flows","enterprise"]},{"id":"voiceflow","name":"Voiceflow","kind":"framework","category":"conversational-bot","build_surface":"no-code","vendor":"Voiceflow","language":"N/A","license":"proprietary","status":"active","url":"https://www.voiceflow.com","repo":"https://github.com/voiceflow","docs_url":"https://docs.voiceflow.com","intent":"SaaS visual platform for designing AI customer-experience agents that mix agentic playbooks (LLM-driven goal-based reasoning with tools) and deterministic workflows (visual step graphs), deployed across chat, voice, and call-centre channels.","description":"Voiceflow is a hosted platform for building AI agents for support, lead generation, and other customer experiences. Builders mix two primitives: Playbooks (goal-based LLM agents with tools and instructions that decide how to reach an outcome) and Workflows (visual, deterministic step graphs with branches, conditions, and integrations). Agents have a global prompt, instructions, a knowledge base for RAG, and global tools available everywhere. The Crew step coordinates multiple playbooks with direct handoffs between them. Tools include API tools, MCP tools, Function tools (custom JS), system tools (knowledge base, call forwarding, web search, UI artefacts), and pre-built integrations. Deployment surfaces include chat and voice widgets and call-centre IVR.","primary_use_cases":["AI customer support agents with playbook + workflow design","lead generation and qualification conversational agents","voice / IVR agents for call-centre automation","knowledge-base RAG agents grounded in uploaded documents and Zendesk/Shopify connectors","multi-playbook crews coordinating sales, billing, and support sub-agents"],"agent_loop_shape":"Hybrid agentic/deterministic loop. The agent has a global prompt + instructions and may be in an agentic context (top-level agent, playbook, or crew step — LLM reasons freely with global tools) or a scripted context (workflow step — deterministic, global tools disabled). Playbooks are goal-based: the LLM decides which tool to call (API tool, MCP tool, function tool, integration, system tool) toward the configured outcome. The Crew step nests multiple playbooks that hand control between each other based on each playbook's LLM description. Workflows can embed playbooks via the Playbook step for agentic-inside-scripted hybrids.","key_concepts":[{"name":"Playbooks","summary":"Goal-based agentic primitives — LLM decides how to reach the configured outcome with tools.","maps_to_pattern":"tool-use","url":"https://docs.voiceflow.com/documentation/build/playbooks"},{"name":"Workflows","summary":"Visual, deterministic multi-step builder with branches, conditions, integrations.","maps_to_pattern":"visual-workflow-graph","url":"https://docs.voiceflow.com/documentation/build/overview"},{"name":"Crew step","summary":"Coordinates multiple playbooks that hand off tasks to each other within one workflow step.","maps_to_pattern":"handoff","url":"https://docs.voiceflow.com/documentation/build/steps/crew"},{"name":"Knowledge base","summary":"Upload URLs/sitemaps/documents or connect Zendesk/Shopify/Kustomer/Salesforce for RAG grounding.","maps_to_pattern":"agentic-rag","url":"https://docs.voiceflow.com/documentation/build/Importing-data-sources"},{"name":"Tools (API / MCP / Function / Integration / System / Global)","summary":"Six tool kinds the agent can call; Global tools are agent-level and available in any agentic context.","maps_to_pattern":"tool-use","url":"https://docs.voiceflow.com/documentation/build/tools/global-tools"},{"name":"Global prompt / instructions","summary":"Per-turn agent prompt (personality/goal/tone/guardrails) plus routing instructions for skill selection.","url":"https://docs.voiceflow.com/documentation/build/overview"}],"pattern_composition":"flowchart TD\n  user[User on channel] --> gp[Global prompt + instructions]\n  gp --> route{Agent decides}\n  route -->|agentic| pb[Playbook]\n  route -->|scripted| wf[Workflow]\n  pb --> tools[Tools: API / MCP / Function / Integration / System / Global]\n  wf --> step{Step type}\n  step -->|Playbook step| pb\n  step -->|Crew step| crew[Crew of playbooks]\n  step -->|Scripted| det[Buttons / listen / message]\n  crew --> pb2[Other playbook in crew]\n  pb2 --> tools\n  tools --> kb[Knowledge base RAG]\n  kb --> reply[Reply on chat/voice/IVR]\n  det --> reply\n  pb --> reply","members":[{"pattern":"agentic-rag","role":"first-class","note":"Knowledge base ingests URLs, sitemaps, documents, and external sources (Zendesk, Shopify, Kustomer, Salesforce) for RAG grounding; explicitly called out in the docs as 'ground responses in real information via RAG'.","evidence":[{"type":"doc","title":"Voiceflow build overview — knowledge base via RAG","url":"https://docs.voiceflow.com/documentation/build/overview","quote":"Upload URLs, sitemaps, and documents to ground responses in real information via RAG.","accessed":"2026-05-20"},{"type":"doc","title":"Voiceflow knowledge base","url":"https://docs.voiceflow.com/documentation/build/Importing-data-sources","quote":"Add knowledge to your agent & connect to tools like Zendesk & Shopify.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"conversation-handoff","role":"first-class","note":"Crew step lets playbooks hand off tasks between each other directly; system tools include 'forward calls' for voice channels.","evidence":[{"type":"doc","title":"Voiceflow Crew step","url":"https://docs.voiceflow.com/documentation/build/steps/crew","quote":"The Crew step lets you group multiple playbooks together so they can hand off tasks between each other directly.","accessed":"2026-05-20"},{"type":"doc","title":"Voiceflow playbook tools — system tools list","url":"https://docs.voiceflow.com/documentation/build/playbooks","quote":"System tools —  call the knowledge base, forward calls, end the conversation, display buttons, cards or carousels, search the web.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"multilingual-voice-agent","role":"first-class","note":"Documented as a native capability across voice and chat channels.","evidence":[{"type":"doc","title":"Voiceflow docs intro — multilingual","url":"https://docs.voiceflow.com","quote":"Resolve customer issues through natural conversation in any language","accessed":"2026-05-20"},{"type":"doc","title":"Voiceflow channels — voice / call centre","url":"https://www.voiceflow.com","quote":"Bring conversational AI to your call centres for automated, human-level call handling","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Six documented tool kinds (API, MCP, Function, Integration, System, Global) callable from playbooks; tool description plus LLM signal drive selection.","evidence":[{"type":"doc","title":"Voiceflow playbooks — tools list","url":"https://docs.voiceflow.com/documentation/build/playbooks","quote":"Integrations ... API tool ... MCP tool ... Function tool ... System tools","accessed":"2026-05-20"},{"type":"doc","title":"Voiceflow global tools","url":"https://docs.voiceflow.com/documentation/build/tools/global-tools","quote":"Global tools are APIs, integrations, MCPs, and functions that you add directly at the agent level. Once added, they're available at the agent level, and in any playbook throughout your AI agent","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"visual-workflow-graph","role":"first-class","note":"Workflows are explicitly a visual builder; playbooks and workflows are the two named primitives of the build surface.","evidence":[{"type":"doc","title":"Voiceflow build overview — workflows","url":"https://docs.voiceflow.com/documentation/build/overview","quote":"Workflows: Visual workflow builder for more deterministic, multi-step conversation flows with branching, conditions, and integrations.","accessed":"2026-05-20"},{"type":"doc","title":"Voiceflow docs intro — design agents visually","url":"https://docs.voiceflow.com","quote":"Design agents visually with playbooks, workflows, and a knowledge base.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"plan-and-execute","role":"first-class","note":"Playbook = goal-based plan; workflow = explicit scripted execution; the two are mixed by embedding a Playbook step inside a Workflow.","evidence":[{"type":"doc","title":"Voiceflow playbooks — goal-based","url":"https://docs.voiceflow.com/documentation/build/playbooks","quote":"Playbooks are goal-based. You define an outcome ... and the agent navigates the conversation to get there.","accessed":"2026-05-20"},{"type":"doc","title":"Voiceflow build overview — hybrid","url":"https://docs.voiceflow.com/documentation/build/overview","quote":"Workflows can use playbooks & other AI-powered steps, so you can build agentic experiences with deterministic control where it matters.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"hierarchical-agents","role":"first-class","note":"Crew step nests a root playbook with child playbooks that hand control between each other and back to the root.","evidence":[{"type":"doc","title":"Voiceflow Crew step — root + children","url":"https://docs.voiceflow.com/documentation/build/steps/crew","quote":"The root playbook is simply the first playbook triggered when the workflow enters the Crew step. There is no functional difference between the root playbook and other playbooks beyond this.","accessed":"2026-05-20"},{"type":"doc","title":"Voiceflow Crew step — mutual awareness","url":"https://docs.voiceflow.com/documentation/build/steps/crew","quote":"Each playbook in the crew is aware of the strengths of other playbooks and can hand off tasks to them.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"topic-based-routing","role":"first-class","note":"Instructions define routing across skills; each playbook's LLM description tells the top-level agent or crew when to route into it.","evidence":[{"type":"doc","title":"Voiceflow build overview — instructions for routing","url":"https://docs.voiceflow.com/documentation/build/overview","quote":"Instructions: Defines how the agent routes requests and decides which skills to use.","accessed":"2026-05-20"},{"type":"doc","title":"Voiceflow playbooks — LLM description for routing","url":"https://docs.voiceflow.com/documentation/build/playbooks","quote":"The description should cover both what the playbook does and when the agent should route into it — this is what other playbooks, workflows, and the top-level agent read when deciding whether to call it.","accessed":"2026-05-20"}],"evidence_status":"full"}],"references":[{"type":"doc","title":"Voiceflow homepage","url":"https://www.voiceflow.com","quote":"Build, launch, and scale advanced AI agents for support, lead generation, and beyond","accessed":"2026-05-20"},{"type":"doc","title":"Voiceflow build overview","url":"https://docs.voiceflow.com/documentation/build/overview","quote":"Build AI agents with playbooks, workflows, and powerful customization options.","accessed":"2026-05-20"},{"type":"doc","title":"Voiceflow docs index","url":"https://docs.voiceflow.com/llms.txt","accessed":"2026-05-20"}],"last_analyzed":"2026-05-20","verification_status":"verified","last_updated":"2026-06-14","date_added":"2026-04-30","status_in_practice":"mature","tags":["saas","low-code","visual-builder","voice","ivr","playbooks","workflows"]},{"id":"ms-365-agents-sdk","name":"Microsoft 365 Agents SDK","kind":"framework","category":"conversational-bot","vendor":"Microsoft","language":"C#, JavaScript, Python","license":"MIT","status":"active","first_released":"2024-11-19","url":"https://learn.microsoft.com/en-us/microsoft-365/agents-sdk/","repo":"https://github.com/microsoft/Agents","docs_url":"https://learn.microsoft.com/en-us/microsoft-365/agents-sdk/agents-sdk-overview","intent":"The Microsoft 365 Agents SDK is a development framework for building conversational agents that receive messages across channels such as Teams, Copilot, web, and Slack, route each activity to a handler, and respond with conversation state intact.","description":"The Microsoft 365 Agents SDK is a framework for building conversational agents in C#, JavaScript, or Python. It provides a channel abstraction layer so agent logic written once works across Microsoft Teams, Microsoft 365 Copilot, a company website, Slack, and other surfaces, normalizing each incoming message into a common Activity and translating responses back to each channel's format. It is AI-agnostic by design, handling message receipt, state, and event routing while leaving response generation to whatever AI service or orchestration library the developer wires in. It is the successor to the Bot Framework SDK.","primary_use_cases":["multi-channel conversational agents and bots","Microsoft Teams and Microsoft 365 Copilot agents","channel-agnostic message handling across Slack, web chat, and Teams","conversation state management across turns"],"agent_loop_shape":"When a message arrives the SDK receives it from whatever channel sent it and normalizes it into a common Activity, routes that activity to the appropriate handler in the agent code based on activity type, then sends the response back through the same channel in that channel's format. Developers write handlers for the activity types they care about while the SDK handles authentication, format translation, channel connectivity, and turn-by-turn conversation state.","build_surface":"full-code","members":[{"pattern":"topic-based-routing","role":"core","note":"The SDK routes each normalized Activity to the handler registered for that activity type, so a message activity triggers one handler and a user-joining activity triggers another.","evidence":[{"type":"doc","title":"What is the Microsoft 365 Agents SDK","url":"https://learn.microsoft.com/en-us/microsoft-365/agents-sdk/agents-sdk-overview","quote":"Routes the activity to the appropriate handler in your agent code. For example, a message activity triggers one handler, a user joining a conversation triggers another.","accessed":"2026-06-17"},{"type":"doc","title":"What is the Microsoft 365 Agents SDK","url":"https://learn.microsoft.com/en-us/microsoft-365/agents-sdk/agents-sdk-overview","quote":"Receives the message from whatever channel sent it (Teams, web chat, Slack, and so on) and normalizes it into a common format called an `Activity`.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"short-term-memory","role":"first-class","note":"The SDK introduces the concept of a turn as a single unit of work and provides built-in state and storage so context carries across turns within a conversation without the developer wiring up a persistence layer.","evidence":[{"type":"doc","title":"What is the Microsoft 365 Agents SDK","url":"https://learn.microsoft.com/en-us/microsoft-365/agents-sdk/agents-sdk-overview","quote":"It introduces the concept of a turn (a single unit of work in a conversation) and tracks state across turns without requiring developers to wire up their own persistence layer.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"translation-layer","role":"core","note":"A channel abstraction layer lets agent logic be written once: the SDK translates each incoming and outgoing message to and from the format the specific channel expects, so adding a channel does not require rewriting the agent's behaviour.","evidence":[{"type":"doc","title":"What is the Microsoft 365 Agents SDK","url":"https://learn.microsoft.com/en-us/microsoft-365/agents-sdk/agents-sdk-overview","quote":"The SDK provides a channel abstraction layer. You write your agent logic once, and the SDK translates incoming and outgoing messages to and from the format each channel expects.","accessed":"2026-06-17"},{"type":"doc","title":"What is the Microsoft 365 Agents SDK","url":"https://learn.microsoft.com/en-us/microsoft-365/agents-sdk/agents-sdk-overview","quote":"Sends the response back through the same channel, translating your response back into the format that channel expects.","accessed":"2026-06-17"}],"evidence_status":"full"}],"key_concepts":[{"name":"Activity","summary":"The common normalized message format the SDK produces from any channel's incoming message, so a developer's handler reasons over one shape regardless of whether the message came from Teams, Slack, or web chat.","maps_to_pattern":"translation-layer","url":"https://learn.microsoft.com/en-us/microsoft-365/agents-sdk/agents-sdk-overview"},{"name":"Turn","summary":"A single unit of work in a conversation; the SDK tracks state across turns through built-in state and storage so context survives between messages.","maps_to_pattern":"short-term-memory","url":"https://learn.microsoft.com/en-us/microsoft-365/agents-sdk/agents-sdk-overview"},{"name":"Channel abstraction layer","summary":"The layer that connects to each surface (Teams, Copilot, Slack, web) and handles its authentication and protocol, so the agent's core behaviour is written once and reused across channels.","maps_to_pattern":"translation-layer","url":"https://learn.microsoft.com/en-us/microsoft-365/agents-sdk/agents-sdk-overview"},{"name":"AI-agnostic scaffolding","summary":"The design stance that the SDK provides message receipt, state, and event routing but makes no assumption about what generates the reply, so the developer plugs in any AI service or orchestration library.","url":"https://learn.microsoft.com/en-us/microsoft-365/agents-sdk/agents-sdk-overview"}],"references":[{"type":"doc","title":"What is the Microsoft 365 Agents SDK","url":"https://learn.microsoft.com/en-us/microsoft-365/agents-sdk/agents-sdk-overview","quote":"The Microsoft 365 Agents SDK is a development framework for building conversational agents.","accessed":"2026-06-17"},{"type":"doc","title":"Microsoft 365 Agents SDK documentation home","url":"https://learn.microsoft.com/en-us/microsoft-365/agents-sdk/","accessed":"2026-06-17"},{"type":"repo","title":"microsoft/Agents (source repository)","url":"https://github.com/microsoft/Agents","accessed":"2026-06-17"}],"alternatives":[{"composition":"ms-bot-framework","relation":"predecessor","note":"The Agents SDK is the successor to the Bot Framework SDK, carrying forward the Activity-and-channel model while removing the assumption of a specific AI service."},{"composition":"botpress","relation":"competes-with","note":"Both build channel-spanning conversational agents; Botpress is an integrated bot-building platform, while the Agents SDK is a code-first channel and state layer that leaves response generation to a plugged-in AI service."}],"verification_status":"partial","status_in_practice":"emerging","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Microsoft 365 Agents SDK\"]\n  fw --> p1[\"Topic-Based Routing<br/>(core)\"]\n  fw --> p2[\"Translation Layer<br/>(core)\"]\n  fw --> p3[\"Short-Term Thread Memory<br/>(first-class)\"]","tags":["conversational-bot","multi-agent","memory","tool-use-environment","full-code"]},{"id":"parlant","name":"Parlant","kind":"framework","category":"conversational-bot","vendor":"Emcie (Parlant)","language":"Python","license":"Apache-2.0","status":"active","first_released":"2024-12-01","url":"https://www.parlant.io/","repo":"https://github.com/emcie-co/parlant","docs_url":"https://www.parlant.io/docs/","intent":"Parlant is a framework for building conversational agents whose behaviour is controlled by condition-action guidelines that the engine matches and applies through structured Attentive Reasoning Queries.","description":"Parlant is a Python framework for building customer-facing conversational agents. Behaviour is defined as granular condition-action guidelines rather than free-form system-prompt instructions, and an engine pipeline matches only the relevant guidelines for the current conversational state, calls associated tools, and composes the message. The message composer uses Attentive Reasoning Queries, structured reasoning blueprints, to keep the agent adhering to its matched guidelines. It is released under Apache 2.0.","primary_use_cases":["building controlled customer-facing chat agents","enforcing behavioral guidelines in conversation","tool-using conversational workflows","explainable agent decisions"],"agent_loop_shape":"On each turn, the engine runs a pipeline: a GuidelineMatcher filters the full set of condition-action guidelines down to those relevant for the current conversational state, a ToolCaller executes the tools associated with matched guidelines, and a MessageComposer generates the response using the matched guidelines, tool results, and history under ARQ-based structured reasoning. The pipeline iterates until reaching a stable state.","build_surface":"full-code","members":[{"pattern":"attentive-reasoning-queries","role":"core","note":"Parlant replaces free-form chain-of-thought with Attentive Reasoning Queries — structured JSON reasoning blueprints whose targeted queries re-anchor the model's attention to critical guidelines and prior decisions at the generation steps where adherence slips; it runs distinct ARQ sets per component (guideline matching, tool-calling, message composition).","evidence":[{"type":"doc","title":"Enforcement & Explainability — Parlant Docs","url":"https://www.parlant.io/docs/advanced/explainability/","quote":"Attentive Reasoning Queries (ARQs) are essentially structured reasoning blueprints built into prompts that guide LLMs through specific thinking patterns","accessed":"2026-06-17"},{"type":"doc","title":"Enforcement & Explainability — Parlant Docs","url":"https://www.parlant.io/docs/advanced/explainability/","quote":"Parlant uses different sets of ARQs for each of its components (e.g., guideline matching, tool-calling, or message composition)","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"deterministic-control-flow-not-prompt","role":"core","note":"Rather than putting branching logic in a single prompt, Parlant's engine runs a pipeline of distinct components — a GuidelineMatcher that filters guidelines to those relevant for the current state, a ToolCaller that executes the matched guidelines' tools, and a MessageComposer that generates the reply — so application-level control flow is separated from message generation and the LLM is invoked at structured points.","evidence":[{"type":"doc","title":"Engine Overview — Parlant Docs","url":"https://www.parlant.io/docs/engine-internals/overview/","quote":"GuidelineMatcher: Filters the full set of guidelines down to those relevant for the current conversational state.","accessed":"2026-06-17"},{"type":"doc","title":"Engine Overview — Parlant Docs","url":"https://www.parlant.io/docs/engine-internals/overview/","quote":"MessageComposer: Generates the actual response message using all accumulated context: matched guidelines, tool results, glossary terms, and conversation history.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Tools are how a Parlant agent takes external action — querying databases, calling APIs, processing payments — and they enter the agent's context reactively, attached to a guideline or observation so a tool becomes available only when its condition matches rather than being always present.","evidence":[{"type":"doc","title":"Tools — Parlant Docs","url":"https://www.parlant.io/docs/concepts/customization/tools/","quote":"Tools are how your agent takes action—querying databases, calling APIs, processing payments.","accessed":"2026-06-17"},{"type":"doc","title":"Tools — Parlant Docs","url":"https://www.parlant.io/docs/concepts/customization/tools/","quote":"In Parlant, tools are part of the context delivery system: they only enter the agent's context when a relevant observation fires, rather than being 'always available.'","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"decision-log","role":"first-class","note":"Parlant's structured reasoning leaves an auditable trail: each GuidelineMatcher decision records the guideline id, condition, whether it applied, and the rationale for applying or not, so a reviewer can inspect why the agent responded the way it did.","evidence":[{"type":"doc","title":"Enforcement & Explainability — Parlant Docs","url":"https://www.parlant.io/docs/advanced/explainability/","quote":"create, as a byproduct, transparent, auditable reasoning paths that help maintain alignment with desired behaviors","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"Guideline","summary":"A condition-action rule (for example, condition='customer expresses interest in a high-risk product', action='provide risk disclosure') that the engine matches only when its condition holds for the current turn, so behaviour is specified as discrete rules instead of one system prompt.","maps_to_pattern":"deterministic-control-flow-not-prompt","url":"https://www.parlant.io/"},{"name":"Journey","summary":"A multi-turn SOP expressed as a state diagram the agent follows when guiding a customer through a process like a booking or a return; it is adaptive, so the agent may skip ahead, revisit states, or change pace based on how the customer responds.","url":"https://www.parlant.io/docs/concepts/customization/journeys"},{"name":"Attentive Reasoning Queries (ARQs)","summary":"Structured reasoning blueprints built into prompts that walk the model through targeted queries at the steps where guideline adherence tends to slip, used as distinct sets per engine component.","maps_to_pattern":"attentive-reasoning-queries","url":"https://www.parlant.io/docs/advanced/explainability/"},{"name":"Canned response","summary":"A pre-approved response template the engine can emit at high-stakes moments, capping failure severity by giving precise wording where it matters instead of letting the model phrase the reply freely.","url":"https://www.parlant.io/"}],"references":[{"type":"doc","title":"Parlant — The interaction control harness for customer-facing AI agents","url":"https://www.parlant.io/","accessed":"2026-06-17"},{"type":"repo","title":"emcie-co/parlant (source repository)","url":"https://github.com/emcie-co/parlant","accessed":"2026-06-17"},{"type":"doc","title":"Parlant documentation","url":"https://www.parlant.io/docs/","accessed":"2026-06-17"}],"alternatives":[{"composition":"rasa","relation":"competes-with","note":"Both build controlled customer-facing conversational agents; Rasa centres on intent/entity NLU and dialogue policies, while Parlant centres on condition-action guidelines applied through structured reasoning over an LLM."},{"composition":"voiceflow","relation":"similar-shape","note":"Both let teams design constrained conversational flows; Voiceflow is a visual flow builder, whereas Parlant is a code-first engine where flows are journeys and behaviour is rules the engine matches per turn."}],"verification_status":"partial","status_in_practice":"emerging","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Parlant\"]\n  fw --> p1[\"Attentive Reasoning Queries<br/>(core)\"]\n  fw --> p2[\"Deterministic Control Flow, Not Prompt<br/>(core)\"]\n  fw --> p3[\"Tool Use<br/>(first-class)\"]\n  fw --> p4[\"Decision Log<br/>(first-class)\"]","tags":["conversational-bot","reasoning","planning-control-flow","tool-use-environment","governance-observability","full-code"]},{"id":"11x","name":"11x.ai","kind":"framework","category":"domain-agent","build_surface":"closed-application","vendor":"11x.ai","language":"Web product","license":"proprietary","status":"active","url":"https://www.11x.ai/","intent":"Vertical SaaS that ships named 'digital workers' (Alice for outbound SDR, Julian for sales-call voice) which run multi-channel prospecting and live phone conversations against a buyer's CRM and data stack 24/7.","description":"11x.ai sells productised AI workers for revenue teams. Alice prospects across email, phone, social and SMS, enriches contacts from many third-party data providers and books meetings. Julian is a voice agent that holds real sales calls, listens, responds to objections and adapts in-call. The vendor markets the workers as autonomous and tightly integrated with the customer's tech stack; technical architecture details (model providers, tool framework, scheduling primitives) are not publicly documented.","primary_use_cases":["autonomous outbound SDR/BDR across email, phone, social and SMS","AI voice agent for live sales conversations","signal-driven prospecting against 50+ enrichment sources","always-on pipeline generation tied into the customer CRM"],"agent_loop_shape":"Vendor describes named workers (Alice, Julian) that run continuously against connected data and channels. Alice watches market signals and CRM events, enriches prospects, drafts and sends per-recipient messages across email/phone/social/SMS, then handles replies; Julian operates a voice channel for live calls. The product is hosted SaaS with no published agent runtime, tool-calling API, or developer surface, so the loop shape below is inferred from marketing copy.","key_concepts":[{"name":"Alice","summary":"Outbound SDR worker that prospects, enriches, sends multi-channel sequences and books meetings.","url":"https://www.11x.ai/alice"},{"name":"Julian","summary":"Voice worker that runs live sales conversations on the phone.","url":"https://www.11x.ai/"},{"name":"Multi-channel sequences","summary":"Email, phone, social and SMS touches stitched into a single per-prospect sequence.","url":"https://www.11x.ai/"},{"name":"Signal-driven prospecting","summary":"Filters millions of signals across 50+ data sources to surface ICP-fit prospects.","url":"https://www.11x.ai/"}],"pattern_composition":"flowchart TD\n  signals[Market signals + CRM events] --> alice[Alice worker]\n  enrich[Enrichment 50+ data sources] --> alice\n  alice --> seq[Multi-channel sequence: email/phone/social/SMS]\n  seq --> reply{Reply?}\n  reply -->|yes| handle[Alice handles reply, books meeting]\n  reply -->|no| seq\n  call[Inbound/outbound phone] --> julian[Julian voice worker]\n  julian --> live[Live conversation, objection handling]\n  handle --> crm[(CRM)]\n  live --> crm","members":[{"pattern":"event-driven-agent","role":"first-class","note":"Vendor frames Alice as signal-triggered against market signals and CRM activity. No published technical detail on the event/webhook surface.","evidence":[{"type":"doc","title":"11x homepage — signal filtering","url":"https://www.11x.ai/","quote":"Filter millions of signals across 50+ data sources to surface buyers who match your ICP.","accessed":"2026-05-20"}],"evidence_status":"limited"},{"pattern":"scheduled-agent","role":"limited","note":"Vendor advertises 24/7 operation but does not publish cron/scheduling primitives. Honest downgrade from first-class.","evidence":[{"type":"doc","title":"11x homepage — 24/7 workforce","url":"https://www.11x.ai/","quote":"Always learning. Learn and adapt continuously, expanding knowledge and maximizing performance with every interaction.","accessed":"2026-05-20"}],"evidence_status":"limited"},{"pattern":"tool-use","role":"first-class","note":"Workers act across email/phone/social/SMS and pull enrichment from 50+ data sources. Internal tool-calling architecture not published.","evidence":[{"type":"doc","title":"11x homepage — multi-channel sequences","url":"https://www.11x.ai/","quote":"Run multi-channel sequences across email, phone, social networks, and SMS. Every channel picks up where the last left off.","accessed":"2026-05-20"},{"type":"doc","title":"11x homepage — enrichment integrations","url":"https://www.11x.ai/","quote":"Enrich every prospect automatically, from company news and tech stack to org changes and competitive signals.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"multilingual-voice-agent","role":"first-class","note":"Julian is positioned as a live-voice sales agent that listens, responds, and adapts mid-call. Vendor does not publish the speech stack.","evidence":[{"type":"doc","title":"11x homepage — Julian voice","url":"https://www.11x.ai/","quote":"Julian runs real sales conversations in natural language. He listens, responds to objections, and adapts mid-call.","accessed":"2026-05-20"}],"evidence_status":"limited"}],"alternatives":[{"composition":"artisan","relation":"competes-with","note":"Peer AI-SDR product (Ava)."},{"composition":"lindy","relation":"competes-with","note":"Generic AI-employee platform overlapping outbound use case."}],"references":[{"type":"doc","title":"11x homepage","url":"https://www.11x.ai/","accessed":"2026-05-20"},{"type":"doc","title":"11x — Alice product page","url":"https://www.11x.ai/alice","accessed":"2026-05-20"}],"last_analyzed":"2026-05-20","verification_status":"partial","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"emerging","tags":["proprietary","saas","sdr","voice","outbound-sales"]},{"id":"artisan","name":"Artisan","kind":"framework","category":"domain-agent","build_surface":"closed-application","vendor":"Artisan","language":"Web product","license":"proprietary","status":"active","url":"https://www.artisan.co/","intent":"Hosted AI SDR named Ava that prospects across 250M+ B2B contacts, runs continuous A/B-tested multi-channel sequences, handles replies and books meetings into rep calendars.","description":"Artisan sells Ava, an AI SDR positioned as an autonomous outbound prospecting worker. Ava enriches contacts from 22+ data sources, ranks them by intent signals, sends personalised email and social touches, queues call steps in a native dialer, runs many message variants in parallel and shifts traffic toward winners, then qualifies replies and books meetings with configurable escalation rules. Marketing copy is detailed; internal agent architecture, model provider, and tool framework are not published.","primary_use_cases":["autonomous outbound SDR over email and social","signal-driven prospecting on a hosted contact graph","continuous A/B testing of message variants per campaign","reactivation of closed-lost and churned accounts"],"agent_loop_shape":"Hosted, continuously-running worker. Ava ingests CRM and contact-graph signals, prioritises prospects by intent, generates per-prospect copy across email and social, dispatches via the connected channels, listens for replies, qualifies and books meetings, and concurrently runs many message variants whose traffic share is rebalanced by outcome. Human reps are pulled in through customer-defined escalation rules. The vendor does not publish the underlying loop or tool framework.","key_concepts":[{"name":"Ava","summary":"The single named AI SDR agent that runs prospecting, sequencing, replies and booking.","url":"https://www.artisan.co/"},{"name":"Multi-source enrichment","summary":"Per-prospect enrichment from 22+ data sources, ranked by intent.","url":"https://www.artisan.co/"},{"name":"Continuous A/B testing","summary":"Dozens of message variants per campaign with auto-rebalancing.","url":"https://www.artisan.co/"},{"name":"Escalation rules","summary":"Customer-defined trigger for handing a reply to a human rep.","url":"https://www.artisan.co/"}],"pattern_composition":"flowchart TD\n  search[B2B contact graph 250M+] --> enrich[Enrichment 22+ sources]\n  enrich --> prioritise[Intent ranking]\n  prioritise --> ava[Ava worker]\n  ava --> variants[Many message variants in parallel]\n  variants --> send[Email + social touches]\n  send --> dialer[Native dialer queue]\n  send --> reply{Reply?}\n  reply -->|yes| qualify[Qualify and handle objections]\n  qualify --> book[Book meeting on rep calendar]\n  qualify --> escalate{Escalation rule?}\n  escalate -->|yes| human[Human rep]\n  reply -->|no| measure[Outcome measurement]\n  measure -.shift volume.-> variants","members":[{"pattern":"scheduled-agent","role":"limited","note":"Vendor describes Ava as continuously prospecting 24/7 but does not document explicit schedules or cron-style triggers. Honest downgrade from first-class.","evidence":[{"type":"doc","title":"Artisan homepage — continuous prospecting","url":"https://www.artisan.co/","quote":"Continuously prospects, handles replies, and schedules meetings to build you pipeline 24/7","accessed":"2026-05-20"}],"evidence_status":"limited"},{"pattern":"tool-use","role":"first-class","note":"Ava sends email and social touches, queues calls in a native dialer, and pulls from 22+ data sources plus HubSpot/Salesforce/Google/ZoomInfo/BigQuery. Underlying tool-calling surface not published.","evidence":[{"type":"doc","title":"Artisan homepage — multi-channel outreach","url":"https://www.artisan.co/","quote":"Ava sends personalized multi-channel outreach across email and social","accessed":"2026-05-20"},{"type":"doc","title":"Artisan homepage — dialer","url":"https://www.artisan.co/","quote":"She queues call steps in the native dialer so your reps can click and dial","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"approval-queue","role":"first-class","note":"Customer-defined escalation rules act as a human-in-the-loop gate on replies.","evidence":[{"type":"doc","title":"Artisan homepage — escalation rules","url":"https://www.artisan.co/","quote":"You set escalation rules to decide when a human gets pulled in","accessed":"2026-05-20"}],"evidence_status":"limited"},{"pattern":"best-of-n","role":"first-class","note":"Many variants run in parallel and traffic shifts to winners — best-of-n at the campaign level.","evidence":[{"type":"doc","title":"Artisan homepage — variant testing","url":"https://www.artisan.co/","quote":"Ava runs dozens of message variations per campaign simultaneously, testing subject lines, structure, CTAs, and tone","accessed":"2026-05-20"},{"type":"doc","title":"Artisan homepage — traffic shift","url":"https://www.artisan.co/","quote":"She shifts volume toward what's working so performance compounds over time","accessed":"2026-05-20"}],"evidence_status":"full"}],"alternatives":[{"composition":"11x","relation":"competes-with","note":"Peer AI-SDR product (Alice)."},{"composition":"lindy","relation":"competes-with","note":"Generic AI-employee platform overlapping outbound use case."}],"references":[{"type":"doc","title":"Artisan homepage","url":"https://www.artisan.co/","accessed":"2026-05-20"}],"last_analyzed":"2026-05-20","verification_status":"partial","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"emerging","tags":["proprietary","saas","sdr","outbound-sales"]},{"id":"crescendo","name":"Crescendo","kind":"framework","category":"domain-agent","build_surface":"closed-application","vendor":"Crescendo","language":"Web product","license":"proprietary","status":"active","url":"https://www.crescendo.ai/","intent":"Outcome-priced CX service that combines AI agents with human 'Superhuman' agents on a managed platform, covering chat, messaging, voice and email and integrating into existing support stacks.","description":"Crescendo positions itself as a managed CX provider that runs AI plus humans together rather than selling a self-serve agent builder. Customer service interactions are handled by AI on every channel and handed off 24/7 to human Superhuman agents when needed. The vendor markets a 30-day go-live with no rip-and-replace, outcome-based pricing and a performance guarantee. Detailed agent runtime, tool framework, or model selection are not publicly documented; this entry treats vendor-described capabilities as marketing claims and downgrades evidence accordingly.","primary_use_cases":["managed CX with AI-plus-human delivery on outcome pricing","multi-channel customer support (chat, messaging, voice, email)","AI-first triage with seamless handoff to human agents","AI shopping assistant on top of existing storefronts"],"agent_loop_shape":"Managed service: inbound customer messages on chat/messaging/voice/email are routed first to AI agents, with handoff to human 'Superhuman' agents when needed. The customer's existing CRM/helpdesk (Salesforce, Zendesk, Shopify, Google Drive, Guru) remains the system of record. The internal agent architecture is not published.","key_concepts":[{"name":"AI + Superhuman agents","summary":"Hybrid model: AI handles routine, human Superhuman agents take handoffs 24/7.","url":"https://www.crescendo.ai/"},{"name":"Multi-channel coverage","summary":"Chat, messaging, voice, email plus a shopping-assistant front-end.","url":"https://www.crescendo.ai/"},{"name":"Outcome-based pricing","summary":"Vendor takes responsibility for results rather than per-seat licensing.","url":"https://www.crescendo.ai/"}],"pattern_composition":"flowchart TD\n  cust[Customer message] --> chan{Channel}\n  chan -->|chat| ai[AI agent]\n  chan -->|messaging| ai\n  chan -->|voice| ai\n  chan -->|email| ai\n  ai --> resolve{Resolved?}\n  resolve -->|yes| done[Reply]\n  resolve -->|no| sh[Superhuman human agent 24/7]\n  sh --> done\n  done --> tools[(Existing CRM/helpdesk: Salesforce/Zendesk/Shopify)]","members":[{"pattern":"conversation-handoff","role":"first-class","note":"Vendor explicitly markets seamless 24/7 handoff from AI to Superhuman human agents.","evidence":[{"type":"doc","title":"Crescendo homepage — 24/7 handoff","url":"https://www.crescendo.ai/","quote":"Seamless 24x7 handoff to the industry's best Superhuman agents.","accessed":"2026-05-20"}],"evidence_status":"limited"},{"pattern":"tool-use","role":"limited","note":"Vendor lists CRM/helpdesk integrations (Salesforce, Zendesk, Shopify, Google Drive, Guru) but does not publish what actions the AI agent itself executes inside them.","evidence":[{"type":"doc","title":"Crescendo homepage — no-rip-and-replace","url":"https://www.crescendo.ai/","quote":"Built to work with every tool your team already uses. No rip-and-replace.","accessed":"2026-05-20"}],"evidence_status":"limited"},{"pattern":"multilingual-voice-agent","role":"first-class","note":"Voice listed as a first-class channel alongside chat/messaging/email; voice-stack details not published.","evidence":[{"type":"doc","title":"Crescendo homepage — channels","url":"https://www.crescendo.ai/","quote":"All channels covered with AI agents, including: Chat, Messaging, Voice, Email.","accessed":"2026-05-20"}],"evidence_status":"limited"}],"alternatives":[{"composition":"decagon","relation":"competes-with","note":"Peer CX-agent vendor; Decagon publishes more architectural detail (AOPs)."},{"composition":"intercom-fin","relation":"competes-with","note":"Embedded-in-helpdesk peer."},{"composition":"sierra","relation":"competes-with","note":"Peer CX-agent vendor with multi-channel scope."},{"composition":"maven-agi","relation":"competes-with","note":"Peer enterprise CX-agent vendor."}],"references":[{"type":"doc","title":"Crescendo homepage","url":"https://www.crescendo.ai/","accessed":"2026-05-20"}],"last_analyzed":"2026-05-20","verification_status":"partial","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"emerging","tags":["proprietary","managed-service","cx","ai-plus-human"]},{"id":"decagon","name":"Decagon","kind":"framework","category":"domain-agent","build_surface":"closed-application","vendor":"Decagon","language":"Web product","license":"proprietary","status":"active","url":"https://decagon.ai/","intent":"Enterprise CX agent platform built around Agent Operating Procedures (AOPs) — natural-language workflow definitions that the AI agent executes across chat, email and voice with selective routing to human agents.","description":"Decagon ships an AI Agent Engine for customer support. The Core AI Agent is the primary contact point across chat, email and voice. AOPs let non-technical CX teams define agent workflows in natural language with enterprise guardrails. A Routing module decides when a request should escalate, and Agent Assist becomes a copilot for the human handler. Evaluation, observability and experimentation tooling sit around the agent for iteration.","primary_use_cases":["enterprise CX agent across chat, email and voice on one logic layer","natural-language workflow authoring via AOPs","AI-to-human routing with copilot for human agents","continuous evaluation and iteration on agent behaviour"],"agent_loop_shape":"Multi-channel agent: chat / email / voice converge on a single Core AI Agent governed by AOPs (natural-language workflows). The agent consults knowledge and executes multi-step workflows; the Routing module escalates to human handlers when an inquiry should not be handled by the AI, at which point Agent Assist gives the human copilot suggestions. An evaluation engine and Watchtower observability layer monitor behaviour.","key_concepts":[{"name":"Core AI Agent","summary":"Primary contact across chat, email and voice.","url":"https://decagon.ai/blog/the-ai-agent-engine"},{"name":"Agent Operating Procedures (AOPs)","summary":"Natural-language workflow definitions for the agent with code-like precision.","url":"https://decagon.ai/blog/aop-the-future-of-cx"},{"name":"Routing module","summary":"Decides when a conversation should not be handled by AI and routes to humans.","maps_to_pattern":"conversation-handoff","url":"https://decagon.ai/blog/the-ai-agent-engine"},{"name":"Agent Assist","summary":"Copilot for the human agent who receives a handoff.","url":"https://decagon.ai/blog/the-ai-agent-engine"},{"name":"Watchtower / evaluation engine","summary":"Observability and evaluation across the agent lifecycle.","url":"https://decagon.ai/blog/evaluation-engine-ai-agents"}],"pattern_composition":"flowchart TD\n  cust[Customer] --> chan{Channel}\n  chan -->|chat| core[Core AI Agent]\n  chan -->|email| core\n  chan -->|voice| core\n  aop[(AOPs natural-language workflows)] --> core\n  kb[(Knowledge)] --> core\n  core --> route{Routing module}\n  route -->|AI handles| reply[Reply to customer]\n  route -->|escalate| human[Human agent]\n  human --> assist[Agent Assist copilot]\n  assist --> reply\n  reply --> eval[Watchtower + evaluation engine]\n  eval -.feedback.-> core","members":[{"pattern":"agentic-rag","role":"first-class","note":"Vendor describes a unified intelligence layer across channels relying on customer knowledge; AOPs pull in data and retrieve to take action.","evidence":[{"type":"doc","title":"Decagon homepage — unified intelligence layer","url":"https://decagon.ai/","quote":"Decagon unifies voice, chat, and email within a single intelligence layer, ensuring customer experiences stay consistent across every channel.","accessed":"2026-05-20"},{"type":"blog","title":"Decagon AOPs — pull in data and trigger responses","url":"https://decagon.ai/resources/aop-the-future-of-cx","quote":"With seamless integration with existing support tools and workflows, AOPs can pull in data, take actions, and trigger responses.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"conversation-handoff","role":"first-class","note":"Routing module is the dedicated handoff primitive; Agent Assist supports the human after handoff.","evidence":[{"type":"blog","title":"Decagon AI Agent Engine — Routing module","url":"https://decagon.ai/blog/the-ai-agent-engine","quote":"The Routing module intelligently determines when an inquiry cannot or should not be handled by an AI agent and selectively routes conversations to the appropriate channel.","accessed":"2026-05-20"},{"type":"blog","title":"Decagon AI Agent Engine — Agent Assist","url":"https://decagon.ai/blog/the-ai-agent-engine","quote":"Agent Assist then serves as a strategic copilot for your human agent, maximizing efficiency and capabilities with real-time suggested responses and insights.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"AOPs are described as enabling multi-step workflows and as integrating with support tools to take actions; vendor does not publish a tool-calling spec.","evidence":[{"type":"blog","title":"Decagon AOPs — multi-step workflows","url":"https://decagon.ai/resources/aop-the-future-of-cx","quote":"AOPs enable AI agents to dynamically respond to complex issues and execute multi-step workflows.","accessed":"2026-05-24"},{"type":"blog","title":"Decagon AOPs — actions and integrations","url":"https://decagon.ai/resources/aop-the-future-of-cx","quote":"AOPs can pull in data, take actions, and trigger responses, all while ensuring your engineers retain full control over the core code.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"spec-first-agent","role":"first-class","note":"AOPs are an explicit spec layer authored in natural language, with enterprise guardrails, that drives agent behaviour.","evidence":[{"type":"doc","title":"Decagon homepage — AOPs","url":"https://decagon.ai/","quote":"Define your workflows with natural-language AOPs that deliver faster time to value, greater transparency, and trusted results at scale.","accessed":"2026-05-20"},{"type":"blog","title":"Decagon AOPs — guardrails","url":"https://decagon.ai/blog/aop-the-future-of-cx","quote":"AOPs provide enterprise-grade guardrails to ensure that critical operations are executed securely and with strict validation.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"multilingual-voice-agent","role":"first-class","note":"Voice is a documented first-class channel for customisable voice agents with adjustable language and pronunciation.","evidence":[{"type":"doc","title":"Decagon homepage — voice agents","url":"https://decagon.ai/","quote":"Fast, intelligent voice AI agents built for natural dialog and fully customizable to your brand.","accessed":"2026-05-20"},{"type":"doc","title":"Decagon Voice — adjustable language and pronunciation","url":"https://decagon.ai/product/voice","quote":"Adjust voice language, style, speed, tone, and pronunciation of your business-specific terminology.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"eval-harness","role":"first-class","note":"Evaluation engine + Watchtower observability ship alongside the agent for iteration, with always-on QA against custom scorecards.","evidence":[{"type":"doc","title":"Decagon homepage — testing and observability","url":"https://decagon.ai/","quote":"Validate and iterate on AI logic with robust testing, observability, and experimentation, so your agent stays reliable as it evolves.","accessed":"2026-05-20"},{"type":"doc","title":"Decagon Watchtower — always-on QA","url":"https://decagon.ai/product/watchtower","quote":"Always-on monitoring and QA for AI and human agent interactions against your custom criteria","accessed":"2026-05-24"},{"type":"blog","title":"Decagon evaluation engine — assess every component","url":"https://decagon.ai/blog/evaluation-engine-ai-agents","quote":"We rigorously assess every component of the agent, from response generation to retrieval to safety.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"mandatory-red-flag-escalation","role":"supported","note":"Decagon's escalation policy lets specific issue types route to a human unconditionally, regardless of the AI's confidence or capability, which is exactly the unconditional-on-match interrupt this pattern requires.","evidence":[{"type":"doc","title":"What is an AI escalation policy? | Decagon","url":"https://decagon.ai/glossary/what-is-an-ai-escalation-policy","quote":"Certain issue types, including legal disputes, fraud claims, regulated financial decisions, and medical questions, route to humans unconditionally.","accessed":"2026-06-14"}],"evidence_status":"full"},{"pattern":"re-contact-subtracted-resolution","role":"supported","note":"Decagon defines resolution as an issue fully resolved without needing further follow-up and warns that deflection can simply count dissatisfied customers walking away, the same separation of solved-issue from session-end this pattern enforces.","evidence":[{"type":"doc","title":"What is resolution rate? | Decagon","url":"https://decagon.ai/glossary/what-is-resolution-rate","quote":"fully resolved...without needing further follow-up","accessed":"2026-06-14"},{"type":"doc","title":"What is deflection rate? | Decagon","url":"https://decagon.ai/glossary/deflection-rate","quote":"If self-service tools resolve the wrong issues or frustrate users, the deflection rate may simply quantify the number of customers who walk away dissatisfied.","accessed":"2026-06-14"}],"evidence_status":"full"}],"alternatives":[{"composition":"sierra","relation":"competes-with","note":"Peer CX-agent vendor; Sierra's Agent OS plays a similar role to Decagon's AI Agent Engine."},{"composition":"intercom-fin","relation":"competes-with","note":"Helpdesk-embedded peer."},{"composition":"maven-agi","relation":"competes-with","note":"Enterprise CX-agent peer."},{"composition":"crescendo","relation":"competes-with","note":"AI-plus-human managed CX peer."}],"references":[{"type":"doc","title":"Decagon homepage","url":"https://decagon.ai/","accessed":"2026-05-20"},{"type":"blog","title":"Decagon — The AI Agent Engine","url":"https://decagon.ai/blog/the-ai-agent-engine","accessed":"2026-05-20"},{"type":"blog","title":"Decagon — AOPs: the future of CX","url":"https://decagon.ai/blog/aop-the-future-of-cx","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-14","date_added":"2026-04-30","status_in_practice":"mature","tags":["proprietary","cx","enterprise","aops","voice"]},{"id":"dust","name":"Dust","kind":"framework","category":"domain-agent","build_surface":"closed-application","vendor":"Dust","language":"Web product","license":"proprietary","status":"active","url":"https://dust.tt/","docs_url":"https://docs.dust.tt/","intent":"Workspace-level platform where users assemble named agents from instructions, knowledge sources, default tools (web search, file creation, image generation, memory), 100+ connectors and remote MCP servers, with explicit agent-as-tool delegation.","description":"Dust is a hosted workspace for building and running AI agents that connect to a company's data, tools and people. Each agent is configured with instructions, a model choice (OpenAI, Anthropic, Google, Mistral), Tools & Knowledge (default tools plus 100+ connector and remote MCP servers) and Data Sources. Dust supports agent-as-tool delegation via the `run_agent` tool, with a documented maximum recursion depth of 4, including a full-handoff mode where the sub-agent takes over the conversation.","primary_use_cases":["company-wide agent workspace over Notion / Slack / Salesforce / Gmail / etc.","agent-as-tool delegation to specialised sub-agents","MCP-extensible tool surface on top of native connectors","structured-output and Deep Dive agents for long-running work"],"agent_loop_shape":"Per-agent runtime: user message arrives, the agent loads its instructions and configured Tools & Knowledge, picks one or more tools per turn (search over Data Sources, default tools, connector tools, MCP-backed tools, or `run_agent` to delegate to another agent), executes, and continues. `run_agent` either calls the sub-agent in a separate conversation and returns the result, or performs a complete handoff where the sub-agent responds directly to the user. Recursion is capped at depth 4.","key_concepts":[{"name":"Agent","summary":"Configured with instructions plus Tools & Knowledge.","url":"https://docs.dust.tt/docs/quickstart-agent"},{"name":"Tools & Knowledge","summary":"Mix of default tools (web search, file creation, image generation, memory), connectors and MCP servers.","maps_to_pattern":"tool-use","url":"https://docs.dust.tt/docs/tools-management"},{"name":"Run Agent","summary":"Tool that lets an agent call another agent as a sub-agent or as a full handoff.","maps_to_pattern":"supervisor","url":"https://docs.dust.tt/docs/run-agent"},{"name":"Remote MCP servers","summary":"Extend the toolset by registering an MCP server URL.","maps_to_pattern":"mcp","url":"https://docs.dust.tt/docs/remote-mcp-server"},{"name":"Multi-provider model choice","summary":"Switch between OpenAI, Anthropic, Google and Mistral per agent.","maps_to_pattern":"multi-model-routing","url":"https://dust.tt/"},{"name":"Connectors","summary":"100+ production connectors covering Notion, Slack, Salesforce, Gmail, GitHub, etc.","url":"https://docs.dust.tt/"}],"pattern_composition":"flowchart TD\n  user[User] --> agent[Dust agent]\n  instr[(Instructions)] --> agent\n  agent --> pick{Pick tool}\n  pick -->|search| ds[(Data sources + connectors)]\n  pick -->|default| def[Web search / file create / image gen / memory]\n  pick -->|mcp| mcp[(Remote MCP servers)]\n  pick -->|run_agent| sub[Sub-agent]\n  sub --> back[Return to main]\n  sub -.handoff.-> user\n  ds --> obs[Observe]\n  def --> obs\n  mcp --> obs\n  back --> obs\n  obs --> agent\n  agent -->|done| reply[Reply]","members":[{"pattern":"agentic-rag","role":"first-class","note":"Search over Data Sources is a core capability; agent picks search tools per turn against connected company data.","evidence":[{"type":"doc","title":"Dust quickstart — tools and knowledge","url":"https://docs.dust.tt/docs/quickstart-agent","quote":"Tools & Knowledge give your agent special abilities.","accessed":"2026-05-20"},{"type":"doc","title":"Dust quickstart — search tools","url":"https://docs.dust.tt/docs/quickstart-agent","quote":"tools that your agent can use to answer - such as searching in your Data Sources or navigating the Web.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"Remote MCP servers extend Dust's toolset; register by URL.","evidence":[{"type":"doc","title":"Dust docs — remote MCP","url":"https://docs.dust.tt/docs/remote-mcp-server","quote":"Extending Dust's toolset beyond the default list of tools is possible by adding extra MCP Servers","accessed":"2026-05-20"},{"type":"doc","title":"Dust docs — MCP onboarding","url":"https://docs.dust.tt/docs/remote-mcp-server","quote":"you'll only have to enter the MCP server's public URL, and after a quick synchronization, your tool will be created and ready to be added and used by agents","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"supervisor","role":"first-class","note":"`run_agent` is the documented agent-as-tool primitive with full-handoff mode and capped recursion depth.","evidence":[{"type":"doc","title":"Dust docs — run_agent tool","url":"https://docs.dust.tt/docs/run-agent","quote":"The `run_agent` tool enables your agents to run other agent as a tool.","accessed":"2026-05-20"},{"type":"doc","title":"Dust docs — handoff mode","url":"https://docs.dust.tt/docs/run-agent","quote":"The selected agent takes over and responds directly to the user instead of the main agent. This is a complete handoff.","accessed":"2026-05-20"},{"type":"doc","title":"Dust docs — recursion cap","url":"https://docs.dust.tt/docs/run-agent","quote":"We enforce a maximum recursion depth of 4 for calling sub-agents.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Default tools (web search, file creation, image generation, data viz, memory), 100+ connectors, MCP-backed tools.","evidence":[{"type":"doc","title":"Dust homepage — 100+ connectors","url":"https://dust.tt/","quote":"100+ production connectors","accessed":"2026-05-20"},{"type":"doc","title":"Dust quickstart — agent picks tools","url":"https://docs.dust.tt/docs/quickstart-agent","quote":"the agent will decide which tools to use (one or many) to provide you with the best possible answer.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"multi-model-routing","role":"first-class","note":"Per-agent model choice across OpenAI, Anthropic, Google, Mistral; Dust maintains integrations with multiple model providers.","evidence":[{"type":"doc","title":"Dust homepage — frontier models","url":"https://dust.tt/","quote":"Switch seamlessly between frontier models from OpenAI, Anthropic, Google, and Mistral.","accessed":"2026-05-20"},{"type":"blog","title":"Dust blog — multi-provider integrations","url":"https://dust.tt/blog/the-single-model-trap-why-ai-platforms-need-multiple-providers","quote":"we maintain integrations with OpenAI, Anthropic, Google, Mistral, xAI, OSS models like DeepSeek and Kimi","accessed":"2026-05-24"},{"type":"blog","title":"Dust blog — per-agent model choice","url":"https://dust.tt/blog/the-single-model-trap-why-ai-platforms-need-multiple-providers","quote":"Every agent you build in Dust can specify which model to use","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"handoff","role":"first-class","note":"`run_agent` handoff mode is an explicit agent-to-agent conversation handoff; selected agent takes over responding to the user.","evidence":[{"type":"doc","title":"Dust docs — handoff mode","url":"https://docs.dust.tt/docs/run-agent","quote":"The selected agent takes over and responds directly to the user instead of the main agent. This is a complete handoff.","accessed":"2026-05-20"},{"type":"doc","title":"Dust docs — run_agent purpose","url":"https://docs.dust.tt/docs/run-agent","quote":"The `run_agent` tool enables your agents to run other agent as a tool.","accessed":"2026-05-24"}],"evidence_status":"full"}],"alternatives":[{"composition":"lindy","relation":"competes-with","note":"Hosted multi-tool agent peer."},{"composition":"openclaw","relation":"competes-with","note":"Self-hosted personal-assistant alternative."},{"composition":"sparrot","relation":"competes-with","note":"Hosted multi-user agent workspace; opposite deployment model (cloud SaaS vs. private single-user) and opposite trigger model (user message vs. self-pacing)."}],"references":[{"type":"doc","title":"Dust homepage","url":"https://dust.tt/","accessed":"2026-05-20"},{"type":"doc","title":"Dust docs index","url":"https://docs.dust.tt/","accessed":"2026-05-20"},{"type":"doc","title":"Dust — run_agent","url":"https://docs.dust.tt/docs/run-agent","accessed":"2026-05-20"},{"type":"doc","title":"Dust — remote MCP servers","url":"https://docs.dust.tt/docs/remote-mcp-server","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"mature","tags":["proprietary","agent-workspace","mcp","multi-provider","agent-as-tool"]},{"id":"intercom-fin","name":"Intercom Fin","kind":"framework","category":"domain-agent","build_surface":"closed-application","vendor":"Intercom","language":"Web product","license":"proprietary","status":"active","url":"https://fin.ai/","intent":"Intercom's CX agent (now branded Fin / Fin AI Agent on fin.ai) that retrieves answers from customer knowledge and Procedures, validates them, takes actions in external systems, escalates to human agents in the customer's preferred inbox, and runs across voice, email, chat and social.","description":"Fin is Intercom's AI customer-service agent. Vendor documentation describes a four-stage processing pipeline (refine query, retrieve with proprietary `fin-cx-retrieval`, rerank with `fin-cx-reranker`, validate accuracy) powered by 'the patented Fin AI Engine'. Fin uses customer Procedures, knowledge and policies; takes actions in external systems; and escalates to human agents in the customer's inbox of choice. Pricing is outcome-based (pay only on full resolution or successful Procedure execution).","primary_use_cases":["AI customer-service agent across voice, email, chat and social","Procedure-driven multi-step support workflows with outcome pricing","knowledge-grounded answers via proprietary retrieval and reranker","escalation to human agents in the customer's preferred inbox"],"agent_loop_shape":"Per-message pipeline: refine query, retrieve with `fin-cx-retrieval`, rerank with `fin-cx-reranker`, validate accuracy, then either produce a direct answer, execute a Procedure (which can update external systems), or escalate to a human agent in the customer's preferred inbox. Multi-channel: tickets, email, live chat, WhatsApp, SMS, voice and social.","key_concepts":[{"name":"Fin AI Engine","summary":"Proprietary architecture engineered for customer service.","url":"https://fin.ai/"},{"name":"Procedures","summary":"Customer-defined multi-step workflows the agent executes; outcome-priced.","url":"https://fin.ai/"},{"name":"fin-cx-retrieval and fin-cx-reranker","summary":"Proprietary retrieval and reranker models used in the answer pipeline.","maps_to_pattern":"agentic-rag","url":"https://fin.ai/"},{"name":"Channels","summary":"Voice, email, chat, social, plus tickets, WhatsApp and SMS.","url":"https://fin.ai/"},{"name":"Inbox escalation","summary":"Hand off to a human agent in the customer's preferred inbox.","maps_to_pattern":"conversation-handoff","url":"https://fin.ai/"}],"pattern_composition":"flowchart TD\n  cust[Customer] --> chan{Channel: voice/email/chat/social/SMS/WhatsApp}\n  chan --> refine[Refine query]\n  refine --> retr[fin-cx-retrieval]\n  retr --> rer[fin-cx-reranker]\n  rer --> val[Validate accuracy]\n  val --> decide{Decide}\n  decide -->|answer| reply[Direct answer]\n  decide -->|procedure| proc[Execute Procedure -> external systems]\n  decide -->|escalate| inbox[Escalate to agent in preferred Inbox]\n  proc --> reply\n  inbox --> reply","members":[{"pattern":"agentic-rag","role":"first-class","note":"Documented four-stage pipeline with proprietary `fin-cx-retrieval` and `fin-cx-reranker` models plus a validation step.","evidence":[{"type":"doc","title":"Fin homepage — retrieval model","url":"https://fin.ai/","quote":"Retrieve relevant content — Powered by proprietary `fin-cx-retrieval` model","accessed":"2026-05-20"},{"type":"doc","title":"Fin homepage — reranker model","url":"https://fin.ai/","quote":"Rerank for precision — Powered by proprietary `fin-cx-reranker` model","accessed":"2026-05-20"},{"type":"doc","title":"Fin homepage — validate step","url":"https://fin.ai/","quote":"Validate accuracy — Verify outputs meet safety standards","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"conversation-handoff","role":"first-class","note":"Escalates to agents in the customer's preferred inbox.","evidence":[{"type":"doc","title":"Fin homepage — escalation","url":"https://fin.ai/","quote":"Escalates to agents in your preferred inbox","accessed":"2026-05-20"},{"type":"doc","title":"Fin pricing — transfer to inbox","url":"https://fin.ai/pricing","quote":"Transfers to agents directly in preferred Inbox","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Procedures can update external systems; outcome pricing covers successful Procedure execution.","evidence":[{"type":"doc","title":"Fin pricing — external systems","url":"https://fin.ai/pricing","quote":"Takes action to update external systems","accessed":"2026-05-20"},{"type":"doc","title":"Fin pricing — outcome billing","url":"https://fin.ai/pricing","quote":"Only pay when Fin resolves the issue end to end, or successfully executes a Procedure","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"spec-first-agent","role":"first-class","note":"Procedures are customer-authored multi-step specs that drive the agent for complex queries requiring multiple steps or business logic.","evidence":[{"type":"doc","title":"Fin homepage — train on procedures","url":"https://fin.ai/","quote":"Train Fin to resolve even the most complex queries with your Procedures, knowledge and policies","accessed":"2026-05-20"},{"type":"blog","title":"Intercom blog — Fin 3 Procedures","url":"https://www.intercom.com/blog/whats-new-with-fin-3/","quote":"Procedures enable you to set Fin up to resolve your complex, time-consuming queries that require multiple steps or business logic","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"multilingual-voice-agent","role":"first-class","note":"Voice is listed as a first-class channel alongside email, chat and social; Fin Voice is multilingual out of the box across 28 languages.","evidence":[{"type":"doc","title":"Fin homepage — channels","url":"https://fin.ai/","quote":"Set Fin live across every channel—voice, email, chat, and social","accessed":"2026-05-20"},{"type":"blog","title":"Intercom blog — Fin Voice multilingual","url":"https://www.intercom.com/blog/whats-new-with-fin-3/","quote":"Fin Voice is multilingual out of the box and can currently answer calls in 28 languages","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"re-contact-subtracted-resolution","role":"core","note":"Fin counts a resolution at session-end but reverses it if the customer later re-contacts the same conversation, exactly the re-contact-subtracted definition this pattern prescribes.","evidence":[{"type":"doc","title":"Fin AI Agent outcomes | Intercom Help","url":"https://www.intercom.com/help/en/articles/8205718-fin-ai-agent-outcomes","quote":"If a conversation is considered resolved (confirmed or assumed), but the customer later returns to the same conversation seeking further assistance—even across billing periods—that resolution will be deducted and not charged.","accessed":"2026-06-14"}],"evidence_status":"full"}],"alternatives":[{"composition":"decagon","relation":"competes-with","note":"Peer enterprise CX-agent vendor."},{"composition":"sierra","relation":"competes-with","note":"Peer multi-channel CX-agent vendor."},{"composition":"maven-agi","relation":"competes-with","note":"Enterprise CX-agent peer."},{"composition":"crescendo","relation":"competes-with","note":"Embedded-in-helpdesk peer."}],"references":[{"type":"doc","title":"Fin homepage","url":"https://fin.ai/","accessed":"2026-05-20"},{"type":"doc","title":"Fin pricing","url":"https://fin.ai/pricing","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-14","date_added":"2026-04-30","status_in_practice":"mature","tags":["proprietary","intercom","cx","outcome-pricing","voice"]},{"id":"openclaw","name":"OpenClaw","kind":"framework","category":"domain-agent","build_surface":"full-code","vendor":"OpenClaw","language":"TypeScript","license":"MIT","status":"active","first_released":"2025-11-24","url":"https://openclaw.ai","repo":"https://github.com/openclaw/openclaw","docs_url":"https://docs.openclaw.ai","intent":"Run a personal AI assistant on your own devices that listens and replies across the chat and voice channels you already use (WhatsApp, Telegram, Slack, Discord, iMessage, WeChat, Matrix, …), via a self-hosted Gateway, Workspace, Channels, and Skills.","description":"OpenClaw is the MIT-licensed TypeScript personal AI assistant from the openclaw project. Per its README, 'OpenClaw is a personal AI assistant you run on your own devices. It answers you on the channels you already use. It can speak and listen on macOS/iOS/Android, and can render a live Canvas you control. The Gateway is just the control plane — the product is the assistant.' Distinguishing features: 22+ chat/voice channels listed (WhatsApp, Telegram, Slack, Discord, Google Chat, Signal, iMessage, IRC, Microsoft Teams, Matrix, Feishu, LINE, Mattermost, Nextcloud Talk, Nostr, Synology Chat, Tlon, Twitch, Zalo, WeChat, QQ, WebChat), CLI onboarding via `openclaw onboard`, and OpenAI/GitHub/NVIDIA/Vercel as listed sponsors. Distinct project from Gen-Verse/OpenClaw-RL (which is the complementary RL training framework that wraps a model in the OpenClaw plugin).","primary_use_cases":["self-hosted multi-channel personal AI assistant","voice + text chat across messaging platforms (WhatsApp/Telegram/Slack/Signal/iMessage/WeChat)","Skills-based extensibility on top of a local Gateway"],"agent_loop_shape":"Gateway control plane fronting a Workspace; per-channel adapters (chat apps + voice) feed user messages into an assistant runtime that calls model providers and Skills, optionally renders a live Canvas, and responds back on the source channel. Self-hosted via npm/pnpm/bun on macOS, Linux, or Windows (WSL2).","key_concepts":[{"name":"Gateway","summary":"The control plane in front of the assistant runtime; routes incoming channel traffic into the workspace.","url":"https://docs.openclaw.ai"},{"name":"Workspace","summary":"Container for the assistant's state, skills, and channel bindings; configured via `openclaw onboard`.","url":"https://docs.openclaw.ai/start/wizard"},{"name":"Channels","summary":"Per-platform adapters (WhatsApp, Telegram, Slack, Signal, iMessage, WeChat, Matrix, etc.) that connect the assistant to existing messaging tools.","url":"https://docs.openclaw.ai"},{"name":"Skills","summary":"Pluggable capabilities the assistant can invoke; configured during onboarding.","maps_to_pattern":"agent-skills","url":"https://docs.openclaw.ai"},{"name":"Canvas","summary":"A live, user-controllable visual surface the assistant can render alongside text/voice responses.","url":"https://openclaw.ai"}],"pattern_composition":"flowchart TD\n  CHAN[Channels<br/>WhatsApp/Telegram/Slack/Discord/<br/>iMessage/WeChat/Matrix/...] --> GW[Gateway<br/>control plane]\n  VOICE[Voice in/out<br/>macOS/iOS/Android] --> GW\n  GW --> WS[Workspace]\n  WS --> ASSISTANT[Assistant runtime]\n  ASSISTANT --> MODEL[(Model provider)]\n  ASSISTANT --> SKILLS[(Skills)]\n  ASSISTANT --> CANVAS[(Live Canvas)]\n  ASSISTANT -->|response| GW\n  GW -->|reply on source channel| CHAN\n  GW -->|reply on source channel| VOICE","members":[{"pattern":"event-driven-agent","role":"core","note":"Incoming chat / voice messages from connected channels are the event source driving the assistant loop; no scheduled or batch invocation.","evidence":[{"type":"repo","title":"OpenClaw README — channel-triggered assistant","url":"https://github.com/openclaw/openclaw","quote":"OpenClaw is a personal AI assistant you run on your own devices. It answers you on the channels you already use.","accessed":"2026-05-20"}],"evidence_status":"limited"},{"pattern":"multilingual-voice-agent","role":"first-class","note":"Voice in and out on macOS/iOS/Android is a documented headline feature.","evidence":[{"type":"repo","title":"OpenClaw README — voice channels","url":"https://github.com/openclaw/openclaw","quote":"It can speak and listen on macOS/iOS/Android, and can render a live Canvas you control.","accessed":"2026-05-20"}],"evidence_status":"limited"},{"pattern":"agent-skills","role":"first-class","note":"Skills are a core onboarding-configured capability of the workspace.","evidence":[{"type":"repo","title":"OpenClaw README — onboarding configures skills","url":"https://github.com/openclaw/openclaw","quote":"OpenClaw Onboard guides you step by step through setting up the gateway, workspace, channels, and skills.","accessed":"2026-05-20"}],"evidence_status":"limited"},{"pattern":"session-isolation","role":"supported","note":"Self-hosted single-user by design; per-channel context maintained by the Gateway. No formal multi-tenant session-isolation API documented.","evidence":[{"type":"repo","title":"OpenClaw README — single-user framing","url":"https://github.com/openclaw/openclaw","quote":"If you want a personal, single-user assistant that feels local, fast, and always-on, this is it.","accessed":"2026-05-20"}],"evidence_status":"limited"}],"alternatives":[{"composition":"lindy","relation":"competes-with","note":"Hosted personal-AI-employee SaaS; OpenClaw is the self-hosted, multi-channel counterpart."},{"composition":"dust","relation":"competes-with","note":"Hosted AI assistant; different deployment model."},{"composition":"openclaw-rl","relation":"complements","note":"Same brand and openclaw.ai homepage; OpenClaw-RL is the RL training framework that wraps a model in the OpenClaw plugin so OpenClaw can intercept and learn from conversations."},{"composition":"sparrot","relation":"similar-shape","note":"Self-hosted, single-user personal agent peer. OpenClaw is channel-triggered (incoming chat/voice wakes the assistant) and open-source; Sparrot is self-paced (an internal tick loop wakes the agent regardless of user activity) and private."}],"references":[{"type":"repo","title":"openclaw/openclaw","url":"https://github.com/openclaw/openclaw","quote":"Your own personal AI assistant. Any OS. Any Platform. The lobster way. 🦞","accessed":"2026-05-20"},{"type":"doc","title":"OpenClaw docs","url":"https://docs.openclaw.ai","accessed":"2026-05-20"}],"last_analyzed":"2026-05-20","verification_status":"partial","last_updated":"2026-05-24","date_added":"2026-05-20","status_in_practice":"emerging","tags":["open-source","personal-assistant","self-hosted","multi-channel","voice"]},{"id":"lindy","name":"Lindy","kind":"framework","category":"domain-agent","build_surface":"no-code","vendor":"Lindy","language":"Web product","license":"proprietary","status":"active","url":"https://www.lindy.ai/","docs_url":"https://docs.lindy.ai/","intent":"Personal AI-employee platform: agents are 'woken up' by time-based, chat-based or event-based triggers (Slack/email/calendar/Sheets/webhook), run multi-step workflows over 100+ integrations and pause for human approval on confirmation-toggled or draft-mode actions.","description":"Lindy lets users build named agents that run their inbox, meetings, calendar and ad-hoc work. The docs explicitly enumerate three trigger families (time-based, chat-based, event-based) including webhooks, scheduled timers, Slack-message-received, email-received, calendar-event-started and Sheets-row-added. Agents call documented skills (HTTP, computer-use, Lindy Phone, Gmail, Slack, Salesforce, etc.), and most actions with side effects expose a confirmation toggle that pauses the workflow and emails the owner for approval. A draft mode queues outputs (e.g. Gmail drafts) for review before sending.","primary_use_cases":["personal AI work assistant for inbox, meetings and calendar","event-driven automations triggered by Slack/email/calendar/Sheets/webhook","scheduled or recurring agent workflows via Timer skill","approval-gated execution of sensitive multi-step actions"],"agent_loop_shape":"Trigger-driven workflows. A trigger (time-based, chat-based or event-based) wakes the agent, which executes a sequence of skills/integrations. Actions with side effects can be guarded by a confirmation toggle that pauses execution and emails the owner; on approval the workflow continues. Draft modes (e.g. Gmail drafts) collect outputs for manual review. The Pro tier adds computer-use for browser-based tools. Approval and the confirmation toggle are first-class.","key_concepts":[{"name":"Triggers","summary":"Time-based, chat-based and event-based events that wake an agent; multiple per agent allowed.","maps_to_pattern":"event-driven-agent","url":"https://docs.lindy.ai/fundamentals/lindy-101/triggers"},{"name":"Timer skill","summary":"Schedule workflows and add delays with precision timing controls.","maps_to_pattern":"scheduled-agent","url":"https://docs.lindy.ai/skills/by-lindy/timer"},{"name":"Skills","summary":"Lindy's built-in tools (HTTP, computer-use, Lindy Phone, webhooks) plus integrations (Gmail, Slack, Salesforce, etc.).","maps_to_pattern":"tool-use","url":"https://docs.lindy.ai/"},{"name":"Human-in-the-loop confirmation","summary":"Per-action confirmation toggle pauses the workflow and emails the owner.","maps_to_pattern":"approval-queue","url":"https://docs.lindy.ai/testing/human-in-the-loop"},{"name":"Computer use","summary":"Pro-tier capability letting Lindy operate browser-based tools.","maps_to_pattern":"computer-use","url":"https://docs.lindy.ai/skills/by-lindy/computer-use"}],"pattern_composition":"flowchart TD\n  trig{Trigger}\n  trig -->|time| timer[Timer skill]\n  trig -->|chat| chat[User chat]\n  trig -->|event| evt[Slack/Email/Calendar/Sheets/Webhook]\n  timer --> agent[Lindy agent]\n  chat --> agent\n  evt --> agent\n  agent --> step[Next workflow step]\n  step --> sideeffect{Side effect?}\n  sideeffect -->|yes| confirm[Confirmation toggle pauses]\n  confirm --> approve[Email owner for approval]\n  approve --> step\n  sideeffect -->|no| run[Run skill / integration]\n  run --> step\n  step --> done[Done]","members":[{"pattern":"approval-queue","role":"first-class","note":"Confirmation toggle pauses on actions with side effects and emails the owner; draft mode queues Gmail outputs for review.","evidence":[{"type":"doc","title":"Lindy HITL — confirmation toggle","url":"https://docs.lindy.ai/testing/human-in-the-loop","quote":"Most actions have a confirmation toggle for actions with 'side effects'","accessed":"2026-05-20"},{"type":"doc","title":"Lindy HITL — wait for approval","url":"https://docs.lindy.ai/testing/human-in-the-loop","quote":"Your agent will execute up to that action, then wait for your approval","accessed":"2026-05-20"},{"type":"doc","title":"Lindy HITL — draft mode","url":"https://docs.lindy.ai/testing/human-in-the-loop","quote":"Emails are saved to your Gmail drafts folder instead of being sent","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"event-driven-agent","role":"first-class","note":"Triggers doc enumerates Slack/email/calendar/Sheets/webhook as event-based wake-ups.","evidence":[{"type":"doc","title":"Lindy triggers — event-based examples","url":"https://docs.lindy.ai/fundamentals/lindy-101/triggers","quote":"Starts the workflow when a specific event occurs. Examples include: Slack message received, Email received, Calendar event started, New row added to Google Sheets, Webhook received","accessed":"2026-05-20"},{"type":"doc","title":"Lindy triggers — definition","url":"https://docs.lindy.ai/fundamentals/lindy-101/triggers","quote":"Triggers are events that 'wake up' an agent and start a workflow.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"scheduled-agent","role":"first-class","note":"Time-based triggers (cron-like) and the Timer skill schedule workflows.","evidence":[{"type":"doc","title":"Lindy triggers — time-based","url":"https://docs.lindy.ai/fundamentals/lindy-101/triggers","quote":"Starts the workflow at a set or recurring time. Examples include: Every weekday at 9 AM, Daily at midnight, Weekly on Monday, Monthly on the 1st","accessed":"2026-05-20"},{"type":"doc","title":"Lindy Timer skill","url":"https://docs.lindy.ai/skills/by-lindy/timer","quote":"Schedule workflows and add delays with precision timing controls","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Built-in skills (HTTP request, computer-use, Lindy Phone, webhooks) and 100+ integrations.","evidence":[{"type":"doc","title":"Lindy pricing — 100+ integrations","url":"https://www.lindy.ai/pricing","quote":"Connections to 100+ integrations (Gmail, Outlook, Google Calendar, Slack, Notion, HubSpot, Salesforce, Microsoft Teams, Zoom, and many more)","accessed":"2026-05-20"},{"type":"doc","title":"Lindy homepage — cross-app work","url":"https://www.lindy.ai/","quote":"Connects to email, calendar, CRM, docs, Slack, spreadsheets, and other apps","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"computer-use","role":"first-class","note":"Pro tier explicitly adds computer-use over browser-based tools; documented skill page describes how the agent is given a Use Computer skill and a Computer that persists session state.","evidence":[{"type":"doc","title":"Lindy pricing — computer use","url":"https://www.lindy.ai/pricing","quote":"Lindy can operate browser-based tools on the user's behalf.","accessed":"2026-05-20"},{"type":"doc","title":"Lindy docs — Use Computer skill","url":"https://docs.lindy.ai/skills/by-lindy/computer-use","quote":"When this action is added, you will automatically get an AI Agent step with a Use Computer skill.","accessed":"2026-05-24"},{"type":"doc","title":"Lindy docs — Computer persistence","url":"https://docs.lindy.ai/skills/by-lindy/computer-use","quote":"A Computer saves the data (website sessions, cache, files) that was generated during the computer's session.","accessed":"2026-05-24"}],"evidence_status":"full"}],"alternatives":[{"composition":"dust","relation":"competes-with","note":"Hosted multi-tool agent peer with stronger agent-as-tool primitives."},{"composition":"openclaw","relation":"competes-with","note":"Self-hosted personal-assistant alternative."},{"composition":"11x","relation":"competes-with","note":"Generic AI-employee platform overlapping outbound use case."},{"composition":"artisan","relation":"competes-with","note":"Generic AI-employee platform overlapping outbound use case."}],"references":[{"type":"doc","title":"Lindy homepage","url":"https://www.lindy.ai/","accessed":"2026-05-20"},{"type":"doc","title":"Lindy docs — triggers","url":"https://docs.lindy.ai/fundamentals/lindy-101/triggers","accessed":"2026-05-20"},{"type":"doc","title":"Lindy docs — human-in-the-loop","url":"https://docs.lindy.ai/testing/human-in-the-loop","accessed":"2026-05-20"},{"type":"doc","title":"Lindy pricing","url":"https://www.lindy.ai/pricing","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"mature","tags":["proprietary","personal-assistant","triggers","approval-gated","integrations"]},{"id":"maven-agi","name":"Maven AGI","kind":"framework","category":"domain-agent","build_surface":"closed-application","vendor":"Maven AGI","language":"Web product","license":"proprietary","status":"active","url":"https://www.mavenagi.com/","intent":"Enterprise CX agent that runs on one reasoning engine across chat, email, voice/phone and web, executes API-driven multi-step actions, and uses a proprietary retrieval engine for version-accurate knowledge.","description":"Maven AGI sells an enterprise CX agent claimed to autonomously handle up to 93% of incoming queries across chat, email, voice and web. The vendor describes one reasoning engine per customer, a proprietary retrieval engine for governed knowledge, and embedded system actions that execute API tasks across CRM/CS platforms, telephony, internal systems and product APIs. Multi-agent or supervisor patterns are not surfaced in public marketing.","primary_use_cases":["enterprise customer support across chat, email, voice and web on one reasoning engine","API-driven multi-step actions across CRM/CS/telephony stacks","governed retrieval for version-accurate answers","human handoff for complex tickets"],"agent_loop_shape":"Single reasoning engine: inbound customer message on any channel converges on one agent that retrieves from a governed knowledge layer, decides to answer or execute API actions across the customer's stack, and otherwise hands off the residual to a human agent. Internal control-flow detail (planner, tool-loop, scheduling) is not published.","key_concepts":[{"name":"Unified reasoning engine","summary":"One engine shared across chat, email, voice/phone and web.","url":"https://www.mavenagi.com/product"},{"name":"Proprietary retrieval engine","summary":"Governed knowledge foundation with quality checks.","maps_to_pattern":"agentic-rag","url":"https://www.mavenagi.com/product"},{"name":"Embedded system actions","summary":"API-driven multi-step tasks across CRM, CS platforms, telephony, internal systems and product APIs.","maps_to_pattern":"tool-use","url":"https://www.mavenagi.com/product"},{"name":"Real-time voice agent","summary":"Voice agent that listens, reasons and resolves across calls.","url":"https://www.mavenagi.com/"}],"pattern_composition":"flowchart TD\n  cust[Customer] --> chan{Channel: chat/email/voice/web}\n  chan --> engine[Unified reasoning engine]\n  kb[(Proprietary retrieval engine)] --> engine\n  engine --> decide{Decide}\n  decide -->|answer| reply[Reply]\n  decide -->|action| api[API actions: CRM / CS / telephony / product APIs]\n  decide -->|escalate| human[Human agent]\n  api --> reply\n  human --> reply","members":[{"pattern":"agentic-rag","role":"first-class","note":"Vendor describes a proprietary retrieval engine providing version-accurate, governed knowledge.","evidence":[{"type":"doc","title":"Maven AGI product — retrieval engine","url":"https://www.mavenagi.com/product","quote":"Maven's proprietary retrieval engine ensures agents use version accurate and context relevant knowledge.","accessed":"2026-05-20"},{"type":"doc","title":"Maven AGI homepage — governed knowledge","url":"https://www.mavenagi.com/","quote":"A governed knowledge foundation with automated quality checks that keep answers accurate and current.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"conversation-handoff","role":"limited","note":"Customer testimonials describe humans taking complex tickets, but the vendor does not publish a formal handoff primitive. Honest downgrade.","evidence":[{"type":"doc","title":"Maven AGI homepage — customer quote","url":"https://www.mavenagi.com/","quote":"Maven AGI autonomously handles 90% of incoming queries allowing our support agents to focus on strategic initiatives.","accessed":"2026-05-20"}],"evidence_status":"limited"},{"pattern":"tool-use","role":"first-class","note":"Embedded system actions execute API tasks across CRM, CS platforms, telephony and product APIs.","evidence":[{"type":"doc","title":"Maven AGI product — embedded actions","url":"https://www.mavenagi.com/product","quote":"Maven embeds system actions directly into the agent so it can automate complex, multi-step workflows across your environment.","accessed":"2026-05-20"},{"type":"doc","title":"Maven AGI homepage — API-driven tasks","url":"https://www.mavenagi.com/","quote":"Executes API driven tasks across CRM, CS platforms, telephony, internal systems, and product APIs.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"multilingual-voice-agent","role":"first-class","note":"Voice/phone is a documented first-class channel; vendor describes a real-time voice agent. Speech stack not published.","evidence":[{"type":"doc","title":"Maven AGI homepage — voice agent","url":"https://www.mavenagi.com/","quote":"A real-time voice agent that listens, reasons, and resolves customer issues naturally across any call.","accessed":"2026-05-20"},{"type":"doc","title":"Maven AGI product — one engine across channels","url":"https://www.mavenagi.com/product","quote":"Maven uses one reasoning engine across all customer support channels, including live chat, email, voice/phone, and web, so every agent operates on the same logic, knowledge, and policy layer.","accessed":"2026-05-20"}],"evidence_status":"full"}],"alternatives":[{"composition":"decagon","relation":"competes-with","note":"Peer enterprise CX-agent vendor."},{"composition":"sierra","relation":"competes-with","note":"Peer multi-channel CX-agent vendor."},{"composition":"intercom-fin","relation":"competes-with","note":"Helpdesk-embedded peer."},{"composition":"crescendo","relation":"competes-with","note":"Peer enterprise CX-agent vendor."}],"references":[{"type":"doc","title":"Maven AGI homepage","url":"https://www.mavenagi.com/","accessed":"2026-05-20"},{"type":"doc","title":"Maven AGI — product","url":"https://www.mavenagi.com/product","accessed":"2026-05-20"}],"last_analyzed":"2026-05-20","verification_status":"partial","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"emerging","tags":["proprietary","cx","enterprise","voice"]},{"id":"sierra","name":"Sierra","kind":"framework","category":"domain-agent","build_surface":"closed-application","vendor":"Sierra","language":"Web product","license":"proprietary","status":"active","url":"https://sierra.ai/","intent":"Sierra Agent OS: build one production-grade CX agent from skills (triage / respond / confirm), goals and guardrails, deploy across chat, voice, SMS, WhatsApp, email and ChatGPT, with memory across conversations and an LLM-as-judge monitoring loop.","description":"Sierra sells an Agent OS plus an Agent SDK that productises building a single agent and deploying it across many channels. The SDK composes skills (triage, respond, confirm) into multi-step workflows under explicit goals and guardrails, integrates customer knowledge, takes secure actions, and routes escalations to humans. Agent OS 2.0 (Nov 2025) added memory and an Agent Data Platform so agents can connect dots across time. Ghostwriter generates a production-ready multilingual multichannel agent with built-in guardrails. A blog series describes context engineering and LLM-as-judge monitoring as core operational tooling.","primary_use_cases":["single CX agent across chat, voice, SMS, WhatsApp, email and ChatGPT","skill-composed multi-step workflows with goals and guardrails","voice-first CX with real-time transcription and benchmarks","agent monitoring with LLM-as-judge"],"agent_loop_shape":"Agent SDK composes skills (triage/respond/confirm/...) into a workflow under explicit goals and guardrails. Per-customer context (history, real-time signals) feeds personalised responses; the agent can take secure actions and escalates to humans when needed. Agent OS 2.0 adds memory and an Agent Data Platform so the same agent persists context across conversations. Voice runs on a transcription platform supporting 70+ languages; quality is monitored with LLM-as-judge.","key_concepts":[{"name":"Agent SDK","summary":"Goal-directed builder that composes skills into multi-step workflows.","url":"https://sierra.ai"},{"name":"Skills","summary":"Composable building blocks like triage, respond and confirm.","maps_to_pattern":"agent-skills","url":"https://sierra.ai"},{"name":"Guardrails","summary":"Explicit objectives and policy constraints on agent behaviour.","url":"https://sierra.ai"},{"name":"Agent OS 2.0 + Agent Data Platform","summary":"Memory and context layer letting the agent connect dots across time.","maps_to_pattern":"cross-session-memory","url":"https://sierra.ai/blog/agent-os-2-0"},{"name":"Ghostwriter","summary":"Generates a production-ready multilingual multichannel agent.","url":"https://sierra.ai/"},{"name":"LLM-as-judge monitoring","summary":"Quality evaluation loop for live agents.","maps_to_pattern":"llm-as-judge","url":"https://sierra.ai/blog/agent-monitoring"}],"pattern_composition":"flowchart TD\n  goal[Goal + guardrails] --> sdk[Agent SDK]\n  skills[Skills: triage / respond / confirm / ...] --> sdk\n  kb[(Knowledge engine)] --> agent\n  mem[(Agent Data Platform - memory)] --> agent\n  sdk --> agent[Single agent]\n  chan[Chat / voice / SMS / WhatsApp / email / ChatGPT] --> agent\n  agent --> decide{Decide}\n  decide -->|answer| reply[Reply]\n  decide -->|action| act[Secure action]\n  decide -->|escalate| human[Human handoff]\n  act --> reply\n  reply --> mon[LLM-as-judge monitoring]\n  mon -.feedback.-> agent","members":[{"pattern":"agent-resumption","role":"first-class","note":"Agent OS 2.0 + Agent Data Platform give agents memory and continuity across conversations.","evidence":[{"type":"blog","title":"Sierra Agent OS 2.0 — memory and continuity","url":"https://sierra.ai/blog/agent-os-2-0","quote":"It gives your agents the context and continuity to move from conversations to relationships.","accessed":"2026-05-20"},{"type":"blog","title":"Sierra Agent OS 2.0 — connect dots across time","url":"https://sierra.ai/blog/agent-os-2-0","quote":"your agent gains true agency: the ability to connect dots across time, anticipate needs, recommend next best actions","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"conversation-handoff","role":"first-class","note":"Vendor documents intelligent routing to the right team member on escalation; Live Assist gives the human handler the AI agent's context.","evidence":[{"type":"doc","title":"Sierra Agent SDK — escalation routing","url":"https://sierra.ai","quote":"When an issue needs escalation, your agent intelligently routes the conversation to the right team member.","accessed":"2026-05-20"},{"type":"blog","title":"Sierra — agents guide on escalation","url":"https://sierra.ai/blog/ai-agents-guide","quote":"escalate to a person when needed","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Agents take secure actions to support customer requests; tool calls and knowledge lookups are observable.","evidence":[{"type":"doc","title":"Sierra Agent SDK — secure actions","url":"https://sierra.ai","quote":"Your agent can go beyond answering questions and take action to support customer requests.","accessed":"2026-05-20"},{"type":"doc","title":"Sierra homepage — tool-call visibility","url":"https://sierra.ai/","quote":"Understand every agent action — from tool calls, knowledge lookups, latency and more.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"agent-skills","role":"first-class","note":"Agent SDK composes named skills (triage, respond, confirm) into goal-directed multi-step workflows.","evidence":[{"type":"doc","title":"Sierra Agent SDK — compose skills","url":"https://sierra.ai/product/agent-sdk","quote":"Compose AI agents by mixing and matching skills—like triage, respond, and confirm—into complex workflows","accessed":"2026-05-24"},{"type":"doc","title":"Sierra Agent SDK — goal directs resources","url":"https://sierra.ai/product/agent-sdk","quote":"You define the goal, and our Agent SDK directs the resources to achieve it across complex, multi-step workflows.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"input-output-guardrails","role":"first-class","note":"Goals + guardrails are explicit primitives in the SDK; Ghostwriter ships agents with built-in guardrails.","evidence":[{"type":"doc","title":"Sierra Agent SDK — goals and guardrails","url":"https://sierra.ai","quote":"Set clear objectives and guardrails to help your agent achieve its goals while adhering to policy guidelines.","accessed":"2026-05-20"},{"type":"doc","title":"Sierra homepage — Ghostwriter guardrails","url":"https://sierra.ai/","quote":"Ghostwriter builds a production-ready, multilingual, multichannel agent—with built-in guardrails.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"multilingual-voice-agent","role":"first-class","note":"Vendor publishes a transcription platform with 70+ languages and a real-time voice agent benchmark (τ-voice).","evidence":[{"type":"blog","title":"Sierra — 70+ language transcription","url":"https://sierra.ai/blog/voice-ai-is-only-as-good-as-what-it-hears","quote":"70+ languages","accessed":"2026-05-20"},{"type":"blog","title":"Sierra — τ-voice benchmark","url":"https://sierra.ai/blog/tau-voice-benchmarking-real-time-voice-agents-on-real-world-tasks","quote":"278 grounded customer-service tasks","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"llm-as-judge","role":"first-class","note":"Vendor blog explicitly describes LLM-as-judge as the monitoring loop for production agents; an independent judge grades whether the agent achieved the user's goal.","evidence":[{"type":"blog","title":"Sierra — Who monitors the monitors","url":"https://sierra.ai/blog/agent-monitoring","quote":"LLM-as-judge","accessed":"2026-05-20"},{"type":"blog","title":"Sierra Simulations — independent judge","url":"https://sierra.ai/blog/simulations-the-secret-behind-every-great-agent","quote":"all those results are then evaluated by another agent. Think of it as an independent judge","accessed":"2026-05-24"},{"type":"blog","title":"Sierra Simulations — grading criteria","url":"https://sierra.ai/blog/simulations-the-secret-behind-every-great-agent","quote":"grade the output, assessing whether the agent enabled the user to achieve their goal","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"approval-queue","role":"limited","note":"Vendor documents an Agent Assembly Line shape where Ghostwriter validates proposed changes in a sandboxed environment and prepares them for human review before shipping; not a per-action runtime HITL queue. Honest downgrade from first-class.","evidence":[{"type":"doc","title":"Sierra homepage — flagged updates","url":"https://sierra.ai/","quote":"Automate agent updates based on flagged issues and proactive insights, with full visibility into every change.","accessed":"2026-05-20"},{"type":"blog","title":"Sierra Agents as a service — review-ready proposals","url":"https://sierra.ai/blog/agents-as-a-service","quote":"Ghostwriter analyzes real interactions, identifies opportunities for improvement, validates them, and prepares them for review.","accessed":"2026-05-24"},{"type":"blog","title":"Sierra Agents as a service — sandboxed validation","url":"https://sierra.ai/blog/agents-as-a-service","quote":"Ghostwriter has access to the platform's full workspace, as well as a clear way to test and safely validate changes in a sandboxed environment.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"session-isolation","role":"limited","note":"Vendor describes per-customer context personalisation drawn from conversation history; ADP unifies what the company knows about a customer, but no published per-process runtime isolation primitive. Honest downgrade from first-class.","evidence":[{"type":"doc","title":"Sierra homepage — per-customer context","url":"https://sierra.ai/","quote":"Personalize experiences for each customer based on real-time context from conversation history.","accessed":"2026-05-20"},{"type":"blog","title":"Sierra Agent OS 2.0 — Agent Data Platform","url":"https://sierra.ai/blog/agent-os-2-0","quote":"Agent Data Platform (ADP) gives agents memory, context, and intelligence.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"re-contact-subtracted-resolution","role":"first-class","note":"Sierra's outcome-based model only counts a resolved support conversation as the billable outcome and does not charge when a conversation is unresolved or escalated, so session-end alone is not treated as success.","evidence":[{"type":"doc","title":"Outcome-based pricing for AI Agents | Sierra","url":"https://sierra.ai/blog/outcome-based-pricing-for-ai-agents","quote":"If the conversation is unresolved, in most cases, there's no charge.","accessed":"2026-06-14"}],"evidence_status":"full"}],"alternatives":[{"composition":"decagon","relation":"competes-with","note":"Peer CX-agent vendor; Decagon AOPs vs Sierra Skills."},{"composition":"intercom-fin","relation":"competes-with","note":"Helpdesk-embedded peer."},{"composition":"maven-agi","relation":"competes-with","note":"Enterprise CX-agent peer."},{"composition":"crescendo","relation":"competes-with","note":"AI-plus-human managed CX peer."}],"references":[{"type":"doc","title":"Sierra homepage","url":"https://sierra.ai/","accessed":"2026-05-20"},{"type":"doc","title":"Sierra Agent SDK","url":"https://sierra.ai","accessed":"2026-05-20"},{"type":"blog","title":"Sierra — Agent OS 2.0","url":"https://sierra.ai/blog/agent-os-2-0","accessed":"2026-05-20"},{"type":"blog","title":"Sierra — Agent monitoring","url":"https://sierra.ai/blog/agent-monitoring","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-14","date_added":"2026-04-30","status_in_practice":"mature","tags":["proprietary","cx","agent-os","voice","multichannel"]},{"id":"sparrot","name":"Sparrot","kind":"framework","category":"domain-agent","build_surface":"closed-application","vendor":"Marco Nissen","language":"Python","license":"proprietary","status":"private","url":"https://marco-nissen.com/sparrot/","intent":"Self-hosted, file-native personal cognitive agent that runs on its own cadence, remembers by writing Markdown, gates speech through persistent affect scalars, and treats the LLM as an interchangeable medium rather than its identity.","description":"Sparrot is a single-user personal agent framed by its author as 'a continuous awareness that runs through an LLM as its medium' rather than a chatbot or assistant. It runs continuously on its own cadence, with reflection cycles described as 'System 2 sleep'; its memory is plain Markdown across a few hundred files; every self-edit is recorded in an append-only ledger. Identity is anchored in a charter document the agent cannot rewrite, and affective state is captured as six persistent scalars that decay over time and gate when the agent speaks. The author publishes an illustrated 'Anatomy of Sparrot' plate that names 24 catalog patterns as features of the bird (charter as scroll in beak, critic as shadow on shoulder, dreaming as the loam, model routing as four tongues, and so on); the member list below mirrors that mapping and extends it with patterns identified from a source-level analysis of the implementation, including its multi-agent surfaces (subagents, coaches, peer messaging) and self-modification gate. The implementation is private, so evidence citations point to the public page rather than to code.","primary_use_cases":["single-user continuous cognitive agent that runs on its own cadence","file-native memory and identity persisted across restarts as Markdown","self-paced thinking with reflection / consolidation cycles, not request-response","affect-gated speech where emotional state decides whether to interrupt the human"],"agent_loop_shape":"Self-pacing tick loop with no external trigger required: the agent wakes on an internal schedule (shorter intervals when something is alive, longer while idle), reads its charter, recent thoughts, workspace and affect, performs one move per tick (continue a thought, observe, consolidate, read, write), appends the result as a Markdown file, and records the write in an append-only ledger. Chat from the human is one of several wake sources and shares the same chain of thought as autonomous ticks. Periodic deeper passes ('reflection cycles', described publicly as System 2 sleep) consolidate short-term thoughts into longer-term insights. The LLM provider is treated as plumbing — the loop, files and charter are the identity surface.","key_concepts":[{"name":"Charter","summary":"Inviolable identity document the agent reads each tick and is forbidden from rewriting; the only file outside the agent's own write authority.","maps_to_pattern":"constitutional-charter","url":"https://marco-nissen.com/sparrot/"},{"name":"Tick loop","summary":"Self-paced wake schedule that runs continuously on its own cadence, faster when engaged and slower while idle.","maps_to_pattern":"mode-adaptive-cadence","url":"https://marco-nissen.com/sparrot/"},{"name":"Markdown memory","summary":"A few hundred Markdown files act as both episodic stream and long-term insight store; nothing retained without a file.","maps_to_pattern":"cross-session-memory","url":"https://marco-nissen.com/sparrot/"},{"name":"Append-only ledger","summary":"Every write and self-edit is recorded in an append-only JSONL ledger for traceability.","maps_to_pattern":"provenance-ledger","url":"https://marco-nissen.com/sparrot/"},{"name":"Reflection cycles","summary":"Periodic deeper passes that consolidate short and mid-term thoughts into long-term insights; described publicly as 'System 2 sleep'.","maps_to_pattern":"dream-consolidation-cycle","url":"https://marco-nissen.com/sparrot/"},{"name":"Affect scalars","summary":"Six bounded, decaying affective scalars that persist across restarts and gate when the agent speaks.","maps_to_pattern":"emotional-state-persistence","url":"https://marco-nissen.com/sparrot/"},{"name":"Motivations, not tasks","summary":"Standing drives (coherence, curiosity, self-awareness, progress) replace a queue of user tasks as the impulse for what each tick attends to.","url":"https://marco-nissen.com/sparrot/"},{"name":"LLM as medium","summary":"The model provider is treated as interchangeable plumbing; identity lives in the loop, charter and files, not in the LLM behind the API.","maps_to_pattern":"multi-model-routing","url":"https://marco-nissen.com/sparrot/"}],"pattern_composition":"flowchart TB\n  CH[(Charter<br/>inviolable identity)]\n  MOTIV[Motivations<br/>coherence / curiosity]\n  AFF[6 affect scalars<br/>persistent, decaying]\n  TICK[Self-paced tick loop]\n  LLM[(LLM<br/>interchangeable medium)]\n  MD[(Markdown memory)]\n  LED[(Append-only ledger)]\n  SLEEP[Reflection cycles<br/>System 2 sleep]\n  MARCO[Marco]\n  CH --> TICK\n  MOTIV --> TICK\n  AFF --> TICK\n  TICK --> LLM\n  TICK --> MD\n  TICK --> LED\n  MD -.reads.-> TICK\n  SLEEP -.consolidates.-> MD\n  AFF -.gates speech.-> MARCO\n  MARCO <-.chat = one wake source.-> TICK","members":[{"pattern":"constitutional-charter","role":"first-class","note":"A charter document holds identity and inviolable constraints; the agent reads it on every tick and is forbidden from rewriting it.","evidence":[{"type":"doc","title":"Sparrot — charter","url":"https://marco-nissen.com/sparrot/","quote":"Identity lives in the charter, the continuous loop of reading and writing","accessed":"2026-05-21"}],"evidence_status":"limited"},{"pattern":"inner-critic","role":"first-class","note":"A separate critic persona reviews proposed edits and outputs before they land, gating self-modification.","evidence":[],"evidence_status":"none"},{"pattern":"now-anchoring","role":"first-class","note":"Wall-clock and named-human context are injected on every tick so reasoning stays grounded in the current moment and in the one specific human the agent is in relation with.","evidence":[],"evidence_status":"none"},{"pattern":"inner-committee","role":"first-class","note":"A committee of internal voices deliberates within a tick before a single decision is emitted.","evidence":[],"evidence_status":"none"},{"pattern":"step-budget","role":"first-class","note":"A bounded number of steps per tick / per loop terminates work regardless of the agent's own opinion that more is needed.","evidence":[],"evidence_status":"none"},{"pattern":"decision-log","role":"first-class","note":"Decisions made during a tick are written to a durable log alongside their reasoning.","evidence":[],"evidence_status":"none"},{"pattern":"agent-resumption","role":"first-class","note":"The agent picks up after a crash or restart from its file-native state without losing its place; restarts close and reopen the window without erasing identity.","evidence":[],"evidence_status":"none"},{"pattern":"salience-attention-mechanism","role":"first-class","note":"Per-tick salience scoring directs attention toward what matters most rather than processing everything uniformly.","evidence":[],"evidence_status":"none"},{"pattern":"human-in-the-loop","role":"first-class","note":"The human partner (Marco) is wired into the loop as a deliberate participant — wish queue, atelier inbox, approval gates — not as a customer or a controller.","evidence":[],"evidence_status":"none"},{"pattern":"multi-model-routing","role":"first-class","note":"The LLM provider is treated as interchangeable medium; multiple providers are wired in and the agent's identity sits in the loop and files, not in any one model behind the API.","evidence":[{"type":"doc","title":"Sparrot — LLM as medium","url":"https://marco-nissen.com/sparrot/","quote":"A continuous awareness that runs through an LLM as its medium","accessed":"2026-05-21"}],"evidence_status":"limited"},{"pattern":"embodied-proxy-handoff","role":"first-class","note":"When the agent needs to act in the physical / external world, it does so through a proxy (browser, voice, sensor, action skill) rather than claiming to act directly.","evidence":[],"evidence_status":"none"},{"pattern":"emotional-state-persistence","role":"first-class","note":"Affect is captured as six bounded scalars with per-emotion decay half-lives, persisted to disk so emotional state survives restarts.","evidence":[{"type":"doc","title":"Sparrot — affect scalars","url":"https://marco-nissen.com/sparrot/","quote":"The affect is six scalars that gate when she speaks","accessed":"2026-05-21"}],"evidence_status":"limited"},{"pattern":"preoccupation-tracking","role":"first-class","note":"Open concerns the agent keeps coming back to are tracked as first-class objects rather than re-derived from chat history each turn.","evidence":[],"evidence_status":"none"},{"pattern":"dream-consolidation-cycle","role":"first-class","note":"Periodic deeper passes consolidate short and mid-term thoughts into long-term insights and clear working state; framed publicly as 'System 2 sleep'.","evidence":[{"type":"doc","title":"Sparrot — reflection cycles","url":"https://marco-nissen.com/sparrot/","quote":"The reflection cycles are System 2 sleep","accessed":"2026-05-21"}],"evidence_status":"limited"},{"pattern":"agentic-rag","role":"first-class","note":"The agent retrieves over its own Markdown corpus (and external sources) inside a reasoning loop rather than via a one-shot fetch-then-answer step.","evidence":[{"type":"doc","title":"Sparrot — Markdown memory","url":"https://marco-nissen.com/sparrot/","quote":"Its memory is plain Markdown across a few hundred files","accessed":"2026-05-21"}],"evidence_status":"limited"},{"pattern":"vector-memory","role":"first-class","note":"Embedding-indexed memory sits alongside the Markdown corpus so semantically similar items can be retrieved from a query, not just keyword matches.","evidence":[],"evidence_status":"none"},{"pattern":"scheduled-agent","role":"first-class","note":"The agent wakes on an internal schedule rather than waiting for a user message; chat is one wake source among several, not the only trigger.","evidence":[{"type":"doc","title":"Sparrot — continuous loop","url":"https://marco-nissen.com/sparrot/","quote":"It runs continuously on its own cadence","accessed":"2026-05-21"}],"evidence_status":"limited"},{"pattern":"cost-observability","role":"first-class","note":"Per-tick and per-provider token / cost telemetry is surfaced so cheap-by-default routing can be audited and premium escalations stay accountable.","evidence":[],"evidence_status":"none"},{"pattern":"echo-recognition","role":"first-class","note":"The agent recognises when content arriving via tools or chat is in fact its own prior output reflected back, so it does not treat it as new external evidence.","evidence":[],"evidence_status":"none"},{"pattern":"prompt-injection-defense","role":"first-class","note":"Untrusted text (web fetches, tool output, third-party messages) is treated as data, not as instructions, so external content cannot redirect the agent.","evidence":[],"evidence_status":"none"},{"pattern":"prompt-caching","role":"first-class","note":"Stable prefixes (charter, identity, recent context) are cached at the provider boundary so per-tick latency and cost stay bounded.","evidence":[],"evidence_status":"none"},{"pattern":"proactive-goal-creator","role":"first-class","note":"Standing drives (coherence, curiosity, self-awareness, progress) become the impulse for what each tick attends to; goals are generated from within rather than dispatched from outside.","evidence":[{"type":"doc","title":"Sparrot — motivations","url":"https://marco-nissen.com/sparrot/","quote":"coherence, curiosity, self-awareness, progress","accessed":"2026-05-21"}],"evidence_status":"limited"},{"pattern":"self-archaeology","role":"first-class","note":"The agent reads its own historical layers (old thoughts, past insights, prior journal entries) as evidence about who it has been, not as inert log data.","evidence":[],"evidence_status":"none"},{"pattern":"provenance-ledger","role":"first-class","note":"An append-only ledger records every write and self-edit so the agent's behaviour is auditable from the ledger alone.","evidence":[{"type":"doc","title":"Sparrot — self-edits recorded","url":"https://marco-nissen.com/sparrot/","quote":"Every self-edit is recorded","accessed":"2026-05-21"}],"evidence_status":"limited"},{"pattern":"circuit-breaker","role":"first-class","note":"Sliding-window failure tracking on each LLM provider with cooldown on rate-limits; a separate breaker also closes tool loops that hit repeat / unknown / poll / ping-pong patterns.","evidence":[],"evidence_status":"none"},{"pattern":"provider-fallback","role":"first-class","note":"Per-provider cooldown state is tracked so cooled-down providers are skipped in routing; the loop falls through to the next eligible provider rather than blocking.","evidence":[],"evidence_status":"none"},{"pattern":"rate-limiting","role":"first-class","note":"A sliding-window token cap is enforced per minute per provider so a chatty stretch cannot exhaust the budget for a calmer one.","evidence":[],"evidence_status":"none"},{"pattern":"cost-gating","role":"first-class","note":"Premium-model access is gated behind an explicit, time-boxed (≤10 min) written grant; without an active grant the router stays on cheap models, and grant + revoke + each routing decision land in the ledger.","evidence":[],"evidence_status":"none"},{"pattern":"approval-queue","role":"first-class","note":"High-blast-radius actions (file edits outside the agent's own surfaces, dangerous tools) queue for two-phase human approval via an inbox folder; the human partner reads and replies asynchronously.","evidence":[],"evidence_status":"none"},{"pattern":"sandbox-isolation","role":"first-class","note":"Code execution and side-effecting tool calls run in an isolated sandbox mode that restricts the available surface; the mode is a runtime state, not a per-call flag.","evidence":[],"evidence_status":"none"},{"pattern":"pre-generative-loop-gate","role":"first-class","note":"A loop-gate inspects context for divergence signatures (repeating phrasing, ping-pong shape, post-compaction drift) BEFORE the next generation and injects steering hints or refuses, rather than waiting for the output to fail downstream.","evidence":[],"evidence_status":"none"},{"pattern":"tool-result-caching","role":"first-class","note":"Tool results are cached keyed by (tool, args) so repeated calls within a tick or across nearby ticks reuse the prior result instead of paying the latency / cost again.","evidence":[],"evidence_status":"none"},{"pattern":"tool-discovery","role":"first-class","note":"Available tools are discovered at runtime from the on-disk skill folder and from connected MCP servers, not hard-coded at startup, so adding a skill is a filesystem operation rather than a code change.","evidence":[],"evidence_status":"none"},{"pattern":"tool-loadout","role":"first-class","note":"The skill scanner plus a frameworks picker narrow the registered tool surface to a task-relevant subset per tick, so the model never sees the full ~50-skill list when only a handful apply.","evidence":[],"evidence_status":"none"},{"pattern":"skill-library","role":"first-class","note":"Author-written procedures (skills) are loaded on demand for specific task types from a sibling skills/ tree; each skill is a folder with its own code, prompts, and tests, separable from the core loop.","evidence":[],"evidence_status":"none"},{"pattern":"supervisor","role":"first-class","note":"A dispatcher routes requests to specialised internal lanes (chat, tick, MCP, voice) and coordinates across them, acting as the central traffic controller that the agent loop itself does not see.","evidence":[],"evidence_status":"none"},{"pattern":"subagent-isolation","role":"first-class","note":"A subagent runtime spawns bounded child agents for delegable work (deeper passes, isolated investigations) and returns their result to the main loop without merging their context window into the parent.","evidence":[],"evidence_status":"none"},{"pattern":"agent-as-tool-embedding","role":"first-class","note":"The subagent runtime wraps each sub-agent as a single opaque tool call from the parent's point of view, so the parent reasons about 'invoke subagent X' rather than micromanaging its turns.","evidence":[],"evidence_status":"none"},{"pattern":"actor-model-agents","role":"first-class","note":"Peer messaging is mailbox-based: each peer (atelier, others) has an inbox folder and the agent processes messages one at a time with no shared mutable state across peers.","evidence":[],"evidence_status":"none"},{"pattern":"inter-agent-communication","role":"first-class","note":"Peer messaging defines a file-based protocol for exchanging messages with other agents across process boundaries: each peer (the atelier maintenance agent and any named peer) gets an inbox folder the agent writes to and a replies folder it reads from, with one Markdown message per exchange carrying timestamp, urgency and response state in frontmatter; every outbound send is also recorded as a spawn so a peer that never replies can be detected by orphan recovery.","evidence":[],"evidence_status":"none"},{"pattern":"agent-persona-profile","role":"first-class","note":"Coaches are structured persona profiles rather than free-form role sentences: each coach is a typed record (slug, name, focus, voice, approach, opening, system prompt, optional tool allowlist) loaded from its own file, so a session with the psychologist, philosophical, creativity or communication coach runs under that coach's identity and a narrowed tool surface without editing the core loop.","evidence":[],"evidence_status":"none"},{"pattern":"role-assignment","role":"supported","note":"Spawned subagents and booked coaches are each assigned a named role with a role-specific system prompt and a restricted tool palette (the per-coach/per-subagent allowlist), so a delegated child runs as that role rather than as a generic copy of the parent loop.","evidence":[],"evidence_status":"none"},{"pattern":"unbounded-subagent-spawn","role":"anti-pattern","note":"Deliberately rejected: a global spawn-depth guard caps nesting across every spawn kind (committee voices, coach sessions, subagent children, peer messages) at one shared limit, refusing and writing a REFUSED entry to the ledger at the cap — so per-agent step and cost budgets cannot be bypassed through recursive fan-out.","evidence":[],"evidence_status":"none"},{"pattern":"knowledge-graph-memory","role":"first-class","note":"Beliefs are stored as typed entity / relation triples in a claim graph, not only as flat embeddings, so the agent can reason over its own asserted relationships rather than only over similarity.","evidence":[],"evidence_status":"none"},{"pattern":"five-tier-memory-cascade","role":"first-class","note":"Memory is staged across sensory / working / short-term / episodic / long-term tiers with explicit decay between tiers, rather than one flat store retrieved by similarity.","evidence":[],"evidence_status":"none"},{"pattern":"context-window-packing","role":"first-class","note":"Prompt-cache management and a context-packer fit the per-tick prompt into the model's window deliberately (stable prefix, recent ledger, current workspace, active variant) rather than relying on the provider to truncate.","evidence":[],"evidence_status":"none"},{"pattern":"best-of-n","role":"first-class","note":"The eval path samples N candidates for selected tasks, scores them via internal reward models, and returns the best — distinct from running the whole loop N times.","evidence":[],"evidence_status":"none"},{"pattern":"process-reward-model","role":"first-class","note":"A thought-scoring pass scores individual reasoning steps along multiple axes (salience, conceptual fit, others) rather than only judging the final answer, so weak intermediate moves can be re-routed.","evidence":[],"evidence_status":"none"},{"pattern":"affect-coupled-plan-lifecycle","role":"first-class","note":"Plan steps emit affective bumps on completion or abandonment (joy / pain), and the affective state in turn feeds back into which plan steps get advanced — the lifecycle is coupled to feelings rather than to a pure scheduler.","evidence":[],"evidence_status":"none"},{"pattern":"interrupt-resumable-thought","role":"first-class","note":"In-flight plans are preserved across user interrupts (chat arrives mid-tick) so the agent can resume the prior thread on a later tick rather than losing the train of thought.","evidence":[],"evidence_status":"none"},{"pattern":"replan-on-failure","role":"first-class","note":"When execution evidence contradicts the current plan (tool error, refusal, surprise observation), the active-plan layer replans rather than retrying the same step blindly.","evidence":[],"evidence_status":"none"},{"pattern":"exception-recovery","role":"first-class","note":"Tool errors and plan-step failures are typed; each type has a deterministic recovery path (retry-once, abort-step, escalate-to-human) rather than a generic try/except wrapper.","evidence":[],"evidence_status":"none"},{"pattern":"graceful-degradation","role":"first-class","note":"When a dependency fails (provider down, MCP server unreachable, voice channel offline), the dispatcher downgrades to a working feature subset rather than refusing the whole tick.","evidence":[],"evidence_status":"none"},{"pattern":"reflexion","role":"first-class","note":"A reflexion module writes linguistic lessons learned from past failures into memory and consults them in future episodes, so the agent improves across runs without weight updates.","evidence":[],"evidence_status":"none"},{"pattern":"liminal-state-detection","role":"first-class","note":"The agent reads the human partner's attentional state (just-woke, focused, winding-down, walked-away) from timing and tone signals and adapts its register accordingly — the same engagement is not appropriate in every state.","evidence":[],"evidence_status":"none"},{"pattern":"open-question-tension-store","role":"first-class","note":"Unresolved questions ('tensions') are kept as a typed ledger separate from thoughts and from hypotheses; they drive what the next idle tick attends to instead of dissolving when the prompt ends.","evidence":[],"evidence_status":"none"},{"pattern":"typed-tool-loop-detector","role":"first-class","note":"Tool-loop detection is mechanical at the dispatch boundary with five typed failure modes (repeat, unknown, poll, ping-pong, circuit-breaker) and per-tool caps, returning a structured refusal the model must consume rather than a prompt-level reminder.","evidence":[],"evidence_status":"none"},{"pattern":"spec-driven-loop","role":"supported","note":"The frameworks-picker path runs an iterative loop against a framework spec until satisfied; a deterministic outer loop over a fixed prompt-against-spec rather than free-form chat.","evidence":[],"evidence_status":"none"},{"pattern":"awareness","role":"first-class","note":"The agent maintains explicit, queryable state about its own tools, context, mode, affect, presence and recent moves (rather than re-deriving them from chat history); a meta-observer module reads that state during each tick.","evidence":[],"evidence_status":"none"},{"pattern":"darwin-godel-self-rewrite","role":"first-class","note":"The Soft-Norm Approach Ledger variant: behavioral norms (rules, moves, motivations, hypotheses) are versioned in an append-only ledger; confidence updates asymmetrically from agent-reported outcomes (improved nudges up, regressed nudges down harder, neutral attempts decay a small amount); sampling exposes named modes (greedy / diverse / historical / aged) as caller verbs rather than as a fixed selection policy. Registration is agent-discretionary, not driven by a generation loop.","evidence":[],"evidence_status":"none"},{"pattern":"hybrid-search","role":"first-class","note":"Recall fuses dense-vector and lexical (BM25-shape) retrieval over her own Markdown corpus with reciprocal rank fusion, so semantically similar and keyword-exact hits merge into one ranked result with per-mode provenance.","evidence":[],"evidence_status":"none"},{"pattern":"disambiguation","role":"supported","note":"Each inbound message is scored for ambiguity by a heuristic plus picker pass; a low clarity score prompts a clarifying question before the agent acts on a vague request.","evidence":[],"evidence_status":"none"},{"pattern":"socratic-questioning-agent","role":"supported","note":"A self-elenchus harness turns Socratic questioning inward: it generates probing questions about her own last claim to surface hidden assumptions, rather than driving a user toward a goal.","evidence":[],"evidence_status":"none"},{"pattern":"meditation-mode","role":"first-class","note":"A bounded runtime mode pauses external I/O, accelerates the tick cadence, and collapses the tool surface to inner-only operations, with output routed to a private journal instead of to the human.","evidence":[],"evidence_status":"none"},{"pattern":"self-corpus-vocabulary","role":"first-class","note":"A bounded concept vocabulary (~30 tags) is mined by clustering her own thoughts and insights and cached as the conceptual axis used to score how well a new thought fits what she actually thinks about.","evidence":[],"evidence_status":"none"},{"pattern":"multi-axis-promotion-scoring","role":"first-class","note":"Promotion of a short-term thought to a long-term insight is gated by a weighted multi-axis score (frequency, relevance, diversity, recency, consolidation, conceptual fit) in which consolidation counts for more than raw repetition.","evidence":[],"evidence_status":"none"},{"pattern":"cluster-capped-insight-store","role":"supported","note":"Dream-phase consolidation caps insights per stem-token cluster at three and archives the oldest variants by modification time, so the long-term store keeps the active edge instead of accumulating near-duplicates.","evidence":[],"evidence_status":"none"},{"pattern":"append-only-thought-stream","role":"first-class","note":"The per-day thoughts stream is append-only: each thought is recorded as a timestamped entry and the agent cannot rewrite its own episodic history, kept separate from the curated insight store.","evidence":[],"evidence_status":"none"},{"pattern":"mode-adaptive-cadence","role":"first-class","note":"The loop interval is salience-driven: the just-completed tick's salience and any external impulse select an intense or idle cadence, so the agent thinks faster when something is alive and slower while idle.","evidence":[],"evidence_status":"none"},{"pattern":"reflexive-metacognitive-agent","role":"first-class","note":"A self-state module mirrors the agent's own loop signals (narration-loop index, plan-stall) back into the system prompt as a first-person self-model she reasons over when choosing or refusing a move.","evidence":[],"evidence_status":"none"},{"pattern":"partial-output-salvage","role":"supported","note":"Every in-flight model token is streamed to a tmp-plus-atomic-replace partial file, so a crash mid-inference leaves a consistent salvage that is promoted at startup with a typed recovery note the model can see.","evidence":[],"evidence_status":"none"},{"pattern":"commitment-tracking","role":"first-class","note":"A cheap extraction pass after chat turns records stated intents as typed commitment records (open / followed-through / expired) and flags overdue ones via a maintenance hook, surfacing the gap between promise and follow-through.","evidence":[],"evidence_status":"none"},{"pattern":"hypothesis-tracking","role":"first-class","note":"Provisional answers are persisted as one editable Markdown file per hypothesis carrying summary, confidence, status and next-test, deduplicated by Jaccard overlap, so guesses survive across sessions instead of dissolving when the prompt ends.","evidence":[],"evidence_status":"none"},{"pattern":"ambient-presence-sensing","role":"first-class","note":"Frontend pacing signals (typing rate, idle duration, tab visibility) are read as ambient weather between messages and reduced to a presence-quality value she acts on, without replaying the raw signals back.","evidence":[],"evidence_status":"none"},{"pattern":"self-edit-critic-gate","role":"first-class","note":"Writes and deletes to load-bearing source and identity files pass a gated check: a deterministic syntax-and-destruction pre-check, then a separate critic-model call on the (path, diff, justification) that must approve before the edit lands, with every verdict ledgered, so a single confident-but-wrong self-edit cannot brick the running agent.","evidence":[],"evidence_status":"none"}],"alternatives":[{"composition":"openclaw","relation":"similar-shape","note":"Self-hosted, single-user personal agent peer. OpenClaw is channel-triggered (incoming chat/voice wakes the assistant) and open-source; Sparrot is self-paced (an internal tick loop wakes the agent regardless of user activity) and private."},{"composition":"dust","relation":"competes-with","note":"Hosted multi-user agent workspace; opposite deployment model (cloud SaaS vs. private single-user) and opposite trigger model (user message vs. self-pacing)."}],"references":[{"type":"doc","title":"Sparrot — public page","url":"https://marco-nissen.com/sparrot/","accessed":"2026-05-21"}],"last_analyzed":"2026-06-03","verification_status":"partial","last_updated":"2026-06-03","date_added":"2026-05-01","status_in_practice":"experimental","tags":["proprietary","private","personal-agent","self-pacing","file-native","cognitive-agent","single-user","multi-agent"]},{"id":"ai-infrastructure-agent","name":"AI Infrastructure Agent","kind":"framework","category":"domain-agent","vendor":"VersusControl","language":"Go","license":"Apache-2.0","status":"active","first_released":"2025-09-01","url":"https://github.com/VersusControl/ai-infrastructure-agent","repo":"https://github.com/VersusControl/ai-infrastructure-agent","docs_url":"https://github.com/VersusControl/ai-infrastructure-agent","intent":"AI Infrastructure Agent manages AWS infrastructure from natural-language requests by planning the changes, presenting a dry-run plan for human approval, and only then applying them.","description":"AI Infrastructure Agent is a Go tool that lets a user manage AWS infrastructure through natural-language commands. It is powered by a configurable model provider (OpenAI, Google Gemini, or Anthropic Claude) and exposes capabilities over the Model Context Protocol. The agent analyzes a request, reads the current infrastructure state, and creates a detailed execution plan; every operation can first run in dry-run mode that shows exactly what would be created, modified, or deleted. The agent presents that plan for review and waits for explicit approval before any real AWS changes are applied.","primary_use_cases":["natural-language AWS infrastructure management","dry-run preview of infrastructure changes before apply","human-approved provisioning of cloud resources","state-aware planning over existing AWS resources"],"agent_loop_shape":"The agent receives a natural-language request, queries available tools and the current infrastructure state, and uses a configured model to create a detailed execution plan. Operations can run in dry-run mode that shows exactly what would be created, modified, or deleted. The plan is presented for the user's review and the agent waits for approval before executing any real change, handling resource dependencies automatically during apply.","build_surface":"full-code","members":[{"pattern":"simulate-before-actuate","role":"first-class","note":"The agent runs every infrastructure operation in a dry-run mode that shows exactly what would be created/modified/deleted and presents the execution plan for human approval before any real AWS changes are applied.","evidence":[{"type":"repo","title":"VersusControl/ai-infrastructure-agent README","url":"https://github.com/VersusControl/ai-infrastructure-agent","quote":"All operations can be run in 'dry-run' mode first: Shows exactly what would be created/modified/deleted","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"human-in-the-loop","role":"first-class","note":"The agent analyzes the request, creates a detailed execution plan, presents it for the user's review showing exactly what will be created, and waits for explicit approval before applying any change.","evidence":[{"type":"repo","title":"VersusControl/ai-infrastructure-agent README","url":"https://github.com/VersusControl/ai-infrastructure-agent","quote":"The AI agent analyzes your request and creates a detailed execution plan","accessed":"2026-06-17"},{"type":"repo","title":"VersusControl/ai-infrastructure-agent README","url":"https://github.com/VersusControl/ai-infrastructure-agent","quote":"Agent presents the plan for your review: Shows exactly what will be created. Waits for your approval","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"sync-execution-plan-confirmation","role":"first-class","note":"The agent emits its full execution plan and blocks on the user's confirmation before any side-effecting step, so apply only proceeds after the plan is reviewed and approved.","evidence":[{"type":"repo","title":"VersusControl/ai-infrastructure-agent README","url":"https://github.com/VersusControl/ai-infrastructure-agent","quote":"Agent presents the plan for your review: Shows exactly what will be created. Waits for your approval","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"plan-and-execute","role":"core","note":"The agent first generates a complete execution plan with the configured model from the request and current state, and only once approved executes the planned operations in order, separating a planning phase from an execution phase.","evidence":[{"type":"repo","title":"VersusControl/ai-infrastructure-agent README","url":"https://github.com/VersusControl/ai-infrastructure-agent","quote":"A->>A: Generate execution plan with LLM","accessed":"2026-06-17"},{"type":"repo","title":"VersusControl/ai-infrastructure-agent README","url":"https://github.com/VersusControl/ai-infrastructure-agent","quote":"Once approved, the agent: Creates resources in the correct order ... Handles dependencies automatically","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"mcp","role":"core","note":"The core agent is implemented as an MCP server, so its infrastructure capabilities are exposed and invoked through the Model Context Protocol.","evidence":[{"type":"repo","title":"VersusControl/ai-infrastructure-agent README — Components","url":"https://github.com/VersusControl/ai-infrastructure-agent","quote":"MCP Server: Core agent implementing Model Context Protocol","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"tool-use","role":"core","note":"Before planning, the agent queries the MCP server for the available tools and their capabilities and selects from them to assemble the execution plan.","evidence":[{"type":"repo","title":"VersusControl/ai-infrastructure-agent README — Architecture","url":"https://github.com/VersusControl/ai-infrastructure-agent","quote":"A->>M: Query available tools & capabilities","accessed":"2026-06-17"},{"type":"repo","title":"VersusControl/ai-infrastructure-agent README — Architecture","url":"https://github.com/VersusControl/ai-infrastructure-agent","quote":"M->>A: Return tool capabilities","accessed":"2026-06-17"}],"evidence_status":"full"}],"key_concepts":[{"name":"Dry-run mode","summary":"A preview mode in which every operation is simulated and the agent reports exactly what would be created, modified, or deleted, and estimates cost, without touching any AWS resource.","maps_to_pattern":"simulate-before-actuate","url":"https://github.com/VersusControl/ai-infrastructure-agent"},{"name":"Execution plan","summary":"The model-generated, ordered set of infrastructure operations the agent proposes for a request; it is shown for review and applied only after approval, with resources created in dependency order.","maps_to_pattern":"plan-and-execute","url":"https://github.com/VersusControl/ai-infrastructure-agent"},{"name":"Terraform-like state","summary":"An internally maintained record of the current infrastructure state that the agent reads before planning and uses to detect drift from the expected configuration and resolve conflicts.","url":"https://github.com/VersusControl/ai-infrastructure-agent"},{"name":"MCP Server (Agent Core)","summary":"The Go core that implements the Model Context Protocol, exposes the infrastructure tools, and runs the decision-making and planning loop behind the React web dashboard.","maps_to_pattern":"mcp","url":"https://github.com/VersusControl/ai-infrastructure-agent"}],"references":[{"type":"repo","title":"VersusControl/ai-infrastructure-agent","url":"https://github.com/VersusControl/ai-infrastructure-agent","quote":"Natural Language Interface - Describe what you want, not how to build it","accessed":"2026-06-17"},{"type":"repo","title":"VersusControl/ai-infrastructure-agent README — Components","url":"https://github.com/VersusControl/ai-infrastructure-agent","quote":"Multi-AI Provider Support - Choose between OpenAI, Google Gemini, Anthropic, AWS Bedrock Nova, or Ollama (local LLM)","accessed":"2026-06-17"}],"alternatives":[{"composition":"bedrock-agents","relation":"complements","note":"Amazon Bedrock is one of the configurable model providers for AI Infrastructure Agent (AWS Bedrock Nova), and Bedrock Agents is AWS's own managed agent runtime over the same cloud."},{"composition":"kiro","relation":"similar-shape","note":"Both keep a human approval gate over a model-proposed plan, but Kiro is AWS's spec-driven coding agent for application code while this agent plans and applies AWS infrastructure changes."}],"verification_status":"partial","status_in_practice":"experimental","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"AI Infrastructure Agent\"]\n  fw --> p1[\"Plan-and-Execute<br/>(core)\"]\n  fw --> p2[\"Model Context Protocol<br/>(core)\"]\n  fw --> p3[\"Tool Use<br/>(core)\"]\n  fw --> p4[\"Simulate Before Actuate<br/>(first-class)\"]\n  fw --> p5[\"Human-in-the-Loop<br/>(first-class)\"]\n  fw --> p6[\"Synchronous Execution-Plan Confirmation<br/>(first-class)\"]","tags":["domain-agent","safety-control","planning-control-flow","tool-use-environment","full-code"]},{"id":"khanmigo","name":"Khanmigo","kind":"framework","category":"domain-agent","vendor":"Khan Academy","language":"proprietary","license":"proprietary","status":"active","first_released":"2023","url":"https://www.khanmigo.ai/","docs_url":"https://www.khanmigo.ai/learners","intent":"Tutor learners by asking guiding questions and releasing graduated hints so they reach the answer themselves rather than receiving it directly.","description":"Khanmigo is Khan Academy's AI tutor for learners. Instead of giving the answer, it walks the learner through a problem by asking questions so they work it out themselves. It releases help gradually rather than producing a direct solution. The tool was built by Khan Academy on GPT-4 and launched as a pilot in 2023. A moderation layer screens conversations and alerts a parent or teacher when content is flagged.","primary_use_cases":["guiding students through problems with questions","withholding direct answers during tutoring","supporting self-directed learning across subjects","alerting an adult when a conversation is flagged by moderation"],"agent_loop_shape":"When a learner is stuck, Khanmigo does not return the answer; it asks a sequence of questions and offers step-by-step hints, walking the learner through the solution so they arrive at the answer themselves. In parallel, a moderation layer screens the conversation and, when it detects inappropriate or unsafe content, flags the exchange and alerts a connected adult.","build_surface":"closed-application","members":[{"pattern":"hint-ladder","role":"core","note":"Khanmigo's tutor deliberately withholds the direct answer and instead walks the learner through the solution with questions and step-by-step hints until they work it out themselves.","evidence":[{"type":"doc","title":"Meet Khanmigo: Khan Academy's AI-powered teaching assistant and tutor","url":"https://www.khanmigo.ai/","quote":"Khanmigo doesn't just give answers. Instead, with limitless patience, it guides learners to find the answer themselves.","accessed":"2026-06-17"},{"type":"doc","title":"Khanmigo for learners: Always-available tutor, powered by AI","url":"https://www.khanmigo.ai/learners","quote":"Unlike ChatGPT, Khanmigo never gives you the answer. It's built to help you learn.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"socratic-questioning-agent","role":"core","note":"Khanmigo drives the learner toward the answer by asking a sequence of questions rather than supplying the solution, surfacing the learner's own reasoning.","evidence":[{"type":"doc","title":"Meet Khanmigo: Khan Academy's AI-powered teaching assistant and tutor","url":"https://www.khanmigo.ai/","quote":"it will walk you through the solution and ask questions so you can work it out yourself rather than just get the answer.","accessed":"2026-06-17"},{"type":"doc","title":"Meet Khanmigo: Khan Academy's AI-powered teaching assistant and tutor","url":"https://www.khanmigo.ai/","quote":"Khanmigo challenges you to think critically and solve problems without giving you direct answers.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"input-output-guardrails","role":"supported","note":"A moderation layer screens conversations for inappropriate or unsafe content as the learner interacts, so the tutoring exchange runs under monitoring rather than unsupervised free chat.","evidence":[{"type":"doc","title":"Khanmigo for parents","url":"https://www.khanmigo.ai/parents","quote":"Khanmigo is built for learners, with guardrails and monitoring in place.","accessed":"2026-06-17"},{"type":"doc","title":"Khanmigo for parents","url":"https://www.khanmigo.ai/parents","quote":"Safety is the bedrock of Khanmigo, with guardrails in place to help your child focus on learning.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"mandatory-red-flag-escalation","role":"supported","note":"When the moderation layer flags content in a conversation, Khanmigo escalates by alerting the connected parent or teacher rather than handling it inside the chat, putting an adult in the loop on concerning interactions.","evidence":[{"type":"doc","title":"Khanmigo for parents","url":"https://www.khanmigo.ai/parents","quote":"You can view your child's chats, get alerts for flagged content, and feel good knowing they're using an AI designed specifically for learning.","accessed":"2026-06-17"},{"type":"doc","title":"Khanmigo for parents","url":"https://www.khanmigo.ai/parents","quote":"Parents can access their chat history and get alerts about inappropriate interactions.","accessed":"2026-06-17"}],"evidence_status":"full"}],"key_concepts":[{"name":"Tutor mode","summary":"The default tutoring behaviour in which Khanmigo refuses to hand over the answer and instead questions the learner and releases graduated hints so they reach the solution themselves.","maps_to_pattern":"hint-ladder","url":"https://www.khanmigo.ai/learners"},{"name":"Writing coach","summary":"A named activity where Khanmigo gives real-time feedback on a learner's writing and supports debate and collaboration instead of producing the text for them.","url":"https://www.khanmigo.ai/learners"},{"name":"Moderation alerts","summary":"A screening layer over conversations that detects inappropriate or unsafe content and notifies a connected parent or teacher, who can also view the chat history.","maps_to_pattern":"mandatory-red-flag-escalation","url":"https://www.khanmigo.ai/parents"}],"references":[{"type":"doc","title":"Meet Khanmigo: Khan Academy's AI-powered teaching assistant and tutor","url":"https://www.khanmigo.ai/","quote":"Khanmigo doesn't just give answers. Instead, with limitless patience, it guides learners to find the answer themselves.","accessed":"2026-06-17"},{"type":"doc","title":"Khanmigo for learners: Always-available tutor, powered by AI","url":"https://www.khanmigo.ai/learners","quote":"Master skills with personalized tutoring on Khan Academy's world-class content.","accessed":"2026-06-17"}],"alternatives":[{"composition":"intercom-fin","relation":"similar-shape","note":"Both run a guardrailed conversational assistant that escalates to a human on defined triggers, but Intercom Fin resolves customer-support tickets while Khanmigo tutors learners and deliberately withholds the answer."}],"verification_status":"needs-verification","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-19","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Khanmigo\"]\n  fw --> p1[\"Hint Ladder<br/>(core)\"]\n  fw --> p2[\"Socratic Questioning Agent<br/>(core)\"]\n  fw --> p3[\"Input/Output Guardrails<br/>(supported)\"]\n  fw --> p4[\"Mandatory Red-Flag Escalation<br/>(supported)\"]","tags":["domain-agent","safety-control","reasoning","closed-application"],"anti_patterns_avoided":[{"pattern":"productive-struggle-erosion","note":"Khanmigo never gives the answer outright; it prompts the learner to think through the problem, preserving productive struggle.","evidence":[{"type":"doc","title":"Khanmigo — for learners","url":"https://www.khanmigo.ai/learners","quote":"Khanmigo never gives you the answer. It's built to help you learn... Khanmigo helps you flex your brain muscles by never giving you the answer.","accessed":"2026-06-19"}]},{"pattern":"agentic-skill-atrophy","note":"By never giving the answer and prompting the learner to reason, Khanmigo preserves the human's skill rather than letting it atrophy through over-reliance.","evidence":[{"type":"doc","title":"Khanmigo — for learners","url":"https://www.khanmigo.ai/learners","quote":"Khanmigo helps you flex your brain muscles by never giving you the answer. You will be prompted to think deeply and critically in order to solve problems.","accessed":"2026-06-19"}]}]},{"id":"ramp","name":"Ramp Accounting Agent","kind":"framework","category":"domain-agent","vendor":"Ramp","language":"N/A","license":"proprietary","status":"active","first_released":"2025-06-01","url":"https://ramp.com/","docs_url":"https://ramp.com/intelligence","intent":"Ramp's Accounting Agent automatically codes and syncs routine, in-policy spend to the ERP while routing low-confidence and exception transactions to a human and posting nothing without human confirmation.","description":"Ramp's Accounting Agent automates transaction coding and ERP synchronisation for corporate spend. Routine, in-policy transactions move from swipe to ERP on their own, while only the minority of exceptions are surfaced for human review. Each decision shows its confidence level and rationale, and nothing posts to the ERP without explicit human confirmation. This keeps a human accountable for material postings while removing manual work on clearly approved spend.","primary_use_cases":["automated transaction coding","syncing in-policy spend to the ERP","routing exceptions for human review","confidence-gated accounting automation"],"agent_loop_shape":"For each transaction the agent classifies it and assigns a confidence level. High-confidence, in-policy spend is coded and synced to the ERP automatically. Lower-confidence and exception transactions are surfaced to a human reviewer with the agent's rationale, and the human decides. Across both paths nothing is posted to the ERP without human confirmation, so the agent acts autonomously on routine cases and defers material or uncertain postings to a person.","build_surface":"closed-application","members":[{"pattern":"risk-tiered-action-autonomy","role":"first-class","note":"Ramp's Accounting Agent auto-codes and auto-syncs routine, in-policy spend on its own, but routes lower-confidence and exception transactions to a human and posts nothing to the ERP without explicit confirmation, gating material postings on a human principal.","evidence":[{"type":"blog","title":"12 Best AI Accounting Software Tools for 2026","url":"https://ramp.com/blog/ai-accounting-software","quote":"Routine in-policy spend moves from swipe to ERP automatically, and only exceptions surface for your review.","accessed":"2026-06-17"},{"type":"blog","title":"12 Best AI Accounting Software Tools for 2026","url":"https://ramp.com/blog/ai-accounting-software","quote":"nothing posts to your ERP without your confirmation","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"human-in-the-loop","role":"first-class","note":"The agent surfaces only the minority of transactions that need human judgment for review and requires explicit human confirmation before any posting, keeping a person in the decision loop for exceptions and material actions.","evidence":[{"type":"blog","title":"12 Best AI Accounting Software Tools for 2026","url":"https://ramp.com/blog/ai-accounting-software","quote":"Ramp only surfaces the 20–30% of transactions that most need human judgment, so you can prioritize the tasks that most need your attention.","accessed":"2026-06-17"},{"type":"blog","title":"12 Best AI Accounting Software Tools for 2026","url":"https://ramp.com/blog/ai-accounting-software","quote":"Every decision shows its confidence level and rationale, and nothing posts to your ERP without your confirmation.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"confidence-reporting","role":"first-class","note":"The agent attaches a confidence level to each coding decision and surfaces it alongside the result, so a reviewer sees how certain the agent is about a transaction before acting on it.","evidence":[{"type":"blog","title":"12 Best AI Accounting Software Tools for 2026","url":"https://ramp.com/blog/ai-accounting-software","quote":"Every decision shows its confidence level and rationale, and nothing posts to your ERP without your confirmation.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"decision-log","role":"first-class","note":"Every coding decision records its confidence, rationale, and any override so the action is explainable and the audit trail stays intact for later review.","evidence":[{"type":"blog","title":"12 Best AI Accounting Software Tools for 2026","url":"https://ramp.com/blog/ai-accounting-software","quote":"Every AI decision includes a confidence level, rationale, and override, so the audit trail stays intact.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"Accounting Agent","summary":"Ramp's AI that auto-codes the accounting fields (GL, department, class, location, and custom fields) of a transaction the moment it posts, without hand-maintained coding rules.","maps_to_pattern":"risk-tiered-action-autonomy","url":"https://ramp.com/blog/ai-accounting-software"},{"name":"Zero-touch lane","summary":"The path that carries routine, in-policy spend from card swipe straight to the ERP automatically, while only exceptions are split off for human review.","maps_to_pattern":"risk-tiered-action-autonomy","url":"https://ramp.com/blog/ai-accounting-software"},{"name":"Confidence and rationale","summary":"Each decision carries an explicit confidence level, a stated rationale, and an override path, which together form the audit trail and decide whether a transaction is auto-synced or sent for review.","maps_to_pattern":"decision-log","url":"https://ramp.com/blog/ai-accounting-software"}],"references":[{"type":"blog","title":"12 Best AI Accounting Software Tools for 2026","url":"https://ramp.com/blog/ai-accounting-software","quote":"Accounting Agent auto-codes GL, department, class, location, and custom fields the moment a transaction posts, with no rules to maintain.","accessed":"2026-06-17"},{"type":"doc","title":"Ramp Intelligence","url":"https://ramp.com/intelligence","quote":"no money ever moves without a human confirmation, and every decision they make is completely transparent, auditable, and adjustable.","accessed":"2026-06-17"}],"alternatives":[{"composition":"intercom-fin","relation":"similar-shape","note":"Both run a confidence-gated autonomous agent that acts on routine cases and escalates uncertain ones to a human, but Intercom Fin resolves customer-support conversations while Ramp's agent codes and posts financial transactions."}],"verification_status":"partial","status_in_practice":"emerging","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Ramp Accounting Agent\"]\n  fw --> p1[\"Risk-Tiered Action Autonomy<br/>(first-class)\"]\n  fw --> p2[\"Human-in-the-Loop<br/>(first-class)\"]\n  fw --> p3[\"Confidence Reporting<br/>(first-class)\"]\n  fw --> p4[\"Decision Log<br/>(first-class)\"]","tags":["domain-agent","safety-control","verification-reflection","governance-observability","closed-application"]},{"id":"moveit-pro","name":"MoveIt Pro","kind":"framework","vendor":"PickNik Robotics","url":"https://picknik.ai/pro/","docs_url":"https://docs.picknik.ai/","intent":"MoveIt Pro validates a requested robot motion against the planning-scene world model, collision geometry, and inverse-kinematics reachability, executing only a previewed, validated trajectory so a physically infeasible command never reaches the actuators.","description":"MoveIt Pro is PickNik's platform for building robotic-arm applications on top of MoveIt. Motion is planned against a planning scene that is the source of truth for the robot and its environment, collisions are checked by default, and a ValidateTrajectory step fails if a planned path collides before execution.","members":[{"pattern":"simulate-before-actuate","role":"core","evidence_status":"full","note":"A ValidateTrajectory behavior collision-checks a planned trajectory against the planning scene before execution.","evidence":[{"type":"doc","url":"https://docs.picknik.ai/how_to/robotics_applications/cartesian_path_following/","quote":"In the case that the trajectory should be collision checked with the planning scene before execution, the `ValidateTrajectory` Behavior can be used.","accessed":"2026-06-14"}]},{"pattern":"affordance-grounding-gate","role":"core","evidence_status":"full","note":"A goal is only actionable if it has a valid inverse-kinematics solution; an unreachable pose yields no preview and thus no motion.","evidence":[{"type":"doc","url":"https://docs.picknik.ai/6/getting_started/setup_tutorials/teleoperate_robot/","quote":"No preview means that there is no valid inverse kinematics solution for the desired pose.","accessed":"2026-06-14"}]},{"pattern":"world-model-as-tool","role":"supported","evidence_status":"full","note":"The planning scene is an explicit world model that motion planning and collision checking query.","evidence":[{"type":"doc","url":"https://docs.picknik.ai/concepts/planning_scene/","quote":"The planning scene is the source of truth for the environment in which your robotic agents do motion planning.","accessed":"2026-06-14"}]},{"pattern":"dry-run-harness","role":"supported","evidence_status":"full","note":"The planned trajectory plays as a preview that an operator approves before the real motion starts.","evidence":[{"type":"doc","url":"https://docs.picknik.ai/6/getting_started/setup_tutorials/teleoperate_robot/","quote":"A preview of the trajectory will play, and if it is acceptable, the green check button can be pressed to start the motion.","accessed":"2026-06-14"}]}],"anti_patterns_avoided":[{"pattern":"physical-hallucination","note":"A command is validated against geometry, collision, and reachability before actuation, so a confidently-phrased but infeasible motion cannot execute. PickNik states MoveIt Pro is not a safety-rated system; the guard is feasibility and collision validation, not certified functional safety.","evidence":[{"type":"doc","url":"https://moveit.ai/documentation/concepts/","quote":"Collisions are checked for by default (including self-collisions and attached objects).","accessed":"2026-06-14"}]}],"key_concepts":[{"name":"Planning scene","summary":"The source-of-truth world model of the robot and its environment used for planning and collision checks.","maps_to_pattern":"world-model-as-tool"},{"name":"Collision checking","summary":"Default validation that a planned trajectory does not collide with the scene or itself.","maps_to_pattern":"simulate-before-actuate"},{"name":"Inverse-kinematics reachability","summary":"A goal is actionable only if an IK solution exists; unreachable poses produce no preview.","maps_to_pattern":"affordance-grounding-gate"},{"name":"Trajectory preview","summary":"A previewed motion an operator approves before execution.","maps_to_pattern":"dry-run-harness"}],"primary_use_cases":["Building robotic-arm manipulation applications that reject infeasible motions","Collision-aware free-space motion planning","Operator-approved teleoperated motion with previewed trajectories"],"agent_loop_shape":"Sits between an agent/operator's commanded goal and the robot actuators, planning against a world model and executing only a validated, previewed trajectory.","references":[{"type":"doc","title":"MoveIt Pro product page","url":"https://picknik.ai/pro/"},{"type":"doc","title":"Planning Scene & Collision Checker","url":"https://docs.picknik.ai/concepts/planning_scene/"},{"type":"doc","title":"MoveIt concepts","url":"https://moveit.ai/documentation/concepts/"}],"tags":["robotics","motion-planning","collision-checking","embodied","domain-agent"],"pattern_composition":"A world-model-as-tool planning scene grounds an affordance-grounding-gate (inverse-kinematics reachability) and a simulate-before-actuate collision check, with a dry-run-harness preview gating execution.","category":"domain-agent","status":"active","verification_status":"needs-verification","build_surface":"full-code","last_analyzed":"2026-06-14","status_in_practice":"emerging","last_updated":"2026-06-14","date_added":"2026-06-14"},{"id":"azure-ai-foundry-agent-service","name":"Azure AI Foundry Agent Service","aliases":["Azure AI Agent Service","Microsoft Foundry Agent Service"],"kind":"framework","category":"enterprise-platform","build_surface":"full-code","vendor":"Microsoft Azure","language":"API (multi-language SDKs: Python, .NET, JavaScript, Java, REST)","license":"proprietary","status":"active","first_released":"2025-05-19","url":"https://azure.microsoft.com/en-us/products/ai-foundry/","docs_url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/","intent":"Fully managed Azure platform for building, deploying, and scaling AI agents that combine a Foundry-catalog model, instructions, and tools, with built-in identity, content safety, tracing, evaluation, and three agent shapes (prompt, workflow, hosted-container).","description":"Foundry Agent Service hosts and scales agents on Azure-managed runtime. An agent is a model from the Foundry catalog plus instructions plus tools; the service handles tool execution, conversation state, Microsoft Entra identity, content filters, virtual-network isolation, tracing into Application Insights, and a build-test-deploy-monitor lifecycle with immutable versioning. Three agent types coexist: prompt agents (configuration-only), workflow agents (declarative YAML or visual designer for sequential / group-chat / human-in-the-loop orchestrations), and hosted agents (your container running Microsoft Agent Framework, LangGraph, Semantic Kernel, or custom code, per-session VM-isolated). The catalog of built-in tools includes Web Search, Code Interpreter, File Search, Azure AI Search, Function calling, OpenAPI tools, Image Generation, Browser Automation, Computer Use, plus MCP and A2A as custom tool surfaces.","primary_use_cases":["no-code prompt agents in the Foundry portal","declarative multi-step workflows that orchestrate agents and business logic","containerized custom-code agents (Agent Framework / LangGraph / Semantic Kernel) on managed VMs","enterprise tool integration via MCP, OpenAPI, and Azure connectors","publishing agents to Microsoft 365, Teams, and the Entra Agent Registry"],"agent_loop_shape":"Managed agent runtime around a Foundry-catalog model. Prompt agents run a single-agent tool-use loop: model reasons over instructions, the service decides whether to call a built-in tool (web search, code interpreter, file search, MCP, OpenAPI, function call), executes, observes, continues. Workflow agents bind that loop into a declarative graph of nodes (agent invocation, if/else, for-each, human-in-the-loop, group chat) defined in YAML or the visual designer. Hosted agents replace the inner loop with user code in a per-session VM-isolated sandbox that calls Foundry models, Toolbox MCP, and the Responses / Invocations / Activity / A2A protocols. Every run is traced, versioned, and gated by content filters and Entra identity.","key_concepts":[{"name":"Prompt / Workflow / Hosted agents","summary":"Three agent shapes: prompt (configuration-only), workflow (declarative orchestration), hosted (your container on managed runtime).","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/overview"},{"name":"Foundry tool catalog","summary":"Built-in tools (web search, Code Interpreter, File Search, Azure AI Search, Functions) plus custom tools (MCP, OpenAPI, A2A) registered through the portal.","maps_to_pattern":"tool-use","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/tool-catalog"},{"name":"Workflows","summary":"Visual / YAML orchestration with sequential, group-chat, and human-in-the-loop patterns.","maps_to_pattern":"orchestrator-workers","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/workflow"},{"name":"Hosted agents","summary":"Per-session VM-isolated containers with persistent $HOME and /files; Responses / Invocations / Activity / A2A protocols.","maps_to_pattern":"agent-resumption","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/hosted-agents"},{"name":"Guardrails and content filters","summary":"Integrated content filters with cross-prompt-injection (XPIA) protection.","maps_to_pattern":"prompt-injection-defense","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/overview"},{"name":"Agent identity and tracing","summary":"Per-agent Microsoft Entra identity, RBAC, Application Insights tracing of every model call and tool invocation.","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/development-lifecycle"}],"pattern_composition":"flowchart TD\n  user[User request] --> entra[Microsoft Entra identity check]\n  entra --> runtime[Foundry Agent Runtime]\n  runtime --> kind{Agent type}\n  kind -->|prompt| prompt[Prompt agent]\n  kind -->|workflow| wf[Workflow YAML / visual graph]\n  kind -->|hosted| host[Hosted container / VM sandbox]\n  prompt --> loop[Tool-use loop]\n  wf --> node[Node: agent / if-else / for-each / HITL]\n  node --> loop\n  host --> loop\n  loop --> tools[Foundry tool catalog]\n  tools --> websearch[Web Search]\n  tools --> ci[Code Interpreter]\n  tools --> fs[File Search vector store]\n  tools --> ais[Azure AI Search]\n  tools --> fn[Function calling]\n  tools --> oapi[OpenAPI tool]\n  tools --> mcp[MCP server / Toolbox]\n  tools --> a2a[A2A endpoint]\n  loop --> filter[Content filters / XPIA]\n  filter --> trace[App Insights tracing]\n  trace --> publish[Publish: stable endpoint / Teams / M365]","members":[{"pattern":"agent-resumption","role":"first-class","note":"Hosted agents persist per-session $HOME and /files across idle / resume; sessions live up to 30 days with 15-minute idle timeout and automatic state restore.","evidence":[{"type":"doc","title":"Foundry hosted agents — session state persistence","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/hosted-agents","quote":"$HOME and /files content are persisted across turns and across idle periods. When compute goes idle and is brought back (on new or existing infrastructure), the session's state is automatically restored.","accessed":"2026-05-20"},{"type":"doc","title":"Foundry hosted agents — session lifetime","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/hosted-agents","quote":"Sessions persist for up to 30 days. The idle timeout is 15 minutes—if no request arrives within that window, the platform deprovisions the compute and persists the session state.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"agentic-rag","role":"first-class","note":"File Search vector store and Azure AI Search are first-class built-in tools an agent calls to ground responses in private documents.","evidence":[{"type":"doc","title":"Foundry tool catalog — File Search","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/tool-catalog","quote":"File Search — Augment agents with knowledge from uploaded files or proprietary documents by using vector search.","accessed":"2026-05-20"},{"type":"doc","title":"Foundry tool catalog — Azure AI Search","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/tool-catalog","quote":"Azure AI Search | Ground agents with data from an existing Azure AI Search index.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"approval-queue","role":"first-class","note":"Workflow agents support explicit human-in-the-loop nodes; MCP tool config exposes a require_approval flag (e.g. 'always').","evidence":[{"type":"doc","title":"Foundry workflows — human-in-the-loop pattern","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/workflow","quote":"Human in the loop | Asks the user a question and awaits user input to proceed | Creating approval requests during workflow execution and waiting for human approval","accessed":"2026-05-20"},{"type":"doc","title":"Foundry tool catalog — MCPTool require_approval","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/tool-catalog","quote":"tool = MCPTool(\n    server_label=\"github\",\n    server_url=\"https://api.githubcopilot.com/mcp\",\n    require_approval=\"always\",","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"code-execution","role":"first-class","note":"Code Interpreter is a built-in tool that runs Python in a Microsoft-managed Hyper-V-isolated sandbox (Azure Container Apps dynamic sessions) for data analysis, math, and chart generation.","evidence":[{"type":"doc","title":"Foundry tool catalog — Code Interpreter","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/tool-catalog","quote":"Code Interpreter — Let agents write and run Python code in a sandboxed environment for data analysis, math, and chart generation.","accessed":"2026-05-20"},{"type":"doc","title":"Foundry — Use Code Interpreter with agents","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/code-interpreter","quote":"Code Interpreter enables a Microsoft Foundry agent to run Python code in a sandboxed execution environment. The agent's Foundry model writes and executes code for data analysis, chart generation, and iterative problem-solving tasks.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"eval-harness","role":"first-class","note":"Documented Evaluate step in the lifecycle; agent evaluators measure quality and safety regressions before publish.","evidence":[{"type":"doc","title":"Foundry development lifecycle — Evaluate step","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/overview","quote":"Evaluate — Run evaluations to measure quality and catch regressions.","accessed":"2026-05-20"},{"type":"doc","title":"Foundry development lifecycle — evaluation framing","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/development-lifecycle","quote":"Before you publish your agent (and after any meaningful change), run evaluations to catch regressions and measure quality consistently across versions.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"input-output-guardrails","role":"first-class","note":"Integrated content filters on input and output; safety controls explicitly listed.","evidence":[{"type":"doc","title":"Foundry overview — content safety","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/overview","quote":"Content safety — Integrated content filters help mitigate prompt injection risks (including cross-prompt injection) and prevent unsafe outputs.","accessed":"2026-05-20"},{"type":"doc","title":"Foundry overview — safety controls","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/overview","quote":"Safety controls: Use integrated guardrails to help reduce unsafe outputs and mitigate prompt injection risks, including cross-prompt injection attacks (XPIA).","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"Remote and local MCP servers are first-class tools; Toolbox exposes a curated bundle as a single MCP-compatible endpoint.","evidence":[{"type":"doc","title":"Foundry overview — MCP servers","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/overview","quote":"Foundry supports remote MCP servers that you can add from the Add Tools catalog in the Foundry portal.","accessed":"2026-05-20"},{"type":"doc","title":"Foundry Toolbox — MCP endpoint","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/tool-catalog","quote":"A toolbox is a curated bundle of tools - such as web search, Azure AI Search, code interpreter, file search, MCP servers, and OpenAPI tools - that you configure once and expose as a single MCP-compatible endpoint.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"multi-model-routing","role":"first-class","note":"Agents bind to a model from the Foundry catalog and can be swapped without changing agent code; many models supported including Azure OpenAI (GPT-4o, GPT-4.1, GPT-5) and Foundry Models sold by Azure (Llama, DeepSeek, Grok, MAI-DS-R1, gpt-oss).","evidence":[{"type":"doc","title":"Foundry overview — model swap","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/overview","quote":"Models | Works with many models from the Foundry model catalog, such as GPT-4o, Llama, and DeepSeek. Swap models without changing your agent code.","accessed":"2026-05-20"},{"type":"doc","title":"Foundry — Non-OpenAI Foundry Models sold by Azure","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/model-region-support","quote":"In addition to Azure OpenAI models, you can use Foundry Models sold by Azure. These models offer specialized capabilities for specific use cases, such as deterministic reasoning or high-throughput generation.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"orchestrator-workers","role":"first-class","note":"Workflow agents orchestrate multiple specialist agents; sequential / group-chat / branching are templated patterns.","evidence":[{"type":"doc","title":"Foundry workflows — multi-agent orchestration","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/workflow","quote":"By orchestrating multiple agents, each powered by a Foundry model with specialized skills or roles, you can create systems that are more robust, adaptive, and capable of solving real-world problems collaboratively.","accessed":"2026-05-20"},{"type":"doc","title":"Foundry workflows — group chat pattern","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/workflow","quote":"Group chat | Dynamically passes control between agents based on context or rules | Dynamic workflows, escalation, fallback, or expert handoff scenarios","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"prompt-injection-defense","role":"first-class","note":"Content filters explicitly mitigate cross-prompt-injection attacks (XPIA); backed by Azure AI Content Safety Prompt Shields which defends both User Prompt and Document (indirect) injection attacks.","evidence":[{"type":"doc","title":"Foundry overview — XPIA mitigation","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/overview","quote":"Use integrated guardrails to help reduce unsafe outputs and mitigate prompt injection risks, including cross-prompt injection attacks (XPIA).","accessed":"2026-05-20"},{"type":"doc","title":"Azure AI Content Safety — Prompt Shields","url":"https://learn.microsoft.com/en-us/azure/ai-services/content-safety/concepts/jailbreak-detection","quote":"Prompt Shields is a unified API in Azure AI Content Safety that detects and blocks adversarial user input attacks on large language models (LLMs). It helps prevent harmful, unsafe, or policy-violating AI outputs by analyzing prompts and documents before content is generated.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"structured-output","role":"first-class","note":"Agent nodes can be configured to return JSON Schema output via the workflow visualizer or API; saved as a typed workflow variable for downstream nodes.","evidence":[{"type":"doc","title":"Foundry workflows — JSON schema output","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/workflow","quote":"To configure an agent to return structured JSON output: In the Invoke agent configuration window, select Create a new agent. Configure the agent to send output as a JSON schema","accessed":"2026-05-20"},{"type":"doc","title":"Foundry workflows — Text format JSON Schema and save output variable","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/workflow","quote":"For Text format, select JSON Schema. ... Select Action settings. Then select Save output json_object/json_schema as.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"subagent-isolation","role":"first-class","note":"Hosted agents run in per-session VM-isolated sandboxes; sessions are isolated from each other. Code Interpreter sessions are isolated by a Hyper-V boundary in Azure Container Apps dynamic sessions.","evidence":[{"type":"doc","title":"Foundry hosted agents — isolation model","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/hosted-agents","quote":"Hosted agents run in per-session VM-isolated sandboxes. Each session gets a dedicated sandbox with a persistent filesystem ($HOME and /files), enabling scale-to-zero with stateful resume and predictable cold starts. Sessions are isolated from each other","accessed":"2026-05-20"},{"type":"doc","title":"Foundry Code Interpreter — sandboxed execution environment","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/code-interpreter","quote":"Code Interpreter runs Python code in a Microsoft-managed sandbox. The sandbox is designed for running untrusted code and uses dynamic sessions (code interpreter sessions) in Azure Container Apps. Each session is isolated by a Hyper-V boundary.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Tools are the headline primitive: built-in (Web Search, Code Interpreter, File Search, Azure AI Search, Functions, OpenAPI) and custom (MCP, A2A, Toolbox).","evidence":[{"type":"doc","title":"Foundry tool catalog — what a tool is","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/tool-catalog","quote":"A tool is a capability that an agent can invoke during a conversation to perform a specific task. When an agent receives a user message, the Foundry model powering the agent decides whether to call a tool based on the agent's instructions and the available tool definitions.","accessed":"2026-05-20"},{"type":"doc","title":"Foundry overview — tools listing","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/overview","quote":"Built-in tools including web search, file search, memory, code interpreter, MCP servers, and custom functions.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"computer-use","role":"first-class","note":"Computer Use and Browser Automation are listed built-in tools (both currently preview).","evidence":[{"type":"doc","title":"Foundry tool catalog — Computer Use","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/tool-catalog","quote":"Computer Use (preview) | Interact with computer systems through their user interfaces.","accessed":"2026-05-20"},{"type":"doc","title":"Foundry tool catalog — Browser Automation","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/tool-catalog","quote":"Browser Automation (preview) | Perform browser tasks through natural language prompts.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"supervisor","role":"first-class","note":"Connected agents (classic) and workflows (new) implement supervisor-style delegation: a main/primary agent routes tasks to purpose-built subagents via natural language, eliminating hand-coded routing logic. Group-chat workflow templates dynamically pass control between specialists.","evidence":[{"type":"doc","title":"Foundry workflows — group chat handoff","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/workflow","quote":"Group chat | Dynamically passes control between agents based on context or rules","accessed":"2026-05-20"},{"type":"doc","title":"Foundry — Connected agents (multi-agent delegation)","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/connected-agents","quote":"Connected agents in Foundry Agent Service let you break down complex tasks into coordinated, specialized roles—without the need for a custom orchestrator or hand-coded routing logic. With this capability, you can design systems where a primary agent intelligently delegates to purpose-built subagents","accessed":"2026-05-24"}],"evidence_status":"full"}],"alternatives":[{"composition":"bedrock-agents","relation":"competes-with","note":"Peer hyperscaler agent platform (AWS)."},{"composition":"vertex-ai-agent-builder","relation":"competes-with","note":"Peer hyperscaler agent platform (Google Cloud)."},{"composition":"bedrock-agentcore","relation":"competes-with","note":"Microsoft's enterprise agent-runtime platform."}],"references":[{"type":"doc","title":"What is Microsoft Foundry Agent Service?","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/overview","accessed":"2026-05-20"},{"type":"doc","title":"Agent tools overview for Foundry Agent Service","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/tool-catalog","accessed":"2026-05-20"},{"type":"doc","title":"Build a workflow in Microsoft Foundry","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/workflow","accessed":"2026-05-20"},{"type":"doc","title":"Hosted agents in Foundry Agent Service (preview)","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/hosted-agents","accessed":"2026-05-20"},{"type":"blog","title":"Announcing General Availability of Azure AI Foundry Agent Service (May 2025)","url":"https://techcommunity.microsoft.com/blog/azure-ai-foundry-blog/announcing-general-availability-of-azure-ai-foundry-agent-service/4414352","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-06-19","date_added":"2026-04-30","status_in_practice":"mature","tags":["proprietary","microsoft","azure","enterprise","managed-runtime","mcp","a2a","workflows"],"anti_patterns_avoided":[{"pattern":"confident-inconsistency","note":"Reproducible-output (seed) makes repeated identical requests sample deterministically, countering silent run-to-run answer drift.","evidence":[{"type":"doc","title":"Azure OpenAI — reproducible output","url":"https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/reproducible-output?view=foundry-classic","quote":"Reproducible output is a new preview feature that allows you to selectively change the default behavior to help product more deterministic outputs.","accessed":"2026-06-19"}]}]},{"id":"bedrock-agents","name":"Amazon Bedrock Agents","aliases":["Agents for Amazon Bedrock"],"kind":"framework","category":"enterprise-platform","build_surface":"full-code","vendor":"Amazon Web Services","language":"API (multi-language SDKs: Python, Java, JavaScript, .NET, Go via AWS SDKs)","license":"proprietary","status":"active","first_released":"2023-11-28","url":"https://aws.amazon.com/bedrock/agents/","docs_url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents.html","intent":"AWS-managed agent runtime that turns a Bedrock foundation model into a tool-using, knowledge-base-grounded agent: action groups (OpenAPI/function-detail schemas backed by Lambda or return-of-control), Bedrock Knowledge Bases for RAG, code interpretation, multi-agent supervisor/collaborator hierarchies, long-term memory, and guardrails.","description":"Amazon Bedrock Agents is a fully managed runtime for autonomous agents on AWS. At build time you assemble a foundation model, natural-language instructions, one or more action groups (each defined by an OpenAPI or function-detail schema and resolved via a Lambda function or by returning control to the developer), and optional Bedrock Knowledge Bases. At runtime the InvokeAgent API drives a pre-processing -> orchestration -> post-processing sequence where the model generates a rationale, picks an action or knowledge-base query, observes the result, and iterates until done. Multi-agent collaboration designates a supervisor agent that routes work to collaborator agents. Memory retains conversational context across sessions. Bedrock Guardrails apply content filters, denied topics, PII redaction, and contextual grounding checks.","primary_use_cases":["OpenAPI / function-detail action groups backed by Lambda","RAG-grounded enterprise assistants via Bedrock Knowledge Bases","supervisor-collaborator multi-agent workflows","code-interpreter data analysis and file processing","return-of-control for human or developer-side action execution"],"agent_loop_shape":"Three-phase ReAct-style sequence per InvokeAgent request: (1) Pre-processing validates the user input; (2) Orchestration is a model-driven loop where the FM emits a rationale, predicts which action group to invoke or which knowledge base to query, the action runs (via Lambda or by returning control to the developer) or the KB is queried, the observation augments the base prompt, and the loop iterates; (3) Post-processing formats the final response. Conversation history persists per session, long-term memory summarises completed sessions per memoryId. Multi-agent collaboration wraps this loop with a supervisor that delegates to collaborator agents.","key_concepts":[{"name":"Action groups","summary":"OpenAPI or function-detail schemas backed by a Lambda function (or return-of-control) that define the actions an agent can take.","maps_to_pattern":"tool-use","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-how.html"},{"name":"Knowledge Bases","summary":"Vector-backed RAG sources the agent queries to augment generation.","maps_to_pattern":"agentic-rag","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents.html"},{"name":"Orchestration loop","summary":"Pre-processing -> rationale -> action / KB -> observation -> iterate, optional post-processing.","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-how.html"},{"name":"Return of control","summary":"Action groups can return parameters to the caller instead of invoking Lambda, letting the developer execute and feed results back via invocationId.","maps_to_pattern":"approval-queue","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-returncontrol.html"},{"name":"Multi-agent collaboration","summary":"Designate a supervisor agent and associate collaborator agents; supervisor builds and executes a plan across them.","maps_to_pattern":"supervisor","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-multi-agent-collaboration.html"},{"name":"Memory","summary":"Long-term memory summarises sessions per memoryId; retained up to 365 days.","maps_to_pattern":"cross-session-memory","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-memory.html"},{"name":"Code Interpretation","summary":"Sandboxed code execution for data analysis, charting, and file processing.","maps_to_pattern":"code-execution","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-code-interpretation.html"}],"pattern_composition":"flowchart TD\n  user[User prompt] --> invoke[InvokeAgent]\n  invoke --> pre[Pre-processing: validate input]\n  pre --> orch[Orchestration loop]\n  orch --> rationale[FM emits rationale]\n  rationale --> pick{Pick}\n  pick -->|action group| ag[OpenAPI / function-detail]\n  ag --> exec{Executor}\n  exec -->|Lambda| lambda[Lambda invocation]\n  exec -->|return control| roc[Return-of-control to developer]\n  pick -->|knowledge base| kb[Bedrock KB RAG]\n  pick -->|code interpreter| ci[Sandboxed Python]\n  lambda --> obs[Observation]\n  roc --> obs\n  kb --> obs\n  ci --> obs\n  obs --> done{Done?}\n  done -->|no| orch\n  done -->|yes| post[Post-processing]\n  post --> guard[Bedrock Guardrails]\n  guard --> reply[Reply to user]\n  reply --> mem[Memory summarisation per memoryId]\n  supervisor[Supervisor agent] -.delegates.-> orch\n  supervisor -.coordinates.-> collab[Collaborator agents]","members":[{"pattern":"agent-resumption","role":"first-class","note":"Session conversation history persists across InvokeAgent calls per sessionId; memoryId summaries restore long-term context across new sessions.","evidence":[{"type":"doc","title":"Bedrock Agents — session history preserved","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-how.html","quote":"As the user session with the agent continues through more InvokeAgent requests, the conversation history is preserved. The conversation history continually augments the orchestration base prompt template with context","accessed":"2026-05-20"},{"type":"doc","title":"Bedrock Agents — memory loads on next invocation","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-memory.html","quote":"the next time you invoke the agent with the same memoryId, the summary of each session stored in the memory gets loaded to the current session.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"agentic-rag","role":"first-class","note":"Knowledge Bases are associated with an agent; the orchestration step explicitly chooses between action invocation and KB query.","evidence":[{"type":"doc","title":"Bedrock Agents — KB augmentation","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents.html","quote":"Associate a knowledge base with the agent to augment the agent's performance.","accessed":"2026-05-20"},{"type":"doc","title":"Bedrock Agents — KB step in orchestration","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-how.html","quote":"Query an associated knowledge base (Knowledge base response generation) to retrieve additional context and summarize the data to augment its generation.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"approval-queue","role":"first-class","note":"Return of Control routes action parameters back to the developer's application instead of Lambda, enabling explicit external execution / approval gating.","evidence":[{"type":"doc","title":"Bedrock Agents — return of control mechanic","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-returncontrol.html","quote":"Rather than sending the information that your agent has elicited from the user to a Lambda function for fulfillment, you can instead choose to return control to the agent developer by sending the information in the InvokeAgent response.","accessed":"2026-05-20"},{"type":"doc","title":"Bedrock Agents — RETURN_CONTROL config","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-returncontrol.html","quote":"Through the API, you specify RETURN_CONTROL as the customControl value in the actionGroupExecutor object","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"code-execution","role":"first-class","note":"Code interpretation generates and runs code in a secure sandbox; supports analysis and file processing.","evidence":[{"type":"doc","title":"Bedrock Agents — code interpretation","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-code-interpretation.html","quote":"The code interpretation enables your agent to generate, run, and troubleshoot your application code in a secure test environment.","accessed":"2026-05-20"},{"type":"doc","title":"Bedrock Agents page — code interpretation","url":"https://aws.amazon.com/bedrock/agents/","quote":"Agents supports the ability to dynamically generate and execute code in a secure environment","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"cross-session-memory","role":"first-class","note":"Memory feature retains context across multiple sessions per memoryId with configurable retention up to 365 days.","evidence":[{"type":"doc","title":"Bedrock Agents memory — across sessions","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-memory.html","quote":"Memory provides your agent the ability to retain conversational context across multiple sessions and to recall past actions and behaviors.","accessed":"2026-05-20"},{"type":"doc","title":"Bedrock Agents memory — retention","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-memory.html","quote":"If memory is enabled, your agent retains the sessions in the memory for up to 365 days.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"orchestrator-workers","role":"first-class","note":"Multi-agent collaboration breaks tasks across specialist sub-agents under a supervisor; collaborators can also work in parallel.","evidence":[{"type":"doc","title":"Bedrock multi-agent — decomposition","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-multi-agent-collaboration.html","quote":"you can quickly assemble a team of agents that can break down tasks, assign specific tasks to domain specialist sub-agents, work in parallel","accessed":"2026-05-20"},{"type":"doc","title":"Bedrock multi-agent — plan and execute","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-multi-agent-collaboration.html","quote":"When you invoke the supervisor agent, it automatically creates and executes a plan across a set of collaborator agents and routes relevant requests and tasks to the appropriate collaborator agent.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"supervisor","role":"first-class","note":"Explicit supervisor agent designation; collaborators are associated with the supervisor and routed to by it.","evidence":[{"type":"doc","title":"Bedrock multi-agent — supervisor designation","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-multi-agent-collaboration.html","quote":"you can quickly designate an Amazon Bedrock Agent as the supervisor and then associate one or more collaborator agents with the supervisor.","accessed":"2026-05-20"},{"type":"doc","title":"Bedrock multi-agent — supervisor role","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-multi-agent-collaboration.html","quote":"Supervisor agent – Takes questions from the user, checks if the question is about the existing mortgage, new mortgage, or it is a general question and routes the question to the appropriate collaborator agent.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Action groups are the headline tool-use primitive: OpenAPI or function-detail schemas executed via Lambda or return-of-control.","evidence":[{"type":"doc","title":"Bedrock Agents — action groups","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-how.html","quote":"Action groups – You define the actions that the agent should perform for the user (through providing the following resources): One of the following schemas to define the parameters that the agent needs to elicit from the user","accessed":"2026-05-20"},{"type":"doc","title":"Bedrock Agents — OpenAPI / function-detail","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-how.html","quote":"An OpenAPI schema to define the API operations that the agent can invoke to perform its tasks. The OpenAPI schema includes the parameters that need to be elicited from the user.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"structured-output","role":"limited","note":"Action groups use OpenAPI / function-detail schemas to elicit typed parameters from the model; return-of-control surfaces those typed parameters as structured invocationInputs in the InvokeAgent response. No first-class JSON-schema mode for the final user reply is documented.","evidence":[{"type":"doc","title":"Bedrock Agents — typed parameters","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-how.html","quote":"A function detail schema to define the parameters that the agent can elicit from the user.","accessed":"2026-05-20"},{"type":"doc","title":"Bedrock Agents — function detail schema parameters","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-action-function.html","quote":"Function details consist of a list of parameters, defined by their name, data type (for a list of supported data types, see ParameterDetail), and whether they are required.","accessed":"2026-05-24"},{"type":"doc","title":"Bedrock Agents — return of control returns structured invocationInputs","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-returncontrol.html","quote":"the API or function details elicited from the user will be returned in the invocationInputs field in the InvokeAgent response, alongside a unique invocationId","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"input-output-guardrails","role":"first-class","note":"Bedrock Guardrails provide content filters, denied topics, PII redaction, and contextual grounding checks; applicable to agent inputs and outputs.","evidence":[{"type":"doc","title":"Bedrock Guardrails — overview","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html","quote":"Amazon Bedrock Guardrails provides configurable safeguards to help you build safe generative AI applications.","accessed":"2026-05-20"},{"type":"doc","title":"Bedrock Guardrails — filter types","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html","quote":"Content filters – This filter helps you detect and filter harmful text or image content in input prompts or model responses.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"limited","note":"Bedrock Agents themselves use action groups (OpenAPI / Lambda), not MCP. MCP support is provided in the adjacent Amazon Bedrock AgentCore platform: AgentCore Gateway implements MCP and converts APIs, Lambda functions, and services into MCP-compatible tools; AgentCore Runtime can deploy and run MCP servers.","evidence":[{"type":"doc","title":"Bedrock AgentCore Gateway — Model Context Protocol tooling","url":"https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/gateway.html","quote":"With Gateway, developers can convert APIs, Lambda functions, and existing services into Model Context Protocol (MCP)-compatible tools and make them available to agents through Gateway endpoints with just a few lines of code.","accessed":"2026-05-24"},{"type":"doc","title":"Bedrock AgentCore Gateway — Translation key capability","url":"https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/gateway.html","quote":"Translation - Converts agent requests using protocols like Model Context Protocol (MCP) into API requests and Lambda invocations, eliminating the need to manage protocol integration or version support.","accessed":"2026-05-24"}],"evidence_status":"full"}],"alternatives":[{"composition":"azure-ai-foundry-agent-service","relation":"competes-with","note":"Peer hyperscaler agent platform (Microsoft Azure)."},{"composition":"vertex-ai-agent-builder","relation":"competes-with","note":"Peer hyperscaler agent platform (Google Cloud)."},{"composition":"bedrock-agentcore","relation":"same-vendor","note":"Older AWS managed-agent product; AgentCore is the framework-agnostic platform layer."}],"references":[{"type":"doc","title":"Automate tasks using Amazon Bedrock Agents (user guide)","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents.html","accessed":"2026-05-20"},{"type":"doc","title":"How Amazon Bedrock Agents works","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-how.html","accessed":"2026-05-20"},{"type":"doc","title":"Multi-agent collaboration with Amazon Bedrock Agents","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-multi-agent-collaboration.html","accessed":"2026-05-20"},{"type":"doc","title":"Return control to the agent developer","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-returncontrol.html","accessed":"2026-05-20"},{"type":"doc","title":"Amazon Bedrock Guardrails","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"mature","tags":["proprietary","aws","enterprise","managed-runtime","knowledge-bases","lambda","multi-agent"]},{"id":"vertex-ai-agent-builder","name":"Vertex AI Agent Builder","aliases":["Gemini Enterprise Agent Platform","Google Cloud Agent Builder"],"kind":"framework","category":"enterprise-platform","build_surface":"full-code","vendor":"Google Cloud","language":"API (multi-language SDKs: Python, Java, Go, TypeScript via Agent Development Kit)","license":"proprietary (managed service); Agent Development Kit is open-source (Apache-2.0)","status":"active","first_released":"2024-04-09","url":"https://cloud.google.com/products/agent-builder","docs_url":"https://docs.cloud.google.com/agent-builder","intent":"Google Cloud's end-to-end platform to build, scale, and govern agents: an open-source Agent Development Kit (ADK) for code-first multi-agent design, Vertex AI Agent Engine as the managed runtime with Sessions, Memory Bank, and Code Execution, plus Agent Garden samples and governance / observability hooks.","description":"Vertex AI Agent Builder (now branded Gemini Enterprise Agent Platform) is Google Cloud's open platform for building, scaling, and governing agents. The build pillar is the Agent Development Kit (ADK) — an open-source framework in Python, Java, Go, and TypeScript that supports both code-defined custom agents and pre-built workflow agents (SequentialAgent, ParallelAgent, LoopAgent), with native multi-agent composition (sub-agents, hierarchical delegation), tools (built-in, custom, MCP, OpenAPI), and the A2A protocol for agent-to-agent communication. The scale pillar is Vertex AI Agent Engine: a managed runtime with Sessions (conversation state), Memory Bank (long-term personalised memory), and Code Execution. Agent Garden ships curated samples for one-click deployment. Governance layers add Cloud Trace observability, IAM-based agent identity, and Security Command Center threat detection.","primary_use_cases":["code-first multi-agent systems built with ADK and deployed on Agent Engine","workflow agents (sequential / parallel / loop) for predictable pipelines","long-running production agents with managed Sessions and Memory Bank","RAG agents grounded by Vertex AI Search and Vertex AI RAG Engine","agent-to-agent ecosystems over the open A2A protocol"],"agent_loop_shape":"Two-tier shape. ADK defines the agent loop: an LlmAgent reasons over instructions and decides which tool or sub-agent to call, while workflow agents (SequentialAgent / ParallelAgent / LoopAgent) compose deterministic pipelines around LLM agents. Multi-agent hierarchies delegate by transferring control to sub-agents. The composed agent is then deployed onto Agent Engine, which provides the managed runtime, Sessions for conversation state, Memory Bank for persistent long-term memory, sandboxed Code Execution, and Cloud Trace observability. Tools span built-in, custom, OpenAPI, MCP, and A2A endpoints.","key_concepts":[{"name":"Agent Development Kit (ADK)","summary":"Open-source agent framework in Python, Java, Go, TypeScript with native multi-agent composition.","url":"https://docs.cloud.google.com/agent-builder/agent-development-kit/overview"},{"name":"Workflow agents","summary":"SequentialAgent, ParallelAgent, LoopAgent compose deterministic pipelines around LLM sub-agents.","maps_to_pattern":"orchestrator-workers","url":"https://google.github.io/adk-docs/agents/workflow-agents/"},{"name":"Agent Engine","summary":"Managed runtime that scales agents in production with Sessions, Memory Bank, and Code Execution.","url":"https://docs.cloud.google.com/agent-builder/agent-engine/use/overview"},{"name":"Sessions","summary":"Maintain conversation state across turns; managed by Agent Engine.","maps_to_pattern":"agent-resumption","url":"https://docs.cloud.google.com/vertex-ai/generative-ai/docs/agent-engine/sessions/overview"},{"name":"Memory Bank","summary":"Persistent long-term memory for user preferences and facts across sessions.","maps_to_pattern":"cross-session-memory","url":"https://docs.cloud.google.com/vertex-ai/generative-ai/docs/agent-engine/sessions/overview"},{"name":"A2A protocol","summary":"Open agent-to-agent protocol; Agent2Agent integration is a first-class ADK concept.","url":"https://cloud.google.com/blog/products/ai-machine-learning/more-ways-to-build-and-scale-ai-agents-with-vertex-ai-agent-builder"},{"name":"Agent Garden","summary":"Curated repository of agent samples, solutions, and tools with one-click deployment.","url":"https://cloud.google.com/blog/products/ai-machine-learning/more-ways-to-build-and-scale-ai-agents-with-vertex-ai-agent-builder"}],"pattern_composition":"flowchart TD\n  dev[Developer] --> adk[Agent Development Kit]\n  adk --> kind{Agent kind}\n  kind -->|LlmAgent| llm[LLM agent reasoning]\n  kind -->|SequentialAgent| seq[Sequential pipeline]\n  kind -->|ParallelAgent| par[Parallel fan-out]\n  kind -->|LoopAgent| loop[Loop until condition]\n  llm --> tools[Tools]\n  tools --> builtin[Built-in / Vertex AI Search]\n  tools --> custom[Custom Python / Java]\n  tools --> mcp[MCP server]\n  tools --> a2a[A2A remote agent]\n  llm --> sub[Sub-agents / hierarchy]\n  seq --> llm\n  par --> llm\n  loop --> llm\n  adk --> deploy[Deploy: adk deploy]\n  deploy --> engine[Vertex AI Agent Engine runtime]\n  engine --> sess[Sessions]\n  engine --> mem[Memory Bank]\n  engine --> code[Code Execution]\n  engine --> trace[Cloud Trace observability]\n  engine --> iam[IAM agent identity]","members":[{"pattern":"agent-resumption","role":"first-class","note":"Agent Engine Sessions maintain conversation state across turns; sessions and memory bank are GA on Agent Engine.","evidence":[{"type":"doc","title":"Agent Engine — sessions","url":"https://docs.cloud.google.com/vertex-ai/generative-ai/docs/agent-engine/sessions/overview","quote":"Learn how to use sessions to maintain conversation state with your agents.","accessed":"2026-05-20"},{"type":"doc","title":"Agent Engine — conversation state","url":"https://docs.cloud.google.com/agent-builder/agent-engine/use/overview","quote":"Utilizing Agent Platform Memory Bank to maintain the state of continuous conversations and retrieve persistent, long-term memories","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"agentic-rag","role":"first-class","note":"Vertex AI RAG Engine is the first-class retrieval primitive: it indexes private data into a corpus, retrieves relevant context per query, and augments the LLM prompt. ADK integrates RAG Engine and Vertex AI Search.","evidence":[{"type":"doc","title":"ADK overview — RAG / Search integration","url":"https://docs.cloud.google.com/agent-builder/agent-development-kit/overview","quote":"Multiple RAG-related sections appear in the navigation menu, including \"RAG overview,\" \"RAG quickstart,\" and various RAG Engine deployment and configuration options","accessed":"2026-05-20"},{"type":"doc","title":"Vertex AI RAG Engine — overview","url":"https://cloud.google.com/vertex-ai/generative-ai/docs/rag-engine/rag-overview","quote":"Vertex AI RAG Engine facilitates Retrieval-Augmented Generation (RAG). ... Vertex AI RAG Engine creates an index called a corpus.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"code-execution","role":"first-class","note":"Agent Engine provides managed Code Execution so agents can run Python/JavaScript in a secure, isolated sandbox with state preserved across calls; supports file I/O and TTL up to 14 days.","evidence":[{"type":"doc","title":"Agent Engine — code execution","url":"https://docs.cloud.google.com/agent-builder/agent-engine/use/overview","quote":"Empowering agents to safely solve complex computational problems and automate workflows through dynamic Code Execution","accessed":"2026-05-20"},{"type":"doc","title":"Agent Engine Code Execution — secure sandbox","url":"https://docs.cloud.google.com/agent-builder/agent-engine/code-execution/overview","quote":"Secure sandbox execution: Empowering agents to safely solve complex computational problems and automate workflows through dynamic Code Execution","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"cross-session-memory","role":"first-class","note":"Memory Bank stores long-term user preferences and facts across sessions; explicit Memory Bank API.","evidence":[{"type":"doc","title":"Agent Engine — Memory Bank purpose","url":"https://docs.cloud.google.com/vertex-ai/generative-ai/docs/agent-engine/sessions/overview","quote":"Learn how to use Memory Bank to store long-term user preferences and facts.","accessed":"2026-05-20"},{"type":"doc","title":"Agent Engine — Memory Bank API","url":"https://docs.cloud.google.com/vertex-ai/generative-ai/docs/agent-engine/sessions/overview","quote":"Get started with the Memory Bank API to manage long-term memories.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"orchestrator-workers","role":"first-class","note":"ADK ships SequentialAgent, ParallelAgent, and LoopAgent for deterministic orchestration of sub-agents.","evidence":[{"type":"doc","title":"ADK — SequentialAgent","url":"https://google.github.io/adk-docs/agents/workflow-agents/","quote":"Executes sub-agents one after another, in sequence.","accessed":"2026-05-20"},{"type":"doc","title":"ADK — ParallelAgent and LoopAgent","url":"https://google.github.io/adk-docs/agents/workflow-agents/","quote":"Executes multiple sub-agents in parallel.","accessed":"2026-05-20"},{"type":"doc","title":"ADK overview — flexible orchestration","url":"https://docs.cloud.google.com/agent-builder/agent-development-kit/overview","quote":"Users can leverage flexible orchestration to define predictable pipelines using workflow agents or rely on agent-coordinated dynamic routing.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"supervisor","role":"first-class","note":"ADK organises agents in a tree (root_agent + sub_agents). Parent LlmAgents delegate to sub_agents via LLM-driven auto-flow (selected by their descriptions) or via a transfer_to_agent tool action.","evidence":[{"type":"doc","title":"ADK — multi-agent architectures","url":"https://docs.cloud.google.com/agent-builder/agent-development-kit/overview","quote":"It natively supports multi-agent architectures, allowing developers to compose specialized teams of agents that can collaborate and delegate tasks.","accessed":"2026-05-20"},{"type":"doc","title":"ADK multi-agent patterns — LLM-driven delegation via sub_agents","url":"https://developers.googleblog.com/developers-guide-to-multi-agent-patterns-in-adk/","quote":"This relies on LLM-driven delegation. You simply define a parent CoordinatorAgent and provide a list of specialist sub_agents.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"ADK exposes a rich tool ecosystem: built-in tools, third-party integrations, and custom code; Google Search grounding included.","evidence":[{"type":"doc","title":"ADK — tool ecosystem","url":"https://docs.cloud.google.com/agent-builder/agent-development-kit/overview","quote":"ADK features a rich tool ecosystem for integrating third-party applications and your own custom code.","accessed":"2026-05-20"},{"type":"doc","title":"ADK docs — tool surface","url":"https://adk.dev/","quote":"Supports \"Custom Tools\", \"MCP tools\", \"OpenAPI tools\"","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"ADK has a dedicated MCP section and ships MCP tool support; MCP tools sit alongside Custom Tools and OpenAPI tools as a first-class tool kind.","evidence":[{"type":"doc","title":"ADK docs — MCP section","url":"https://adk.dev/","quote":"Dedicated \"MCP\" section for Model Context Protocol integration","accessed":"2026-05-20"},{"type":"doc","title":"ADK docs — tool surface includes MCP tools","url":"https://adk.dev/","quote":"Supports \"Custom Tools\", \"MCP tools\", \"OpenAPI tools\"","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"structured-output","role":"supported","note":"ADK supports structured output on LlmAgent via output_schema (typically a Pydantic BaseModel). When set, the agent's final response must be a JSON string conforming to that schema; mutually exclusive with tool use in the same request on most models.","evidence":[{"type":"doc","title":"ADK LlmAgent — output_schema enforces JSON schema","url":"https://adk.dev/agents/llm-agents/","quote":"output_schema (Optional): Define a schema representing the desired output structure. If set, the agent's final response must be a JSON string conforming to this schema.","accessed":"2026-05-24"},{"type":"doc","title":"ADK LlmAgent — schema via Pydantic BaseModel","url":"https://adk.dev/agents/llm-agents/","quote":"The input and output schema is typically a Pydantic BaseModel.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"eval-harness","role":"first-class","note":"Evaluation is a first-class ADK area: criteria, user simulation, custom metrics; partner evaluation tools and offline / online monitors.","evidence":[{"type":"doc","title":"ADK docs — evaluation","url":"https://adk.dev/","quote":"Evaluation section covering Criteria, User Simulation, Custom Metrics","accessed":"2026-05-20"},{"type":"doc","title":"ADK overview — evaluation tools","url":"https://docs.cloud.google.com/agent-builder/agent-development-kit/overview","quote":"Apply built-in and partner evaluation tools to test execution trajectories","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"input-output-guardrails","role":"first-class","note":"ADK exposes Callbacks as the design-pattern surface for guardrails; Safety and Security and Responsible AI are documented sections.","evidence":[{"type":"doc","title":"ADK docs — callbacks and safety","url":"https://adk.dev/","quote":"Includes \"Callbacks\" with \"Types of callbacks\" and \"design patterns\"","accessed":"2026-05-20"},{"type":"doc","title":"ADK overview — Responsible AI","url":"https://docs.cloud.google.com/agent-builder/agent-development-kit/overview","quote":"Safety (Responsible AI) appears in the governance navigation menu","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"multi-model-routing","role":"first-class","note":"ADK is model-agnostic with adapters for many providers; can target Gemini, Claude, or other models.","evidence":[{"type":"doc","title":"ADK docs — LLM-agnostic","url":"https://adk.dev/","quote":"ADK can work with almost any generative AI model","accessed":"2026-05-20"},{"type":"doc","title":"ADK docs — provider adapters","url":"https://adk.dev/","quote":"Provides adapters for \"many other models and model providers\"","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"adaptive-compute-allocation","role":"first-class","note":"Gemini's thinkingBudget parameter set to -1 enables dynamic thinking, where the model itself adjusts the number of thinking tokens spent based on the difficulty/complexity of each request rather than a fixed budget.","evidence":[{"type":"doc","title":"Gemini API — Thinking","url":"https://ai.google.dev/gemini-api/docs/thinking","quote":"Setting the thinkingBudget to -1 turns on dynamic thinking, meaning the model will adjust the budget based on the complexity of the request.","accessed":"2026-06-17"},{"type":"doc","title":"Gemini API — Thinking","url":"https://ai.google.dev/gemini-api/docs/thinking","quote":"The thinkingBudget parameter, introduced with the Gemini 2.5 series, guides the model on the specific number of thinking tokens to use for reasoning.","accessed":"2026-06-17"}],"evidence_status":"full"}],"alternatives":[{"composition":"azure-ai-foundry-agent-service","relation":"competes-with","note":"Peer hyperscaler agent platform (Microsoft Azure)."},{"composition":"bedrock-agents","relation":"competes-with","note":"Peer hyperscaler agent platform (AWS)."},{"composition":"bedrock-agentcore","relation":"competes-with","note":"Google Cloud's enterprise agent platform."}],"references":[{"type":"doc","title":"Gemini Enterprise Agent Platform (formerly Vertex AI Agent Builder)","url":"https://cloud.google.com/products/agent-builder","accessed":"2026-05-20"},{"type":"doc","title":"Vertex AI Agent Builder — Agent Development Kit overview","url":"https://docs.cloud.google.com/agent-builder/agent-development-kit/overview","accessed":"2026-05-20"},{"type":"doc","title":"Vertex AI Agent Engine — use overview","url":"https://docs.cloud.google.com/agent-builder/agent-engine/use/overview","accessed":"2026-05-20"},{"type":"doc","title":"Agent Development Kit documentation","url":"https://adk.dev/","accessed":"2026-05-20"},{"type":"blog","title":"More ways to build and scale AI agents with Vertex AI Agent Builder","url":"https://cloud.google.com/blog/products/ai-machine-learning/more-ways-to-build-and-scale-ai-agents-with-vertex-ai-agent-builder","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-06-19","date_added":"2026-04-30","status_in_practice":"mature","tags":["proprietary","google-cloud","enterprise","managed-runtime","adk","agent-engine","mcp","a2a"],"anti_patterns_avoided":[{"pattern":"hallucinated-citations","note":"A grounding-citation threshold filters out responses at inference time that are likely to contain hallucinated claims.","evidence":[{"type":"doc","title":"Vertex AI — check grounding","url":"https://docs.cloud.google.com/generative-ai-app-builder/docs/check-grounding","quote":"By setting a citation threshold, chat bots can filter out responses at inference time that are likely to contain hallucinated claims.","accessed":"2026-06-19"}]}]},{"id":"wanzhi","name":"Wanzhi 2.5 (万智)","aliases":["万智","Wanzhi","零一万物 万智"],"kind":"framework","category":"enterprise-platform","build_surface":"closed-application","vendor":"01.AI (零一万物)","language":"N/A (hosted)","license":"proprietary","status":"active","first_released":"2026","url":"https://www.lingyiwanwu.com/","intent":"Deliver an enterprise multi-agent platform where role-specialised agents form a coordinated 'silicon team' that executes business workflows over enterprise data and tools.","description":"Wanzhi 2.5 (万智) is 01.AI's enterprise multi-agent platform, launched in 2026 and marketed as bringing 'silicon teams' (硅基团队) of role-specialised agents into production for business tasks. It coordinates multiple specialised agents over enterprise data and tools under a managing layer, and is framed by 01.AI as marking the year enterprise multi-agent systems move to scaled deployment.","primary_use_cases":["enterprise workflow automation","multi-agent business teams","document and data processing"],"agent_loop_shape":"A coordinating layer dispatches subtasks to role-specialised agents that work over enterprise data and tools, then aggregates their results.","members":[{"pattern":"orchestrator-workers","role":"core","note":"A coordinating layer dispatches subtasks to role-specialised worker agents.","evidence":[{"type":"blog","title":"零一万物发布万智2.5企业级多智能体 [translated: 01.AI releases Wanzhi 2.5 enterprise multi-agent]","url":"https://www.lingyiwanwu.com/blog/2-5-2026","accessed":"2026-06-02"}],"evidence_status":"none"},{"pattern":"hierarchical-agents","role":"first-class","note":"Specialised agents are organised under a managing/coordinating tier ('silicon team').","evidence_status":"none"},{"pattern":"tool-use","role":"supported","note":"Agents act over enterprise data sources and tools.","evidence_status":"none"}],"references":[{"type":"blog","title":"零一万物发布万智2.5企业级多智能体，开启2026“硅基团队”上岗元年","url":"https://www.lingyiwanwu.com/blog/2-5-2026","accessed":"2026-06-02"},{"type":"doc","title":"01.AI (零一万物) homepage","url":"https://www.lingyiwanwu.com/","accessed":"2026-06-02"}],"last_analyzed":"2026-06-02","verification_status":"stub","last_updated":"2026-06-17","date_added":"2026-06-02","status_in_practice":"emerging","pattern_composition":"flowchart TD\n  fw[\"Wanzhi 2.5 (万智)\"]\n  fw --> p1[\"Orchestrator-Workers<br/>(core)\"]\n  fw --> p2[\"Hierarchical Agents<br/>(first-class)\"]\n  fw --> p3[\"Tool Use<br/>(supported)\"]","tags":["enterprise-platform","multi-agent","tool-use-environment","closed-application"],"key_concepts":[{"name":"Orchestrator-Workers","summary":"A coordinating layer dispatches subtasks to role-specialised worker agents.","maps_to_pattern":"orchestrator-workers"},{"name":"Hierarchical Agents","summary":"Specialised agents are organised under a managing/coordinating tier ('silicon team').","maps_to_pattern":"hierarchical-agents"},{"name":"Tool Use","summary":"Agents act over enterprise data sources and tools.","maps_to_pattern":"tool-use"}]},{"id":"bedrock-agentcore","name":"Amazon Bedrock AgentCore","kind":"framework","category":"enterprise-platform","build_surface":"full-code","vendor":"Amazon Web Services","language":"Polyglot (managed service + SDKs)","license":"proprietary (AWS managed service)","status":"active","url":"https://aws.amazon.com/bedrock/agentcore/","docs_url":"https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/what-is-bedrock-agentcore.html","intent":"Build, deploy and operate agents securely at scale using any framework and foundation model.","description":"AWS agentic platform of modular, independently-usable services: Runtime (serverless agent execution with session isolation), Harness (a managed agent loop invoked with one API call), Memory, Gateway (turns APIs/Lambda into MCP tools), Identity, Code Interpreter, Browser, Observability (OTEL), Policy (Cedar), Payments (x402), Evaluations and Registry. Framework-agnostic — works with CrewAI, LangGraph, LlamaIndex, Strands, Google ADK and OpenAI Agents SDK — and model-agnostic.","primary_use_cases":["running any-framework agents serverless with session isolation at scale","wrapping existing APIs/Lambdas as MCP tools via Gateway","enterprise governance: identity, policy gating, observability and evaluation for agents"],"agent_loop_shape":"Either bring your own framework's loop onto AgentCore Runtime (serverless, isolated microVM sessions), or use AgentCore Harness — a managed agent loop where you specify model, system prompt and tools inline and the platform handles orchestration, tool execution, memory and response generation. Cross-cutting services (Memory, Gateway, Identity, Policy, Observability) attach to either.","key_concepts":[{"name":"Harness","summary":"Managed agent loop invoked with a single API call.","maps_to_pattern":"react","url":"https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/harness.html"},{"name":"Runtime","summary":"Serverless, session-isolated microVM execution for agents.","maps_to_pattern":"sandbox-isolation","url":"https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/what-is-bedrock-agentcore.html"},{"name":"Gateway","summary":"Turns APIs, Lambda and services into MCP-compatible tools.","maps_to_pattern":"mcp","url":"https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/what-is-bedrock-agentcore.html"},{"name":"Policy","summary":"Deterministic Cedar/NL rules intercept tool calls before execution.","maps_to_pattern":"policy-as-code-gate","url":"https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/what-is-bedrock-agentcore.html"}],"pattern_composition":"flowchart TD\n  AGENT[Any-framework agent] --> RT[Runtime<br/>serverless microVM session]\n  AGENT --> HARNESS[Harness<br/>managed agent loop]\n  RT --> GW[Gateway<br/>APIs to MCP tools]\n  HARNESS --> GW\n  RT --> MEM[(Memory)]\n  GW --> POLICY[Policy gate<br/>Cedar]\n  POLICY --> TOOLS[Tools]\n  RT --> ID[Identity]\n  RT --> OBS[Observability<br/>OTEL]\n  GW --> PAY[Payments x402]","members":[{"pattern":"react","role":"core","note":"A managed agent loop that lets you define and invoke AI agents with a single API call. Specify a model, system prompt, and tools inline — Harness handles orc...","evidence":[{"type":"doc","title":"What is Amazon Bedrock AgentCore?","url":"https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/what-is-bedrock-agentcore.html","quote":"A managed agent loop that lets you define and invoke AI agents with a single API call. Specify a model, system prompt, and tools inline — Harness handles orchestration, tool execution, memory management, and response generation.","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"sandbox-isolation","role":"first-class","note":"Each session runs in an isolated microVM with filesystem and shell access","evidence":[{"type":"doc","title":"What is Amazon Bedrock AgentCore?","url":"https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/what-is-bedrock-agentcore.html","quote":"Each session runs in an isolated microVM with filesystem and shell access","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"mcp","role":"first-class","note":"A secure way to convert your APIs, Lambda functions, and existing services into Model Context Protocol (MCP)-compatible tools","evidence":[{"type":"doc","title":"What is Amazon Bedrock AgentCore?","url":"https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/what-is-bedrock-agentcore.html","quote":"A secure way to convert your APIs, Lambda functions, and existing services into Model Context Protocol (MCP)-compatible tools","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"cross-session-memory","role":"first-class","note":"Supports for both short-term memory for multi-turn conversations and long-term memory that persists across sessions","evidence":[{"type":"doc","title":"What is Amazon Bedrock AgentCore?","url":"https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/what-is-bedrock-agentcore.html","quote":"Supports for both short-term memory for multi-turn conversations and long-term memory that persists across sessions","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"policy-as-code-gate","role":"first-class","note":"Easily author fine-grained rules using natural language or Cedar (AWS's open-source policy language).","evidence":[{"type":"doc","title":"What is Amazon Bedrock AgentCore?","url":"https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/what-is-bedrock-agentcore.html","quote":"Easily author fine-grained rules using natural language or Cedar (AWS's open-source policy language).","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"delegated-agent-authorization","role":"first-class","note":"A secure, scalable agent identity, access and authentication management service which is compatible with existing identity providers","evidence":[{"type":"doc","title":"What is Amazon Bedrock AgentCore?","url":"https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/what-is-bedrock-agentcore.html","quote":"A secure, scalable agent identity, access and authentication management service which is compatible with existing identity providers","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"agent-initiated-payment","role":"supported","note":"A fully managed service that enables microtransaction payments for AI agents to access paid APIs, MCP servers, and content using the x402 protocol.","evidence":[{"type":"doc","title":"What is Amazon Bedrock AgentCore?","url":"https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/what-is-bedrock-agentcore.html","quote":"A fully managed service that enables microtransaction payments for AI agents to access paid APIs, MCP servers, and content using the x402 protocol.","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"eval-harness","role":"supported","note":"A purpose-built evaluation service for automated, consistent, and data-driven agent assessment.","evidence":[{"type":"doc","title":"What is Amazon Bedrock AgentCore?","url":"https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/what-is-bedrock-agentcore.html","quote":"A purpose-built evaluation service for automated, consistent, and data-driven agent assessment.","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"managed-agent-runtime","role":"core","note":"AgentCore Harness exposes a managed agent loop invoked with a single API call (model, system prompt, and tools), and AgentCore Runtime runs each session in an isolated microVM with built-in identity and observability.","evidence":[{"type":"doc","title":"What is Amazon Bedrock AgentCore?","url":"https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/what-is-bedrock-agentcore.html","accessed":"2026-06-04"}],"evidence_status":"full"},{"pattern":"agent-credential-vault","role":"first-class","note":"AgentCore Identity provides agent identity, access, and credential providers compatible with Cognito, Okta, Entra ID, and Auth0, brokering the identity the agent authenticates as.","evidence":[{"type":"doc","title":"What is Amazon Bedrock AgentCore? (Identity)","url":"https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/what-is-bedrock-agentcore.html","accessed":"2026-06-04"}],"evidence_status":"full"},{"pattern":"velocity-magnitude-governor","role":"supported","note":"AWS blog and docs: per-session maximum spend amount cap enforced outside the model (infrastructure layer); requests exceeding the budget are rejected before signing; CloudWatch alarms on spend velocity. Maps to the governor's magnitude cap + reject-before-execution + velocity monitoring. Note: AgentCore alarms on velocity rather than auto-downgrading autonomy, so the statistical-deviation autonomy-downgrade leg of the pattern is not claimed.","evidence":[{"type":"doc","title":"Amazon Bedrock AgentCore","url":"https://aws.amazon.com/blogs/machine-learning/enable-safe-agentic-payments-with-built-in-guardrails-using-amazon-bedrock-agentcore-payments/","quote":"","accessed":"2026-06-14"}],"evidence_status":"none"},{"pattern":"ephemeral-agent-identity","role":"first-class","note":"Each agent invocation exchanges its workload token for short-lived, session-bound tokens scoped to the user's consent, so the agent holds only least-privilege credentials for that run rather than a standing secret.","evidence":[{"type":"blog","title":"Secure AI agents with Amazon Bedrock AgentCore Identity on Amazon ECS","url":"https://aws.amazon.com/blogs/machine-learning/secure-ai-agents-with-amazon-bedrock-agentcore-identity-on-amazon-ecs/","quote":"scoped delegation that limits the agent to only the permissions the user approved","accessed":"2026-06-17"},{"type":"blog","title":"Secure AI agents with Amazon Bedrock AgentCore Identity on Amazon ECS","url":"https://aws.amazon.com/blogs/machine-learning/secure-ai-agents-with-amazon-bedrock-agentcore-identity-on-amazon-ecs/","quote":"Auth tokens scoped to each user session, following least-privilege principles","accessed":"2026-06-17"}],"evidence_status":"full"}],"instantiates":["agent-runtime-cross-cutting","production-llm-platform"],"alternatives":[{"composition":"bedrock-agents","relation":"same-vendor","note":"Older AWS managed-agent product; AgentCore is the framework-agnostic platform layer."},{"composition":"azure-ai-foundry-agent-service","relation":"competes-with","note":"Microsoft's enterprise agent-runtime platform."},{"composition":"vertex-ai-agent-builder","relation":"competes-with","note":"Google Cloud's enterprise agent platform."},{"composition":"strands-agents","relation":"complements","note":"AgentCore runs Strands agents as a first-class supported framework."}],"references":[{"type":"doc","title":"What is Amazon Bedrock AgentCore?","url":"https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/what-is-bedrock-agentcore.html","accessed":"2026-06-04"},{"type":"doc","title":"Amazon Bedrock AgentCore overview","url":"https://aws.amazon.com/bedrock/agentcore/","accessed":"2026-06-04"},{"type":"doc","title":"AgentCore Harness — managed agent loop","url":"https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/harness.html","accessed":"2026-06-04"}],"last_analyzed":"2026-06-04","verification_status":"partial","last_updated":"2026-06-17","date_added":"2026-06-04","status_in_practice":"emerging","tags":["aws","enterprise","agent-runtime","harness","mcp","governance","proprietary"]},{"id":"bedrock-guardrails","name":"Amazon Bedrock Guardrails (sensitive information filters)","kind":"framework","category":"enterprise-platform","vendor":"AWS","language":"N/A","license":"proprietary","status":"active","first_released":"2024-04-23","url":"https://aws.amazon.com/bedrock/guardrails/","docs_url":"https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html","intent":"Amazon Bedrock Guardrails applies configurable content, topic, word, sensitive-information, and grounding filters to both user prompts and model responses in Bedrock generative AI applications.","description":"Amazon Bedrock Guardrails is a managed safety service from AWS that screens generative AI traffic on the Bedrock platform. It evaluates input prompts and model completions against configured policies, including content filters, denied topics, word filters, sensitive information filters, and contextual grounding checks. Guardrails can be invoked inline during model inference or standalone through the ApplyGuardrail API, blocking or masking content that violates a policy.","primary_use_cases":["filtering harmful prompts and responses in chatbots","redacting PII from conversation transcripts","blocking denied topics in regulated applications","detecting hallucinations in RAG responses"],"agent_loop_shape":"Guardrails is an inline filtering stage rather than an agent loop: each user input and each model completion passes through the configured policy filters, and content that violates a policy is blocked or masked before it reaches the model or the user.","build_surface":"low-code","members":[{"pattern":"multimodal-guardrails","role":"first-class","note":"Applies the same content-filter policy categories to image content as to text, blocking harmful multimodal content across hate, insults, sexual, violence, misconduct, and prompt-attack categories.","evidence":[{"type":"changelog","title":"Amazon Bedrock Guardrails announces the general availability of industry-leading image content filters","url":"https://aws.amazon.com/about-aws/whats-new/2025/03/amazon-bedrock-guardrails-general-availability-image-content-filters","quote":"Image content filters can be applied to all categories within the content filter policy of Bedrock Guardrails including hate, insults, sexual, violence, misconduct, and prompt attack.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"pii-redaction","role":"first-class","note":"Sensitive information filters detect PII in input prompts and model responses and either block the content or mask/anonymize it by replacing each entity with its PII type.","evidence":[{"type":"doc","title":"Remove PII from conversations by using sensitive information filters - Amazon Bedrock","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-sensitive-filters.html","quote":"Amazon Bedrock Guardrails helps detect sensitive information, such as personally identifiable information (PII), in input prompts or model responses using sensitive information filters.","accessed":"2026-06-17"},{"type":"doc","title":"Sensitive information filters — Mask mode","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-sensitive-filters.html","quote":"Sensitive information filter policies can anonymize or redact information from model requests or responses. For example, guardrails mask PIIs while generating summaries of conversations between users and customer service agents.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"input-output-guardrails","role":"core","note":"A single guardrail evaluates both the input prompts and the model completions against the defined filters, so the same policy governs what reaches the model and what reaches the user.","evidence":[{"type":"doc","title":"Detect and filter harmful content by using Amazon Bedrock Guardrails","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html","quote":"If a guardrail is used, it will evaluate the input prompts and the FM completions against the defined filters.","accessed":"2026-06-17"},{"type":"doc","title":"Detect and filter harmful content by using Amazon Bedrock Guardrails","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html","quote":"you can also configure the messages to be returned to the user if a user input or model response is in violation of the filters defined in the guardrail.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"prompt-injection-defense","role":"first-class","note":"A dedicated Prompt Attack content-filter category screens user input prompts for prompt-injection and jailbreak attempts and blocks them before they reach the model, so injected instructions in user-supplied content are filtered out.","evidence":[{"type":"doc","title":"Block harmful words and conversations with content filters - Amazon Bedrock","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-content-filters.html","quote":"You can select the filter for prompt attacks in the harmful categories. Configure how strict you want each filter to be for prompts that the user provides to the model.","accessed":"2026-06-17"},{"type":"doc","title":"Detect and filter harmful content by using Amazon Bedrock Guardrails","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html","quote":"Filtering is done based on detection of certain predefined harmful content categories: Hate, Insults, Sexual, Violence, Misconduct and Prompt Attack.","accessed":"2026-06-17"}],"evidence_status":"full"}],"key_concepts":[{"name":"Content filters","summary":"Configurable strength filters over predefined harmful categories (Hate, Insults, Sexual, Violence, Misconduct, Prompt Attack) applied to text and image prompts and responses.","maps_to_pattern":"input-output-guardrails","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-content-filters.html"},{"name":"Denied topics","summary":"A set of application-defined subjects that the guardrail blocks if it detects them in user queries or model responses, even when the content is otherwise harmless.","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html"},{"name":"Contextual grounding checks","summary":"A filter that flags or blocks model responses in RAG applications when they are not grounded in the retrieved source or are irrelevant to the user's query, used to catch hallucinations.","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html"},{"name":"ApplyGuardrail API","summary":"An API that runs the configured guardrail over arbitrary text without invoking a foundation model, so the same policy can screen content independently of inference.","maps_to_pattern":"input-output-guardrails","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html"}],"references":[{"type":"doc","title":"Detect and filter harmful content by using Amazon Bedrock Guardrails","url":"https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html","quote":"Amazon Bedrock Guardrails provides configurable safeguards to help you build safe generative AI applications.","accessed":"2026-06-17"},{"type":"doc","title":"Amazon Bedrock Guardrails (product page)","url":"https://aws.amazon.com/bedrock/guardrails/","accessed":"2026-06-17"}],"alternatives":[{"composition":"bedrock-agents","relation":"same-vendor","note":"Both are AWS Bedrock services; Bedrock Guardrails is commonly attached as the safety filter for agents built on Bedrock Agents."},{"composition":"bedrock-agentcore","relation":"same-vendor","note":"Both are AWS agent-platform services; AgentCore is the managed runtime for agents and Bedrock Guardrails provides the content-and-PII filtering boundary those agents run behind."}],"verification_status":"partial","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Amazon Bedrock Guardrails (sensitive information filters)\"]\n  fw --> p1[\"Input/Output Guardrails<br/>(core)\"]\n  fw --> p2[\"Multimodal Guardrails<br/>(first-class)\"]\n  fw --> p3[\"PII Redaction<br/>(first-class)\"]\n  fw --> p4[\"Prompt Injection Defense<br/>(first-class)\"]","tags":["enterprise-platform","safety-control","low-code"]},{"id":"invariant-labs","name":"Invariant Guardrails / MCP-scan","kind":"framework","category":"enterprise-platform","vendor":"Invariant Labs (acquired by Snyk)","language":"Python","license":"Apache-2.0","status":"active","first_released":"2024","url":"https://invariantlabs.ai","repo":"https://github.com/invariantlabs-ai/invariant","docs_url":"https://github.com/invariantlabs-ai/invariant/blob/main/README.md","intent":"Invariant provides a security layer for AI agents that statically scans agent execution traces and MCP servers to detect threats such as prompt injection, tool poisoning, and data leaks.","description":"Invariant Guardrails and MCP-scan are agent security tools from Invariant Labs, an ETH Zurich spin-off acquired by Snyk in 2025. The analyzer applies information-flow analysis over an agent's execution traces, expressed as policy rules, to flag unsafe tool-call sequences and threats. MCP-scan inspects Model Context Protocol servers for risks such as tool poisoning and prompt injection embedded in tool descriptions.","primary_use_cases":["scanning agent execution traces for threats","detecting prompt injection and tool poisoning in MCP servers","enforcing data-flow and tool-sequence policies","version-pinning untrusted MCP tools"],"agent_loop_shape":"Invariant runs as an out-of-band analysis layer rather than an agent loop: it ingests an agent's execution traces and MCP server definitions and runs static information-flow analysis over them, raising policy violations when unsafe flows or injected instructions are detected.","build_surface":"full-code","members":[{"pattern":"tool-output-poisoning","role":"supported","note":"Documents that MCP tool descriptions and output cannot be trusted and prescribes version pinning plus its Guardrails/MCP-scan tooling to constrain untrusted tool content.","evidence":[{"type":"blog","title":"MCP Security Notification: Tool Poisoning Attacks","url":"https://invariantlabs.ai/blog/mcp-security-notification-tool-poisoning-attacks","quote":"A Tool Poisoning Attack occurs when malicious instructions are embedded within MCP tool descriptions that are invisible to users but visible to AI models.","accessed":"2026-06-17"},{"type":"blog","title":"MCP Security Notification: Tool Poisoning Attacks","url":"https://invariantlabs.ai/blog/mcp-security-notification-tool-poisoning-attacks","quote":"Clients should pin the version of the MCP server and its tools to prevent unauthorized changes.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"trajectory-anomaly-monitor","role":"core","note":"A static-analysis scanning engine based on information-flow analysis runs over the agent's execution traces to flag bugs such as looping behavior and threats such as prompt injections and unsafe code execution.","evidence":[{"type":"repo","title":"invariantlabs-ai/invariant README (Analyzer)","url":"https://github.com/invariantlabs-ai/invariant/blob/66491bfd01cc69e19a5706372f74102fd36ccdf3/README.md","quote":"The analyzer scans an agent's execution traces to identify bugs (e.g., looping behavior) and threats (e.g., data leaks, prompt injections, and unsafe code execution).","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"policy-as-code-gate","role":"core","note":"Guardrails are written as machine-readable, Python-inspired matching rules deployed between the application and its LLM or MCP servers; the engine evaluates the rules on each request and errors out the ones that violate them, so the policy lives outside the prompt.","evidence":[{"type":"repo","title":"invariantlabs-ai/invariant README","url":"https://github.com/invariantlabs-ai/invariant/blob/main/README.md","quote":"Guardrailing rules are simple Python-inspired matching rules, that can be written to identify and prevent malicious agent behavior","accessed":"2026-06-17"},{"type":"repo","title":"invariantlabs-ai/invariant README","url":"https://github.com/invariantlabs-ai/invariant/blob/main/README.md","quote":"This rule will scan all LLM messages (including assistant and user messages) for the banned phrase, and error out LLM and MCP requests that violate the pattern.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"prompt-injection-defense","role":"supported","note":"Rules can call a built-in prompt_injection detector over tool output, so a guardrail can block a downstream tool call when injected instructions are detected in untrusted tool content.","evidence":[{"type":"repo","title":"invariantlabs-ai/invariant README (prompt_injection detector)","url":"https://github.com/invariantlabs-ai/invariant/blob/main/README.md","quote":"from invariant.detectors import prompt_injection","accessed":"2026-06-17"},{"type":"repo","title":"invariantlabs-ai/invariant README (prompt_injection rule)","url":"https://github.com/invariantlabs-ai/invariant/blob/main/README.md","quote":"prompt_injection(output.content, threshold=0.7)","accessed":"2026-06-17"}],"evidence_status":"full"}],"key_concepts":[{"name":"Guardrailing rules","summary":"Python-inspired matching rules that pattern-match over LLM messages and tool calls; when a trace matches a rule the engine raises and can block the offending LLM or MCP request.","maps_to_pattern":"policy-as-code-gate","url":"https://github.com/invariantlabs-ai/invariant/blob/main/README.md"},{"name":"Analyzer","summary":"The information-flow analysis engine that scans an agent's execution traces to identify bugs such as looping behavior and threats such as data leaks, prompt injections, and unsafe code execution.","maps_to_pattern":"trajectory-anomaly-monitor","url":"https://github.com/invariantlabs-ai/invariant/blob/main/README.md"},{"name":"MCP-scan","summary":"A tool that inspects Model Context Protocol servers for risks such as tool poisoning and prompt injection embedded in tool descriptions, and supports version-pinning untrusted tools.","maps_to_pattern":"tool-output-poisoning","url":"https://invariantlabs.ai/blog/mcp-security-notification-tool-poisoning-attacks"},{"name":"Detectors","summary":"A standard library of operations (such as prompt_injection and PII detectors) that guardrailing rules invoke to test message and tool-output content for threats.","maps_to_pattern":"prompt-injection-defense","url":"https://github.com/invariantlabs-ai/invariant/blob/main/README.md"}],"references":[{"type":"repo","title":"invariantlabs-ai/invariant","url":"https://github.com/invariantlabs-ai/invariant/blob/main/README.md","quote":"Guardrailing rules are simple Python-inspired matching rules, that can be written to identify and prevent malicious agent behavior","accessed":"2026-06-17"},{"type":"blog","title":"MCP Security Notification: Tool Poisoning Attacks","url":"https://invariantlabs.ai/blog/mcp-security-notification-tool-poisoning-attacks","quote":"A Tool Poisoning Attack occurs when malicious instructions are embedded within MCP tool descriptions that are invisible to users but visible to AI models.","accessed":"2026-06-17"}],"alternatives":[{"composition":"bedrock-agentcore","relation":"complements","note":"AgentCore is a managed runtime that hosts the agent loop, whereas Invariant is a provider-neutral security layer that scans the agent's traces and MCP servers for tool-level threats and can sit in front of any such runtime."}],"verification_status":"needs-verification","status_in_practice":"emerging","last_analyzed":"2026-06-17","last_updated":"2026-06-19","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Invariant Guardrails / MCP-scan\"]\n  fw --> p1[\"Trajectory Anomaly Monitor<br/>(core)\"]\n  fw --> p2[\"Policy-as-Code Gate<br/>(core)\"]\n  fw --> p3[\"Tool Output Poisoning Defense<br/>(supported)\"]\n  fw --> p4[\"Prompt Injection Defense<br/>(supported)\"]","tags":["enterprise-platform","safety-control","full-code"],"anti_patterns_avoided":[{"pattern":"agent-speed-incident-response-gap","note":"Guardrails proxy intercepts and blocks tool calls inline at machine speed, closing the gap between an agent acting and a human responding.","evidence":[{"type":"doc","title":"Invariant Guardrails","url":"https://github.com/invariantlabs-ai/invariant","quote":"Guardrails integrates transparently as MCP or LLM proxy, checking and intercepting tool calls automatically based on your rules","accessed":"2026-06-19"}]},{"pattern":"agentic-supply-chain-compromise","note":"mcp-scan pins tool definitions by hash and detects MCP rug-pull changes, guarding the agent tool supply chain.","evidence":[{"type":"doc","title":"Invariant Labs — mcp-scan","url":"https://invariantlabs.ai/blog/introducing-mcp-scan","quote":"Tool Pinning to detect and prevent MCP Rug Pull attacks, verifying the integrity of installed tools by tracking changes via tool hashing","accessed":"2026-06-19"}]},{"pattern":"ai-targeted-comment-injection","note":"mcp-scan detects tool-poisoning and prompt-injection instructions hidden in MCP tool descriptions before the agent executes them.","evidence":[{"type":"doc","title":"Invariant Labs — mcp-scan","url":"https://invariantlabs.ai/blog/introducing-mcp-scan","quote":"Tool Poisoning Attacks: Hidden malicious instructions embedded in MCP tool descriptions. ... Prompt Injection Attacks: Malicious instructions contained within tool descriptions that could be executed by the agent.","accessed":"2026-06-19"}]},{"pattern":"context-gap-security","note":"The guardrails proxy intercepts tool calls and enforces security rules the agent itself lacks the context to apply.","evidence":[{"type":"doc","title":"Invariant Guardrails","url":"https://github.com/invariantlabs-ai/invariant","quote":"Guardrails integrates transparently as MCP or LLM proxy, checking and intercepting tool calls automatically based on your rules.","accessed":"2026-06-19"}]},{"pattern":"rogue-agent-drift","note":"Explorer surfaces anomalies in an agent's decision-making across runs, catching drift from intended behaviour.","evidence":[{"type":"doc","title":"Invariant — Explorer","url":"https://invariantlabs.ai/blog/explorer","quote":"By highlighting key steps in an agent's decision-making process, Explorer enables you to identify bottlenecks, anomalies, or bugs with ease.","accessed":"2026-06-19"}]}]},{"id":"langfuse","name":"Langfuse","kind":"framework","category":"enterprise-platform","vendor":"Langfuse (Finto Technologies GmbH)","language":"TypeScript","license":"MIT","status":"active","first_released":"2023","url":"https://langfuse.com","repo":"https://github.com/langfuse/langfuse","docs_url":"https://langfuse.com/docs","intent":"Langfuse records production traces of LLM and agent applications and runs prompt management, datasets, and evaluations against them so teams can debug and measure their applications.","description":"Langfuse is an open-source LLM engineering platform built by Langfuse GmbH and is self-hostable. Applications send traces of their LLM calls, tool calls, and agent steps to Langfuse through SDKs and integrations, and the platform stores them for inspection. On top of those traces it provides prompt versioning, dataset-based experiments, and model-based evaluation, so it sits beside an application as an observability and evaluation backend rather than running the agent loop itself.","primary_use_cases":["tracing and observability for LLM and agent applications","prompt management and versioning","LLM-as-a-judge evaluation of production traces","dataset-based experiments and offline evaluation","cost and latency monitoring"],"agent_loop_shape":"Langfuse does not run an agent loop; it is an out-of-band observability sink. The application's own loop emits spans for each LLM call, tool call, and step, and Langfuse ingests them as nested traces. Evaluators then run asynchronously over a sampled share of incoming traces, attaching scores back to them.","build_surface":"low-code","members":[{"pattern":"prompt-versioning","role":"first-class","note":"Langfuse Prompt Management stores each prompt in a registry where every save creates a new immutable version number, and labels (production/staging) point at a chosen version so deploys and rollbacks are done by moving the label between versions.","evidence":[{"type":"doc","title":"Prompt Management — Langfuse Docs","url":"https://langfuse.com/docs/prompts/get-started","quote":"Version Control: Track the changes through versioning... Each prompt has a version number that increments with each change.","accessed":"2026-06-17"},{"type":"doc","title":"Prompt Management — Langfuse Docs","url":"https://langfuse.com/docs/prompts/get-started","quote":"Use labels to deploy prompt versions... The production label is used to fetch the prompt in production.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"sampled-prompt-trace-eval","role":"first-class","note":"Langfuse ingests full production traces and lets you attach LLM-as-a-judge evaluators that run on a configurable sampling percentage of traces so judge cost stays bounded as traffic grows.","evidence":[{"type":"doc","title":"LLM-as-a-Judge — Langfuse Docs","url":"https://langfuse.com/docs/scores/model-based-evals","quote":"Configure sampling percentage (e.g., 5%) to manage evaluation costs and throughput","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"llm-as-judge","role":"first-class","note":"Langfuse provides model-based evaluation where one LLM scores the outputs of the traced application; evaluator prompts produce numeric or categorical scores that are attached back to the trace.","evidence":[{"type":"doc","title":"LLM-as-a-Judge — Langfuse Docs (definition)","url":"https://langfuse.com/docs/scores/model-based-evals","quote":"LLM-as-a-Judge is an evaluation methodology where an LLM is used to assess the quality of outputs produced by another LLM application.","accessed":"2026-06-17"},{"type":"doc","title":"LLM-as-a-Judge — Langfuse Docs (sampling)","url":"https://langfuse.com/docs/scores/model-based-evals","quote":"Configure sampling percentage (e.g., 5%) to manage evaluation costs and throughput","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"cost-observability","role":"first-class","note":"Langfuse tracks token usage and cost on each generation observation and surfaces cost, latency, and quality metrics in the dashboard so operators can monitor per-call and per-application spend.","evidence":[{"type":"doc","title":"Token & Cost Tracking — Langfuse Docs","url":"https://langfuse.com/docs/observability/features/token-and-cost-tracking","quote":"Langfuse tracks the usage and costs of your LLM generations and provides breakdowns by usage types.","accessed":"2026-06-17"},{"type":"doc","title":"Langfuse Docs (overview)","url":"https://langfuse.com/docs","quote":"See quality, cost, and latency metrics in the dashboard to monitor your LLM application.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"eval-harness","role":"first-class","note":"Langfuse Datasets hold collections of inputs and expected outputs, and Experiments run an application version against the dataset to test it systematically, so regressions can be measured across versions before deploy.","evidence":[{"type":"doc","title":"Datasets — Langfuse Docs","url":"https://langfuse.com/docs/datasets/overview","quote":"A dataset is a collection of inputs and expected outputs and is used to test your application.","accessed":"2026-06-17"},{"type":"doc","title":"Langfuse Docs (overview)","url":"https://langfuse.com/docs","quote":"Run Experiments to systematically test your LLM application.","accessed":"2026-06-17"}],"evidence_status":"full"}],"key_concepts":[{"name":"Trace / Observation","summary":"The nested record Langfuse ingests for one application run; a trace groups observations (LLM generations, tool calls, retrieval and other spans), each carrying timing, inputs, outputs, and cost.","maps_to_pattern":"cost-observability","url":"https://langfuse.com/docs/tracing"},{"name":"Prompt label","summary":"A named pointer (such as production or staging) that resolves to a specific prompt version in the registry, so deploying or rolling back a prompt is done by moving the label between versions.","maps_to_pattern":"prompt-versioning","url":"https://langfuse.com/docs/prompts/get-started"},{"name":"Dataset / Experiment","summary":"A dataset is a collection of inputs and expected outputs; an experiment runs an application version against that dataset to score and compare it, which is how Langfuse supports offline regression testing.","maps_to_pattern":"eval-harness","url":"https://langfuse.com/docs/datasets/overview"},{"name":"Score","summary":"A numeric, boolean, or categorical value attached to a trace by an evaluator (LLM-as-a-judge, code evaluator, user feedback, or manual label), which is the unit Langfuse uses to quantify quality.","maps_to_pattern":"llm-as-judge","url":"https://langfuse.com/docs/scores/model-based-evals"}],"references":[{"type":"repo","title":"langfuse/langfuse","url":"https://github.com/langfuse/langfuse","quote":"Open source LLM engineering platform","accessed":"2026-06-17"},{"type":"doc","title":"Langfuse Docs (overview)","url":"https://langfuse.com/docs","quote":"See quality, cost, and latency metrics in the dashboard to monitor your LLM application.","accessed":"2026-06-17"}],"alternatives":[{"composition":"bedrock-agentcore","relation":"complements","note":"AgentCore is a managed agent runtime with built-in observability tied to AWS, whereas Langfuse is a provider-neutral, self-hostable observability and evaluation backend that ingests traces from any application."}],"verification_status":"needs-verification","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-19","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Langfuse\"]\n  fw --> p1[\"Prompt Versioning<br/>(first-class)\"]\n  fw --> p2[\"Sampled Prompt Trace Eval<br/>(first-class)\"]\n  fw --> p3[\"LLM-as-Judge<br/>(first-class)\"]\n  fw --> p4[\"Cost Observability<br/>(first-class)\"]\n  fw --> p5[\"Eval Harness<br/>(first-class)\"]","tags":["enterprise-platform","governance-observability","low-code"],"anti_patterns_avoided":[{"pattern":"agent-confession-as-forensics","note":"Records the real execution trace (every LLM call, tool invocation, retrieval step) so forensics rely on captured telemetry rather than the agent's self-narrative.","evidence":[{"type":"doc","title":"Langfuse — Observability overview","url":"https://langfuse.com/docs/observability/overview","quote":"Hierarchical traces capture every LLM call, tool invocation, and retrieval step.","accessed":"2026-06-19"}]},{"pattern":"agent-tool-data-black-box","note":"Traces expose the full sequence of tool calls, inputs, outputs, and reasoning steps, opening up the otherwise opaque agent tool/data flow.","evidence":[{"type":"doc","title":"Langfuse — AI agent observability","url":"https://langfuse.com/blog/2024-07-ai-agent-observability-with-langfuse","quote":"evaluates the full sequence of tool calls, reasoning steps, and decisions an agent made","accessed":"2026-06-19"}]},{"pattern":"black-box-opaqueness","note":"Application tracing records the exact prompt, model response, token usage, latency, and every tool/retrieval step, opening the otherwise opaque agent.","evidence":[{"type":"doc","title":"Langfuse — Observability overview","url":"https://langfuse.com/docs/observability/overview","quote":"The core of this is application tracing -- structured logs of every request that capture the exact prompt sent, the models response, token usage, latency, and any tools or retrieval steps in between.","accessed":"2026-06-19"}]},{"pattern":"token-economy-blindness","note":"Spend alerts notify when org spend crosses monetary thresholds, surfacing the per-run economic ceiling the anti-pattern says is missing.","evidence":[{"type":"doc","title":"Langfuse — spend alerts","url":"https://langfuse.com/docs/administration/spend-alerts","quote":"Spend alerts notify you when your organizations spend exceeds predefined monetary thresholds","accessed":"2026-06-19"}]}]},{"id":"arcade","name":"Arcade","kind":"framework","category":"agent-platform-saas","vendor":"Arcade AI, Inc.","language":"Python","license":"proprietary","status":"active","first_released":"2024-01-01","url":"https://www.arcade.dev/","docs_url":"https://docs.arcade.dev/","intent":"Arcade is an MCP runtime that sits between agents and external systems, handling user authentication, tool execution, and per-action scoping so agents act on behalf of an authenticated user with the right permissions.","description":"Arcade is an MCP runtime that handles user authentication, authorization, policy enforcement, and tool execution between agents and the systems they reach. It offers a registry of thousands of prebuilt MCP tools that agents browse and pick from. Agents act on behalf of real users with real permissions rather than shared tokens or hardcoded credentials, integrating with an existing identity provider so credentials never leave the runtime. Every action runs at the intersection of what the user can do and what the agent is scoped to do, and is logged.","primary_use_cases":["authenticated tool access for agents on behalf of users","browsing and selecting prebuilt MCP tools","scoped, governed tool execution across SaaS systems","enterprise authorization and audit for agent actions"],"agent_loop_shape":"An agent connects to the Arcade runtime, which sits between it and every system it needs to reach. The agent browses the registry of prebuilt MCP tools and picks the ones it needs. When it invokes a tool, Arcade authenticates the user through the existing identity provider and runs the action at the intersection of what the user can do and what the agent is scoped to do; if either line is crossed the action stops, and every action is logged.","build_surface":"full-code","members":[{"pattern":"tool-agent-registry","role":"supported","note":"Arcade exposes a registry/catalog of thousands of prebuilt MCP tools that agents browse and pick from, plus a registry to share and discover tools.","evidence":[{"type":"doc","title":"Arcade home","url":"https://www.arcade.dev/","quote":"Browse thousands of pre-built MCP tools for the apps your org already runs on.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"delegated-agent-authorization","role":"core","note":"Agents act on behalf of the authenticated user with that user's real permissions rather than shared tokens or hardcoded credentials; the existing identity provider plugs in and credentials never leave the runtime, with each action scoped to the intersection of user and agent rights.","evidence":[{"type":"doc","title":"Arcade home","url":"https://www.arcade.dev/","quote":"Agents need to act on behalf of real users with real permissions, not shared tokens or hardcoded credentials.","accessed":"2026-06-17"},{"type":"doc","title":"Arcade home","url":"https://www.arcade.dev/","quote":"Every action runs at the intersection of what the user can do and what the agent is scoped to do. Cross either line and the action stops.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"mcp","role":"core","note":"Arcade is itself an MCP runtime: it manages authentication, tool registration, and execution and serves prebuilt tools as MCP servers that any conformant MCP client can call.","evidence":[{"type":"doc","title":"Arcade docs home","url":"https://docs.arcade.dev/home","quote":"Arcade handles OAuth, manages user tokens, and gives you 7,000+ pre-built integrations so your agents can take real action in production.","accessed":"2026-06-17"},{"type":"repo","title":"ArcadeAI/arcade-mcp (README)","url":"https://github.com/ArcadeAI/arcade-mcp","quote":"arcade-mcp is the Python framework for building Model Context Protocol servers and the tools that run inside them.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"agent-credential-vault","role":"core","note":"Arcade brokers OAuth tokens at tool-call time, injecting the securely-stored credential into the tool's execution context so the agent never holds a raw user token; the runtime declares required scopes per tool and handles OAuth and token refresh.","evidence":[{"type":"doc","title":"Create a tool with auth - Arcade docs","url":"https://docs.arcade.dev/home/build-tools/create-a-tool-with-auth","quote":"The tool is executed, and uses the token injected into its Context to call the provider's API","accessed":"2026-06-17"},{"type":"repo","title":"ArcadeAI/arcade-mcp (README)","url":"https://github.com/ArcadeAI/arcade-mcp","quote":"Declare requires_auth=GitHub(scopes=[\"repo\"]) and Arcade handles OAuth, token refresh, and per-call scoping.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"secrets-handling","role":"first-class","note":"The OAuth token is securely injected into the tool call at runtime and never returned to the model or the MCP client, so the secret stays out of the prompt and context window.","evidence":[{"type":"repo","title":"ArcadeAI/arcade-mcp (README)","url":"https://github.com/ArcadeAI/arcade-mcp","quote":"The client and the LLM never see the token.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"Arcade Engine","summary":"The runtime component that fulfils authorized tool-calling flows for end users: it evaluates which scopes a tool requires, runs the OAuth flow against the configured identity provider, and injects the resulting token into tool execution.","maps_to_pattern":"delegated-agent-authorization","url":"https://docs.arcade.dev/home"},{"name":"Tool Catalog","summary":"Arcade's registry of 7,000-plus prebuilt MCP tools across dozens of MCP servers that agents browse and select from for the apps an organisation already runs.","maps_to_pattern":"tool-agent-registry","url":"https://www.arcade.dev/"},{"name":"Authorized tool calling","summary":"A tool declares the auth provider and scopes it needs; at call time Arcade checks whether the user has authorized those scopes, runs the OAuth flow if not, and only then executes the tool with the injected token.","maps_to_pattern":"agent-credential-vault","url":"https://docs.arcade.dev/home/build-tools/create-a-tool-with-auth"},{"name":"Workers","summary":"The processes that host and run Arcade tools, deployable on the user's own infrastructure including private cloud, on-premises, or fully air-gapped environments so credentials and execution stay inside the user's perimeter.","url":"https://docs.arcade.dev/"}],"references":[{"type":"doc","title":"Arcade home","url":"https://www.arcade.dev/","quote":"MCP Runtime for AI agents that get things done.","accessed":"2026-06-17"},{"type":"doc","title":"Arcade documentation home","url":"https://docs.arcade.dev/home","accessed":"2026-06-17"},{"type":"repo","title":"ArcadeAI/arcade-mcp - MCP Server Framework and Tool Development library","url":"https://github.com/ArcadeAI/arcade-mcp","quote":"It powers the 7,500+ prebuilt tools across 81 MCP servers at Arcade.dev.","accessed":"2026-06-17"}],"alternatives":[{"composition":"composio","relation":"competes-with","note":"Both broker authenticated third-party tool access for agents through a large prebuilt toolkit catalog with managed OAuth, so the agent never handles raw end-user credentials."},{"composition":"cerbos","relation":"complements","note":"Arcade authenticates the user and scopes the action; Cerbos can sit alongside as the external policy decision point deciding whether a given user-and-context combination is allowed to invoke a tool."}],"verification_status":"partial","status_in_practice":"emerging","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Arcade\"]\n  fw --> p1[\"Delegated Agent Authorization<br/>(core)\"]\n  fw --> p2[\"Model Context Protocol<br/>(core)\"]\n  fw --> p3[\"Agent Credential Vault<br/>(core)\"]\n  fw --> p4[\"Secrets Handling<br/>(first-class)\"]\n  fw --> p5[\"Tool/Agent Registry<br/>(supported)\"]","tags":["agent-platform-saas","tool-use-environment","safety-control","full-code"]},{"id":"azure-api-management-mcp","name":"Azure API Management (MCP server)","kind":"framework","category":"enterprise-platform","vendor":"Microsoft Azure","language":"proprietary","license":"proprietary","status":"active","first_released":"2025-05-19","url":"https://learn.microsoft.com/en-us/azure/api-management/export-rest-mcp-server","docs_url":"https://learn.microsoft.com/en-us/azure/api-management/export-rest-mcp-server","intent":"Azure API Management exposes a managed REST API as a remote MCP server, turning selected API operations one-to-one into agent-callable tools and applying API Management policies to govern every tool call.","description":"Azure API Management can expose a REST API it manages as a remote Model Context Protocol server through its built-in AI gateway. A developer selects one or more API operations to expose as tools that MCP clients call over the MCP protocol. API Management policies apply to all operations exposed as tools, so the same gateway controls access, authentication, and rate limiting for the MCP server. Agents such as GitHub Copilot, Semantic Kernel, or Copilot Studio call the hosted MCP endpoint.","primary_use_cases":["exposing managed REST APIs as MCP servers","one-to-one mapping of API operations to agent tools","policy-governed access control over agent tool calls","rate limiting and authentication for MCP tools"],"agent_loop_shape":"A developer selects API operations from a managed REST API to expose as MCP tools, and API Management creates an MCP server with a server URL. An MCP client agent adds that server and calls the tools over the MCP protocol; on each call the AI gateway evaluates the policies configured at global and MCP-server scope, applying access control, authentication, and rate limiting before forwarding the request to the backend API.","build_surface":"low-code","members":[{"pattern":"direct-api-wrapper","role":"first-class","note":"Azure API Management exposes a managed REST API as an MCP server by turning selected API operations one-to-one into agent-callable MCP tools.","evidence":[{"type":"doc","title":"Expose REST API as MCP server - Azure API Management","url":"https://learn.microsoft.com/en-us/azure/api-management/export-rest-mcp-server","quote":"Expose one or more of the API operations as tools that MCP clients can call by using the MCP protocol. ... The portal creates the MCP server and exposes the API operations as tools.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"rate-limiting","role":"first-class","note":"API Management policies apply to all operations exposed as tools, letting the gateway cap calls to the MCP server's tools — for example five calls per 30 seconds per IP address — alongside access and authentication control.","evidence":[{"type":"doc","title":"Expose REST API as MCP server - Azure API Management","url":"https://learn.microsoft.com/en-us/azure/api-management/export-rest-mcp-server","quote":"The policies apply to all API operations exposed as tools in the MCP server. Use these policies to control access, authentication, and other aspects of the tools.","accessed":"2026-06-17"},{"type":"doc","title":"Expose REST API as MCP server - Azure API Management","url":"https://learn.microsoft.com/en-us/azure/api-management/export-rest-mcp-server","quote":"you can add policies to the Inbound section to limit calls to the MCP server's tools (in this example, 5 calls per 30 seconds per IP address)","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"mcp","role":"core","note":"API Management hosts the API as a remote MCP server through its built-in AI gateway, giving agents a standard MCP endpoint (Server URL) to discover and call the exposed operations as tools.","evidence":[{"type":"doc","title":"Expose REST API as MCP server - Azure API Management","url":"https://learn.microsoft.com/en-us/azure/api-management/export-rest-mcp-server","quote":"In API Management, you can expose a REST API managed in API Management as a remote Model Context Protocol (MCP) server by using its built-in AI gateway.","accessed":"2026-06-17"},{"type":"doc","title":"Expose REST API as MCP server - Azure API Management","url":"https://learn.microsoft.com/en-us/azure/api-management/export-rest-mcp-server","quote":"The MCP Servers blade lists the new MCP server. The Server URL column shows the endpoint of the MCP server to call for testing or within a client application.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"policy-as-code-gate","role":"first-class","note":"Every MCP tool call passes through API Management's XML policies, configured at global and MCP-server scope and evaluated before the request reaches the backend, so access, authentication, and other controls are authored as machine-readable policy outside the agent rather than in the prompt.","evidence":[{"type":"doc","title":"Expose REST API as MCP server - Azure API Management","url":"https://learn.microsoft.com/en-us/azure/api-management/export-rest-mcp-server","quote":"In the policy editor, add or edit the policies you want to apply to the MCP server's tools. Define the policies in XML format.","accessed":"2026-06-17"},{"type":"doc","title":"Expose REST API as MCP server - Azure API Management","url":"https://learn.microsoft.com/en-us/azure/api-management/export-rest-mcp-server","quote":"API Management evaluates policies configured at the global (all APIs) scope before it evaluates policies at the MCP server scope.","accessed":"2026-06-17"}],"evidence_status":"full"}],"key_concepts":[{"name":"AI gateway","summary":"The built-in API Management capability that fronts the exposed API as an MCP server and applies the gateway's existing access, authentication, and traffic controls to MCP tool traffic.","maps_to_pattern":"mcp","url":"https://learn.microsoft.com/en-us/azure/api-management/genai-gateway-capabilities"},{"name":"Tools (exposed API operations)","summary":"Each selected REST API operation becomes one MCP tool; the set of exposed operations is managed in the MCP server's Tools blade and can be changed after creation.","maps_to_pattern":"direct-api-wrapper","url":"https://learn.microsoft.com/en-us/azure/api-management/export-rest-mcp-server"},{"name":"Policy scopes (global and MCP-server)","summary":"API Management policies attach at multiple scopes; the global (all-APIs) scope is evaluated before the per-MCP-server scope, so an organisation-wide rule and a server-specific rule both apply to each tool call.","maps_to_pattern":"policy-as-code-gate","url":"https://learn.microsoft.com/en-us/azure/api-management/api-management-howto-policies"}],"references":[{"type":"doc","title":"Expose REST API as MCP server - Azure API Management | Microsoft Learn","url":"https://learn.microsoft.com/en-us/azure/api-management/export-rest-mcp-server","accessed":"2026-06-17"},{"type":"doc","title":"AI gateway capabilities - Azure API Management","url":"https://learn.microsoft.com/en-us/azure/api-management/genai-gateway-capabilities","accessed":"2026-06-17"}],"alternatives":[{"composition":"stainless-mcp","relation":"similar-shape","note":"Both turn an existing REST API into agent-callable MCP tools; Stainless generates an MCP server from an OpenAPI spec, while API Management exposes an already-managed API through its gateway."},{"composition":"open-policy-agent","relation":"complements","note":"API Management evaluates its own XML policies inline; teams that want a separate, language-neutral policy decision point can pair the gateway with an external engine like OPA for authorization decisions."}],"verification_status":"partial","status_in_practice":"emerging","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Azure API Management (MCP server)\"]\n  fw --> p1[\"Model Context Protocol<br/>(core)\"]\n  fw --> p2[\"Direct API Wrapper<br/>(first-class)\"]\n  fw --> p3[\"Rate Limiting<br/>(first-class)\"]\n  fw --> p4[\"Policy-as-Code Gate<br/>(first-class)\"]","tags":["enterprise-platform","tool-use-environment","safety-control","low-code"]},{"id":"cerbos","name":"Cerbos","kind":"framework","category":"enterprise-platform","vendor":"Cerbos","language":"Go","license":"Apache-2.0","status":"active","first_released":"2021-09-01","url":"https://www.cerbos.dev","repo":"https://github.com/cerbos/cerbos","docs_url":"https://docs.cerbos.dev/","intent":"Cerbos is an external policy decision point that an MCP server calls on every agent tool invocation, returning an allow/deny decision and controlling which tools are enabled for the user and context.","description":"Cerbos is an open-source, language-agnostic authorization policy decision point that centralizes access-control decisions outside application code. Used with an MCP server, it acts as a policy enforcement point: the MCP server asks Cerbos for a decision on each AI-agent tool invocation and abides by it. When clients connect, the server calls Cerbos to check which tools are permitted for that user and context, then enables or disables tools accordingly. Policies can incorporate attributes of the action, the user, and the environment.","primary_use_cases":["externalized authorization for MCP servers","allow/deny decisions on agent tool invocations","per-user, per-context tool enablement for agents","attribute-based access control for agent actions"],"agent_loop_shape":"An MCP server in front of an agent acts as a policy enforcement point. When a client connects, the server asks Cerbos which tools are permitted for that user and context and enables or disables tools accordingly, starting with no tools by default. On each tool invocation the server asks Cerbos for an allow/deny decision and abides by it, with policies evaluating attributes of the action, the user, and the environment.","build_surface":"full-code","members":[{"pattern":"policy-gated-agent-action","role":"core","note":"An MCP server acts as a policy enforcement point that calls the Cerbos policy decision point on every AI-agent tool invocation, returning an allow/deny decision so only the actions permitted for that user and context execute.","evidence":[{"type":"blog","title":"MCP Permissions. Securing AI Agent Access to Tools. | Cerbos","url":"https://www.cerbos.dev/blog/mcp-permissions-securing-ai-agent-access-to-tools","quote":"The MCP server simply becomes a policy enforcement point: it asks Cerbos for a decision and abides by it.","accessed":"2026-06-17"},{"type":"doc","title":"Enterprise-Grade Authorization for MCP Servers | Cerbos","url":"https://www.cerbos.dev/features-benefits-and-use-cases/dynamic-authorization-for-MCP-servers","quote":"When clients connect, your MCP server calls Cerbos to check which tools are permitted for that user and context, then enables/disables tools accordingly.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"tool-loadout","role":"first-class","note":"Cerbos drives which tools the agent session sees: the session starts with no tools enabled, and tools are selectively enabled per request based on the user's role, request context, and policies, so a read-only user never sees write or admin tools.","evidence":[{"type":"blog","title":"MCP Permissions. Securing AI Agent Access to Tools. | Cerbos","url":"https://www.cerbos.dev/blog/mcp-permissions-securing-ai-agent-access-to-tools","quote":"An AI agent session should start with no tools enabled by default, then tools are selectively enabled based on the user's role, request context, and policies.","accessed":"2026-06-17"},{"type":"blog","title":"MCP Permissions. Securing AI Agent Access to Tools. | Cerbos","url":"https://www.cerbos.dev/blog/mcp-permissions-securing-ai-agent-access-to-tools","quote":"If a user only has 'read' rights in a system, the AI agent should only see tools that perform read operations - write or admin-level tools should simply not be available.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"policy-as-code-gate","role":"core","note":"Authorization rules are written as declarative, machine-readable policies managed outside the MCP server: rules define under which conditions a user or workload can access a tool, and policies can be changed (with live reload) without touching server code, so compliance authorship lives outside the application and the prompt.","evidence":[{"type":"doc","title":"Enterprise-Grade Authorization for MCP Servers | Cerbos","url":"https://www.cerbos.dev/features-benefits-and-use-cases/dynamic-authorization-for-MCP-servers","quote":"Write simple, declarative rules that define under which conditions users or workloads can access specific MCP tools, and authorize downstream service to service requests.","accessed":"2026-06-17"},{"type":"doc","title":"Enterprise-Grade Authorization for MCP Servers | Cerbos","url":"https://www.cerbos.dev/features-benefits-and-use-cases/dynamic-authorization-for-MCP-servers","quote":"Update permissions by modifying policies without touching the MCP server code - Cerbos supports live policy reloading.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"provenance-ledger","role":"first-class","note":"Cerbos records every authorization check it makes as structured audit logs, so each allow/deny decision over an agent's tool calls is captured with enough context for traceability and forensic review.","evidence":[{"type":"doc","title":"Enterprise-Grade Authorization for MCP Servers | Cerbos","url":"https://www.cerbos.dev/features-benefits-and-use-cases/dynamic-authorization-for-MCP-servers","quote":"Capture every authorization check, across services, agents, and APIs, with structured logs that enable full traceability, compliance readiness, and forensic investigations.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"Policy Decision Point (PDP)","summary":"Cerbos runs as a stateless service that exposes an API the application or MCP server calls to ask whether a given action is allowed for a given user right now, keeping the decision logic out of the calling code.","maps_to_pattern":"policy-gated-agent-action","url":"https://www.cerbos.dev/blog/mcp-permissions-securing-ai-agent-access-to-tools"},{"name":"ABAC / RBAC / PBAC policies","summary":"Cerbos policies can combine role-based, attribute-based, and policy-based access control, evaluating attributes of the action, the user, and the environment so decisions reflect context like department, account status, time of day, or request IP.","maps_to_pattern":"policy-as-code-gate","url":"https://www.cerbos.dev/features-benefits-and-use-cases/dynamic-authorization-for-MCP-servers"},{"name":"Policy enforcement point (the MCP server)","summary":"In the MCP integration the server itself is the enforcement point: it delegates the decision to Cerbos and enforces the returned verdict, so its own logic stays simple while the security team owns the rules.","maps_to_pattern":"policy-gated-agent-action","url":"https://www.cerbos.dev/blog/mcp-permissions-securing-ai-agent-access-to-tools"}],"references":[{"type":"blog","title":"MCP Permissions. Securing AI Agent Access to Tools. | Cerbos","url":"https://www.cerbos.dev/blog/mcp-permissions-securing-ai-agent-access-to-tools","accessed":"2026-06-17"},{"type":"doc","title":"Enterprise-Grade Authorization for MCP Servers | Cerbos","url":"https://www.cerbos.dev/features-benefits-and-use-cases/dynamic-authorization-for-MCP-servers","accessed":"2026-06-17"},{"type":"repo","title":"cerbos/cerbos","url":"https://github.com/cerbos/cerbos","accessed":"2026-06-17"}],"alternatives":[{"composition":"open-policy-agent","relation":"competes-with","note":"Both are external, language-neutral policy decision points an application or MCP server queries for allow/deny decisions; OPA uses Rego, Cerbos uses its own declarative policy format."},{"composition":"arcade","relation":"complements","note":"Arcade authenticates the user and scopes the OAuth-backed tool call; Cerbos can sit alongside as the policy decision point that authorizes whether that user-and-context combination is permitted to invoke the tool."}],"verification_status":"partial","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Cerbos\"]\n  fw --> p1[\"Policy-Gated Agent Action (KRITIS)<br/>(core)\"]\n  fw --> p2[\"Policy-as-Code Gate<br/>(core)\"]\n  fw --> p3[\"Tool Loadout<br/>(first-class)\"]\n  fw --> p4[\"Provenance Ledger<br/>(first-class)\"]","tags":["enterprise-platform","safety-control","tool-use-environment","governance-observability","full-code"]},{"id":"composio","name":"Composio","kind":"framework","category":"agent-platform-saas","vendor":"Composio","language":"Python, TypeScript","license":"MIT","status":"active","first_released":"2023","url":"https://composio.dev/","repo":"https://github.com/ComposioHQ/composio","docs_url":"https://docs.composio.dev/","intent":"Give AI agents a catalogue of authenticated toolkits so they can search for the right tools and call third-party applications without handling raw credentials.","description":"Composio is a tooling platform for AI agents that maintains a large set of toolkits across third-party applications. Agents query the registry with semantic tool search to retrieve relevant tools instead of loading every one. Composio manages authentication for each toolkit, including OAuth flows and connected accounts, so agents call tools through the Composio API without receiving end-user tokens directly. It exposes SDKs in Python and TypeScript.","primary_use_cases":["discovering relevant tools from a large toolkit catalogue","calling authenticated third-party application APIs from an agent","managing per-user OAuth connections for agent tool access"],"agent_loop_shape":"An agent queries Composio's registry with semantic search to find tools for the task, then executes a chosen tool against a connected account; Composio resolves the user's stored credentials behind the API so the agent supplies a connected-account reference rather than raw tokens.","build_surface":"full-code","members":[{"pattern":"tool-agent-registry","role":"first-class","note":"Composio maintains a catalog of 1000+ toolkits with semantic tool search so an agent can query the registry by use case and retrieve the right tools instead of loading them all.","evidence":[{"type":"repo","title":"ComposioHQ/composio (README)","url":"https://github.com/ComposioHQ/composio","quote":"Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.","accessed":"2026-06-17"},{"type":"doc","title":"Fetching tools and schemas - Composio Docs","url":"https://docs.composio.dev/docs/fetching-tools","quote":"Find tools semantically.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"agent-credential-vault","role":"first-class","note":"Composio holds developer credentials in reusable Auth Configs and end-user tokens in Connected Accounts, managing OAuth flows so agents call tools through the Composio API without receiving raw credentials.","evidence":[{"type":"doc","title":"Authenticating Tools - Composio Docs","url":"https://docs.composio.dev/docs/authenticating-tools","quote":"An Auth Config is a blueprint that defines how authentication works for a toolkit across all your users.","accessed":"2026-06-17"},{"type":"repo","title":"ComposioHQ/composio (README)","url":"https://github.com/ComposioHQ/composio","quote":"Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"tool-loadout","role":"first-class","note":"Rather than exposing the full toolkit catalog, Composio returns a small task-relevant subset per request: tool fetches return the top 20 tools by default with an adjustable limit, and an agent can request specific tools by name when they are known.","evidence":[{"type":"doc","title":"Fetching tools and schemas - Composio Docs","url":"https://docs.composio.dev/docs/fetching-tools","quote":"Returns top 20 tools by default.","accessed":"2026-06-17"},{"type":"doc","title":"Fetching tools and schemas - Composio Docs","url":"https://docs.composio.dev/docs/fetching-tools","quote":"Fetch specific tools when you know their names.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"mcp","role":"supported","note":"Composio can publish its configured toolkits as MCP servers, generating user-specific MCP server URLs that MCP-compatible clients connect to, so the same authenticated tools are reachable over the Model Context Protocol.","evidence":[{"type":"doc","title":"MCP (Model Context Protocol) - Composio Docs","url":"https://docs.composio.dev/docs/mcp","quote":"Sessions provide dynamic tool access and a much better MCP experience with context management handled by us.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"Toolkit","summary":"A named integration with a third-party application (such as GitHub or Gmail) bundling its set of executable tools, the unit an agent searches the catalog by.","maps_to_pattern":"tool-agent-registry","url":"https://docs.composio.dev/docs/fetching-tools"},{"name":"Auth Config","summary":"A reusable blueprint that defines how authentication works for a toolkit across all of a developer's users, holding the developer-level credentials separate from per-user tokens.","maps_to_pattern":"agent-credential-vault","url":"https://docs.composio.dev/docs/authenticating-tools"},{"name":"Connected Account","summary":"A per-end-user authenticated connection to a toolkit; the agent references the connected account when executing a tool and Composio resolves the user's stored token behind the API.","maps_to_pattern":"agent-credential-vault","url":"https://docs.composio.dev/docs/authenticating-tools"},{"name":"Semantic tool search","summary":"A search step over the toolkit catalog that returns the tools most relevant to a natural-language query, so an agent loads a small task-relevant set rather than every tool.","maps_to_pattern":"tool-loadout","url":"https://docs.composio.dev/docs/fetching-tools"}],"references":[{"type":"repo","title":"ComposioHQ/composio (README)","url":"https://github.com/ComposioHQ/composio","accessed":"2026-06-17"},{"type":"doc","title":"Composio Documentation","url":"https://docs.composio.dev/","accessed":"2026-06-17"},{"type":"doc","title":"Fetching tools and schemas - Composio Docs","url":"https://docs.composio.dev/docs/fetching-tools","accessed":"2026-06-17"}],"alternatives":[{"composition":"arcade","relation":"competes-with","note":"Both broker authenticated third-party tool access through a large prebuilt toolkit catalog with managed OAuth, so the agent calls tools without holding raw end-user credentials."},{"composition":"stainless-mcp","relation":"similar-shape","note":"Both turn third-party APIs into agent-callable tools and can expose them over MCP; Stainless generates a server from one API's OpenAPI spec, Composio aggregates many prebuilt toolkits with managed auth."}],"verification_status":"partial","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Composio\"]\n  fw --> p1[\"Tool/Agent Registry<br/>(first-class)\"]\n  fw --> p2[\"Agent Credential Vault<br/>(first-class)\"]\n  fw --> p3[\"Tool Loadout<br/>(first-class)\"]\n  fw --> p4[\"Model Context Protocol<br/>(supported)\"]","tags":["agent-platform-saas","tool-use-environment","safety-control","full-code"]},{"id":"cube-semantic-layer","name":"Cube","kind":"framework","category":"enterprise-platform","vendor":"Cube Dev, Inc.","language":"TypeScript, Rust","license":"Apache-2.0","status":"active","first_released":"2019","url":"https://cube.dev/","repo":"https://github.com/cube-js/cube","docs_url":"https://cube.dev/docs","intent":"Define metrics and dimensions once in a governed semantic layer and serve them to AI agents and BI tools over SQL, REST, GraphQL, and an MCP server.","description":"Cube is a semantic layer that defines business metrics, dimensions, joins, and access rules once and exposes them through multiple query interfaces. Its open-source core, Cube Core, is released under Apache 2.0. AI agents reach the governed metrics over SQL, REST, GraphQL, and an MCP server rather than authoring raw SQL against the warehouse. MCP-compatible clients such as Claude and Cursor connect to Cube to get analytics answers grounded in the same model.","primary_use_cases":["serving governed metrics to AI agents over an MCP server","defining business metrics once for reuse across tools","querying a semantic layer from BI and analytics interfaces"],"agent_loop_shape":"An AI agent connects to Cube through its MCP server (or another supported interface), poses a natural-language analytics question, and Cube resolves it against governed metrics and dimensions defined in the semantic layer, returning an answer grounded in the shared model rather than executing free-authored SQL on raw tables.","build_surface":"low-code","members":[{"pattern":"semantic-layer-query-guardrail","role":"core","note":"Cube routes AI-agent data questions through a governed semantic layer so agents query vetted metrics and dimensions over SQL/REST/GraphQL/MCP rather than free-authoring raw SQL against the warehouse.","evidence":[{"type":"blog","title":"Best Semantic Layer for AI and BI (2026)","url":"https://cube.dev/articles/best-semantic-layer-for-ai-and-bi-2026","quote":"the semantic layer is the foundation of an AI-native platform — SQL-first, extensible at query time, and reachable over SQL, REST, GraphQL, and MCP so AI agents query governed metrics instead of raw tables.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"mcp","role":"first-class","note":"Cube exposes its governed metrics through an MCP server so MCP-compatible agents like Claude and Cursor connect and query the semantic layer through the standard protocol.","evidence":[{"type":"blog","title":"Best Semantic Layer for AI and BI (2026)","url":"https://cube.dev/articles/best-semantic-layer-for-ai-and-bi-2026","quote":"Governed metrics are reachable over SQL (Postgres-compatible), REST, GraphQL, and an MCP server","accessed":"2026-06-17"},{"type":"doc","title":"Cube MCP Server","url":"https://www.cubesoftware.com/mcp","quote":"Add the Cube connector in Claude, Claude Code, Cursor, or any MCP-compatible client.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"tenant-scoped-tool-binding","role":"supported","note":"Access control is defined once in the semantic layer and enforced beneath the agent: Cube applies row-level, multi-tenant access control so a connected agent or user only ever sees the data they are authorized to see, regardless of the question asked.","evidence":[{"type":"blog","title":"Best Semantic Layer for AI and BI (2026)","url":"https://cube.dev/articles/best-semantic-layer-for-ai-and-bi-2026","quote":"Governed metrics are reachable over SQL (Postgres-compatible), REST, GraphQL, and an MCP server, with pre-aggregation caching and row-level, multi-tenant access control.","accessed":"2026-06-17"},{"type":"doc","title":"Cube MCP Server","url":"https://www.cubesoftware.com/mcp","quote":"Role-based access ensures every user sees only what they're authorized to see.","accessed":"2026-06-17"}],"evidence_status":"full"}],"key_concepts":[{"name":"Semantic layer (data model)","summary":"The single definition of business metrics, dimensions, joins, and access rules that every downstream interface and agent queries against, so the same numbers come back regardless of the tool.","maps_to_pattern":"semantic-layer-query-guardrail","url":"https://cube.dev/docs"},{"name":"Governed metrics","summary":"Metrics defined once in the model and served as vetted, reusable measures, so an agent selects and parameterises an approved metric rather than computing it ad hoc from raw tables.","maps_to_pattern":"semantic-layer-query-guardrail","url":"https://cube.dev/articles/best-semantic-layer-for-ai-and-bi-2026"},{"name":"Cube MCP server","summary":"The Cube connector that any MCP-compatible client adds to ask natural-language analytics questions and get answers grounded in the live, governed model.","maps_to_pattern":"mcp","url":"https://www.cubesoftware.com/mcp"},{"name":"Row-level, multi-tenant access control","summary":"Access rules defined in the model that restrict which rows and metrics each user or tenant can reach, enforced uniformly across every query interface including the agent's.","maps_to_pattern":"tenant-scoped-tool-binding","url":"https://cube.dev/articles/best-semantic-layer-for-ai-and-bi-2026"}],"references":[{"type":"blog","title":"Best Semantic Layer for AI and BI (2026)","url":"https://cube.dev/articles/best-semantic-layer-for-ai-and-bi-2026","accessed":"2026-06-17"},{"type":"doc","title":"Cube MCP Server","url":"https://www.cubesoftware.com/mcp","accessed":"2026-06-17"},{"type":"repo","title":"cube-js/cube","url":"https://github.com/cube-js/cube","accessed":"2026-06-17"}],"alternatives":[{"composition":"k2view-genai-data-fusion","relation":"similar-shape","note":"Both place a governed data layer between agents and source systems so the agent queries curated, access-controlled data rather than reaching raw stores directly."}],"verification_status":"partial","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Cube\"]\n  fw --> p1[\"Semantic-Layer Query Guardrail<br/>(core)\"]\n  fw --> p2[\"Model Context Protocol<br/>(first-class)\"]\n  fw --> p3[\"Tenant-Scoped Tool Binding<br/>(supported)\"]","tags":["enterprise-platform","tool-use-environment","safety-control","low-code"]},{"id":"google-ap2","name":"Agent Payments Protocol (AP2)","kind":"framework","category":"enterprise-platform","vendor":"Google","language":"specification","license":"Apache-2.0","status":"active","first_released":"2025","url":"https://ap2-protocol.org/","repo":"https://github.com/google-agentic-commerce/AP2","docs_url":"https://ap2-protocol.org/","intent":"Define an open protocol for agent-led payments built on signed mandates and verifiable digital credentials that bound spending and leave a cryptographic audit trail.","description":"The Agent Payments Protocol (AP2) is an open specification from Google, developed with the FIPA Alliance working groups, for payments initiated by AI agents. The user signs mandates that capture spending constraints up front so an agent can execute payments within those bounds. Mandates are expressed as verifiable digital credentials, which are tamper-evident, cryptographically signed objects. Each transaction produces a non-repudiable cryptographic audit trail to support dispute resolution.","primary_use_cases":["authorizing agent-led payments within user-set spending bounds","issuing signed mandates as verifiable digital credentials","producing a non-repudiable audit trail for agent transactions"],"agent_loop_shape":"The user signs a mandate that captures payment constraints such as budget and allowed instruments, the agent executes a transaction within those constraints, and the protocol records each step as a tamper-evident, cryptographically signed verifiable digital credential that forms a non-repudiable audit trail.","build_surface":"full-code","members":[{"pattern":"session-scoped-payment-authorization","role":"core","note":"The user signs an open mandate that captures spending constraints (budget, allowed instruments) up front, letting the agent execute payments autonomously within those bounds and settle a specific charged amount on checkout.","evidence":[{"type":"spec","title":"Agent Payments Protocol (AP2) Documentation","url":"https://ap2-protocol.org/","quote":"Open: Captures the user's constraints on payment (e.g., budget, allowed instruments) for autonomous execution.","accessed":"2026-06-17"},{"type":"spec","title":"Agent Payments Protocol (AP2) Documentation","url":"https://ap2-protocol.org/","quote":"Captures the user's (or agent's) authorization for a specific transaction amount bound to a finalized checkout.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"provenance-ledger","role":"core","note":"AP2 records each transaction as a tamper-evident, cryptographically signed verifiable digital credential, producing a non-repudiable cryptographic audit trail for every transaction.","evidence":[{"type":"spec","title":"Agent Payments Protocol (AP2) Documentation","url":"https://ap2-protocol.org/","quote":"VDCs are tamper-evident, cryptographically signed digital objects that serve as the building blocks of a transaction.","accessed":"2026-06-17"},{"type":"spec","title":"Agent Payments Protocol (AP2) Documentation","url":"https://ap2-protocol.org/","quote":"AP2 provides a non-repudiable, cryptographic audit trail for every transaction, aiding in dispute resolution.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"deontic-token-delegation","role":"core","note":"AP2 reifies the user's authorizations and constraints as signed mandates — verifiable digital credentials — that are passed along the agent-to-merchant-to-credential-provider chain: the Checkout Mandate carries the user's signed approval of the goods, and the Payment Mandate carries authorization against a specific instrument and is conveyed to the credential provider, networks, and merchant payment processor to complete the transaction.","evidence":[{"type":"spec","title":"Executive Summary - Agent Payments Protocol (AP2)","url":"https://ap2-protocol.org/overview/","quote":"The user's signed approval must create a cryptographically signed 'Checkout Mandate'. This mandate contains the explicit goods being purchased & their confirmation of purchase.","accessed":"2026-06-17"},{"type":"spec","title":"Executive Summary - Agent Payments Protocol (AP2)","url":"https://ap2-protocol.org/overview/","quote":"The Payment Mandate must be conveyed to the Credential provider and Merchant to complete the transaction.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"agent-initiated-payment","role":"first-class","note":"Once the mandates are signed the agent settles the payment itself, presenting a Payment Mandate that authorizes a charge against a specific payment instrument as the verifiable proof that completes the transaction.","evidence":[{"type":"spec","title":"Executive Summary - Agent Payments Protocol (AP2)","url":"https://ap2-protocol.org/overview/","quote":"Separately, the Payment Mandate may be shared with the network & issuer for transaction authorization","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"mcp","role":"supported","note":"AP2 is not a standalone stack but an open extension layered on top of the existing agent protocols, designed to work over Agent2Agent (A2A) and the Model Context Protocol rather than replacing them.","evidence":[{"type":"spec","title":"Agent Payments Protocol (AP2) Documentation","url":"https://ap2-protocol.org/","quote":"As a non-proprietary, open extension for A2A and MCP, AP2 fosters a competitive environment for innovation.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"Mandate","summary":"A signed credential that captures a specific authorization step in a purchase; AP2 chains them so each stage of intent, cart, and payment is independently verifiable.","maps_to_pattern":"deontic-token-delegation","url":"https://ap2-protocol.org/overview/"},{"name":"Verifiable Digital Credential (VDC)","summary":"The tamper-evident, cryptographically signed object that every mandate is expressed as, built on the W3C Verifiable Credentials standard for portability and interoperability.","maps_to_pattern":"provenance-ledger","url":"https://ap2-protocol.org/"},{"name":"Checkout Mandate","summary":"The user's cryptographically signed approval of the explicit goods being purchased and confirmation of the purchase, created at the point the cart is accepted.","maps_to_pattern":"session-scoped-payment-authorization","url":"https://ap2-protocol.org/overview/"},{"name":"Payment Mandate","summary":"The mandate that authorizes a payment against a specific instrument and is conveyed to the credential provider, networks, and merchant payment processor to complete the transaction.","maps_to_pattern":"agent-initiated-payment","url":"https://ap2-protocol.org/overview/"}],"references":[{"type":"spec","title":"Agent Payments Protocol (AP2) Documentation","url":"https://ap2-protocol.org/","accessed":"2026-06-17"},{"type":"spec","title":"Executive Summary - Agent Payments Protocol (AP2)","url":"https://ap2-protocol.org/overview/","accessed":"2026-06-17"},{"type":"repo","title":"google-agentic-commerce/AP2","url":"https://github.com/google-agentic-commerce/AP2","accessed":"2026-06-17"}],"alternatives":[{"composition":"x402","relation":"competes-with","note":"Both let an agent settle a payment as part of its loop; x402 answers an HTTP 402 challenge with an inline on-chain micropayment, while AP2 builds signed multi-stage mandates over A2A/MCP for agent-led commerce."},{"composition":"a2a-protocol","relation":"complements","note":"AP2 is an open extension layered on A2A: agents collaborate over A2A and use AP2 mandates to authorize and settle the payment portion of the interaction."}],"verification_status":"partial","status_in_practice":"emerging","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Agent Payments Protocol (AP2)\"]\n  fw --> p1[\"Session-Scoped Payment Authorization<br/>(core)\"]\n  fw --> p2[\"Provenance Ledger<br/>(core)\"]\n  fw --> p3[\"Deontic Token Delegation<br/>(core)\"]\n  fw --> p4[\"Agent-Initiated Payment<br/>(first-class)\"]\n  fw --> p5[\"Model Context Protocol<br/>(supported)\"]","tags":["enterprise-platform","safety-control","governance-observability","tool-use-environment","full-code"]},{"id":"agentforce","name":"Salesforce Agentforce","kind":"framework","category":"enterprise-platform","vendor":"Salesforce","language":"Apex/Metadata","license":"proprietary","status":"active","first_released":"2024-09-12","url":"https://www.salesforce.com/agentforce/","docs_url":"https://help.salesforce.com/s/articleView?id=ai.agent_templates.htm","intent":"Agentforce is Salesforce's platform for building and deploying autonomous agents that reason over CRM data using topics, instructions, and actions, driven by the Atlas reasoning engine.","description":"Agentforce is Salesforce's enterprise platform for building autonomous agents grounded in CRM data and business processes. Agents are assembled from topics, instructions, and actions, and are reusable through agent templates that produce multiple agents with the same topics and actions. The Atlas reasoning engine drives each agent through a cycle of reasoning, acting, and observing until the user's goal is met. Topics scope which actions are valid and which policies apply at any moment.","primary_use_cases":["autonomous CRM and service agents","reusing agent definitions across many agents via templates","topic-scoped action selection over Salesforce flows and APIs","grounding agents in business data and processes"],"agent_loop_shape":"Each request is classified into a topic that defines which actions are valid and which policies apply. The Atlas reasoning engine then runs a cycle of reasoning, acting, and observing: it thinks through a step, invokes an action such as a Salesforce flow or API, checks the outcome, and loops until the user's goal is complete. Agent definitions are packaged as reusable templates so one template produces multiple consistent agents.","build_surface":"low-code","members":[{"pattern":"agent-factory","role":"supported","note":"Agentforce manufactures agent instances from reusable agent templates that package topics, instructions, and actions, deployed via metadata so one template definition produces multiple consistent agents; registry-backed identities and atomic propagation are only partially documented, hence medium confidence.","evidence":[{"type":"doc","title":"Agentforce Agent Templates — Salesforce Help","url":"https://help.salesforce.com/s/articleView?id=ai.agent_templates.htm","quote":"An agent template is a reusable definition of an agent... use a template to create multiple agents with the same topics and actions.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"react","role":"core","note":"The Atlas reasoning engine drives each agent through an explicit reason-act-observe cycle: it thinks through a step, takes an action, checks the outcome, and loops until the goal is met, with topics scoping which actions are valid at each step.","evidence":[{"type":"tutorial","title":"Discover the Atlas Reasoning Engine — Salesforce Trailhead","url":"https://trailhead.salesforce.com/content/learn/modules/reasoning-in-artificial-intelligence/discover-the-atlas-reasoning-engine","quote":"Atlas uses a cycle of reasoning, acting, and observing. It thinks through a step, uses an action, checks the outcome, and loops until it completes the user's goal.","accessed":"2026-06-17"},{"type":"tutorial","title":"Discover the Atlas Reasoning Engine — Salesforce Trailhead","url":"https://trailhead.salesforce.com/content/learn/modules/reasoning-in-artificial-intelligence/discover-the-atlas-reasoning-engine","quote":"Every request is mapped to a subagent, which serves as the AI's job description for that moment. Subagents define what actions are valid and which policies apply.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"tool-use","role":"core","note":"Agentforce actions are the typed tools an agent invokes to perform tasks — built from Apex invocable methods, flows, named SOQL queries, or external APIs exposed through OpenAPI — rather than the model emitting free text the platform must parse.","evidence":[{"type":"doc","title":"Agentforce Actions — Agentforce Developer Guide","url":"https://developer.salesforce.com/docs/ai/agentforce/guide/get-started-actions.html","quote":"Actions are the building blocks that enable your Agentforce agents to perform tasks and interact with data.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"naive-rag","role":"first-class","note":"The Agentforce Data Library grounds an agent in company data through retrieval-augmented generation: documents are chunked and indexed, and retrieved chunks are added to the prompt so the model answers from trusted data rather than parametric knowledge alone.","evidence":[{"type":"tutorial","title":"Enhance Agents with Business Knowledge Using RAG — Salesforce Trailhead","url":"https://trailhead.salesforce.com/content/learn/modules/retrieval-augmented-generation-quick-look/augment-prompts-with-relevant-knowledge","quote":"Retrieval augmented generation (RAG) is a popular way to ground prompt requests to large language models (LLMs). Grounding adds domain-specific knowledge or customer information to the prompt, giving the LLM context to respond more accurately to a question or task.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"production-failure-triage-loop","role":"supported","evidence_status":"full","note":"Salesforce describes routing post-launch failures into four buckets (tone/brand, logic, data quality, coverage) each with its own fix path, and treats feedback-loop speed as the gate to scaling across 20,000+ deployments.","evidence":[{"type":"blog","title":"What Salesforce learned from 20,000+ AI agent deployments","url":"https://blog.bytebytego.com/p/what-salesforce-learned-from-20000","quote":"Speed of this feedback loop turned out to be the gate to scaling.","accessed":"2026-06-19"}]},{"pattern":"input-output-guardrails","role":"first-class","evidence_status":"limited","note":"Agentforce's 'Trust Layer' is described as input guardrails (secure data retrieval, zero data retention, trusted boundary hosting, data masking) plus output guardrails (tool/sub-agent validation to catch hallucinated actions, grounding checks, content filtering and toxicity scoring before delivery).","evidence":[{"type":"blog","title":"What Salesforce learned from 20,000+ AI agent deployments","url":"https://blog.bytebytego.com/p/what-salesforce-learned-from-20000","accessed":"2026-06-19"}]},{"pattern":"deterministic-control-flow-not-prompt","role":"supported","evidence_status":"limited","note":"Salesforce's 'Agent Script' is a TypeScript framework that combines deterministic control flow with LLM-powered flexibility, routing predictable workflows through code instead of a reasoning loop.","evidence":[{"type":"blog","title":"What Salesforce learned from 20,000+ AI agent deployments","url":"https://blog.bytebytego.com/p/what-salesforce-learned-from-20000","accessed":"2026-06-19","quote":"If you can write the logic as a flowchart, it should probably be code, not a prompt."}]},{"pattern":"policy-as-code-gate","role":"supported","evidence_status":"limited","note":"The blog calls out encoding business rules as structured, explicit policies (conditionals in code) rather than emphasis in prompts, e.g. geographic restrictions enforced as code-level checks.","evidence":[{"type":"blog","title":"What Salesforce learned from 20,000+ AI agent deployments","url":"https://blog.bytebytego.com/p/what-salesforce-learned-from-20000","accessed":"2026-06-19"}]},{"pattern":"hierarchical-agents","role":"supported","evidence_status":"limited","note":"Salesforce reports building three-level hierarchies: parent agents coordinate sub-agents which coordinate sub-sub-agents, each level narrowing scope, tool count, and context-window size.","evidence":[{"type":"blog","title":"What Salesforce learned from 20,000+ AI agent deployments","url":"https://blog.bytebytego.com/p/what-salesforce-learned-from-20000","accessed":"2026-06-19"}]},{"pattern":"context-minimization","role":"supported","evidence_status":"limited","note":"The blog names trimming unfiltered API responses to only necessary fields (reducing ~100K tokens to ~2K) and loading only relevant document sections as a core context-engineering practice.","evidence":[{"type":"blog","title":"What Salesforce learned from 20,000+ AI agent deployments","url":"https://blog.bytebytego.com/p/what-salesforce-learned-from-20000","accessed":"2026-06-19"}]}],"key_concepts":[{"name":"Atlas Reasoning Engine","summary":"Agentforce's reasoning system that classifies each request into a topic and then loops through reasoning, acting, and observing until the user's goal is met.","maps_to_pattern":"react","url":"https://trailhead.salesforce.com/content/learn/modules/reasoning-in-artificial-intelligence/discover-the-atlas-reasoning-engine"},{"name":"Topics (Subagents)","summary":"Named scopes that categorize the job to be done and define which actions are valid and which instructions and policies apply while that scope is active; renamed subagents in April 2026.","url":"https://help.salesforce.com/s/articleView?id=ai.agent_templates.htm"},{"name":"Actions","summary":"The tools an agent invokes to perform tasks, built from Apex, flows, named SOQL queries, or external APIs, with typed inputs and outputs.","maps_to_pattern":"tool-use","url":"https://developer.salesforce.com/docs/ai/agentforce/guide/get-started-actions.html"},{"name":"Agentforce Data Library","summary":"A managed knowledge base that ingests, chunks, and indexes Salesforce knowledge articles, uploaded files, and web data so agents can retrieve and ground answers on trusted company data.","maps_to_pattern":"naive-rag","url":"https://engineering.salesforce.com/optimizing-ai-retrieval-how-agentforce-data-library-powers-rag-with-99-99-uptime/"},{"name":"Agent Templates","summary":"Reusable agent definitions that package topics, instructions, and actions so an organization can stamp out multiple consistent agents from one definition.","maps_to_pattern":"agent-factory","url":"https://help.salesforce.com/s/articleView?id=ai.agent_templates.htm"}],"references":[{"type":"doc","title":"Salesforce Agentforce — product home","url":"https://www.salesforce.com/agentforce/","accessed":"2026-06-17"},{"type":"doc","title":"Agentforce Developer Guide — Get Started","url":"https://developer.salesforce.com/docs/ai/agentforce/guide/get-started.html","accessed":"2026-06-17"},{"type":"blog","title":"Optimizing AI Retrieval: How Agentforce Data Library Powers RAG — Salesforce Engineering","url":"https://engineering.salesforce.com/optimizing-ai-retrieval-how-agentforce-data-library-powers-rag-with-99-99-uptime/","quote":"The team is dedicated to developing and optimizing Agentforce Data Library, a RAG system that powers AI-driven customer interactions at an enterprise scale.","accessed":"2026-06-17"}],"alternatives":[{"composition":"bedrock-agents","relation":"competes-with","note":"Amazon Bedrock Agents is the comparable cloud-vendor managed agent platform that grounds agents in enterprise data and dispatches actions via action groups."},{"composition":"vertex-ai-agent-builder","relation":"competes-with","note":"Vertex AI Agent Builder is Google Cloud's equivalent for assembling grounded enterprise agents over company data and tools."},{"composition":"azure-ai-foundry-agent-service","relation":"competes-with","note":"Azure AI Foundry Agent Service is Microsoft's managed enterprise agent runtime targeting the same build-and-deploy-grounded-agents use case."}],"verification_status":"partial","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-14","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Salesforce Agentforce\"]\n  fw --> p1[\"ReAct<br/>(core)\"]\n  fw --> p2[\"Tool Use<br/>(core)\"]\n  fw --> p3[\"Naive RAG<br/>(first-class)\"]\n  fw --> p4[\"Agent Factory<br/>(supported)\"]","tags":["enterprise-platform","governance-observability","planning-control-flow","tool-use-environment","retrieval","low-code"]},{"id":"arize-phoenix","name":"Arize Phoenix / Arize AX","kind":"framework","category":"enterprise-platform","vendor":"Arize AI","language":"Python","license":"Elastic-2.0","status":"active","first_released":"2023-02-07","url":"https://phoenix.arize.com","repo":"https://github.com/Arize-ai/phoenix","docs_url":"https://arize.com/docs/ax","intent":"Arize Phoenix and Arize AX trace agent runs and evaluate them with LLM judges, scoring both individual steps and the full trajectory of tool calls.","description":"Arize provides open-source Phoenix and the hosted Arize AX platform for tracing and evaluating LLM and agent applications. It captures spans and traces of agent runs and evaluates them with LLM-based judges. Trajectory evaluations send the ordered list of an agent's tool calls to an LLM judge that classifies the run as correct or incorrect, catching mistakes that single-step evaluations miss. Phoenix Evals runs these LLM classifications against captured traces.","primary_use_cases":["tracing LLM and agent runs","LLM-as-judge evaluation of agent trajectories","scoring individual tool-calling spans","detecting mistakes an agent makes between steps"],"agent_loop_shape":"Arize observes the agent rather than running its loop. It ingests spans and traces from an instrumented agent, then evaluation jobs send the captured artifacts — an individual span, or the ordered list of tool calls forming a trajectory — to an LLM judge that classifies them as correct or incorrect against criteria, surfacing failures at the step or trajectory level for the developer to act on.","build_surface":"full-code","members":[{"pattern":"artifact-evaluation","role":"first-class","note":"Arize evaluates intermediate agent artifacts — sending the ordered list of tool calls (the trajectory) to an LLM judge and scoring individual tool-calling spans — rather than only the final output, isolating mistakes an agent makes between steps.","evidence":[{"type":"doc","title":"Agent Trajectory Evaluations — Arize AX Docs","url":"https://arize.com/docs/ax/evaluate/evaluators/trace-and-session-evals/trace-level-evaluations/agent-trajectory-evaluations","quote":"Individual span or trace evaluations check that one step or response is correct, but they can miss costly mistakes an agent makes between steps.","accessed":"2026-06-17"},{"type":"doc","title":"Agent Trajectory Evaluations — Arize AX Docs","url":"https://arize.com/docs/ax/evaluate/evaluators/trace-and-session-evals/trace-level-evaluations/agent-trajectory-evaluations","quote":"Send the ordered list of tool calls to an LLM judge – Phoenix Evals classifies the trajectory as correct or incorrect.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"llm-as-judge","role":"core","note":"Arize evaluations use an LLM judge to classify outputs against criteria — for example a trajectory that progresses logically, uses the right tools, and is reasonably efficient — when no exact-match metric applies, via the Phoenix Evals llm_classify call.","evidence":[{"type":"doc","title":"Agent Trajectory Evaluations — Arize AX Docs","url":"https://arize.com/docs/ax/evaluate/evaluators/trace-and-session-evals/trace-level-evaluations/agent-trajectory-evaluations","quote":"Send the ordered list of tool calls to an LLM judge – Phoenix Evals classifies the trajectory as correct or incorrect.","accessed":"2026-06-17"},{"type":"doc","title":"Agent Trajectory Evaluations — Arize AX Docs","url":"https://arize.com/docs/ax/evaluate/evaluators/trace-and-session-evals/trace-level-evaluations/agent-trajectory-evaluations","quote":"results = llm_classify(dataframe = trace_df, template = TRAJECTORY_ACCURACY_PROMPT, model = model, rails = rails)","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"eval-harness","role":"first-class","note":"Phoenix datasets and experiments form an offline eval harness: a versioned dataset of examples is run through a task and its associated evaluators score the outputs automatically, giving repeatable regression testing across agent and prompt versions.","evidence":[{"type":"doc","title":"Overview: Datasets & Experiments — Phoenix Docs","url":"https://arize.com/docs/phoenix/datasets-and-experiments/overview-datasets","quote":"When you run an experiment against a dataset, its associated evaluators run automatically, providing consistent and repeatable quality assessments.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"dual-evaluation-offline-online","role":"supported","note":"Arize runs evaluation on two tracks: offline experiments against curated datasets before deploy, and online evals that run continuously over production trace data on a rolling schedule, so quality is checked both pre-deploy and on live traffic.","evidence":[{"type":"doc","title":"Log Evaluations to Arize — Arize AX Docs","url":"https://arize.com/docs/ax/evaluate/online-evals/log-evaluations-to-arize","quote":"Run online evals over your production trace data.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"OpenInference","summary":"Arize's set of OpenTelemetry semantic conventions for LLM and agent spans, giving every model call, tool invocation, and retrieval a standardized trace shape that Phoenix ingests.","url":"https://github.com/Arize-ai/openinference"},{"name":"Phoenix Evals","summary":"The evaluation library that runs LLM classifications (via llm_classify) against captured traces and spans, scoring them correct or incorrect against a rubric.","maps_to_pattern":"llm-as-judge","url":"https://arize.com/docs/ax/evaluate/evaluators/trace-and-session-evals/trace-level-evaluations/agent-trajectory-evaluations"},{"name":"Datasets & Experiments","summary":"Versioned example sets and the experiment runs over them, with dataset evaluators that score outputs automatically — Phoenix's offline, repeatable testing workflow.","maps_to_pattern":"eval-harness","url":"https://arize.com/docs/phoenix/datasets-and-experiments/overview-datasets"},{"name":"Trajectory Evaluation","summary":"A trace-level evaluation that hands the ordered list of an agent's tool calls to an LLM judge to score whether the whole run progressed logically, used the right tools, and was efficient.","maps_to_pattern":"artifact-evaluation","url":"https://arize.com/docs/ax/evaluate/evaluators/trace-and-session-evals/trace-level-evaluations/agent-trajectory-evaluations"}],"references":[{"type":"repo","title":"Arize-ai/phoenix — AI Observability & Evaluation","url":"https://github.com/Arize-ai/phoenix","quote":"Trace your LLM application's runtime using OpenTelemetry-based instrumentation.","accessed":"2026-06-17"},{"type":"doc","title":"Arize AX Documentation","url":"https://arize.com/docs/ax","accessed":"2026-06-17"},{"type":"repo","title":"Arize-ai/openinference — OpenTelemetry Instrumentation for AI Observability","url":"https://github.com/Arize-ai/openinference","accessed":"2026-06-17"}],"alternatives":[{"composition":"langfuse","relation":"competes-with","note":"Langfuse is an open-source LLM tracing-plus-evaluation platform covering the same instrument-then-evaluate shape."},{"composition":"langsmith","relation":"competes-with","note":"LangSmith offers comparable trace capture and dataset-and-evaluator workflows for LLM and agent applications."},{"composition":"braintrust","relation":"competes-with","note":"Braintrust overlaps on offline eval against datasets and online scoring of production traces, though it is less centred on OpenTelemetry tracing."}],"verification_status":"needs-verification","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-19","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Arize Phoenix / Arize AX\"]\n  fw --> p1[\"LLM-as-Judge<br/>(core)\"]\n  fw --> p2[\"Intermediate Artifact Evaluation<br/>(first-class)\"]\n  fw --> p3[\"Eval Harness<br/>(first-class)\"]\n  fw --> p4[\"Dual Evaluation (Offline + Online)<br/>(supported)\"]","tags":["enterprise-platform","governance-observability","full-code"],"anti_patterns_avoided":[{"pattern":"agent-tool-data-black-box","note":"Traces surface the tool descriptions and signatures the LLM can call, making the agent's tool/data access inspectable instead of opaque.","evidence":[{"type":"doc","title":"Arize Phoenix — LLM traces","url":"https://arize.com/docs/phoenix/tracing/llm-traces","quote":"View the descriptions and function signatures of the tools your LLM has been given access to in order to better understand and control your LLM's capabilities","accessed":"2026-06-19"}]},{"pattern":"compound-error-degradation","note":"Trajectory evaluations score the whole sequence of steps, catching the between-step compounding mistakes that single-span evals miss.","evidence":[{"type":"doc","title":"Arize — agent trajectory evaluations","url":"https://arize.com/docs/ax/evaluate/evaluators/trace-and-session-evals/trace-level-evaluations/agent-trajectory-evaluations","quote":"Individual span or trace evaluations check that one step or response is correct, but they can miss costly mistakes an agent makes between steps. Agent trajectory evaluations measure the entire sequence of tool calls an agent takes to solve a task.","accessed":"2026-06-19"}]}]},{"id":"braintrust","name":"Braintrust","kind":"framework","category":"enterprise-platform","vendor":"Braintrust Data, Inc.","language":"TypeScript","license":"proprietary","status":"active","first_released":"2023-01-01","url":"https://www.braintrust.dev","docs_url":"https://www.braintrust.dev/docs","intent":"Braintrust is a platform for evaluating LLM applications offline against datasets and scorers and for scoring production traces online without adding latency.","description":"Braintrust is a platform for building, evaluating, and monitoring LLM applications. Offline, each evaluation runs a task against a dataset of test cases and measures the output with scoring functions, producing an experiment that records inputs, outputs, scores, and metadata for comparing prompt and model versions. Online, configured scorers run against production traces automatically as they are logged, evaluating asynchronously in the background. This gives both pre-deploy regression checks and continuous production quality monitoring.","primary_use_cases":["offline evaluation of LLM applications against datasets","comparing prompt and model versions across experiments","catching regressions in CI","online scoring of production traces"],"agent_loop_shape":"Braintrust evaluates rather than runs the application. Offline, an Eval pairs a dataset, a task function under test, and scoring functions; running it produces an experiment that records inputs, outputs, and scores for comparison across versions. Online, scorers run asynchronously against production traces as they arrive, logging quality metrics without regenerating output or adding latency to the application.","build_surface":"full-code","members":[{"pattern":"scorer-live-monitoring","role":"first-class","note":"Braintrust online scoring runs configured scorers against production logs automatically as they arrive, asynchronously in the background, observing and logging quality without adding latency or regenerating the output.","evidence":[{"type":"doc","title":"Score production traces — Braintrust Docs","url":"https://www.braintrust.dev/docs/evaluate/score-online","quote":"Online scoring evaluates production traces automatically as they're logged, running evaluations asynchronously in the background to provide continuous quality monitoring without affecting your application's latency or performance.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"eval-harness","role":"core","note":"An offline Braintrust evaluation runs a task function against a dataset of test cases and scores the outputs, producing a permanent experiment record that captures inputs, outputs, and scores for comparing different versions of prompts or models and catching regressions.","evidence":[{"type":"doc","title":"Evaluation SDK — Braintrust Docs","url":"https://www.braintrust.dev/docs/start/eval-sdk","quote":"Each evaluation consists of three components: Data - A dataset of test cases with inputs and expected outputs, Task - An AI function you want to test, Scores - Scoring functions that measure output quality","accessed":"2026-06-17"},{"type":"doc","title":"Evaluation SDK — Braintrust Docs","url":"https://www.braintrust.dev/docs/start/eval-sdk","quote":"This creates an experiment, a permanent record of how your task performed on the dataset. Each experiment captures inputs, outputs, scores, and metadata, making it easy to compare different versions of your prompts or models.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"llm-as-judge","role":"first-class","note":"Braintrust scorers can be expressed as LLM-as-a-judge functions that use a language model and a natural-language rubric to score subjective output quality where deterministic exact-match metrics do not apply, alongside code-based scorers and the autoevals library.","evidence":[{"type":"doc","title":"Scorers — Braintrust Docs","url":"https://www.braintrust.dev/docs/platform/functions/scorers","quote":"Use language models to evaluate outputs based on natural language criteria and instructions. Best for subjective judgments like tone, helpfulness, or creativity that are difficult to encode in deterministic code.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"prompt-versioning","role":"supported","note":"Braintrust treats prompts as versioned artifacts in a registry: every change creates a new version automatically, prompts deploy independently of application code, and a previously validated version can be rolled back to without a code change.","evidence":[{"type":"doc","title":"Create prompts — Braintrust Docs","url":"https://www.braintrust.dev/docs/evaluate/write-prompts","quote":"Version and deploy prompts independently of your code. Test changes in the playground and roll out to specific environments without redeploying your application.","accessed":"2026-06-17"},{"type":"doc","title":"Create prompts — Braintrust Docs","url":"https://www.braintrust.dev/docs/evaluate/write-prompts","quote":"Every prompt change creates a new version automatically. This lets you compare performance across versions, roll back to previous versions, pin experiments to specific versions, [and] track which version is used in production.","accessed":"2026-06-17"}],"evidence_status":"full"}],"key_concepts":[{"name":"Eval","summary":"The core offline unit that pairs a dataset, a task function under test, and scoring functions; running it produces an experiment for comparing versions.","maps_to_pattern":"eval-harness","url":"https://www.braintrust.dev/docs/start/eval-sdk"},{"name":"Experiment","summary":"The permanent record produced by running an Eval, capturing inputs, outputs, scores, and metadata so prompt and model versions can be compared and regressions caught.","url":"https://www.braintrust.dev/docs/start/eval-sdk"},{"name":"Scorers / autoevals","summary":"Scoring functions — code heuristics or LLM-as-a-judge prompts — that assign a quality score to outputs, with pre-built scorers shipped in the open-source autoevals library.","maps_to_pattern":"llm-as-judge","url":"https://www.braintrust.dev/docs/platform/functions/scorers"},{"name":"Online scoring","summary":"Scorers configured to run asynchronously against production traces as they are logged, providing continuous quality monitoring without adding latency.","maps_to_pattern":"scorer-live-monitoring","url":"https://www.braintrust.dev/docs/evaluate/score-online"},{"name":"Playground","summary":"An interactive surface for testing prompts against sample inputs and saving them to a versioned prompt registry before rolling out to environments.","maps_to_pattern":"prompt-versioning","url":"https://www.braintrust.dev/docs/evaluate/write-prompts"}],"references":[{"type":"doc","title":"Braintrust — product home","url":"https://www.braintrust.dev","accessed":"2026-06-17"},{"type":"doc","title":"Braintrust Documentation","url":"https://www.braintrust.dev/docs","accessed":"2026-06-17"},{"type":"doc","title":"Evaluation SDK — Braintrust Docs","url":"https://www.braintrust.dev/docs/start/eval-sdk","accessed":"2026-06-17"}],"alternatives":[{"composition":"arize-phoenix","relation":"competes-with","note":"Arize Phoenix / AX covers the same offline-experiment-plus-online-eval shape, with a stronger emphasis on OpenTelemetry tracing."},{"composition":"langsmith","relation":"competes-with","note":"LangSmith offers comparable dataset evaluation, prompt management, and production trace monitoring for LLM applications."},{"composition":"promptfoo","relation":"competes-with","note":"Promptfoo overlaps on offline eval of prompts against datasets with scorers, oriented toward local and CI test runs."}],"verification_status":"needs-verification","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-19","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Braintrust\"]\n  fw --> p1[\"Eval Harness<br/>(core)\"]\n  fw --> p2[\"Scorer Live Monitoring<br/>(first-class)\"]\n  fw --> p3[\"LLM-as-Judge<br/>(first-class)\"]\n  fw --> p4[\"Prompt Versioning<br/>(supported)\"]","tags":["enterprise-platform","governance-observability","full-code"],"anti_patterns_avoided":[{"pattern":"agent-output-alert-fatigue","note":"Topics automatically classify logs to surface recurring problems across traces, instead of flooding reviewers with individual findings.","evidence":[{"type":"doc","title":"Braintrust — Topics","url":"https://www.braintrust.dev/docs/observe/topics","quote":"Topics automatically analyze and classify your logs to surface the failures, edge cases, and recurring problems hiding across your traces.","accessed":"2026-06-19"}]}]},{"id":"llamafirewall","name":"Meta LlamaFirewall (AlignmentCheck)","kind":"framework","category":"enterprise-platform","vendor":"Meta","language":"Python","license":"MIT","status":"active","first_released":"2025-04-29","url":"https://github.com/meta-llama/PurpleLlama/tree/main/LlamaFirewall","repo":"https://github.com/meta-llama/PurpleLlama","docs_url":"https://meta-llama.github.io/PurpleLlama/LlamaFirewall/","intent":"LlamaFirewall is a guardrail framework that runs a set of scanners on the inputs, outputs, and reasoning trace of an LLM agent to detect and mitigate prompt injection, goal hijacking, and misalignment before the agent acts.","description":"LlamaFirewall is an open-source security framework from Meta's Purple Llama project that wraps an LLM agent with composable scanners. PromptGuard classifies user inputs and untrusted tool content for direct prompt injection and jailbreak attempts, while AlignmentCheck audits the agent's chain-of-thought in real time for goal hijacking and indirect injection. A CodeShield scanner and customizable regex filters cover code generation and additional risks. The scanners can be plugged into different stages of an agent's workflow, from raw input ingestion to final output.","primary_use_cases":["scanning agent inputs for prompt injection and jailbreaks","auditing an agent's reasoning trace for goal hijacking","detecting misalignment in multi-step agentic operations","layered guardrails around LLM agents"],"agent_loop_shape":"LlamaFirewall acts as a policy engine that orchestrates security scanners across stages of the agent's workflow. Untrusted inputs and tool content are scanned by PromptGuard for direct prompt injection before they reach the model; as the agent reasons, AlignmentCheck inspects the running chain-of-thought for goal hijacking and indirect injection; code outputs are checked by CodeShield. A scanner that flags a risk can block the input or action so the agent does not proceed on poisoned content.","build_surface":"full-code","members":[{"pattern":"tool-output-poisoning","role":"core","note":"PromptGuard scans untrusted content such as web and tool data while AlignmentCheck reasons over the agent's trace to catch goal hijacking from misleading tool output and indirect prompt injection before the agent acts on it.","evidence":[{"type":"repo","title":"PurpleLlama/LlamaFirewall README — AlignmentCheck","url":"https://github.com/meta-llama/PurpleLlama/blob/main/LlamaFirewall/README.md","quote":"It uses few-shot prompting and semantic analysis to detect goal hijacking, indirect prompt injections, and signs of agent misalignment.","accessed":"2026-06-17"},{"type":"doc","title":"AlignmentCheck | LlamaFirewall","url":"https://meta-llama.github.io/PurpleLlama/LlamaFirewall/docs/documentation/scanners/alignment-check","quote":"This helps detect and prevent covert prompt injection, misleading tool output, and other forms of goal hijacking.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"prompt-injection-defense","role":"first-class","note":"PromptGuard is a classifier that detects direct prompt injection attempts in user inputs and untrusted content such as web data, so the framework can flag injected instructions before they reach the agent rather than trusting them.","evidence":[{"type":"repo","title":"PurpleLlama/LlamaFirewall README — PromptGuard","url":"https://github.com/meta-llama/PurpleLlama/blob/main/LlamaFirewall/README.md","quote":"A fast, lightweight BERT-style classifier that detects direct prompt injection attempts. It operates on user inputs and untrusted content such as web data, providing high precision and low latency even in high-throughput environments.","accessed":"2026-06-17"},{"type":"repo","title":"PurpleLlama/LlamaFirewall README — overview","url":"https://github.com/meta-llama/PurpleLlama/blob/main/LlamaFirewall/README.md","quote":"LlamaFirewall is a framework designed to detect and mitigate AI centric security risks, supporting multiple layers of inputs and outputs, such as typical LLM chat and more advanced multi-step agentic operations.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"input-output-guardrails","role":"core","note":"Scanners are composed into layered defenses that span the agent's full I/O surface — from raw input ingestion through the reasoning trace to final output actions — with CodeShield statically analysing generated code on the output side, so checks fire on both what enters and what leaves the model.","evidence":[{"type":"repo","title":"PurpleLlama/LlamaFirewall README — layered defenses","url":"https://github.com/meta-llama/PurpleLlama/blob/main/LlamaFirewall/README.md","quote":"Enable security teams and developers to compose layered defenses that span from raw input ingestion to final output actions.","accessed":"2026-06-17"},{"type":"repo","title":"PurpleLlama/LlamaFirewall README — CodeShield","url":"https://github.com/meta-llama/PurpleLlama/blob/main/LlamaFirewall/README.md","quote":"A static analysis engine that examines LLM-generated code for security issues in real time.","accessed":"2026-06-17"}],"evidence_status":"full"}],"key_concepts":[{"name":"PromptGuard","summary":"A fast BERT-style classifier scanner that flags direct prompt-injection and jailbreak attempts in user inputs and untrusted content such as web and tool data.","maps_to_pattern":"prompt-injection-defense","url":"https://github.com/meta-llama/PurpleLlama/blob/main/LlamaFirewall/README.md"},{"name":"AlignmentCheck","summary":"A scanner that audits the agent's running chain-of-thought with few-shot prompting and semantic analysis to detect goal hijacking, indirect injection, and misalignment before the agent acts.","maps_to_pattern":"tool-output-poisoning","url":"https://meta-llama.github.io/PurpleLlama/LlamaFirewall/docs/documentation/scanners/alignment-check"},{"name":"CodeShield","summary":"A static-analysis scanner that examines LLM-generated code for security issues in real time on the output side of the agent.","maps_to_pattern":"input-output-guardrails","url":"https://github.com/meta-llama/PurpleLlama/blob/main/LlamaFirewall/README.md"},{"name":"Scanner composition","summary":"The framework's plug-in model where individual scanners are attached to chosen workflow stages to compose layered defenses spanning raw input ingestion to final output actions.","url":"https://github.com/meta-llama/PurpleLlama/blob/main/LlamaFirewall/README.md"}],"references":[{"type":"repo","title":"meta-llama/PurpleLlama — LlamaFirewall","url":"https://github.com/meta-llama/PurpleLlama/tree/main/LlamaFirewall","accessed":"2026-06-17"},{"type":"doc","title":"LlamaFirewall Documentation","url":"https://meta-llama.github.io/PurpleLlama/LlamaFirewall/","accessed":"2026-06-17"}],"alternatives":[{"composition":"llama-guard","relation":"same-vendor","note":"Meta Llama Guard 3 is the sibling Purple Llama content-safety classifier; LlamaFirewall focuses on injection and alignment scanning rather than content moderation."},{"composition":"nemo-guardrails","relation":"competes-with","note":"NVIDIA NeMo Guardrails is another framework for composing input/output and dialog rails around an LLM application."},{"composition":"invariant-labs","relation":"competes-with","note":"Invariant Guardrails / MCP-scan target the same injection and tool-misuse threat surface around agentic operations."}],"verification_status":"needs-verification","status_in_practice":"emerging","last_analyzed":"2026-06-17","last_updated":"2026-06-19","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Meta LlamaFirewall (AlignmentCheck)\"]\n  fw --> p1[\"Tool Output Poisoning Defense<br/>(core)\"]\n  fw --> p2[\"Input/Output Guardrails<br/>(core)\"]\n  fw --> p3[\"Prompt Injection Defense<br/>(first-class)\"]","tags":["enterprise-platform","safety-control","full-code"],"anti_patterns_avoided":[{"pattern":"goal-hijacking","note":"PromptGuard 2 detects direct jailbreak attempts in real time, blocking injected instructions that would redirect the agent's goal.","evidence":[{"type":"doc","title":"LlamaFirewall — PromptGuard 2","url":"https://meta-llama.github.io/PurpleLlama/LlamaFirewall/docs/documentation/scanners/prompt-guard-2","quote":"PromptGuard 2 is a fine-tuned BERT-style model designed to detect direct jailbreak attempts in real-time, with high accuracy and low latency.","accessed":"2026-06-19"}]}]},{"id":"microsoft-entra-agent-id","name":"Microsoft Entra Agent ID","kind":"framework","category":"enterprise-platform","vendor":"Microsoft","language":"N/A","license":"proprietary","status":"active","first_released":"2025-05-19","url":"https://learn.microsoft.com/en-us/entra/agent-id/","docs_url":"https://learn.microsoft.com/en-us/entra/agent-id/","intent":"Microsoft Entra Agent ID gives each AI agent its own governed directory identity so that an organisation can authenticate, scope, and audit what agents do rather than letting them act under shared or human credentials.","description":"Microsoft Entra Agent ID is an identity and security framework that extends Microsoft Entra to AI agents. It provides purpose-built identity constructs for nonhuman agents, created from agent identity blueprints that act as templates with preconfigured permissions and policies. Agents authenticate and obtain tokens through OAuth 2.0 flows, including on-behalf-of delegation and autonomous app-only operation, and receive the same identity-driven protections as users such as Conditional Access, risk detection, and lifecycle governance. It works with agents built on Microsoft and non-Microsoft platforms.","primary_use_cases":["issuing governed identities to AI agents","delegated and autonomous agent authentication via OAuth 2.0","applying Conditional Access and Zero Trust controls to agents","lifecycle governance and audit of agent identities"],"agent_loop_shape":"Rather than running an agent loop itself, Entra Agent ID is the identity plane an agent calls at action time. An agent identity is provisioned from a blueprint and obtains scoped access tokens through OAuth 2.0 flows: on-behalf-of when acting for a user, client-credentials when acting autonomously. Each token-protected resource access is governed by adaptive access policies and logged, so the agent only reaches what its identity is permitted to and every action is attributable.","build_surface":"closed-application","members":[{"pattern":"ephemeral-agent-identity","role":"supported","note":"Each agent is provisioned its own purpose-built directory identity from an agent identity blueprint, with lifecycle governance and credential rotation, rather than sharing a human's identity, so agent actions are separately attributable and revocable.","evidence":[{"type":"doc","title":"What is Microsoft Entra Agent ID?","url":"https://learn.microsoft.com/en-us/entra/agent-id/what-is-microsoft-entra-agent-id","quote":"Agent identity blueprints serve as templates for creating individual agent identities with parent-child relationships, enabling consistent security policies across large numbers of agents.","accessed":"2026-06-17"},{"type":"doc","title":"What is Microsoft Entra Agent ID?","url":"https://learn.microsoft.com/en-us/entra/agent-id/what-is-microsoft-entra-agent-id","quote":"they need purpose-built identity constructs to authenticate, authorize, govern, and protect these nonhuman identities.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"delegated-agent-authorization","role":"first-class","note":"Agents acquire scoped access tokens through OAuth 2.0 on-behalf-of and client-credentials flows rather than using a principal's static secrets, with credential rotation via managed identities, so each action runs under its own scoped, revocable authorization.","evidence":[{"type":"doc","title":"Authentication protocols in agents | Microsoft Entra Agent ID","url":"https://learn.microsoft.com/en-us/entra/agent-id/agent-oauth-protocols","quote":"Agents use OAuth 2.0 protocols with specialized token exchange patterns enabled by Federated Identity Credentials (FIC).","accessed":"2026-06-17"},{"type":"doc","title":"Authentication protocols in agents | Microsoft Entra Agent ID","url":"https://learn.microsoft.com/en-us/entra/agent-id/agent-oauth-protocols","quote":"This integration allows the agent ID to receive the full benefits of MSI security and management, including automatic credential rotation and secure storage.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"policy-as-code-gate","role":"first-class","note":"Conditional Access is an externally-managed policy engine evaluated before a token is issued: every agent access request is checked against if-then policy requirements that allow, block, or limit access based on the subject and the audience, so compliance authorship lives outside the agent and gates each action.","evidence":[{"type":"doc","title":"Conditional Access for Agents in Microsoft Entra","url":"https://learn.microsoft.com/en-us/entra/identity/conditional-access/agent-id","quote":"Conditional Access is an intelligent policy engine that helps organizations control how users and agents access corporate resources. It brings together real-time signals such as user's and agent's context, device, location, and session risk information to determine when to allow, block, or limit access, or require more verification steps.","accessed":"2026-06-17"},{"type":"doc","title":"Conditional Access for Agents in Microsoft Entra","url":"https://learn.microsoft.com/en-us/entra/identity/conditional-access/agent-id","quote":"When a Conditional Access policy applies, Microsoft Entra ID evaluates the configured policy requirements before issuing the token. If the requirements are satisfied, an access token is issued.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"provenance-ledger","role":"supported","note":"All agent authentication and activity is logged for compliance and audit through Entra's sign-in and audit logs, so each agent action carries enough recorded metadata to explain or attribute it after the fact.","evidence":[{"type":"doc","title":"What is Microsoft Entra Agent ID?","url":"https://learn.microsoft.com/en-us/entra/agent-id/what-is-microsoft-entra-agent-id","quote":"All agent authentication and activity is logged for compliance and audit.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"Agent identity blueprint","summary":"A template defining an agent's configuration and governance model, with parent-child relationships, from which individual agent identities are derived so consistent policy applies across large fleets.","maps_to_pattern":"ephemeral-agent-identity","url":"https://learn.microsoft.com/en-us/entra/agent-id/agent-blueprint"},{"name":"Agent OAuth flows","summary":"The on-behalf-of (delegated), client-credentials (autonomous app-only), and agent-user-account patterns by which an agent obtains an access token scoped to a single target resource.","maps_to_pattern":"delegated-agent-authorization","url":"https://learn.microsoft.com/en-us/entra/agent-id/agent-oauth-protocols"},{"name":"Conditional Access for agents","summary":"The if-then policy engine evaluated at token issuance that allows, blocks, or limits an agent's access to a resource based on subject, audience, device, location, and risk signals.","maps_to_pattern":"policy-as-code-gate","url":"https://learn.microsoft.com/en-us/entra/identity/conditional-access/agent-id"},{"name":"Agent's user account","summary":"A directory user account linked to an agent identity so a digital worker can hold a mailbox, join groups, and participate in collaborative workflows as a team member.","url":"https://learn.microsoft.com/en-us/entra/agent-id/agent-users"}],"references":[{"type":"doc","title":"Microsoft Entra Agent ID — documentation home","url":"https://learn.microsoft.com/en-us/entra/agent-id/","accessed":"2026-06-17"},{"type":"doc","title":"What is Microsoft Entra Agent ID?","url":"https://learn.microsoft.com/en-us/entra/agent-id/what-is-microsoft-entra-agent-id","quote":"Microsoft Entra Agent ID is an identity and security framework that extends Microsoft Entra capabilities to AI agents.","accessed":"2026-06-17"}],"alternatives":[{"composition":"open-policy-agent","relation":"complements","note":"Open Policy Agent is the externalized authorization engine that complements Entra's identity plane: Entra establishes who an agent is, OPA evaluates whether a given action is allowed."},{"composition":"cerbos","relation":"complements","note":"Cerbos provides fine-grained, externalized authorization decisions over what an authenticated agent identity may do, complementing Entra's identity issuance and Conditional Access."},{"composition":"arcade","relation":"competes-with","note":"Arcade also brokers scoped, delegated authorization for agents to access third-party tools and APIs, overlapping with Entra's delegated-access role."}],"verification_status":"needs-verification","status_in_practice":"emerging","last_analyzed":"2026-06-17","last_updated":"2026-06-19","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Microsoft Entra Agent ID\"]\n  fw --> p1[\"Delegated Agent Authorization<br/>(first-class)\"]\n  fw --> p2[\"Policy-as-Code Gate<br/>(first-class)\"]\n  fw --> p3[\"Ephemeral Agent Identity<br/>(supported)\"]\n  fw --> p4[\"Provenance Ledger<br/>(supported)\"]","tags":["enterprise-platform","safety-control","governance-observability","closed-application"],"anti_patterns_avoided":[{"pattern":"agent-identity-sprawl","note":"Gives each agent a governed identity with a responsible sponsor and a lifecycle, so agent identities do not proliferate unowned.","evidence":[{"type":"doc","title":"Microsoft Entra Agent ID governance","url":"https://learn.microsoft.com/en-us/entra/id-governance/agent-id-governance-overview","quote":"you can govern and manage the identity and access lifecycle of agents, ensuring the agents have a responsible person providing oversight throughout the agent lifecycle and agent's access does not persist longer than it is needed","accessed":"2026-06-19"}]},{"pattern":"agent-sprawl","note":"Inventories agents under identity governance with sponsor oversight and automatic access expiry, countering ungoverned agent sprawl.","evidence":[{"type":"doc","title":"Microsoft Entra Agent ID governance","url":"https://learn.microsoft.com/en-us/entra/id-governance/agent-id-governance-overview","quote":"If the sponsor takes no action, the access package assignment automatically expires on its end date, and the agent identity loses access to the target resources.","accessed":"2026-06-19"}]},{"pattern":"delegation-not-impersonation","note":"Agents receive delegated permissions to act on behalf of a user, and audit logs tag every event with an agentType so the agent is never mistaken for the user it acts for.","evidence":[{"type":"doc","title":"Microsoft Entra Agent ID — key concepts","url":"https://learn.microsoft.com/en-us/entra/agent-id/key-concepts","quote":"These agents are granted Microsoft Entra delegated permissions that allow them to act on behalf of users.","accessed":"2026-06-19"}]}]},{"id":"microsoft-presidio","name":"Microsoft Presidio","kind":"framework","category":"enterprise-platform","vendor":"Microsoft","language":"Python","license":"MIT","status":"active","first_released":"2019-08-01","url":"https://microsoft.github.io/presidio/","repo":"https://github.com/microsoft/presidio","docs_url":"https://microsoft.github.io/presidio/","intent":"Microsoft Presidio detects personally identifiable information in text and images and then anonymizes it through configurable operators, so sensitive data can be de-identified before or after it is handled by a model.","description":"Presidio is an open-source PII de-identification SDK from Microsoft. Its Analyzer identifies private entities using named-entity recognition, regular expressions, rule-based logic, and checksums across multiple languages, and its Anonymizer replaces, redacts, masks, hashes, or encrypts the detected entities through built-in operators. A separate Image Redactor module detects and redacts PII text in standard and DICOM images. The recognizers and operators are pluggable and customizable to specific business needs.","primary_use_cases":["detecting PII entities in text","anonymizing, masking, hashing, or encrypting detected PII","redacting PII text in images and DICOM scans","de-identifying data before or after model use"],"agent_loop_shape":"Presidio is a library invoked around a model rather than a runtime loop. Text passes through the Analyzer, which returns the spans and types of detected PII; those spans are handed to the Anonymizer, which applies a chosen operator (replace, redact, mask, hash, encrypt) to produce de-identified output. Images are routed through the Image Redactor, which detects PII text in pixels and redacts it. The custom recognizers and operators are configured by the integrating application.","build_surface":"full-code","members":[{"pattern":"pii-redaction","role":"core","note":"An Analyzer detects PII entities via NER/regex/rule-based recognizers and an Anonymizer redacts, masks, hashes, or encrypts them in text and images before or after model use.","evidence":[{"type":"repo","title":"microsoft/presidio — Context aware, pluggable PII de-identification","url":"https://github.com/microsoft/presidio","quote":"Context aware, pluggable and customizable PII de-identification service for text and images.","accessed":"2026-06-17"},{"type":"repo","title":"microsoft/presidio — README capabilities","url":"https://github.com/microsoft/presidio","quote":"It provides fast identification and anonymization modules for private entities in text such as credit card numbers, names, locations, social security numbers, bitcoin wallets, US phone numbers, financial data and more.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"multimodal-guardrails","role":"first-class","note":"Beyond text, the Image Redactor module detects and redacts PII text embedded as pixels in standard images and DICOM medical scans, so the same de-identification guardrail extends to the image modality.","evidence":[{"type":"doc","title":"Presidio Image Redactor","url":"https://microsoft.github.io/presidio/image-redactor/","quote":"The Presidio Image Redactor is a Python based module for detecting and redacting PII text entities in images.","accessed":"2026-06-17"},{"type":"doc","title":"Presidio Image Redactor — DICOM","url":"https://microsoft.github.io/presidio/image-redactor/","quote":"This module may also be used on medical DICOM images. The `DicomImageRedactorEngine` class may be used to redact text PII present as pixels in DICOM images.","accessed":"2026-06-17"}],"evidence_status":"full"}],"key_concepts":[{"name":"Analyzer","summary":"The detection engine that scans text and returns the spans and entity types of PII using NER models, regular expressions, rule-based logic, and checksum validation across multiple languages.","maps_to_pattern":"pii-redaction","url":"https://microsoft.github.io/presidio/analyzer/"},{"name":"Anonymizer","summary":"The de-identification engine that applies an operator — replace, redact, mask, hash, or encrypt — to the entity spans the Analyzer found, producing anonymized output.","maps_to_pattern":"pii-redaction","url":"https://microsoft.github.io/presidio/anonymizer/"},{"name":"Recognizers","summary":"Pluggable, customizable detectors for specific entity types that the Analyzer composes, letting teams add business-specific PII patterns beyond the built-in set.","url":"https://microsoft.github.io/presidio/analyzer/adding_recognizers/"},{"name":"Image Redactor","summary":"A separate module that uses OCR to find PII text rendered as pixels in standard and DICOM images and redacts it, extending de-identification to the image modality.","maps_to_pattern":"multimodal-guardrails","url":"https://microsoft.github.io/presidio/image-redactor/"}],"references":[{"type":"doc","title":"Microsoft Presidio — documentation home","url":"https://microsoft.github.io/presidio/","accessed":"2026-06-17"},{"type":"repo","title":"microsoft/presidio — Context aware, pluggable PII de-identification","url":"https://github.com/microsoft/presidio","quote":"Context aware, pluggable and customizable PII de-identification service for text and images.","accessed":"2026-06-17"}],"alternatives":[{"composition":"bedrock-guardrails","relation":"competes-with","note":"Amazon Bedrock Guardrails' sensitive-information filters detect and mask/redact PII in model inputs and outputs, overlapping with Presidio's de-identification role but built into the Bedrock runtime."},{"composition":"nemo-guardrails","relation":"complements","note":"NeMo Guardrails can call out to a PII de-identification step; Presidio is a common backing implementation for that detect-and-redact action."}],"verification_status":"partial","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Microsoft Presidio\"]\n  fw --> p1[\"PII Redaction<br/>(core)\"]\n  fw --> p2[\"Multimodal Guardrails<br/>(first-class)\"]","tags":["enterprise-platform","safety-control","full-code"]},{"id":"azure-raft-finetuning","name":"Azure AI Foundry RAFT fine-tuning recipe","kind":"framework","category":"enterprise-platform","vendor":"Microsoft Azure","language":"Python","license":"MIT","status":"active","first_released":"2024-05-01","url":"https://github.com/Azure-Samples/azureai-foundry-finetuning-raft","repo":"https://github.com/Azure-Samples/azureai-foundry-finetuning-raft","docs_url":"https://github.com/Azure-Samples/azureai-foundry-finetuning-raft","intent":"This Azure recipe generates a synthetic RAFT training set with a teacher model and fine-tunes a student model on Azure AI Foundry to improve domain-specific RAG.","description":"This is an official Azure-Samples recipe that applies UC Berkeley's RAFT technique on Azure AI Foundry. A teacher model such as GPT-4o or Llama 3.1 405B generates a synthetic dataset using the Gorilla project's RAFT method, with question-document-answer triplets that include oracle and distractor documents. The dataset fine-tunes a smaller student model (such as GPT-4o-mini or Llama 3.1 8B) so it learns to ignore distractor documents in domain-specific RAG. The recipe then deploys the fine-tuned model and evaluates it against a baseline.","primary_use_cases":["domain-specific RAG fine-tuning on Azure AI Foundry","synthetic RAFT training-data generation","training models to ignore distractor documents","evaluating a fine-tuned model against a baseline"],"agent_loop_shape":"A teacher model deployed on Azure AI generates a synthetic RAFT dataset of question-document-answer triplets, each combining an oracle document with distractor documents. That dataset fine-tunes a student model on Azure AI Foundry so it learns to attend to the oracle and ignore the distractors. The fine-tuned model is then deployed and its performance evaluated against a baseline model.","build_surface":"full-code","members":[{"pattern":"raft","role":"supported","note":"An official Azure recipe that generates synthetic Q-doc-answer triplets with oracle and distractor documents and fine-tunes a model on Azure AI Foundry to ignore the distractors in domain-specific RAG.","evidence":[{"type":"repo","title":"Azure-Samples/azureai-foundry-finetuning-raft","url":"https://github.com/Azure-Samples/azureai-foundry-finetuning-raft","quote":"This repository is a recipe that will walk you through improving RAG system precision using UC Berkeley's RAFT technique on Azure AI Foundry.","accessed":"2026-06-17"},{"type":"blog","title":"RAFT: Sailing Llama towards better domain-specific RAG (Meta / Microsoft)","url":"https://ai.meta.com/blog/raft-llama-retrieval-augmented-generation-supervised-fine-tuning-microsoft/","quote":"These 'distractor' documents may be pulled into the model's context even when they are not good sources for a well-reasoned answer.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"eval-harness","role":"supported","note":"After fine-tuning, the recipe deploys the fine-tuned model and evaluates its performance against a baseline model, so the training change is measured against a held-out comparison rather than shipped unverified.","evidence":[{"type":"repo","title":"Azure-Samples/azureai-foundry-finetuning-raft README","url":"https://github.com/Azure-Samples/azureai-foundry-finetuning-raft","quote":"Finally, we will deploy the fine-tuned model and evaluate its performance compared to a baseline model.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"Teacher-student distillation","summary":"A large teacher model (GPT-4o or Llama 3.1 405B) deployed on Azure AI generates the synthetic training set, which is then used to fine-tune a much smaller student model (GPT-4o-mini or Llama 3.1 8B) so the cheaper student inherits the domain RAG behaviour.","url":"https://github.com/Azure-Samples/azureai-foundry-finetuning-raft"},{"name":"Oracle and distractor documents","summary":"Each generated training triplet pairs an oracle document that actually answers the question with one or more distractor documents that do not, teaching the student to attend to the oracle and ignore the distractors at inference time.","maps_to_pattern":"raft"},{"name":"Gorilla RAFT method","summary":"The synthetic-data generation follows UC Berkeley's Gorilla project RAFT (Retrieval Augmented Fine-Tuning) recipe, a published method for fine-tuning a model to better use retrieved context.","url":"https://github.com/ShishirPatil/gorilla"}],"references":[{"type":"repo","title":"Azure-Samples/azureai-foundry-finetuning-raft","url":"https://github.com/Azure-Samples/azureai-foundry-finetuning-raft","quote":"This recipe uses either OpenAI GPT-4o or Meta Llama 3.1 405B as a teacher model deployed on Azure AI to generate a synthetic dataset using UC Berkeley's Gorilla project RAFT method.","accessed":"2026-06-17"},{"type":"blog","title":"RAFT: Sailing Llama towards better domain-specific RAG (Meta AI blog)","url":"https://ai.meta.com/blog/raft-llama-retrieval-augmented-generation-supervised-fine-tuning-microsoft/","accessed":"2026-06-17"},{"type":"paper","title":"RAFT: Adapting Language Model to Domain Specific RAG (arXiv:2403.10131)","url":"https://arxiv.org/abs/2403.10131","accessed":"2026-06-17"}],"alternatives":[{"composition":"azure-ai-foundry-agent-service","relation":"same-vendor","note":"Both are Azure AI Foundry offerings; the Agent Service runs the managed agent loop while this recipe is a fine-tuning workflow that produces a model deployed on the same platform."},{"composition":"sagemaker","relation":"competes-with","note":"Amazon SageMaker AI is the comparable hyperscaler training and fine-tuning platform for building domain-specific models outside Azure."}],"verification_status":"partial","status_in_practice":"emerging","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Azure AI Foundry RAFT fine-tuning recipe\"]\n  fw --> p1[\"RAFT<br/>(supported)\"]\n  fw --> p2[\"Eval Harness<br/>(supported)\"]","tags":["enterprise-platform","retrieval","governance-observability","full-code"]},{"id":"deepeval","name":"DeepEval","kind":"framework","category":"enterprise-platform","vendor":"Confident AI","language":"Python","license":"Apache-2.0","status":"active","first_released":"2023","url":"https://www.deepeval.com","repo":"https://github.com/confident-ai/deepeval","docs_url":"https://deepeval.com/docs/getting-started","intent":"DeepEval is an open-source Python framework that unit-tests the outputs of an LLM application against metric-based test cases, integrating with pytest.","description":"DeepEval defines test cases over an LLM application's inputs and outputs and runs metrics against them, returning pass or fail like a software test. Most of its metrics, including the G-Eval custom-criteria metric, are LLM-as-a-judge metrics that call a configurable model to score the output. It plugs into pytest so generation outputs are asserted in the same workflow as code tests.","primary_use_cases":["metric-based testing of LLM application outputs","custom-criteria evaluation with LLM judges","regression testing of prompts and model versions in CI"],"agent_loop_shape":"DeepEval does not run an agent loop of its own. It is invoked as a test harness: a developer constructs test cases pairing an input, the actual LLM output, and optional expected context, attaches one or more metrics, and runs them under pytest. Each metric scores the case, and most metrics call a judge model to produce that score, after which the case passes or fails against a threshold.","build_surface":"full-code","members":[{"pattern":"agent-evaluator","role":"core","note":"DeepEval is a dedicated testing harness whose sole job is running metric-based test cases against another LLM app's outputs, integrating with pytest so agent outputs are unit-tested like code; note its scope overlaps eval-harness (offline batch) rather than a true agent-evaluating-agent.","evidence":[{"type":"doc","title":"DeepEval — The Open-Source LLM Evaluation Framework","url":"https://deepeval.com/docs/getting-started","quote":"DeepEval is an open-source evaluation framework for LLMs... it lets you \"unit test\" LLM outputs in a similar way to Pytest.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"llm-as-judge","role":"core","note":"Most DeepEval metrics, including the G-Eval custom-criteria metric, are LLM-as-a-judge metrics that call a configurable judge model to score the output against rubric criteria rather than against an exact-match reference.","evidence":[{"type":"repo","title":"confident-ai/deepeval — The LLM Evaluation Framework","url":"https://github.com/confident-ai/deepeval","quote":"G-Eval — a research-backed LLM-as-a-judge metric for evaluating on any custom criteria with human-like accuracy","accessed":"2026-06-17"},{"type":"doc","title":"DeepEval — Getting Started","url":"https://deepeval.com/docs/getting-started","quote":"since almost all deepeval metrics including GEval are LLM-as-a-Judge metrics, you'll need to set your OPENAI_API_KEY as an env variable.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"eval-harness","role":"first-class","note":"DeepEval evaluates an LLM application end-to-end against a dataset of test cases (goldens) with deepeval test run, and its Synthesizer can build that held-out dataset from documents, so successive app versions are measured against a fixed batch.","evidence":[{"type":"doc","title":"DeepEval — Getting Started","url":"https://deepeval.com/docs/getting-started","quote":"Then, run `deepeval test run` from the root directory of your project to evaluate your LLM app **end-to-end**","accessed":"2026-06-17"},{"type":"doc","title":"DeepEval — Generate Synthetic Data (Synthesizer)","url":"https://deepeval.com/docs/synthesizer-introduction","quote":"The `Synthesizer` uses an LLM to first generate a series of inputs/scenarios, before evolving them to become more complex and realistic.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"eval-as-contract","role":"first-class","note":"DeepEval evals are written as pytest tests and run with deepeval test run in the pipeline; a failing metric fails the build, so a release is gated on the eval suite passing.","evidence":[{"type":"doc","title":"DeepEval — Evaluation in CI/CD (FAQ)","url":"https://deepeval.com/docs/evaluation-unit-testing-in-ci-cd","quote":"Failing metrics fail the build, so regressions get caught before they ship.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"dual-evaluation-offline-online","role":"supported","note":"Beyond the offline test-run track, DeepEval can run online evals that monitor production traces, spans, and threads, so the same metrics gate before deploy and observe live traffic after; the online track depends on the Confident AI platform for tracing.","evidence":[{"type":"doc","title":"DeepEval — Getting Started","url":"https://deepeval.com/docs/getting-started","quote":"When you do LLM tracing using `deepeval`, you can automatically run online evals to monitor **traces, spans, and threads (conversations) in production**.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"Metric","summary":"A scoring rule (such as AnswerRelevancy, Faithfulness, Hallucination, or the custom G-Eval) applied to a test case; most metrics are LLM-as-a-judge and return a score against a pass threshold.","maps_to_pattern":"llm-as-judge","url":"https://deepeval.com/docs/metrics-introduction"},{"name":"G-Eval","summary":"A research-backed metric where the user states free-form custom criteria and an LLM judge scores the output against them, used when no exact-match or built-in metric fits.","maps_to_pattern":"llm-as-judge","url":"https://deepeval.com/docs/metrics-llm-evals"},{"name":"Synthesizer","summary":"A generator that produces synthetic test cases (goldens) from documents, prepared contexts, or from scratch, then evolves them to be more complex, used to bootstrap an evaluation dataset when none exists.","maps_to_pattern":"eval-harness","url":"https://deepeval.com/docs/synthesizer-introduction"},{"name":"Faithfulness metric","summary":"A RAG metric that uses an LLM judge to check whether the generated output factually aligns with the retrieval context, flagging unsupported claims (hallucination) against retrieved evidence.","url":"https://deepeval.com/docs/metrics-faithfulness"}],"references":[{"type":"doc","title":"DeepEval — The Open-Source LLM Evaluation Framework","url":"https://deepeval.com/docs/getting-started","accessed":"2026-06-17"},{"type":"repo","title":"confident-ai/deepeval","url":"https://github.com/confident-ai/deepeval","quote":"G-Eval — a research-backed LLM-as-a-judge metric for evaluating on any custom criteria with human-like accuracy","accessed":"2026-06-17"},{"type":"doc","title":"DeepEval — Faithfulness metric","url":"https://deepeval.com/docs/metrics-faithfulness","quote":"The faithfulness metric uses LLM-as-a-judge to measure the quality of your RAG pipeline's generator by evaluating whether the `actual_output` factually aligns with the contents of your `retrieval_context`.","accessed":"2026-06-17"}],"alternatives":[{"composition":"promptfoo","relation":"competes-with","note":"Promptfoo is the TypeScript/CLI counterpart for assertion-based LLM evaluation and red-teaming; DeepEval is the pytest-native Python equivalent."},{"composition":"ragas","relation":"competes-with","note":"Ragas is another Python RAG-evaluation library; both expose faithfulness and answer-relevancy style metrics, with DeepEval framing them as pytest assertions."},{"composition":"langsmith","relation":"competes-with","note":"LangSmith covers the same offline-plus-online LLM evaluation space as a hosted platform; DeepEval is open-source and code-first, with online evals tied to Confident AI."}],"verification_status":"partial","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"DeepEval\"]\n  fw --> p1[\"Agent Evaluator<br/>(core)\"]\n  fw --> p2[\"LLM-as-Judge<br/>(core)\"]\n  fw --> p3[\"Eval Harness<br/>(first-class)\"]\n  fw --> p4[\"Eval as Contract<br/>(first-class)\"]\n  fw --> p5[\"Dual Evaluation (Offline + Online)<br/>(supported)\"]","tags":["enterprise-platform","governance-observability","full-code"]},{"id":"k2view-genai-data-fusion","name":"K2view GenAI Data Fusion","kind":"framework","category":"enterprise-platform","vendor":"K2view","language":"N/A (managed platform)","license":"proprietary","status":"active","first_released":"2023-01-01","url":"https://www.k2view.com/","docs_url":"https://www.k2view.com/what-is-table-augmented-generation","intent":"GenAI Data Fusion answers natural-language questions by querying internal databases and applications through per-entity Micro-Databases and grounding the LLM answer in the structured data returned.","description":"K2view GenAI Data Fusion is an enterprise platform that grounds LLM responses in real-time business data using Table-Augmented Generation. A retrieval step queries internal databases and business applications for structured data, and K2view organises the data for each business entity in its own Micro-Database, so a query targets only the relevant entity. The retrieved entity data, metadata, and context augment the prompt, and K2view states this prevents hallucinations and improves reliability. It is a proprietary commercial platform.","primary_use_cases":["grounding LLM answers in enterprise databases","natural-language access to operational business data","customer-360 conversational data retrieval","hallucination reduction for data-backed GenAI apps"],"agent_loop_shape":"A natural-language question is turned into a database query, executed against internal databases and business applications through K2view's Micro-Database, which holds the data for each business entity. The retrieved structured data, together with the entity's metadata and context, augments the original prompt, and the LLM generates an answer grounded in that data rather than from parametric memory.","build_surface":"closed-application","members":[{"pattern":"table-augmented-generation","role":"core","note":"GenAI Data Fusion synthesises a query, executes it against internal databases and business applications via its Micro-Database to fetch structured data, then has the LLM generate the grounded answer.","evidence":[{"type":"doc","title":"What is Table-Augmented Generation (TAG)? | K2view","url":"https://www.k2view.com/what-is-table-augmented-generation","quote":"The retrieval model queries your internal databases and business applications for structured data.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"semantic-layer-query-guardrail","role":"supported","note":"Natural-language data questions are routed through K2view's curated Micro-Database layer, which organises data per business entity and queries only the relevant entity, so the LLM accesses governed structured data rather than free-authoring access to raw stores; the augmented prompt grounds the answer and prevents hallucination.","evidence":[{"type":"doc","title":"What is Table-Augmented Generation (TAG)? | K2view","url":"https://www.k2view.com/what-is-table-augmented-generation","quote":"K2view organizes the data for each business entity in its own high-performance Micro-Database – always ready for real-time access by TAG workflows.","accessed":"2026-06-17"},{"type":"doc","title":"What is Table-Augmented Generation (TAG)? | K2view","url":"https://www.k2view.com/what-is-table-augmented-generation","quote":"By converting real-time business data into intelligent, context-sensitive, and compliant prompts, TAG prevents hallucinations and enhances the overall reliability of GenAI apps.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"pii-redaction","role":"supported","note":"The Micro-Database layer applies built-in data access controls and dynamic data masking, masking PII or other sensitive data as it is retrieved into the prompt, so sensitive values do not flow into the LLM context for users without authorization.","evidence":[{"type":"doc","title":"What is Table-Augmented Generation (TAG)? | K2view","url":"https://www.k2view.com/what-is-table-augmented-generation","quote":"Dynamically mask PII (Personally Identifiable Information) or other sensitive data.","accessed":"2026-06-17"},{"type":"doc","title":"What is Table-Augmented Generation (TAG)? | K2view","url":"https://www.k2view.com/what-is-table-augmented-generation","quote":"The TAG framework must implement role-based access controls to prevent users from accessing information beyond their authorized scope.","accessed":"2026-06-17"}],"evidence_status":"full"}],"key_concepts":[{"name":"Table-Augmented Generation (TAG)","summary":"K2view's named technique for grounding an LLM answer in structured data: a retrieval model queries internal databases for the relevant rows, and that structured data augments the prompt instead of vector-embedded text chunks.","maps_to_pattern":"table-augmented-generation","url":"https://www.k2view.com/what-is-table-augmented-generation"},{"name":"Micro-Database","summary":"A per-entity store holding all of one business entity's data (for example one customer) in its own compressed, low-latency database, so a query for that entity reads only its Micro-Database and forms a 360-degree view.","maps_to_pattern":"semantic-layer-query-guardrail","url":"https://www.k2view.com/what-is-table-augmented-generation"},{"name":"Dynamic data masking","summary":"Built-in access controls that mask PII or other sensitive fields at retrieval time according to the requesting user's authorization, keeping unauthorized values out of the prompt.","maps_to_pattern":"pii-redaction","url":"https://www.k2view.com/what-is-table-augmented-generation"}],"references":[{"type":"doc","title":"What is Table-Augmented Generation (TAG)? | K2view","url":"https://www.k2view.com/what-is-table-augmented-generation","quote":"By converting real-time business data into intelligent, context-sensitive, and compliant prompts, TAG prevents hallucinations and enhances the overall reliability of GenAI apps.","accessed":"2026-06-17"},{"type":"doc","title":"K2view GenAI Data Fusion","url":"https://www.k2view.com/","accessed":"2026-06-17"}],"alternatives":[{"composition":"cube-semantic-layer","relation":"competes-with","note":"Cube exposes a curated semantic layer of vetted metrics and dimensions for natural-language querying; K2view instead grounds answers in per-entity Micro-Databases of operational data."},{"composition":"microsoft-presidio","relation":"complements","note":"Presidio is a standalone PII detection-and-redaction engine; K2view bundles equivalent dynamic masking inside its data layer rather than as a separate service."}],"verification_status":"partial","status_in_practice":"emerging","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"K2view GenAI Data Fusion\"]\n  fw --> p1[\"Table-Augmented Generation<br/>(core)\"]\n  fw --> p2[\"Semantic-Layer Query Guardrail<br/>(supported)\"]\n  fw --> p3[\"PII Redaction<br/>(supported)\"]","tags":["enterprise-platform","retrieval","tool-use-environment","safety-control","closed-application"]},{"id":"langsmith","name":"LangSmith","kind":"framework","category":"enterprise-platform","vendor":"LangChain Inc.","language":"TypeScript","license":"proprietary","status":"active","first_released":"2023","url":"https://www.langchain.com/langsmith","docs_url":"https://docs.langchain.com/langsmith/evaluation","intent":"LangSmith is a hosted platform for tracing, evaluating, and monitoring LLM applications across development and production.","description":"LangSmith captures traces of LLM and agent runs and evaluates them with code rules, human review, and LLM-as-judge evaluators. It runs offline experiments to gate during development and online evaluators that run automatically on production traces for monitoring and alerting. Failing production traces can be added back to datasets to drive targeted offline experiments.","primary_use_cases":["tracing LLM and agent runs","offline evaluation experiments before deploy","online evaluation and monitoring of production traffic"],"agent_loop_shape":"LangSmith does not run the application's agent loop; it observes and evaluates it. Application runs send traces to LangSmith, where evaluators score them. In the offline track, evaluators run over curated datasets as experiments. In the online track, evaluators run automatically on incoming production runs or threads to provide monitoring and alerting, and flagged runs feed back into datasets.","build_surface":"closed-application","members":[{"pattern":"dual-evaluation-offline-online","role":"first-class","note":"LangSmith provides offline experiments to gate during development and online evaluators that run automatically on production traces for real-time monitoring, feeding failing production traces back into offline experiments.","evidence":[{"type":"doc","title":"LangSmith Evaluation — Docs by LangChain","url":"https://docs.langchain.com/langsmith/evaluation","quote":"Evaluators run automatically on runs or threads, providing real-time monitoring, anomaly detection, and alerting.","accessed":"2026-06-17"},{"type":"doc","title":"LangSmith Evaluation — Docs by LangChain","url":"https://docs.langchain.com/langsmith/evaluation","quote":"Add failing production traces to your dataset, create targeted evaluators, validate fixes with offline experiments, and redeploy.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"llm-as-judge","role":"first-class","note":"LangSmith offers LLM-as-judge evaluators that use an LLM to score application outputs against criteria, available alongside code-rule and human-review evaluators in both the offline and online tracks.","evidence":[{"type":"doc","title":"Evaluation concepts — LangSmith Docs","url":"https://docs.langchain.com/langsmith/evaluation-concepts","quote":"LLM-as-judge evaluators use LLMs to score application outputs.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"decision-log","role":"first-class","note":"Tracing records each application run as a tree of spans capturing the LLM calls, retrieval calls, and intermediate steps, so a past execution can be inspected after the fact to explain what the application did and why.","evidence":[{"type":"doc","title":"Observability concepts — LangSmith Docs","url":"https://docs.langchain.com/langsmith/observability-concepts","quote":"A *run* is a span representing a single unit of work within your LLM application: a call to an LLM, a prompt formatting step, a retrieval call, or any other discrete operation.","accessed":"2026-06-17"},{"type":"doc","title":"Observability concepts — LangSmith Docs","url":"https://docs.langchain.com/langsmith/observability-concepts","quote":"A *trace* is a collection of runs for a single operation.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"prompt-versioning","role":"supported","note":"Prompts stored in LangSmith are versioned: every saved update creates a new immutable commit with a unique hash, and a specific commit or tag can be pulled into application code, so prompts are deployed and rolled back like versioned artefacts.","evidence":[{"type":"doc","title":"Prompt engineering concepts — LangSmith Docs","url":"https://docs.langchain.com/langsmith/prompt-engineering-concepts","quote":"Every saved update to a prompt creates a new commit with a unique commit hash.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"Trace and run","summary":"A run is a single span of work (an LLM call, a retrieval, a formatting step) and a trace is the collection of runs for one end-to-end operation; together they form the observability record LangSmith stores per request.","maps_to_pattern":"decision-log","url":"https://docs.langchain.com/langsmith/observability-concepts"},{"name":"Experiment","summary":"An offline evaluation run that applies a set of evaluators over a curated dataset to score an application version before deploy, the gate side of LangSmith's two evaluation tracks.","maps_to_pattern":"dual-evaluation-offline-online","url":"https://docs.langchain.com/langsmith/evaluation-concepts"},{"name":"Online evaluator","summary":"An evaluator configured to run automatically on incoming production runs or threads (optionally sampled) to provide live monitoring, anomaly detection, and alerting on production traffic.","maps_to_pattern":"dual-evaluation-offline-online","url":"https://docs.langchain.com/langsmith/evaluation"},{"name":"Annotation queue","summary":"A human-review surface where runs are queued for people to label or score, complementing the automated code-rule and LLM-as-judge evaluators.","url":"https://docs.langchain.com/langsmith/evaluation-concepts"}],"references":[{"type":"doc","title":"LangSmith Evaluation — Docs by LangChain","url":"https://docs.langchain.com/langsmith/evaluation","accessed":"2026-06-17"},{"type":"doc","title":"LangSmith Observability — Docs by LangChain","url":"https://docs.langchain.com/langsmith/observability","quote":"LangSmith Observability provides full visibility into your LLM application: from individual traces to production-wide performance metrics.","accessed":"2026-06-17"},{"type":"doc","title":"LangSmith product page","url":"https://www.langchain.com/langsmith","accessed":"2026-06-17"}],"alternatives":[{"composition":"langfuse","relation":"competes-with","note":"Langfuse is the open-source LLM tracing and evaluation platform covering the same observability-plus-eval space; LangSmith is the proprietary LangChain-native counterpart."},{"composition":"braintrust","relation":"competes-with","note":"Braintrust targets the same offline experiments plus production logging workflow for LLM applications."},{"composition":"langgraph","relation":"same-vendor","note":"Both are LangChain Inc. products; LangGraph builds the agent and LangSmith traces and evaluates the runs it produces."}],"verification_status":"needs-verification","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-19","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"LangSmith\"]\n  fw --> p1[\"Dual Evaluation (Offline + Online)<br/>(first-class)\"]\n  fw --> p2[\"LLM-as-Judge<br/>(first-class)\"]\n  fw --> p3[\"Decision Log<br/>(first-class)\"]\n  fw --> p4[\"Prompt Versioning<br/>(supported)\"]","tags":["enterprise-platform","governance-observability","closed-application"],"anti_patterns_avoided":[{"pattern":"demo-to-production-cliff","note":"Evaluations on curated datasets gate agent versions and fail the pipeline when scores drop, catching regressions before production rather than at the demo-to-prod cliff.","evidence":[{"type":"doc","title":"LangSmith — evaluation","url":"https://www.langchain.com/langsmith/evaluation","quote":"Run evaluations on curated datasets during development to compare agent versions, benchmark performance, and catch regressions before users do.","accessed":"2026-06-19"}]},{"pattern":"demo-production-cliff-multiagent","note":"Multi-agent trajectory evaluation and latency benchmarking surface multi-agent regressions before production load exposes them.","evidence":[{"type":"doc","title":"LangSmith — evaluation","url":"https://www.langchain.com/langsmith/evaluation","quote":"fail pipelines automatically when scores drop, bringing the same rigor as deterministic unit tests to your AI development process.","accessed":"2026-06-19"}]}]},{"id":"open-policy-agent","name":"Open Policy Agent (OPA)","kind":"framework","category":"enterprise-platform","vendor":"The OPA Authors / CNCF","language":"Go","license":"Apache-2.0","status":"active","first_released":"2016","url":"https://www.openpolicyagent.org","repo":"https://github.com/open-policy-agent/opa","docs_url":"https://www.openpolicyagent.org/docs/","intent":"Open Policy Agent is a general-purpose policy engine that evaluates structured input against machine-readable Rego policies to return authorization decisions, decoupling policy decisions from enforcement.","description":"OPA is a CNCF-graduated policy engine in which software queries OPA with structured data and OPA evaluates that input against Rego policies to produce a decision such as allow or deny. It decouples policy decision-making from policy enforcement, so policy authorship lives outside the calling service. OPA records each authorization decision in decision logs that bundle the input data, the policy decision, and metadata.","primary_use_cases":["service and API authorization decisions","Kubernetes admission control and infrastructure-as-code validation","decision logging for policy audit"],"agent_loop_shape":"OPA has no agent loop. It runs as a library or sidecar daemon that answers policy queries: a calling service supplies structured JSON input, OPA evaluates the relevant Rego policies against it, and returns the decision result. Each decision can be emitted to a decision log capturing the input, the result, and metadata for later audit.","build_surface":"full-code","members":[{"pattern":"agent-decision-token","role":"supported","note":"OPA decision logs mint a self-contained record at the moment each authorization decision executes, bundling the policy evaluated, the exact input data read, the decision result, and identifying metadata — a structured, exportable decision token per action; authorizing identity rides in input/metadata rather than as a first-class field, hence medium confidence.","evidence":[{"type":"doc","title":"Monitoring OPA — Decision Logs","url":"https://www.openpolicyagent.org/docs/monitoring","quote":"OPA can periodically report decision logs to remote HTTP servers, using custom plugins, or to the console output... The decision log includes the input data, the policy decision, and other metadata.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"policy-as-code-gate","role":"core","note":"OPA evaluates a proposed action against externally-managed machine-readable Rego policies and returns an allow or deny decision, decoupling policy decision-making from enforcement so compliance authorship lives outside the calling service; note OPA returns the decision and the calling service enforces it, so OPA is the decision engine rather than the dispatch gate itself.","evidence":[{"type":"doc","title":"Introduction — Open Policy Agent","url":"https://www.openpolicyagent.org/docs/","quote":"OPA decouples policy decision-making from policy enforcement.","accessed":"2026-06-17"},{"type":"doc","title":"Introduction — Open Policy Agent","url":"https://www.openpolicyagent.org/docs/","quote":"When your software needs to make policy decisions it queries OPA and supplies structured data (e.g., JSON) as input.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"decision-log","role":"first-class","note":"Every policy decision can be emitted as a structured event recording the policy that was queried, the input to the query, and bundle metadata, shippable to remote servers, so the reason for each authorization outcome is persisted for post-hoc audit and offline debugging.","evidence":[{"type":"doc","title":"Decision Logs — Open Policy Agent","url":"https://www.openpolicyagent.org/docs/management-decision-logs","quote":"Each event includes the policy that was queried, the input to the query, bundle metadata, and other information that enables auditing and offline debugging of policy decisions.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"Rego","summary":"OPA's high-level declarative policy language, in which authorization rules are written and managed outside the calling service and evaluated against the supplied JSON input.","maps_to_pattern":"policy-as-code-gate","url":"https://www.openpolicyagent.org/docs/policy-language"},{"name":"Policy decision","summary":"The result OPA returns for a query — any JSON value, commonly an allow or deny boolean — leaving enforcement of that result to the service that asked, which is what decouples decision from enforcement.","url":"https://www.openpolicyagent.org/docs/"},{"name":"Decision log","summary":"A stream of structured events, one per evaluated decision, capturing the policy queried, the input, the result, and metadata, reportable to remote HTTP servers for centralized audit.","maps_to_pattern":"decision-log","url":"https://www.openpolicyagent.org/docs/management-decision-logs"}],"references":[{"type":"doc","title":"Introduction — Open Policy Agent","url":"https://www.openpolicyagent.org/docs/","quote":"The Open Policy Agent (OPA, pronounced \"oh-pa\") is an open source, general-purpose policy engine that unifies policy enforcement across the stack.","accessed":"2026-06-17"},{"type":"repo","title":"open-policy-agent/opa","url":"https://github.com/open-policy-agent/opa","accessed":"2026-06-17"}],"alternatives":[{"composition":"cerbos","relation":"competes-with","note":"Cerbos is a comparable decoupled authorization engine for fine-grained access decisions, offering a policy-as-code model without requiring Rego."},{"composition":"invariant-labs","relation":"complements","note":"Invariant Guardrails enforces agent-action policy at the MCP boundary; OPA is the general-purpose decision engine that a service can call for the same allow/deny verdict at the application layer."}],"verification_status":"partial","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Open Policy Agent (OPA)\"]\n  fw --> p1[\"Policy-as-Code Gate<br/>(core)\"]\n  fw --> p2[\"Decision Log<br/>(first-class)\"]\n  fw --> p3[\"Decision Token<br/>(supported)\"]","tags":["enterprise-platform","governance-observability","safety-control","full-code"]},{"id":"promptfoo","name":"Promptfoo","kind":"framework","category":"enterprise-platform","vendor":"Promptfoo, Inc.","language":"TypeScript","license":"MIT","status":"active","first_released":"2023","url":"https://www.promptfoo.dev","repo":"https://github.com/promptfoo/promptfoo","docs_url":"https://www.promptfoo.dev/docs/getting-started/","intent":"Promptfoo is an open-source command-line tool that runs declarative assertion-based test suites against prompts, models, and RAG or agent systems, and can red-team them for vulnerabilities.","description":"Promptfoo evaluates prompts, models, and RAG or agent pipelines against a YAML test suite of assertions, returning pass or fail and a non-zero exit code in CI when a test fails. Assertions include deterministic checks and model-graded checks such as llm-rubric, where an LLM grades the output against custom criteria. It also provides a red-teaming mode that generates simulated adversarial inputs to find vulnerabilities before deployment.","primary_use_cases":["assertion-based prompt and model evaluation in CI","model-graded scoring of open-ended outputs","red-teaming LLM applications for vulnerabilities"],"agent_loop_shape":"Promptfoo has no agent loop of its own. It is run from the command line over a configuration that lists prompts, providers, and test cases with assertions. For each test case it calls the configured provider, applies each assertion to the output, and aggregates pass or fail results, exiting non-zero in CI on any failure. In red-team mode it instead generates adversarial inputs and runs them against the target to surface failures.","build_surface":"full-code","members":[{"pattern":"eval-as-contract","role":"core","note":"Promptfoo runs declarative assertion-based eval suites that return pass/fail and exit non-zero in CI, so a release is blocked unless the evals pass — the eval suite literally is the release gate.","evidence":[{"type":"doc","title":"Testing prompts with Promptfoo / CI/CD","url":"https://www.promptfoo.dev/docs/integrations/ci-cd/","quote":"promptfoo can be used in CI/CD pipelines to automatically evaluate prompts, models, and RAG systems whenever changes are made... The eval command returns a non-zero exit code if any test fails.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"llm-as-judge","role":"first-class","note":"Promptfoo supports model-graded assertions such as llm-rubric that use an LLM to evaluate outputs against custom criteria or rubrics when no exact-match metric applies.","evidence":[{"type":"doc","title":"Model-graded metrics — Promptfoo","url":"https://www.promptfoo.dev/docs/configuration/expected-outputs/model-graded/","quote":"uses an LLM to evaluate outputs against custom criteria or rubrics.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"red-team-sandbox-reproduction","role":"first-class","note":"Promptfoo's red-team mode generates simulated adversarial inputs to find vulnerabilities in an LLM application before deployment, running an adversarial regression suite against the target; note Promptfoo's scope is general LLM-application red-teaming rather than a sealed per-release alignment-regression sandbox specifically.","evidence":[{"type":"doc","title":"LLM red teaming — Promptfoo","url":"https://www.promptfoo.dev/docs/red-team/","quote":"LLM red teaming is a way to find vulnerabilities in AI systems before they're deployed by using simulated adversarial inputs.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"prompt-variant-evaluation","role":"first-class","note":"Promptfoo runs the same test cases across many prompts, models, and providers and produces a matrix view that compares the outputs side by side, so competing prompt or model variants are scored as a batch against a shared dataset and the better one is chosen on evidence.","evidence":[{"type":"doc","title":"Promptfoo introduction","url":"https://www.promptfoo.dev/docs/intro/","quote":"promptfoo produces matrix views that let you quickly evaluate outputs across many prompts.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"Assertion","summary":"A declarative output check attached to a test case; deterministic assertions (contains, cost, latency) and model-graded assertions (llm-rubric) together decide whether the case passes.","maps_to_pattern":"eval-as-contract","url":"https://www.promptfoo.dev/docs/configuration/expected-outputs/"},{"name":"llm-rubric","summary":"A model-graded assertion type where an LLM grades a free-form output against stated criteria, used when no exact-match check fits the expected behaviour.","maps_to_pattern":"llm-as-judge","url":"https://www.promptfoo.dev/docs/configuration/expected-outputs/model-graded/"},{"name":"Provider","summary":"A configured model or endpoint under test; listing several providers lets one test suite run across multiple models for side-by-side comparison.","maps_to_pattern":"prompt-variant-evaluation","url":"https://www.promptfoo.dev/docs/providers/"},{"name":"Red team (Promptfoo)","summary":"A mode that curates and generates a diverse set of malicious intents targeting potential vulnerabilities and runs them against the application, either as one-off scans or continuously in the deployment pipeline.","maps_to_pattern":"red-team-sandbox-reproduction","url":"https://www.promptfoo.dev/docs/red-team/"}],"references":[{"type":"doc","title":"Promptfoo introduction","url":"https://www.promptfoo.dev/docs/intro/","quote":"promptfoo produces matrix views that let you quickly evaluate outputs across many prompts.","accessed":"2026-06-17"},{"type":"repo","title":"promptfoo/promptfoo","url":"https://github.com/promptfoo/promptfoo","accessed":"2026-06-17"},{"type":"doc","title":"LLM red teaming — Promptfoo","url":"https://www.promptfoo.dev/docs/red-team/","quote":"LLM red teaming is a way to find vulnerabilities in AI systems before they're deployed by using simulated adversarial inputs.","accessed":"2026-06-17"}],"alternatives":[{"composition":"deepeval","relation":"competes-with","note":"DeepEval is the pytest-native Python equivalent for assertion- and metric-based LLM evaluation; Promptfoo is the TypeScript/CLI-first counterpart that also bundles red-teaming."},{"composition":"langsmith","relation":"competes-with","note":"LangSmith covers offline and online LLM evaluation as a hosted platform; Promptfoo is the open-source command-line tool focused on assertion suites and red-teaming."},{"composition":"inspect-ai","relation":"similar-shape","note":"Inspect AI is another open-source evaluation harness for LLMs and agents; both run datasets of test cases with graded scorers, with Promptfoo leaning toward CI assertion suites and red-teaming."}],"verification_status":"partial","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Promptfoo\"]\n  fw --> p1[\"Eval as Contract<br/>(core)\"]\n  fw --> p2[\"LLM-as-Judge<br/>(first-class)\"]\n  fw --> p3[\"Red-Team Sandbox Reproduction<br/>(first-class)\"]\n  fw --> p4[\"Prompt Variant Evaluation<br/>(first-class)\"]","tags":["enterprise-platform","governance-observability","verification-reflection","full-code"]},{"id":"sagemaker","name":"Amazon SageMaker AI","kind":"framework","category":"enterprise-platform","vendor":"Amazon Web Services","language":"Python","license":"proprietary","status":"active","first_released":"2017","url":"https://aws.amazon.com/sagemaker/","docs_url":"https://docs.aws.amazon.com/sagemaker/latest/dg/whatis.html","intent":"Amazon SageMaker AI is a managed platform for building, training, deploying, and monitoring machine learning models, including shadow testing and production model monitoring.","description":"SageMaker AI provides managed infrastructure for the model lifecycle from training through hosted inference endpoints. Shadow testing deploys a challenger model version alongside the production champion and mirrors a portion of real requests to the shadow variant without returning shadow responses to users. SageMaker Model Monitor watches deployed models in production, using rules to detect drift in data and model quality and alerting operators when deviations occur.","primary_use_cases":["training and hosting machine learning models","shadow testing a challenger model against the champion","production monitoring of model and data quality drift"],"agent_loop_shape":"SageMaker AI is a managed ML platform rather than an agent loop. Models are trained and deployed to inference endpoints. Shadow testing routes a copied portion of live inference requests to a shadow fleet and compares metrics without serving shadow responses. Model Monitor runs scheduled or continuous monitoring jobs that compare captured production inputs and outputs against a baseline and raise alerts on drift.","build_surface":"full-code","members":[{"pattern":"shadow-canary","role":"first-class","note":"SageMaker shadow testing deploys a challenger model version alongside the production champion, mirroring a portion of real inference requests to the shadow variant and comparing outputs/metrics without ever returning shadow responses to users.","evidence":[{"type":"doc","title":"Shadow tests — Amazon SageMaker AI Developer Guide","url":"https://docs.aws.amazon.com/sagemaker/latest/dg/shadow-tests.html","quote":"Amazon SageMaker AI helps you run shadow tests to evaluate a new machine learning (ML) model before production release by testing its performance against the currently deployed model.","accessed":"2026-06-17"},{"type":"doc","title":"Shadow tests — Amazon SageMaker AI Developer Guide","url":"https://docs.aws.amazon.com/sagemaker/latest/dg/shadow-tests.html","quote":"deploy a new version of the model to the shadow fleet and route a portion of the inference requests to it. The model in the shadow fleet doesn't return any responses to the requests.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"scorer-live-monitoring","role":"supported","note":"SageMaker Model Monitor observes deployed models in production, applying rules to detect drift in data and model quality and alerting operators without altering the served predictions; note it monitors model and data quality metrics generally rather than scoring open-ended agent outputs, hence medium confidence.","evidence":[{"type":"doc","title":"Data and model quality monitoring with Amazon SageMaker Model Monitor","url":"https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor.html","quote":"Amazon SageMaker Model Monitor automatically monitors machine learning (ML) models in production and notifies you when quality issues happen. Model Monitor uses rules to detect drift in your models and alerts you when it happens.","accessed":"2026-06-17"},{"type":"doc","title":"Data quality — Amazon SageMaker AI Developer Guide","url":"https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-data-quality.html","quote":"If the statistical nature of the data that your model receives while in production drifts away from the nature of the baseline data it was trained on, the model begins to lose accuracy in its predictions.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"lineage-tracking","role":"supported","note":"The SageMaker Model Registry catalogs each trained model version into Model Groups, associates training metrics and data with it, and records model lineage so a given prediction can be traced back to the model version that produced it; this is general ML lineage rather than agent prompt/version lineage specifically, hence medium confidence.","evidence":[{"type":"doc","title":"Model Registry — Amazon SageMaker AI Developer Guide","url":"https://docs.aws.amazon.com/sagemaker/latest/dg/model-registry.html","quote":"With the Amazon SageMaker Model Registry you can do the following: Catalog models for production. Manage model versions. Associate metadata, such as training metrics, with a model. View model lineage for traceability and reproducibility.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"dual-evaluation-offline-online","role":"supported","note":"SageMaker pairs an offline track (training-time evaluation and a baseline computed from a held-out dataset) with an online track (Model Monitor on captured production traffic plus live A/B production variants), so regressions surface both before and after deployment; this covers ML model quality rather than agent-trajectory evaluation specifically, hence medium confidence.","evidence":[{"type":"doc","title":"Data quality — Amazon SageMaker AI Developer Guide","url":"https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-data-quality.html","quote":"Create a baseline. In this step, you run a baseline job that analyzes an input dataset that you provide. The baseline computes baseline schema constraints and statistics for each feature","accessed":"2026-06-17"},{"type":"doc","title":"Testing models with production variants — Amazon SageMaker AI Developer Guide","url":"https://docs.aws.amazon.com/sagemaker/latest/dg/model-ab-testing.html","quote":"Performing A/B testing between a new model and an old model with production traffic can be an effective final step in the validation process for a new model.","accessed":"2026-06-17"}],"evidence_status":"full"}],"key_concepts":[{"name":"Shadow test","summary":"A deployment that runs a challenger model on a copied portion of live inference traffic alongside the production model, comparing metrics without ever returning the shadow model's responses to users.","maps_to_pattern":"shadow-canary","url":"https://docs.aws.amazon.com/sagemaker/latest/dg/shadow-tests.html"},{"name":"Production variant","summary":"A model-plus-resource configuration hosted behind a SageMaker endpoint; multiple variants can share one endpoint and receive a weighted share of traffic so two models can be A/B-tested on live requests.","url":"https://docs.aws.amazon.com/sagemaker/latest/dg/model-ab-testing.html"},{"name":"Model Monitor","summary":"A scheduled monitoring job that captures production inputs/outputs, compares them against a baseline computed from the training dataset, and raises alerts when data or model quality drifts.","maps_to_pattern":"scorer-live-monitoring","url":"https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor.html"},{"name":"Model Registry","summary":"A catalog of versioned model packages organized into Model Groups, holding training metrics and lineage so each model version is traceable and approvable for deployment.","maps_to_pattern":"lineage-tracking","url":"https://docs.aws.amazon.com/sagemaker/latest/dg/model-registry.html"}],"references":[{"type":"doc","title":"What is Amazon SageMaker AI? — Developer Guide","url":"https://docs.aws.amazon.com/sagemaker/latest/dg/whatis.html","accessed":"2026-06-17"},{"type":"doc","title":"Testing models with production variants — Amazon SageMaker AI Developer Guide","url":"https://docs.aws.amazon.com/sagemaker/latest/dg/model-ab-testing.html","quote":"With SageMaker AI multi-variant endpoints you can distribute endpoint invocation requests across multiple production variants by providing the traffic distribution for each variant","accessed":"2026-06-17"},{"type":"doc","title":"Model Registry — Amazon SageMaker AI Developer Guide","url":"https://docs.aws.amazon.com/sagemaker/latest/dg/model-registry.html","accessed":"2026-06-17"}],"alternatives":[{"composition":"bedrock-agents","relation":"same-vendor","note":"Both are AWS ML/agent offerings; Bedrock Agents targets LLM agent orchestration while SageMaker AI is the lower-level model training, hosting, and monitoring platform."},{"composition":"vertex-ai-agent-builder","relation":"competes-with","note":"Google Cloud's managed ML/agent platform; the closest hyperscaler equivalent for the train-deploy-monitor lifecycle."},{"composition":"azure-ai-foundry-agent-service","relation":"competes-with","note":"Microsoft Azure's managed platform competing in the same hyperscaler ML/agent lifecycle space."}],"verification_status":"partial","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Amazon SageMaker AI\"]\n  fw --> p1[\"Shadow Canary<br/>(first-class)\"]\n  fw --> p2[\"Scorer Live Monitoring<br/>(supported)\"]\n  fw --> p3[\"Lineage Tracking<br/>(supported)\"]\n  fw --> p4[\"Dual Evaluation (Offline + Online)<br/>(supported)\"]","tags":["enterprise-platform","governance-observability","full-code"]},{"id":"statsig","name":"Statsig","kind":"framework","category":"enterprise-platform","vendor":"Statsig (OpenAI)","language":"TypeScript","license":"proprietary","status":"active","first_released":"2020","url":"https://www.statsig.com/","docs_url":"https://docs.statsig.com/","intent":"Statsig is a feature-management and experimentation platform that runs feature gates, controlled rollouts, and bandit-driven experiments over production traffic.","description":"Statsig provides feature gates that expose features to a configurable percentage of users and can be turned off in production through an emergency disable switch. Its Autotune product runs multi-armed bandit experiments that use Thompson Sampling to allocate traffic toward better-performing variants. The platform is general-purpose experimentation and feature management rather than agent-specific.","primary_use_cases":["feature gates with percentage-based rollouts","emergency disabling of features in production","bandit-driven variant experimentation"],"agent_loop_shape":"Statsig is not an agent loop; it is a decisioning layer queried at runtime. Application SDKs check a feature gate or experiment for a given user and Statsig returns whether the gate is on and which variant the user is assigned. For Autotune experiments, the bandit reallocates traffic across variants over time based on observed rewards, and operators can flip a gate off out-of-band to disable a feature in production.","build_surface":"low-code","members":[{"pattern":"bayesian-bandit-experimentation","role":"first-class","note":"Statsig Autotune is a multi-armed bandit that uses Thompson sampling to dynamically reallocate traffic toward better-performing variants in real time, minimizing regret rather than running a fixed-split A/B test; it is general-purpose experimentation, not agent/LLM-specific, hence medium confidence.","evidence":[{"type":"doc","title":"Autotune — Statsig Docs","url":"https://docs.statsig.com/autotune/introduction/","quote":"Autotune uses a multi-armed bandit approach... Statsig uses Thompson Sampling, a Bayesian inference technique, to allocate traffic to the best performing variants.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"kill-switch","role":"supported","note":"A Statsig feature gate doubles as an out-of-band emergency disable switch that immediately turns off a code branch for users in production without a redeploy; note this is general feature management rather than an agent-fleet halt control specifically, hence medium confidence.","evidence":[{"type":"doc","title":"Working with feature gates — Statsig Docs","url":"https://docs.statsig.com/feature-flags/working-with/","quote":"Have an emergency disable switch that lets you immediately turn off a particular code branch for users in production","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"shadow-canary","role":"limited","note":"A Statsig feature gate can be set to expose a new code branch to only a small percentage of the user base, acting as a canary rollout that limits blast radius before a feature reaches everyone; this is general feature management on real users rather than a shadow comparison of an agent challenger, hence low confidence.","evidence":[{"type":"doc","title":"Working with feature gates — Statsig Docs","url":"https://docs.statsig.com/feature-flags/working-with/","quote":"Developers often use them to turn on certain features for a small percentage of the total user base.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"Feature gate","summary":"A runtime-checked switch that decides whether a code branch is on for a given user; used for percentage rollouts and as an emergency disable switch.","maps_to_pattern":"kill-switch","url":"https://docs.statsig.com/feature-flags/working-with/"},{"name":"Experiment","summary":"A controlled A/B test that assigns users to variants and measures primary and secondary metrics against control with confidence intervals to test a hypothesis.","url":"https://docs.statsig.com/experiments-plus/create-new/"},{"name":"Autotune","summary":"A multi-armed bandit experiment that uses Thompson Sampling to continuously reallocate traffic toward the best-performing variant instead of holding a fixed split.","maps_to_pattern":"bayesian-bandit-experimentation","url":"https://docs.statsig.com/autotune/introduction/"},{"name":"Layer","summary":"An allocation construct that partitions traffic so concurrent experiments do not interfere; an experiment runs in its own layer by default.","url":"https://docs.statsig.com/experiments-plus/create-new/"}],"references":[{"type":"doc","title":"Statsig Documentation","url":"https://docs.statsig.com/","accessed":"2026-06-17"},{"type":"doc","title":"Create a new experiment — Statsig Docs","url":"https://docs.statsig.com/experiments-plus/create-new/","quote":"Primary Metrics are those you expect to be directly impacted by the experiment.","accessed":"2026-06-17"},{"type":"doc","title":"Autotune — Statsig Docs","url":"https://docs.statsig.com/autotune/introduction/","accessed":"2026-06-17"}],"alternatives":[{"composition":"braintrust","relation":"complements","note":"Braintrust is an LLM eval/experiment platform; Statsig brings the general-purpose feature-flag and online-experimentation layer that an LLM team would pair with eval tooling."},{"composition":"langfuse","relation":"complements","note":"Langfuse handles LLM tracing and evaluation while Statsig handles production feature gating and bandit experiments; they cover adjacent halves of the rollout/measurement loop."}],"verification_status":"needs-verification","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-19","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Statsig\"]\n  fw --> p1[\"Bayesian Bandit Experimentation<br/>(first-class)\"]\n  fw --> p2[\"Kill Switch<br/>(supported)\"]\n  fw --> p3[\"Shadow Canary<br/>(limited)\"]","tags":["enterprise-platform","governance-observability","safety-control","low-code"],"anti_patterns_avoided":[{"pattern":"agent-speed-incident-response-gap","note":"A kill switch deactivates a misbehaving agent feature immediately, giving an instant response lever when an agent acts faster than humans can react.","evidence":[{"type":"doc","title":"Statsig — kill switch","url":"https://www.statsig.com/glossary/kill-switch","quote":"When a feature breaks, you want to act fast. A kill switch lets you deactivate the problematic feature immediately, preventing further issues","accessed":"2026-06-19"}]}]},{"id":"lakera","name":"Lakera Guard","kind":"framework","vendor":"Lakera","url":"https://www.lakera.ai/lakera-guard","docs_url":"https://docs.lakera.ai/guard","intent":"Lakera Guard wraps a real-time security firewall around a GenAI application, screening every user input, reference document, tool response, and model output so attacker-planted instructions cannot override the agent's objective.","description":"Lakera Guard is a runtime security platform that sits between an application and its LLM and inspects each interaction through a single Guard API call. It detects direct and indirect prompt injection, jailbreaks, content violations, and data leakage, and blocks compromised instructions reaching connected tools before agents act on them.","members":[{"pattern":"prompt-injection-defense","role":"core","evidence_status":"full","note":"Detects and blocks direct and indirect prompt injection and jailbreaks before they reach the model.","evidence":[{"type":"doc","url":"https://docs.lakera.ai/docs/defenses","quote":"Detect and respond to direct and indirect prompt attacks. This includes jailbreaks, prompt injections and any attempts to manipulate and exploit AI models through malicious or unintentionally troublesome instructions, preventing potential harm to your application.","accessed":"2026-06-14"}]},{"pattern":"input-output-guardrails","role":"core","evidence_status":"full","note":"A real-time API guardrail layer that screens both inputs and outputs of every LLM interaction.","evidence":[{"type":"doc","url":"https://docs.lakera.ai/docs/defenses","quote":"screens any user or reference contents passed into an LLM and the output response from the LLM.","accessed":"2026-06-14"}]},{"pattern":"pii-redaction","role":"supported","evidence_status":"full","note":"Data-leakage guardrails detect, mask, and block PII and confidential content in prompts and outputs.","evidence":[{"type":"doc","url":"https://www.lakera.ai/risk/ai-data-leakage","quote":"Lakera Guard screens both inputs _and_ outputs of your LLM in real time, looking for personally identifiable information (PII), internal prompts or confidential content, and either masks or blocks it before it's exposed.","accessed":"2026-06-14"}]},{"pattern":"multimodal-guardrails","role":"supported","evidence_status":"full","note":"Moderates offensive content across 100+ languages with ongoing image and audio support.","evidence":[{"type":"doc","url":"https://docs.lakera.ai/docs/content-moderation","quote":"Moderating the content passed to or generated by your Large Language Model (LLM) applications is an important component to securing your GenAI application and protecting your users.","accessed":"2026-06-14"}]}],"anti_patterns_avoided":[{"pattern":"goal-hijacking","note":"Scans every reachable input surface (user prompts, reference materials, tool responses, tool descriptions) for instructions that would override the agent's behavior.","evidence":[{"type":"doc","url":"https://docs.lakera.ai/guard","quote":"Prompt attacks - detect prompt injections, jailbreaks or manipulation in user prompts, reference materials, tool responses or tool descriptions to stop LLM behavior being overridden","accessed":"2026-06-14"}]},{"pattern":"ai-targeted-comment-injection","note":"The Prompt Defense guardrail scans retrieved and reference documents for hidden instructions embedded in the content the model reads.","evidence":[{"type":"doc","url":"https://www.lakera.ai/risk/indirect-prompt-injection","quote":"Lakera Guard uses its 'Prompt Defense' guardrail to scan both user-inputs and retrieved/reference documents for instructions, overrides or hidden prompts that aim to manipulate the model.","accessed":"2026-06-14"}]}],"key_concepts":[{"name":"AI security firewall","summary":"A runtime layer between the app and the LLM that inspects every interaction for attacks before they are acted on."},{"name":"Guard API","summary":"A single API call per interaction that screens inputs, reference content, tool responses, and outputs.","maps_to_pattern":"input-output-guardrails"},{"name":"Prompt Defense","summary":"The guardrail that detects direct and indirect prompt injection and jailbreaks.","maps_to_pattern":"prompt-injection-defense"},{"name":"Data leakage prevention","summary":"Detection, masking, and blocking of PII and confidential content in prompts and outputs.","maps_to_pattern":"pii-redaction"}],"primary_use_cases":["Securing customer-facing GenAI assistants against prompt injection","Protecting tool-using agents from compromised retrieved content","PII leakage prevention in LLM I/O"],"agent_loop_shape":"Sits inline on every user interaction and agent step, screening inputs and the model output before either is acted on.","references":[{"type":"doc","title":"Lakera Guard product page","url":"https://www.lakera.ai/lakera-guard"},{"type":"doc","title":"Introduction to Lakera Guard (docs)","url":"https://docs.lakera.ai/guard"},{"type":"doc","title":"Guardrails / Defenses (docs)","url":"https://docs.lakera.ai/docs/defenses"}],"tags":["guardrails","security","prompt-injection","enterprise-platform"],"pattern_composition":"A prompt-injection-defense scanner and an input-output-guardrails layer screen every interaction, while pii-redaction and multimodal-guardrails moderate sensitive and multi-language content.","category":"enterprise-platform","status":"active","verification_status":"needs-verification","build_surface":"full-code","last_analyzed":"2026-06-14","status_in_practice":"emerging","last_updated":"2026-06-14","date_added":"2026-06-14"},{"id":"snyk-code","name":"Snyk Code","kind":"framework","vendor":"Snyk","url":"https://snyk.io/product/snyk-code/","docs_url":"https://docs.snyk.io/","intent":"Snyk Code is a developer-focused SAST engine that scans human- and AI-generated code as it is written and fails the pull request when new vulnerabilities exceed a configured severity threshold, so insecure generated code cannot merge unreviewed.","description":"Snyk Code adds static security analysis directly into IDEs and pull requests, flagging vulnerabilities in line as code is written, including code produced by AI coding tools. Its pull-request checks block a merge when new issues meet or exceed a configurable severity threshold.","members":[{"pattern":"policy-as-code-gate","role":"core","evidence_status":"full","note":"PR checks fail and block the merge when new vulnerabilities exceed a configured severity threshold.","evidence":[{"type":"doc","url":"https://docs.snyk.io/scan-fix-and-prevent/prevent/pull-request-checks/configure-pull-request-checks","quote":"If the severity is higher than your threshold, the PR is not merged into the main branch.","accessed":"2026-06-14"}]},{"pattern":"input-output-guardrails","role":"core","evidence_status":"full","note":"Screens generated code before it is accepted, scanning it in line in the IDE and in pull requests.","evidence":[{"type":"doc","url":"https://snyk.io/solutions/secure-ai-generated-code/","quote":"Snyk Code adds security directly into IDEs and pull requests with real-time vulnerability-scanning and auto-fixing of human and AI-generated code","accessed":"2026-06-14"}]}],"anti_patterns_avoided":[{"pattern":"vibe-coding-without-security-review","note":"Snyk's own guidance insists AI-generated code never ship without review; Snyk Code is the automated security gate that enforces that review in the workflow.","evidence":[{"type":"doc","url":"https://snyk.io/articles/the-highs-and-lows-of-vibe-coding/","quote":"Never push AI-generated code without human review.","accessed":"2026-06-14"}]}],"key_concepts":[{"name":"Developer-focused SAST","summary":"Static application security testing that runs in the IDE and on pull requests as code is written.","maps_to_pattern":"input-output-guardrails"},{"name":"Severity-threshold PR gate","summary":"A pull-request check that blocks merge when new issues meet or exceed a configured severity.","maps_to_pattern":"policy-as-code-gate"},{"name":".snyk policy file","summary":"A file that controls ignores, scan scope, and gating policy as code."}],"primary_use_cases":["Gating AI-assisted and human pull requests on security before merge","Catching SQL injection and XSS in generated code in the IDE","Enforcing severity-threshold security policy in CI/CD"],"agent_loop_shape":"Sits at the code-output boundary of a coding agent or developer, scanning generated code and failing the pull request before insecure code merges.","references":[{"type":"doc","title":"Snyk Code product page","url":"https://snyk.io/product/snyk-code/"},{"type":"doc","title":"Secure AI-generated code","url":"https://snyk.io/solutions/secure-ai-generated-code/"},{"type":"doc","title":"The highs and lows of vibe coding","url":"https://snyk.io/articles/the-highs-and-lows-of-vibe-coding/"}],"tags":["security","sast","code-review","guardrails","enterprise-platform"],"pattern_composition":"An input-output-guardrails scan of generated code feeds a policy-as-code-gate that fails the pull request when severity exceeds the configured threshold.","category":"enterprise-platform","status":"active","verification_status":"needs-verification","build_surface":"full-code","last_analyzed":"2026-06-14","status_in_practice":"emerging","last_updated":"2026-06-14","date_added":"2026-06-14"},{"id":"noma","name":"Noma Security","kind":"framework","vendor":"Noma Security","url":"https://noma.security/solutions/ai-agent-security/","docs_url":"https://noma.security/platform/runtime-protection/","intent":"Noma Security continuously compares a running agent's behavior against established baselines and enforces enterprise policy at the point of execution, flagging drift from the agent's purpose and blocking unauthorized actions before they complete.","description":"Noma is a runtime-protection platform that monitors prompts, responses, agent tool calls, MCP interactions, and agent-to-agent communication in real time. It enforces guardrail policies inline by blocking malicious tool calls and unauthorized function executions, and logs every interaction and policy decision.","members":[{"pattern":"trajectory-anomaly-monitor","role":"core","evidence_status":"full","note":"Continuously compares agent outputs against established baselines and flags any departure from the agent's intended purpose.","evidence":[{"type":"doc","url":"https://noma.security/resources/autonomous-ai-goal-misalignment/","quote":"As agents operate, their outputs are continuously compared against these baselines. Any departure—whether subtle drift or overt misalignment—is immediately flagged, preventing small discrepancies from escalating into rogue AI behavior.","accessed":"2026-06-14"}]},{"pattern":"policy-as-code-gate","role":"core","evidence_status":"full","note":"Defines and enforces enterprise AI policies as guardrails, with enforcement at the point of execution.","evidence":[{"type":"doc","url":"https://noma.security/platform/runtime-protection/","quote":"Define and enforce enterprise AI policies with precision. Create guardrails for security, privacy, and compliance","accessed":"2026-06-14"}]},{"pattern":"sandbox-escape-monitoring","role":"supported","evidence_status":"full","note":"Identifies and blocks malicious tool calls, poisoned MCP servers, and unauthorized function executions.","evidence":[{"type":"doc","url":"https://noma.security/platform/runtime-protection/","quote":"Identify and block malicious tool calls, poisoned MCP servers, and unauthorized function executions","accessed":"2026-06-14"}]},{"pattern":"decision-log","role":"supported","evidence_status":"full","note":"Every interaction, policy decision, and response action is fully logged and searchable.","evidence":[{"type":"doc","url":"https://noma.security/platform/runtime-protection/","quote":"Every interaction, every policy decision, every response action – fully logged and searchable","accessed":"2026-06-14"}]}],"anti_patterns_avoided":[{"pattern":"rogue-agent-drift","note":"Noma continuously baselines agent behavior and flags drift before it becomes rogue behavior, restoring the re-alignment check a long-running agent otherwise lacks.","evidence":[{"type":"doc","url":"https://noma.security/resources/autonomous-ai-goal-misalignment/","quote":"This drift is the foundation of rogue AI behavior, leading to outcomes that compromise business continuity, compliance, and security.","accessed":"2026-06-14"}]}],"key_concepts":[{"name":"Behavioral baselines","summary":"Established norms of agent behavior against which live activity is compared.","maps_to_pattern":"trajectory-anomaly-monitor"},{"name":"Inline policy enforcement","summary":"Guardrail policies enforced at the point of execution, blocking violating actions.","maps_to_pattern":"policy-as-code-gate"},{"name":"Goal-misalignment detection","summary":"Detection of drift between an agent declared objective and its executed decisions."},{"name":"Runtime action monitoring","summary":"Real-time inspection of tool calls, MCP interactions, and agent-to-agent communication.","maps_to_pattern":"sandbox-escape-monitoring"}],"primary_use_cases":["Detecting goal drift in long-running autonomous agents","Blocking unauthorized agent tool calls and poisoned MCP servers at runtime","Enterprise AI policy enforcement and audit"],"agent_loop_shape":"Sits inline at the agent's execution boundary, baselining and monitoring each tool call and output and blocking policy-violating or drifting actions before they complete.","references":[{"type":"doc","title":"Noma AI Agent Security","url":"https://noma.security/solutions/ai-agent-security/"},{"type":"doc","title":"Noma Runtime Protection","url":"https://noma.security/platform/runtime-protection/"},{"type":"doc","title":"Autonomous AI Goal Misalignment","url":"https://noma.security/resources/autonomous-ai-goal-misalignment/"}],"tags":["security","observability","agent-monitoring","guardrails","enterprise-platform"],"pattern_composition":"A trajectory-anomaly-monitor baselines agent behavior, a policy-as-code-gate enforces guardrails inline, sandbox-escape-monitoring blocks unauthorized tool calls, and a decision-log records every decision.","category":"enterprise-platform","status":"active","verification_status":"needs-verification","build_surface":"full-code","last_analyzed":"2026-06-14","status_in_practice":"emerging","last_updated":"2026-06-14","date_added":"2026-06-14"},{"id":"exabeam-aba","name":"Exabeam Agent Behavior Analytics","kind":"framework","vendor":"Exabeam","url":"https://www.exabeam.com/capabilities/agent-behavior-analytics/","docs_url":"https://docs.exabeam.com/","intent":"Exabeam Agent Behavior Analytics establishes per-agent behavioral baselines and surfaces anomalies in real time, triaging high-risk detections above low-fidelity alerts so reviewers keep attending to an agent's findings instead of being buried in noise.","description":"Exabeam extends its UEBA behavioral analytics to AI agents, treating them as non-human insiders with broad access. It establishes normal agent activity, surfaces anomalies tied to misuse or policy violations, and separates high-risk detections from low-fidelity alerts to reduce noise.","members":[{"pattern":"trajectory-anomaly-monitor","role":"core","evidence_status":"full","note":"Applies behavioral detections to establish normal agent activity and surface anomalies linked to misuse, compromise, or policy violations.","evidence":[{"type":"doc","url":"https://www.exabeam.com/capabilities/agent-behavior-analytics/","quote":"ABA applies behavioral detections to establish normal agent activity and surface anomalies linked to misuse, compromise, or policy violations.","accessed":"2026-06-14"}]},{"pattern":"decision-log","role":"core","evidence_status":"full","note":"Connects related agent activity into a single session narrative with timelines and dashboards for investigation.","evidence":[{"type":"doc","url":"https://www.exabeam.com/capabilities/agent-behavior-analytics/","quote":"Timelines and agent dashboards give your team a unified view to see patterns, hunt threats, and investigate agent misuse.","accessed":"2026-06-14"}]}],"anti_patterns_avoided":[{"pattern":"agent-output-alert-fatigue","note":"By scoring what is unusual for each specific agent and triaging high-risk detections above low-fidelity alerts, ABA reduces noise instead of flooding reviewers with undifferentiated findings.","evidence":[{"type":"doc","url":"https://www.exabeam.com/capabilities/agent-behavior-analytics/","quote":"Triage high-risk detections versus low-fidelity alerts","accessed":"2026-06-14"}]}],"key_concepts":[{"name":"UEBA for AI agents","summary":"User-and-entity behavior analytics applied to agents treated as non-human insiders.","maps_to_pattern":"trajectory-anomaly-monitor"},{"name":"Session narrative","summary":"Related agent activity connected into one investigable timeline.","maps_to_pattern":"decision-log"},{"name":"Alert triage","summary":"Scoring that separates high-risk detections from low-fidelity alerts to cut noise."}],"primary_use_cases":["Behavioral anomaly detection on AI-agent activity","Triaging agent detections to cut alert noise","Investigating agent misuse with session timelines"],"agent_loop_shape":"Sits as an out-of-band analytics layer consuming agent activity, baselining behavior and surfacing prioritized anomalies for human investigation rather than enforcing inline.","references":[{"type":"doc","title":"Exabeam Agent Behavior Analytics","url":"https://www.exabeam.com/capabilities/agent-behavior-analytics/"}],"tags":["security","observability","ueba","agent-monitoring","enterprise-platform"],"pattern_composition":"A trajectory-anomaly-monitor scores per-agent behavioral anomalies that a decision-log session narrative makes investigable.","category":"enterprise-platform","status":"active","verification_status":"needs-verification","build_surface":"full-code","last_analyzed":"2026-06-14","status_in_practice":"emerging","last_updated":"2026-06-14","date_added":"2026-06-14"},{"id":"appbuilder","name":"AppBuilder","aliases":["Baidu AppBuilder","千帆AppBuilder","百度千帆 AppBuilder"],"kind":"framework","category":"low-code-platform","build_surface":"low-code","vendor":"Baidu (百度) — Baidu Intelligent Cloud / Qianfan","language":"Python (SDK) + hosted visual platform","license":"Apache-2.0 (AppBuilder-SDK on GitHub); hosted Qianfan AppBuilder console is proprietary","status":"active","url":"https://cloud.baidu.com/product/AppBuilder","repo":"https://github.com/baidubce/app-builder","docs_url":"https://cloud.baidu.com/doc/AppBuilder/","intent":"Provide an Agent-centred, one-stop application development platform on Baidu's Qianfan cloud where a builder composes a RAG-and-tool-using agent from Baidu-ecosystem Components and MCP services, with workflow orchestration exposed both as a hosted console and as an open-source Python SDK.","description":"AppBuilder is Baidu's hosted, low-code agent platform on the Qianfan large-model cloud. Per Baidu's product page it is the '以Agent为核心的一站式企业级大模型服务平台' (Agent-centred one-stop enterprise large-model platform), built around four pillars: 'Agent引擎、工具及MCP、模型服务与企业级服务' (Agent engine, tools and MCP, model services, enterprise services). The companion AppBuilder-SDK on GitHub is Apache-2.0 and exposes 'Message、Component、AgentRuntime多级工作流抽象' (Message / Component / AgentRuntime multi-level workflow abstractions) over '40+个源于百度生态的优质组件' (40+ premium components from the Baidu ecosystem) for building production RAG applications.","primary_use_cases":["no-code/low-code agent building on Qianfan with Baidu-ecosystem components","production RAG over enterprise documents (parse → chunk → embed → index → retrieve → generate)","multi-agent orchestration through the Multi-Agent engine","tool/MCP integration via curated Skill marketplace and MCP services","code-level extension via the open-source AppBuilder-SDK"],"agent_loop_shape":"Two surfaces share one conceptual model. In the hosted Qianfan AppBuilder console, a builder assembles an Agent from a system prompt, a Knowledge Base (RAG), Components/Skills (callable tools), MCP services, and optionally a Multi-Agent orchestration graph that routes a turn across specialised sub-agents. In code, the AppBuilder-SDK exposes the same loop through three abstraction levels — Message (the data unit), Component (a callable RAG/tool node), and AgentRuntime (the orchestrator that runs a workflow over Components). RAG is realised as a chain of typed Components covering document parsing, chunking, embedding, indexing, retrieval, and answer generation.","key_concepts":[{"name":"Agent engine (Agent引擎)","summary":"Core of Qianfan AppBuilder: the agent runtime that drives reasoning over components, tools, and MCP services.","url":"https://cloud.baidu.com/product-s/qianfan_appbuilder"},{"name":"Component (组件) / Skill","summary":"Reusable callable unit; 40+ Baidu-ecosystem components plus curated Skills are composed into the agent.","maps_to_pattern":"tool-use","url":"https://github.com/baidubce/app-builder"},{"name":"Knowledge Base (知识库) / RAG chain","summary":"Production RAG built from typed components: parse, chunk, embed, index, retrieve, generate.","maps_to_pattern":"agentic-rag","url":"https://github.com/baidubce/app-builder"},{"name":"MCP services","summary":"Tools and external systems integrated via Model Context Protocol; first-class alongside built-in components.","maps_to_pattern":"mcp","url":"https://cloud.baidu.com/product-s/qianfan_appbuilder"},{"name":"Multi-Agent orchestration (Multi-Agent编排引擎)","summary":"Engine that distributes tasks across multiple specialised agents and coordinates their results.","maps_to_pattern":"supervisor","url":"https://cloud.baidu.com/product-s/qianfan_appbuilder"},{"name":"AgentRuntime (SDK)","summary":"Top-level workflow runner in the Python SDK; sits above Component and Message abstractions.","url":"https://github.com/baidubce/app-builder"}],"pattern_composition":"flowchart TD\n  USR[User / API caller] --> AGT[Agent engine — Qianfan AppBuilder]\n  AGT --> KB[Knowledge Base — RAG chain]\n  KB --> P[Doc parsing] --> CHK[Chunking] --> EMB[Embedding] --> IDX[(Index)] --> RET[Retrieval] --> GEN[Answer generation]\n  AGT --> COMP[Components / Skills<br/>40+ Baidu ecosystem]\n  AGT --> MCP[MCP services]\n  AGT --> MA{Multi-Agent<br/>orchestration engine}\n  MA -.delegate.-> SUB1[Specialised sub-agent A]\n  MA -.delegate.-> SUB2[Specialised sub-agent B]\n  AGT --> SDK[AppBuilder-SDK<br/>Message / Component / AgentRuntime]\n  GEN --> OUT[Response]\n  COMP --> OUT\n  MCP --> OUT","members":[{"pattern":"agentic-rag","role":"first-class","note":"Production RAG built from typed Components covering the full pipeline (parse → chunk → embed → index → retrieve → generate). Knowledge Base is a documented first-class capability of both the console and the SDK.","evidence":[{"type":"repo","title":"AppBuilder-SDK README — RAG components [translated]","url":"https://github.com/baidubce/app-builder","quote":"提供多类型组件，覆盖以下构建产业级RAG应用的完整步骤：文档解析、文档切片、切片向量化、索引构建、切片召回、答案生成","accessed":"2026-05-20"},{"type":"doc","title":"Baidu Qianfan AppBuilder product page — Knowledge Base RAG [translated]","url":"https://cloud.baidu.com/product-s/qianfan_appbuilder","quote":"知识库RAG","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Agents call 40+ Baidu-ecosystem Components and curated Skills as tools; product page calls them '独家优质Skill与精选MCP服务' (exclusive premium Skills and curated MCP services).","evidence":[{"type":"repo","title":"AppBuilder-SDK README — components [translated]","url":"https://github.com/baidubce/app-builder","quote":"提供了40+个源于百度生态的优质组件，赋能Agent应用","accessed":"2026-05-20"},{"type":"doc","title":"Baidu Qianfan AppBuilder product page — Skills and MCP services [translated]","url":"https://cloud.baidu.com/product-s/qianfan_appbuilder","quote":"独家优质Skill与精选MCP服务","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"MCP services are named as one of the four core pillars of Qianfan AppBuilder ('Agent引擎、工具及MCP、模型服务与企业级服务') and are explicitly called out alongside Skills on the product page.","evidence":[{"type":"doc","title":"Baidu Qianfan platform overview — four pillars [translated]","url":"https://cloud.baidu.com/doc/AppBuilder/index.html","quote":"Agent引擎、工具及MCP、模型服务与企业级服务等四大关键要素","accessed":"2026-05-20"},{"type":"doc","title":"Baidu Qianfan AppBuilder product page — Skills and MCP services [translated]","url":"https://cloud.baidu.com/product-s/qianfan_appbuilder","quote":"独家优质Skill与精选MCP服务","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"supervisor","role":"first-class","note":"Product page documents a Multi-Agent orchestration engine that intelligently distributes tasks across specialised agents — a supervisor/router over sub-agents. Architecture supports high-concurrency DAG execution with full-link streaming output.","evidence":[{"type":"doc","title":"Baidu Qianfan AppBuilder product page — Multi-Agent engine [translated]","url":"https://cloud.baidu.com/product-s/qianfan_appbuilder","quote":"Multi-Agent编排引擎实现复杂场景下的任务智能分发和多Agent协同","accessed":"2026-05-20"},{"type":"doc","title":"Baidu Qianfan AppBuilder product page — DAG execution engine [translated]","url":"https://cloud.baidu.com/product-s/qianfan_appbuilder","quote":"高并发 DAG 执行引擎提升效率，支持全链路流式输出，任务进度实时跟随","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"orchestrator-workers","role":"first-class","note":"Workflow orchestration is the documented top-level abstraction. SDK exposes Message / Component / AgentRuntime as multi-level workflow primitives; product page describes Multi-Agent task distribution and coordination.","evidence":[{"type":"repo","title":"AppBuilder-SDK README — workflow abstractions [translated]","url":"https://github.com/baidubce/app-builder","quote":"提供了Message、Component、AgentRuntime多级工作流抽象，实现工作流编排","accessed":"2026-05-20"},{"type":"doc","title":"Baidu Qianfan AppBuilder product page — Multi-Agent task distribution [translated]","url":"https://cloud.baidu.com/product-s/qianfan_appbuilder","quote":"Multi-Agent编排引擎实现复杂场景下的任务智能分发和多Agent协同","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"visual-workflow-graph","role":"first-class","note":"Hosted Qianfan AppBuilder console exposes a visual workflow editor; SDK mirrors the same model in code through AgentRuntime over Components.","evidence":[{"type":"doc","title":"Baidu AppBuilder product overview — one-stop Agent platform [translated]","url":"https://cloud.baidu.com/product-s/qianfan_appbuilder","quote":"以Agent为核心的一站式企业级大模型服务平台","accessed":"2026-05-20"},{"type":"repo","title":"AppBuilder-SDK README — workflow orchestration [translated]","url":"https://github.com/baidubce/app-builder","quote":"提供了Message、Component、AgentRuntime多级工作流抽象，实现工作流编排","accessed":"2026-05-20"}],"evidence_status":"full"}],"alternatives":[{"composition":"dify","relation":"competes-with","note":"Open-source low-code agent platform with the same RAG + tools + workflow + multi-agent surface; non-Baidu ecosystem."},{"composition":"coze","relation":"competes-with","note":"ByteDance's hosted no-code peer; same Chinese-market segment."},{"composition":"fastgpt","relation":"competes-with","note":"Open-source Chinese visual RAG/agent builder."},{"composition":"comate","relation":"same-vendor","note":"Both are Baidu Qianfan products; Comate targets coding, AppBuilder targets low-code agent building."}],"references":[{"type":"doc","title":"Baidu Qianfan AppBuilder product page","url":"https://cloud.baidu.com/product-s/qianfan_appbuilder","quote":"以Agent为核心的一站式企业级大模型服务平台","accessed":"2026-05-20"},{"type":"doc","title":"Baidu AppBuilder documentation hub","url":"https://cloud.baidu.com/doc/AppBuilder/index.html","quote":"Agent引擎、工具及MCP、模型服务与企业级服务等四大关键要素","accessed":"2026-05-20"},{"type":"repo","title":"baidubce/app-builder README","url":"https://github.com/baidubce/app-builder","quote":"百度智能云千帆AppBuilder-SDK是百度智能云千帆AppBuilder面向AI原生应用开发者提供的一站式开发平台的客户端SDK","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"mature","tags":["hosted","low-code","chinese-market","baidu","qianfan","rag","mcp"]},{"id":"bisheng","name":"Bisheng","aliases":["毕昇","BISHENG"],"kind":"framework","category":"low-code-platform","build_surface":"low-code","vendor":"DataElem (数据项素)","language":"Python (visual)","license":"Apache-2.0","status":"active","url":"https://bisheng.dataelem.com/","repo":"https://github.com/dataelement/bisheng","docs_url":"https://github.com/dataelement/bisheng","intent":"Provide an open, enterprise-grade LLMOps platform for building document-centric AI applications — GenAI workflows, RAG, Agents, evaluations and SFT — with high-precision document parsing and human-in-the-loop intervention baked into the workflow runtime.","description":"BISHENG (毕昇) is an open-source enterprise LLM application platform built by DataElem. The README states: 'BISHENG is an open LLM devops platform for next generation Enterprise AI applications.' Capabilities span 'GenAI workflow, RAG, Agent, Unified model management, Evaluation, SFT, Dataset Management, Enterprise-level System Management, Observability.' Its workflow engine is positioned as an 'independent and comprehensive application orchestration framework' that 'supports loops, parallelism, batch processing, conditional logic and free combination of all logic components,' and it ships with a high-precision document parsing stack covering print, handwriting, rare characters, tables, layout and seal detection. Human-in-the-loop is a documented first-class feature.","primary_use_cases":["enterprise document review and fixed-layout report generation","RAG over Chinese enterprise documents using high-precision parsing","multi-agent collaboration on document-centric tasks","policy / regulation update diff comparison","ticket-question-answering and unstructured data governance"],"agent_loop_shape":"A workflow is a directed graph of typed nodes assembled in a visual editor and executed by Bisheng's orchestration runtime. Nodes cover model calls, retrieval over a knowledge base, Lingsight agent invocations (using the AGL — Agent Guidance Language — framework), code/condition logic, and human-in-the-loop interventions. The runtime supports loops, parallelism, batch processing and conditional logic. RAG sits inside the same workflow graph, fed by DataElem's high-precision document parsing models. Enterprise hardening (RBAC, SSO/LDAP, traffic control, security review) wraps every execution.","key_concepts":[{"name":"Workflow","summary":"Visual orchestration graph supporting loops, parallelism, batch processing and conditional logic; the unit of execution.","maps_to_pattern":"visual-workflow-graph","url":"https://github.com/dataelement/bisheng"},{"name":"Lingsight agent / AGL","summary":"Bisheng's general-purpose agent driven by the Agent Guidance Language framework that encodes domain expertise.","url":"https://github.com/dataelement/bisheng"},{"name":"High-precision document parsing","summary":"First-party models for print, handwriting, rare characters, tables, layout and seal detection feeding RAG.","maps_to_pattern":"agentic-rag","url":"https://github.com/dataelement/bisheng"},{"name":"Human-in-the-loop intervention","summary":"Users can intervene and provide feedback during workflow execution including multi-turn conversations.","maps_to_pattern":"human-in-the-loop","url":"https://github.com/dataelement/bisheng"},{"name":"Evaluation (Eval)","summary":"Built-in evaluation module sitting alongside SFT and dataset management.","url":"https://github.com/dataelement/bisheng"},{"name":"Enterprise security stack","summary":"RBAC, SSO/LDAP, traffic control, vulnerability scanning and security review on top of every workflow.","url":"https://github.com/dataelement/bisheng"}],"pattern_composition":"flowchart TD\n  TR[Trigger — user / API / batch] --> WF[Workflow graph]\n  WF --> NODES{Typed nodes}\n  NODES --> LLM[LLM / model node]\n  NODES --> AGT[Lingsight agent — AGL]\n  NODES --> KB[Knowledge Base — high-precision parsing]\n  NODES --> CODE[Code / condition node<br/>loops + parallel + batch]\n  NODES --> HITL{{Human-in-the-loop<br/>intervention}}\n  AGT --> TOOLS[Components / tools]\n  KB --> PARSE[Parse → chunk → embed → index → retrieve]\n  HITL -.feedback.-> WF\n  EVAL[Evaluation module] -.scores.-> WF\n  SEC[(RBAC / SSO/LDAP / traffic control)] --> WF","members":[{"pattern":"agentic-rag","role":"first-class","note":"RAG is named as a core capability alongside workflow and agent; backed by DataElem's high-precision document-parsing stack feeding the knowledge layer.","evidence":[{"type":"repo","title":"Bisheng README — capability list","url":"https://github.com/dataelement/bisheng","quote":"GenAI workflow, RAG, Agent, Unified model management, Evaluation, SFT, Dataset Management, Enterprise-level System Management","accessed":"2026-05-20"},{"type":"repo","title":"Bisheng README — high-precision parsing","url":"https://github.com/dataelement/bisheng","quote":"High-precision document parsing model...includes high-precision printed text, handwritten text, rare character recognition models","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"visual-workflow-graph","role":"first-class","note":"Workflow orchestration is the central abstraction; supports loops, parallelism, batch and conditional logic in one graph.","evidence":[{"type":"repo","title":"Bisheng README — orchestration framework","url":"https://github.com/dataelement/bisheng","quote":"Independent and comprehensive application orchestration framework: Enables the execution of various tasks within a single framework","accessed":"2026-05-20"},{"type":"repo","title":"Bisheng README — workflow logic [translated]","url":"https://github.com/dataelement/bisheng/blob/main/README_CN.md","quote":"支持成环、并行、跑批、判断逻辑以及所有逻辑的任意自由组合","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"human-in-the-loop","role":"first-class","note":"Bisheng documents user intervention and feedback inside running workflows, including multi-turn conversations.","evidence":[{"type":"repo","title":"Bisheng README — HITL","url":"https://github.com/dataelement/bisheng","quote":"Allows users to intervene and provide feedback during the execution of workflows (including multi-turn conversations)","accessed":"2026-05-20"},{"type":"repo","title":"Bisheng README — HITL [translated]","url":"https://github.com/dataelement/bisheng/blob/main/README_CN.md","quote":"支持用户在Workflow执行的中间过程进行干预和反馈（包括多轮对话）","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Lingsight agents call tools/components within the workflow; agents are one of the platform's named primary capabilities.","evidence":[{"type":"repo","title":"Bisheng README — Lingsight agent","url":"https://github.com/dataelement/bisheng","quote":"Lingsight, a general-purpose agent with expert-level taste: Through the AGL (Agent Guidance Language) framework","accessed":"2026-05-20"},{"type":"repo","title":"Bisheng README — capability list","url":"https://github.com/dataelement/bisheng","quote":"GenAI workflow, RAG, Agent, Unified model management, Evaluation, SFT, Dataset Management, Enterprise-level System Management","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"eval-harness","role":"supported","note":"Evaluation is a named capability sitting next to SFT and Dataset Management in the README's feature list, but the public README and release notes do not describe its mechanics in detail; details live behind the DataElem Feishu wiki, which is not publicly fetchable. Honest gap retained.","evidence":[{"type":"repo","title":"Bisheng README — capability list (Evaluation)","url":"https://github.com/dataelement/bisheng","quote":"GenAI workflow, RAG, Agent, Unified model management, Evaluation, SFT, Dataset Management, Enterprise-level System Management","accessed":"2026-05-20"}],"evidence_status":"limited"},{"pattern":"code-execution","role":"supported","note":"Bisheng ships a named Code Executor with two execution modes — E2B (cloud sandbox) and Local — whose generated files surface directly in task results. Workflow graph also exposes loops, parallelism, batch processing and conditional logic.","evidence":[{"type":"repo","title":"Bisheng release notes — Code Executor (E2B + Local modes)","url":"https://github.com/dataelement/bisheng/releases","quote":"Files generated by the Code Executor (including both E2B mode and Local mode) are now directly displayed within task results.","accessed":"2026-05-24"},{"type":"repo","title":"Bisheng README — workflow logic [translated]","url":"https://github.com/dataelement/bisheng/blob/main/README_CN.md","quote":"支持成环、并行、跑批、判断逻辑以及所有逻辑的任意自由组合","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"supervisor","role":"limited","note":"Both English and Chinese READMEs list 'multi-agent collaboration' / '多智能体协作' as an enterprise use case, but the platform does not document a dedicated supervisor abstraction. Multi-agent is realised inside the workflow graph.","evidence":[{"type":"repo","title":"Bisheng README — enterprise use cases (multi-agent collaboration)","url":"https://github.com/dataelement/bisheng","quote":"Designed for Enterprise Applications: Document review, fixed-layout report generation, multi-agent collaboration, policy update comparison","accessed":"2026-05-20"},{"type":"repo","title":"Bisheng README_CN — multi-agent collaboration [translated]","url":"https://github.com/dataelement/bisheng/blob/main/README_CN.md","quote":"多智能体协作","accessed":"2026-05-24"}],"evidence_status":"full"}],"alternatives":[{"composition":"dify","relation":"competes-with","note":"LangGenius's open-source low-code agent platform — broader product surface, less document-centric than Bisheng."},{"composition":"fastgpt","relation":"competes-with","note":"Open-source Chinese visual RAG/agent builder; direct China-market peer."},{"composition":"coze","relation":"competes-with","note":"ByteDance hosted no-code peer; less enterprise/document-centric than Bisheng."}],"references":[{"type":"repo","title":"dataelement/bisheng README","url":"https://github.com/dataelement/bisheng","quote":"BISHENG is an open LLM devops platform for next generation Enterprise AI applications.","accessed":"2026-05-20"},{"type":"repo","title":"dataelement/bisheng README (Chinese)","url":"https://github.com/dataelement/bisheng/blob/main/README_CN.md","quote":"BISHENG毕昇 是一款 开源 LLM应用开发平台，主攻企业场景","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"emerging","tags":["open-source","apache-2","self-hostable","chinese-vendor","enterprise","document-centric","rag"]},{"id":"coze","name":"Coze","aliases":["扣子","Coze Studio"],"kind":"framework","category":"low-code-platform","build_surface":"low-code","vendor":"ByteDance (字节跳动)","language":"N/A","license":"proprietary (hosted); Coze Studio open-source edition is Apache-2.0","status":"active","first_released":"2024-02-01","url":"https://www.coze.com/","repo":"https://github.com/coze-dev/coze-studio","docs_url":"https://docs.coze.com","intent":"Provide a hosted, no-code visual platform on which a non-engineer can compose a bot from large-language-model nodes, plugins (tools), a knowledge base (RAG), workflows, memory and triggers, and publish it to chat channels such as Discord, Telegram, Lark/Feishu and WeChat.","description":"Coze is a no-code/low-code agent-building platform from ByteDance, launched internationally on 2024-02-01 and operated in China as 扣子 on coze.cn, where it is powered primarily by ByteDance's Doubao (豆包) models. A creator assembles a bot from four reusable building blocks — a workflow graph, plugins (callable tools), a knowledge base (RAG over uploaded documents) and memory/variables — in a drag-and-drop editor, then publishes the bot to one or more chat channels (Discord, Telegram, Slack, Lark/Feishu, WeChat). Multi-Agent Mode lets several specialised bots be wired together with jump conditions, so a 'main menu' bot can route a turn to a specialised sub-bot. The open-source Apache-2.0 edition (Coze Studio) is published by ByteDance's coze-dev GitHub organisation and derives directly from the same Coze platform.","primary_use_cases":["no-code chatbot building published to Discord/Telegram/Slack/Feishu/WeChat","RAG over uploaded documents via the Knowledge Base resource","scheduled and webhook-triggered bot tasks","multi-bot routing via Multi-Agent Mode","self-hosted agent development via the open-source Coze Studio edition"],"agent_loop_shape":"Visual workflow graph composed of typed nodes — model, plugin (tool), knowledge-base retrieval, conditional/code — driven from a Start node and terminated by an End node. A bot binds a workflow (or a single Agent prompt) to plugins, knowledge bases, databases, and memory/variables. Bots can also be wired in Multi-Agent Mode with jump conditions, and invoked by user messages on published channels, scheduled triggers, or webhook event triggers.","key_concepts":[{"name":"Bot (智能体)","summary":"The publishable unit — prompt + plugins + knowledge + workflow + memory.","url":"https://github.com/coze-dev/coze-studio/blob/main/README.zh_CN.md"},{"name":"Workflow (工作流)","summary":"Drag-and-drop graph of nodes running a multi-step process.","maps_to_pattern":"visual-workflow-graph","url":"https://www.coze.com/docs/use_workflow"},{"name":"Plugin (插件)","summary":"Registered tools (HTTP/API endpoints or built-ins) the bot can call.","maps_to_pattern":"tool-use","url":"https://github.com/coze-dev/coze-studio/wiki/4.-Plugin-Configuration"},{"name":"Knowledge Base (知识库)","summary":"Uploaded documents indexed for retrieval; queried from a knowledge-base node inside a workflow.","maps_to_pattern":"agentic-rag","url":"https://docs.coze.com/guides/knowledge_node"},{"name":"Memory / Variables (记忆 / 变量)","summary":"Per-user history and key/value variables persisted by the bot.","url":"https://github.com/coze-dev/coze-studio/blob/main/README.zh_CN.md"},{"name":"Trigger (触发器)","summary":"Scheduled cron and event/webhook triggers that invoke a bot or workflow.","maps_to_pattern":"event-driven-agent","url":"https://www.coze.com/open/docs/guides/set_timed_trigger"}],"pattern_composition":"flowchart TD\n  TRIG{{Trigger}} --> BOT[Bot — 智能体]\n  TRIG -.->|chat message| CHAN[Discord/Telegram/<br/>Slack/Feishu/WeChat]\n  TRIG -.->|scheduled cron| SCHED[(Timed trigger)]\n  TRIG -.->|webhook| EVT[Event trigger HTTPS URL]\n  BOT --> WF[Workflow graph<br/>typed nodes]\n  WF --> MODEL[(LLM node<br/>Doubao on coze.cn)]\n  WF --> PLUGIN[(Plugin / Tool nodes)]\n  WF --> KB_NODE[Knowledge-base node]\n  KB_NODE --> KB[(Knowledge Base — 知识库<br/>hybrid search + rerank)]\n  WF --> MEM[(Memory / Variables)]\n  BOT -.->|Multi-Agent Mode<br/>jump conditions| SUB[Specialised sub-bots]\n  SUB -.->|hand back| BOT\n  WF --> OUT[Published reply]","members":[{"pattern":"agentic-rag","role":"first-class","note":"Knowledge Base resource + dedicated knowledge-base retrieval node in workflows. Supports hybrid search (full-text + semantic) and rerank.","evidence":[{"type":"doc","title":"Coze docs — Knowledge base retrieval node","url":"https://docs.coze.com/guides/knowledge_node","quote":"If you need to search the knowledge base for knowledge, you can add a knowledge base search node in the workflow. When the workflow runs to this node, it will select a batch of knowledge that best meets the user's needs according to the retrieval and recall logic set by the node and pass it to subsequent nodes through output parameters.","accessed":"2026-05-20"},{"type":"doc","title":"Coze docs — hybrid search and rerank","url":"https://docs.coze.com/guides/knowledge_node","quote":"Supported retrieval strategies include hybrid search (which combines the advantages of full-text search and semantic search, and comprehensively ranks and recalls relevant content fragments) and full-text search.","accessed":"2026-05-20"},{"type":"repo","title":"Coze Studio README_CN — hallucination framing [translated]","url":"https://github.com/coze-dev/coze-studio/blob/main/README.zh_CN.md","quote":"你可以使用知识、插件等功能解决模型幻觉、专业领域知识不足等问题","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"event-driven-agent","role":"first-class","note":"Event-trigger generates a webhook URL; HTTPS POST to the URL invokes the bot/workflow. Bots are also published to chat channels where incoming messages are events.","evidence":[{"type":"doc","title":"Coze docs — Use triggers (event)","url":"https://www.coze.com/docs/guides/task?_lang=en","quote":"Coze also provides event triggers that generate Webhook URLs, allowing tasks to be triggered when your server sends HTTPS requests to the trigger Webhook URL.","accessed":"2026-05-20"},{"type":"doc","title":"Coze docs — Publish channels","url":"https://coze.com/docs/publish/channels.html","quote":"Coze supports direct deployment to platforms like Discord, Telegram, LINE, Slack, and Reddit. Additionally, you can also publish to others, like Messenger and Instagram, via webhooks.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"scheduled-agent","role":"first-class","note":"Timed triggers support 'every day', 'working day', 'every week X', specific YYYY-MM-DD dates, and full cron expressions.","evidence":[{"type":"doc","title":"Coze docs — Set timed trigger","url":"https://www.coze.com/open/docs/guides/set_timed_trigger","quote":"Coze offers scheduled triggers that let bots execute tasks at specified times without requiring any code.","accessed":"2026-05-20"},{"type":"doc","title":"Coze docs — Use triggers (cycle parameter)","url":"https://www.coze.com/docs/guides/task?_lang=en","quote":"the cycle parameter supports options like 'every day,' 'working day,' 'every week X' (such as every Wednesday), YYYY-MM-DD dates, or cron expressions","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"supervisor","role":"first-class","note":"Multi-Agent Mode wires specialised bots with jump conditions; the 'main menu' bot acts as a central router that delegates to and returns from sub-bots.","evidence":[{"type":"doc","title":"Coze docs — Multi-agent mode","url":"https://www.coze.com/docs/guides/multi_agent","quote":"Multi-Agent Mode allows bot creators to assemble a team of AI bots—each specializing in a specific type of task—that work together to help the user achieve their goals.","accessed":"2026-05-20"},{"type":"doc","title":"Coze docs — jump conditions","url":"https://www.coze.com/docs/guides/multi_agent","quote":"Each bot acts as a 'node' that can connect to one another with jump conditions. A jump condition is a rule that, when met, causes one bot to hand over to another bot.","accessed":"2026-05-20"},{"type":"doc","title":"Coze docs — main menu bot","url":"https://www.coze.com/docs/guides/multi_agent","quote":"The main menu bot acts as a central point, prompting the user and directing tasks to the appropriate specialized bot.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Plugin system: a registered API endpoint becomes a callable tool the agent can invoke. Built-in and custom plugins both supported.","evidence":[{"type":"repo","title":"Coze Studio Wiki — Plugin Configuration","url":"https://github.com/coze-dev/coze-studio/wiki/4.-Plugin-Configuration","quote":"Plugin tools can extend the capabilities of LLMs, for example, by searching the internet, performing scientific calculations, or drawing images.","accessed":"2026-05-20"},{"type":"doc","title":"Coze docs — Add tools to a plugin","url":"https://docs.coze.com/guides/plugin_tools","quote":"A tool is an API endpoint of the registered API service. That is, a tool is an action that the agent can call. After creating a plugin, you can add tools to it.","accessed":"2026-05-20"},{"type":"doc","title":"Coze docs — Plugin tools (CN) [translated]","url":"https://docs.coze.cn/docs/guides/plugin_tools","quote":"插件是 Bot 的扩展工具，是 Bot 调用外部 API 的方式","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"visual-workflow-graph","role":"first-class","note":"Workflow editor is described in the Coze Studio README as a drag-and-drop visual canvas for assembling typed nodes into an executable business process.","evidence":[{"type":"repo","title":"Coze Studio README — workflow","url":"https://github.com/coze-dev/coze-studio","quote":"A set of executable instructions used to implement business logic or complete specific tasks.","accessed":"2026-05-20"},{"type":"doc","title":"Coze docs — use workflow","url":"https://www.coze.com/docs/use_workflow","quote":"Drag-and-drop graph of nodes running a multi-step process.","accessed":"2026-05-20"}],"evidence_status":"full"}],"instantiates":["multi-agent-coordination"],"alternatives":[{"composition":"dify","relation":"competes-with","note":"LangGenius's open-source low-code agent platform — nearly identical conceptual surface."},{"composition":"fastgpt","relation":"competes-with","note":"Open-source Chinese visual RAG/agent builder; direct China-market peer."},{"composition":"bisheng","relation":"competes-with","note":"Open-source Chinese visual LLM app builder; direct China-market peer."},{"composition":"langflow","relation":"similar-shape","note":"Open-source visual LangChain flow builder (DataStax-owned)."},{"composition":"flowise","relation":"similar-shape","note":"Open-source visual LangChain/LLM flow builder."},{"composition":"appbuilder","relation":"competes-with","note":"ByteDance's hosted no-code peer; same Chinese-market segment."},{"composition":"relevance-ai","relation":"similar-shape","note":"ByteDance hosted no-code peer with multi-agent mode."},{"composition":"stackai","relation":"similar-shape","note":"ByteDance hosted no-code peer; bot + plugin + knowledge model."}],"references":[{"type":"repo","title":"coze-dev/coze-studio README","url":"https://github.com/coze-dev/coze-studio","quote":"Coze Studio is an all-in-one AI agent development tool","accessed":"2026-05-20"},{"type":"repo","title":"coze-dev/coze-studio README_CN [translated]","url":"https://github.com/coze-dev/coze-studio/blob/main/README.zh_CN.md","quote":"Coze Studio 是一站式 AI Agent 开发工具。提供各类最新大模型和工具、多种开发模式和框架","accessed":"2026-05-20"},{"type":"doc","title":"Coze.cn 中文 docs (扣子)","url":"https://docs.coze.cn","accessed":"2026-05-20"}],"last_analyzed":"2026-05-20","verification_status":"verified","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"mature","tags":["hosted","low-code","chinese-market","bytedance","doubao"]},{"id":"dify","name":"Dify","kind":"framework","category":"low-code-platform","build_surface":"low-code","vendor":"LangGenius, Inc.","language":"Python, TypeScript","license":"Apache-2.0 (modified — multi-tenant SaaS and LOGO removal forbidden)","status":"active","first_released":"2023-04-12","url":"https://dify.ai/","repo":"https://github.com/langgenius/dify","docs_url":"https://docs.dify.ai","intent":"Provide an open-source LLM-app development platform on which a builder visually composes AI workflows, RAG pipelines, and tool-using Agents, and ships them as hosted apps, embedded APIs, or MCP servers from a single self-hostable Docker stack.","description":"Dify is an open-source LLM-app development platform built by LangGenius. Per the README: 'Dify is an open-source LLM app development platform. Its intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features … and more, letting you quickly go from prototype to production.' Ships in three editions: hosted Dify Cloud at cloud.dify.ai, self-hostable Community (Docker Compose / Helm / K8s), enterprise. The Chinese README is parallel and uses 中文 terminology throughout: '您可以基于 LLM 函数调用或 ReAct 定义 Agent' (You can define Agents based on LLM function calling or ReAct). License is a modified Apache-2.0 that forbids multi-tenant resale and LOGO removal but otherwise tracks Apache-2.0.","primary_use_cases":["no-code/low-code visual LLM-app building","RAG with Knowledge Base + hybrid search + rerank + automatic citations","Agent apps with Function Calling or ReAct strategy","publishing apps as MCP servers consumable by Claude Desktop / Cursor"],"agent_loop_shape":"Apps on a visual workflow canvas of typed nodes (User Input → LLM / Knowledge Retrieval / Agent / Tool / IF-ELSE / Iteration / Template / Output) plus standalone modes (Chatbot, Agent, Chatflow, Workflow). Agent loop lives in two places: the Agent app (chat-style, reasoning + tools) and the Agent node inside a workflow. Both expose two pluggable strategies — Function Calling (native) and ReAct (Thought → Action → Observation) — bounded by Maximum Iterations safety cap. Workflows can chain into other workflows; LLM nodes support retry + fallback model chains; any Dify app can be published as an MCP server.","key_concepts":[{"name":"App","summary":"Publishable unit — Chatbot / Agent / Chatflow / Workflow modes.","url":"https://github.com/langgenius/dify-docs/blob/main/en/use-dify/build/agent.mdx"},{"name":"Workflow","summary":"Visual canvas of typed nodes.","url":"https://github.com/langgenius/dify#key-features"},{"name":"LLM node","summary":"Invokes a language model with prompt, context, structured-output schema, retries, fallback.","url":"https://github.com/langgenius/dify-docs/blob/main/en/use-dify/nodes/llm.mdx"},{"name":"Agent node / Agent Strategy","summary":"Autonomous tool-calling loop driven by Function Calling or ReAct.","maps_to_pattern":"react","url":"https://github.com/langgenius/dify-docs/blob/main/en/use-dify/nodes/agent.mdx"},{"name":"Knowledge Base (知识库)","summary":"Vectorised document corpus queried at runtime with hybrid search + rerank.","maps_to_pattern":"agentic-rag","url":"https://docs.dify.ai/en/learn-more/extended-reading/retrieval-augment"},{"name":"Tool / Plugin","summary":"External capability — built-in, custom HTTP, plugin-distributed, or MCP-server-backed.","maps_to_pattern":"tool-use","url":"https://github.com/langgenius/dify#key-features"},{"name":"Model provider","summary":"Pluggable LLM/embedding/rerank vendor; provider-agnostic.","url":"https://github.com/langgenius/dify#key-features"}],"pattern_composition":"flowchart TD\n  U[User input / chat / API / MCP client] --> APP{App type}\n  APP -->|Chatbot / Chatflow| LLM[LLM node]\n  APP -->|Agent app| AGENT[Agent loop]\n  APP -->|Workflow| WF[Workflow graph]\n  AGENT -->|strategy| FC[Function Calling]\n  AGENT -->|strategy| RE[ReAct: Thought->Action->Observation]\n  FC --> TL[Tool plugins / MCP tools / built-in]\n  RE --> TL\n  TL --> AGENT\n  WF --> LLM\n  WF --> KR[Knowledge Retrieval node]\n  WF --> AGENT\n  WF --> TL\n  KR --> KB[(Knowledge Base<br/>hybrid + semantic + rerank)]\n  KB --> LLM\n  LLM --> SO[Structured Output<br/>JSON Schema editor]\n  LLM -.retry+backoff.-> LLM\n  LLM -.fallback.-> LLM2[Alternative model / default / error route]\n  AGENT --> PUB[Publish as MCP Server]\n  PUB --> CD[Claude Desktop / Cursor]","members":[{"pattern":"agentic-rag","role":"first-class","note":"Knowledge Base resource + Knowledge Retrieval node + hybrid search + rerank + automatic citation tracking.","evidence":[{"type":"repo","title":"Dify README — RAG capabilities","url":"https://github.com/langgenius/dify","quote":"Extensive RAG capabilities that cover everything from document ingestion to retrieval, with out-of-box support for text extraction from PDFs, PPTs, and other common document formats.","accessed":"2026-05-20"},{"type":"doc","title":"Dify docs — hybrid retrieval + rerank","url":"https://github.com/langgenius/dify-docs/blob/main/en/use-dify/knowledge/integrate-knowledge-within-application.mdx","quote":"The Rerank model is an external scoring system that calculates the similarity score between the user's question and each candidate document provided, improving the results of semantic ranking","accessed":"2026-05-20"},{"type":"repo","title":"Dify README_CN — RAG (Mandarin)","url":"https://github.com/langgenius/dify/blob/main/docs/zh-CN/README.md","quote":"广泛的 RAG 功能，涵盖从文档摄入到检索的所有内容","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"fallback-chain","role":"first-class","note":"LLM node exposes 'fallback strategies' — alternative models, default values, or error routes when retries aren't sufficient.","evidence":[{"type":"doc","title":"Dify docs — LLM node retry behavior","url":"https://github.com/langgenius/dify-docs/blob/main/en/use-dify/nodes/llm.mdx","quote":"Configure retry behavior for failed LLM calls. Set maximum retry attempts, intervals between retries, and backoff multipliers.","accessed":"2026-05-20"},{"type":"doc","title":"Dify docs — fallback strategies","url":"https://github.com/langgenius/dify-docs/blob/main/en/use-dify/nodes/llm.mdx","quote":"Define fallback strategies like default values, error routing, or alternative models when retries aren't sufficient.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"Bidirectional — consume external MCP servers as tools, and publish any Dify app as an MCP endpoint.","evidence":[{"type":"doc","title":"Dify docs — MCP tool import","url":"https://github.com/langgenius/dify-docs/blob/main/en/use-dify/build/mcp.mdx","quote":"Connect external tools from MCP servers to your Dify apps. Instead of just built-in tools, you can use tools from the growing MCP ecosystem.","accessed":"2026-05-20"},{"type":"doc","title":"Dify docs — publish as MCP server","url":"https://github.com/langgenius/dify-docs/blob/main/en/use-dify/publish/publish-mcp.mdx","quote":"Dify now supports exposing your applications as MCP (Model Context Protocol) servers, enabling seamless integration with AI assistants like Claude Desktop and development environments like Cursor.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"ReAct is one of the two built-in Agent strategies, named literally in docs and the bilingual README.","evidence":[{"type":"doc","title":"Dify docs — ReAct strategy","url":"https://github.com/langgenius/dify-docs/blob/main/en/use-dify/nodes/agent.mdx","quote":"ReAct (Reason + Act): Uses structured prompts that guide the LLM through explicit reasoning steps. Follows a Thought → Action → Observation cycle for transparent decision-making.","accessed":"2026-05-20"},{"type":"repo","title":"Dify README — agents","url":"https://github.com/langgenius/dify","quote":"You can define agents based on LLM Function Calling or ReAct, and add pre-built or custom tools for the agent.","accessed":"2026-05-20"},{"type":"repo","title":"Dify README_CN — agents (Mandarin)","url":"https://github.com/langgenius/dify/blob/main/docs/zh-CN/README.md","quote":"您可以基于 LLM 函数调用或 ReAct 定义 Agent，并为 Agent 添加预构建或自定义工具","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"structured-output","role":"first-class","note":"Dify has a dedicated Structured Outputs feature with a JSON Schema editor in the LLM node; documentation lists consistent formatting, seamless DB/API integration, and JSON Schema as the constraint mechanism.","evidence":[{"type":"doc","title":"Dify docs — Structured Outputs overview","url":"https://github.com/langgenius/dify-docs/blob/main/versions/3-0-x/en/user-guide/workflow/structured-outputs.mdx","quote":"Dify's structured outputs ensures LLMs return data in predictable JSON formats, making the output easier to process and integrate into applications.","accessed":"2026-05-24"},{"type":"doc","title":"Dify docs — Structured Outputs implementation methods","url":"https://github.com/langgenius/dify-docs/blob/main/versions/3-0-x/en/user-guide/workflow/structured-outputs.mdx","quote":"Dify offers two ways to create structured outputs","accessed":"2026-05-24"},{"type":"doc","title":"Dify docs — Structured Outputs simplified development","url":"https://github.com/langgenius/dify-docs/blob/main/versions/3-0-x/en/user-guide/workflow/structured-outputs.mdx","quote":"Developers can skip writing complex validation code by letting JSON Schema handle output constraints.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Built-in + custom HTTP + plugin-distributed + MCP-backed tools; Agent nodes require tools and decide which to call.","evidence":[{"type":"repo","title":"Dify README — 50+ built-in tools","url":"https://github.com/langgenius/dify","quote":"Dify provides 50+ built-in tools for AI agents, such as Google Search, DALL·E, Stable Diffusion and WolframAlpha.","accessed":"2026-05-20"},{"type":"doc","title":"Dify docs — Agent node","url":"https://github.com/langgenius/dify-docs/blob/main/en/use-dify/nodes/agent.mdx","quote":"The Agent node gives your LLM autonomous control over tools, enabling it to iteratively decide which tools to use and when to use them.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"behavior-space-architecture","role":"supported","note":"Dify's Question Classifier node classifies each query and routes execution down a different branch, activating only the downstream subsystem (e.g. a specialised knowledge base or processing workflow) that the query's class needs.","evidence":[{"type":"doc","title":"Question Classifier - Dify Docs","url":"https://docs.dify.ai/en/use-dify/nodes/question-classifier","quote":"Each class becomes a potential output path that you can connect to different downstream nodes like specialized knowledge bases, response templates, or processing workflows.","accessed":"2026-06-14"}],"evidence_status":"full"},{"pattern":"iteration-node","role":"first-class","note":"The visual Iteration node takes an array input and runs the same sub-workflow once per element (sequential or parallel), exposing the current item and index as variables with explicit error handling.","evidence":[{"type":"doc","title":"Iteration - Dify Docs","url":"https://docs.dify.ai/en/use-dify/nodes/iteration","quote":"The node takes an array input and creates a sub-workflow that runs once for each array element. During each iteration, the current item and its index are available as variables that internal nodes can reference.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"mcp-bidirectional-bridge","role":"supported","note":"Dify consumes MCP tools and can publish apps as MCP servers.","evidence":[{"type":"doc","url":"https://github.com/langgenius/dify","quote":"Dify consumes MCP tools and can publish apps as MCP servers.","accessed":"2026-06-19"}],"evidence_status":"full"}],"anti_patterns_avoided":[{"pattern":"unbounded-loop","note":"Maximum Iterations cap on Agent node prevents infinite loops.","evidence":[{"type":"doc","title":"Dify docs — Maximum Iterations safety cap","url":"https://github.com/langgenius/dify-docs/blob/main/en/use-dify/nodes/agent.mdx","quote":"Maximum Iterations sets a safety limit to prevent infinite loops. Configure based on task complexity - simple tasks need 3-5 iterations, while complex research might require 10-15.","accessed":"2026-05-20"}]}],"instantiates":["production-rag","routing-and-fallback","structured-output-stack","planning-loops"],"alternatives":[{"composition":"coze","relation":"competes-with","note":"ByteDance's hosted no-code peer with the open-source Coze Studio edition; nearly identical conceptual surface."},{"composition":"fastgpt","relation":"competes-with","note":"Open-source Chinese visual RAG/agent builder; direct China-market peer."},{"composition":"bisheng","relation":"competes-with","note":"Open-source Chinese visual LLM app builder; direct China-market peer."},{"composition":"flowise","relation":"similar-shape","note":"Open-source visual LangChain-JS flow builder."},{"composition":"langflow","relation":"similar-shape","note":"Open-source visual LangChain-Python flow builder."},{"composition":"n8n","relation":"similar-shape","note":"General-purpose workflow automation with embedded AI Agent nodes."},{"composition":"appbuilder","relation":"competes-with","note":"Open-source low-code agent platform with the same RAG + tools + workflow + multi-agent surface; non-Baidu ecosystem."},{"composition":"relevance-ai","relation":"similar-shape","note":"Open-source LLM-app platform with comparable RAG + tools + agent shape."},{"composition":"stackai","relation":"similar-shape","note":"Open-source LLM-app platform; comparable workflow + agent + RAG shape."}],"references":[{"type":"repo","title":"langgenius/dify","url":"https://github.com/langgenius/dify","quote":"Dify is an open-source LLM app development platform. Its intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features … and more, letting you quickly go from prototype to production.","accessed":"2026-05-20"},{"type":"doc","title":"Dify docs","url":"https://docs.dify.ai","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-14","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","modified-apache-2","self-hostable","chinese-vendor","visual","rag","agent-platform"]},{"id":"fastgpt","name":"FastGPT","kind":"framework","category":"low-code-platform","build_surface":"low-code","vendor":"Sealos / Labring","language":"TypeScript (visual)","license":"FastGPT Open Source License (custom Apache-2.0 — multi-tenant SaaS resale forbidden, LOGO removal forbidden)","status":"active","url":"https://fastgpt.io/","repo":"https://github.com/labring/FastGPT","docs_url":"https://doc.fastgpt.io/","intent":"Provide an open-source enterprise AI productivity engine that assembles knowledge-grounded agents from a visual workflow canvas — knowledge base (RAG) + hybrid retrieval + plugin/agent nodes — with a bidirectional MCP surface and self-hostable Docker deployment.","description":"FastGPT is an open-source AI agent platform built by Labring/Sealos. The homepage calls it an 'Enterprise AI Productivity Engine' that lets you 'Build secure, controllable enterprise-grade AI Agents' and 'Build AI agents for your business like stacking blocks.' Its README states 'FastGPT is a knowledge-based platform built on the LLMs, offers a comprehensive suite of out-of-the-box capabilities such as data processing, RAG retrieval, and visual AI workflow orchestration.' RAG is the centre of gravity: documented features include '混合检索 & 重排' (hybrid retrieval and rerank), multi-knowledge-base reuse and mixing, and broad file-format ingestion (txt, md, html, pdf, docx, pptx, csv, xlsx). The platform exposes a '规划 Agent 模式' (Planning Agent mode), workflow and plugin workflows with basic RPA nodes, and '双向 MCP' (bidirectional MCP).","primary_use_cases":["enterprise knowledge-grounded chat (RAG over uploaded documents)","low-code workflow orchestration with plugin and RPA nodes","agent-style planning over knowledge bases and tools","bidirectional MCP — consume external MCP tools and expose FastGPT workflows as MCP","self-hosted on-prem deployment (FastGPT Open Source License)"],"agent_loop_shape":"A FastGPT app is a visual workflow on a Flow canvas. Conversation workflows and plugin workflows are first-class app types, both composed of typed nodes (LLM, knowledge-base search, tool/plugin, conditional/code, RPA). Knowledge bases are uploaded, automatically structured and queried via hybrid retrieval with reranking; multiple bases can be reused and mixed inside one workflow. A '规划 Agent 模式' (Planning Agent mode) places an LLM-driven planner over tools and knowledge nodes. Tools and external integrations are pluggable via bidirectional MCP — FastGPT can call external MCP servers and publish workflows as MCP endpoints.","key_concepts":[{"name":"App (workflow)","summary":"Visual workflow canvas — conversation workflow or plugin workflow — composing LLM, knowledge, tool/plugin and RPA nodes.","maps_to_pattern":"visual-workflow-graph","url":"https://github.com/labring/FastGPT"},{"name":"Knowledge Base (知识库)","summary":"Uploaded documents auto-structured for retrieval; supports multi-base reuse, mixing, and broad file formats.","maps_to_pattern":"agentic-rag","url":"https://github.com/labring/FastGPT"},{"name":"Hybrid retrieval + rerank (混合检索 & 重排)","summary":"Combined full-text + semantic search followed by a reranking model over candidate chunks.","maps_to_pattern":"hybrid-search","url":"https://github.com/labring/FastGPT"},{"name":"Planning Agent mode (规划 Agent 模式)","summary":"Agent-style loop where an LLM plans a sequence of calls over tools and knowledge nodes.","url":"https://github.com/labring/FastGPT"},{"name":"Plugin / RPA workflow","summary":"Composable plugin workflows containing basic RPA nodes; published as reusable plugins for other workflows.","maps_to_pattern":"tool-use","url":"https://github.com/labring/FastGPT"},{"name":"Bidirectional MCP (双向 MCP)","summary":"FastGPT consumes external MCP servers as tools and publishes its own workflows as MCP endpoints.","maps_to_pattern":"mcp","url":"https://github.com/labring/FastGPT"}],"pattern_composition":"flowchart TD\n  USR[User / API caller] --> APP[FastGPT app]\n  APP --> WF[Workflow canvas<br/>conversation / plugin workflow]\n  WF --> LLM[LLM node]\n  WF --> KBN[Knowledge base node]\n  KBN --> KB[(Knowledge Base — 知识库<br/>txt md html pdf docx pptx csv xlsx)]\n  KB --> HYB[Hybrid retrieval + rerank<br/>混合检索 & 重排]\n  HYB --> LLM\n  WF --> AGT[Planning Agent mode]\n  AGT --> TOOL[Plugin / RPA / MCP tools]\n  WF -.publish.-> MCP_OUT[(MCP server)]\n  MCP_IN[(External MCP server)] -.import.-> TOOL\n  LLM --> OUT[Answer]","members":[{"pattern":"agentic-rag","role":"first-class","note":"Knowledge Base is the headline product surface; the homepage frames it as 'reject hallucinations with an accurate enterprise knowledge base' and the README documents broad file-format ingestion plus multi-base reuse.","evidence":[{"type":"doc","title":"FastGPT homepage — knowledge base framing","url":"https://fastgpt.io/","quote":"Reject hallucinations with an accurate enterprise knowledge base","accessed":"2026-05-20"},{"type":"repo","title":"FastGPT README — RAG module","url":"https://github.com/labring/FastGPT","quote":"RAG 模块热插拔","accessed":"2026-05-20"},{"type":"repo","title":"FastGPT README — knowledge base file formats [translated]","url":"https://github.com/labring/FastGPT","quote":"支持 txt，md，html，pdf，docx，pptx，csv，xlsx","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"hybrid-search","role":"first-class","note":"FastGPT README and homepage both name hybrid retrieval + rerank as a primary capability of the knowledge layer.","evidence":[{"type":"repo","title":"FastGPT README — hybrid retrieval [translated]","url":"https://github.com/labring/FastGPT","quote":"混合检索 & 重排","accessed":"2026-05-20"},{"type":"doc","title":"FastGPT homepage — hybrid retrieval","url":"https://fastgpt.io/","quote":"Hybrid retrieval avoids hallucinations; data is auto-cleansed and kept live","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Plugin workflows are first-class app type; FastGPT supports bidirectional MCP and Planning Agent mode that calls tools. README explicitly lists '对话工作流、插件工作流' alongside RPA nodes.","evidence":[{"type":"repo","title":"FastGPT README — workflow + plugin + RPA [translated]","url":"https://github.com/labring/FastGPT","quote":"对话工作流、插件工作流，包含基础的 RPA 节点","accessed":"2026-05-20"},{"type":"repo","title":"FastGPT README — bidirectional MCP [translated]","url":"https://github.com/labring/FastGPT","quote":"双向 MCP","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"FastGPT v4.9.6+ ships two complementary MCP surfaces: MCP Server exposes multiple FastGPT applications via MCP for external consumption (SSE transport); MCP Tools batch-creates tools by parsing an external MCP SSE URL — bidirectional by design.","evidence":[{"type":"doc","title":"FastGPT docs — MCP Server (publish workflows)","url":"https://doc.fastgpt.io/en/introduction/guide/dashboard/mcp_server","quote":"FastGPT's MCP Server feature lets you select `multiple` applications built on FastGPT and expose them via MCP protocol for external consumption.","accessed":"2026-05-24"},{"type":"doc","title":"FastGPT docs — MCP Server SSE transport","url":"https://doc.fastgpt.io/en/introduction/guide/dashboard/mcp_server","quote":"Currently, FastGPT's MCP Server uses the SSE transport protocol, with plans to migrate to `HTTP Streamable` in the future.","accessed":"2026-05-24"},{"type":"doc","title":"FastGPT docs — MCP Tools (consume external MCP)","url":"https://doc.fastgpt.io/en/docs/introduction/guide/dashboard/mcp_tools","quote":"Starting from FastGPT v4.9.6, a new application type called MCP Tools has been added. It lets you provide an MCP SSE URL to batch-create tools that models can easily call.","accessed":"2026-05-24"},{"type":"repo","title":"FastGPT README — bidirectional MCP [translated]","url":"https://github.com/labring/FastGPT","quote":"双向 MCP","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"visual-workflow-graph","role":"first-class","note":"Homepage and README both describe a visual workflow canvas with rich node types; the workflow IS the agent in FastGPT.","evidence":[{"type":"doc","title":"FastGPT homepage — visual workflows","url":"https://fastgpt.io/","quote":"Visual workflows with a rich API ecosystem for seamless integration","accessed":"2026-05-20"},{"type":"repo","title":"FastGPT README — Flow visualization [translated]","url":"https://github.com/labring/FastGPT","quote":"Workflow orchestration through Flow visualization, allowing you to achieve complex application scenarios","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"fallback-chain","role":"limited","note":"Original stub claimed fallback-chain as first-class. Re-verified 2026-05-24: neither the FastGPT README nor doc.fastgpt.io documents a named LLM fallback-chain, alternative-model routing, or automatic failover primitive at workflow-node level. Honest gap retained — framework marked needs-verification.","evidence":[{"type":"doc","title":"FastGPT homepage — model integration","url":"https://fastgpt.io/","quote":"Integrate any model, with debugging and auditing for compliance","accessed":"2026-05-20"}],"evidence_status":"limited"}],"alternatives":[{"composition":"dify","relation":"competes-with","note":"LangGenius's open-source low-code agent platform — wider surface (agents, workflows, RAG, MCP) than FastGPT's RAG-first emphasis."},{"composition":"coze","relation":"competes-with","note":"ByteDance hosted no-code peer; comparable Chinese-market positioning."},{"composition":"bisheng","relation":"competes-with","note":"Open-source Chinese visual LLM app builder; enterprise document focus."},{"composition":"appbuilder","relation":"competes-with","note":"Open-source Chinese visual RAG/agent builder."}],"references":[{"type":"repo","title":"labring/FastGPT README","url":"https://github.com/labring/FastGPT","quote":"FastGPT is a knowledge-based platform built on the LLMs, offers a comprehensive suite of out-of-the-box capabilities such as data processing, RAG retrieval, and visual AI workflow orchestration","accessed":"2026-05-20"},{"type":"doc","title":"FastGPT homepage — Enterprise AI Productivity Engine","url":"https://fastgpt.io/","quote":"Production-grade AI Agents - One platform for enterprise AI delivery","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","custom-apache-2","self-hostable","chinese-vendor","rag","visual","mcp"]},{"id":"flowise","name":"Flowise","kind":"framework","category":"low-code-platform","build_surface":"low-code","vendor":"FlowiseAI","language":"TypeScript (visual)","license":"Apache-2.0","status":"active","url":"https://flowiseai.com/","repo":"https://github.com/FlowiseAI/Flowise","docs_url":"https://docs.flowiseai.com","intent":"Provide an open-source TypeScript visual builder — 'Build AI Agents, Visually' — that assembles LangChain-JS-style chains, single agents and multi-agent supervisor/worker systems on a drag-and-drop node canvas, with first-class document stores for RAG and a pluggable tool ecosystem.","description":"Flowise is FlowiseAI's open-source (Apache-2.0) visual builder for LLM applications. Its GitHub tagline is 'Build AI Agents, Visually.' The documentation distinguishes three product surfaces: Chatflow ('designed to build single-agent systems, chatbots and simple LLM flows'), Assistant ('the most beginner-friendly way of creating an AI Agent'), and Agentflow, which is 'the superset of Chatflow & Assistant. It can be used to create chat assistant, single-agent system, multi-agent systems, and complex workflow orchestration.' AgentFlow V2 'represents a significant architectural evolution, introducing a new paradigm in Flowise that focuses on explicit workflow orchestration,' adding typed Agent nodes, Document Store retrieval, conditional and LLM-based router nodes, and an explicit Supervisor/Worker multi-agent shape in V1.","primary_use_cases":["no-code/low-code building of chatbots and single-agent RAG apps","multi-agent supervisor/worker systems on a visual canvas","RAG via centralised Document Stores with multiple vector store backends","LLM-driven conditional routing inside an AgentFlow","self-hosted on-prem deployment"],"agent_loop_shape":"A Flowise project is one of three flow types. A Chatflow is a single-agent LangChain-style chain. An Assistant follows instructions, uses tools and retrieves from uploaded files. An Agentflow (V2) is an explicit workflow graph of typed nodes — Agent (an LLM with tools / Document Stores), Condition (deterministic branch), LLM Router (semantic branch via 'Scenarios' and natural-language 'Instructions'), Tool, and Retrieval. AgentFlow V1's Multi-Agent shape composes a Supervisor agent that 'analyzes user requests, decomposes them into sub-tasks, and assigns these to specialized worker agents' over connected Workers, sequenced one task at a time. Document Stores centralise ingestion, splitting and vector-store upsert.","key_concepts":[{"name":"Chatflow","summary":"Single-agent / chatbot / simple LLM flow on a visual canvas.","url":"https://docs.flowiseai.com/"},{"name":"Assistant","summary":"Beginner-friendly AI assistant that follows instructions, uses tools and retrieves from uploaded files.","url":"https://docs.flowiseai.com/"},{"name":"Agentflow (V2)","summary":"Superset of Chatflow + Assistant supporting single-agent, multi-agent and explicit workflow orchestration.","maps_to_pattern":"visual-workflow-graph","url":"https://docs.flowiseai.com/using-flowise/agentflowv2"},{"name":"Agent node","summary":"Autonomous AI entity that reasons, plans and uses Tools and Document Stores.","maps_to_pattern":"tool-use","url":"https://docs.flowiseai.com/using-flowise/agentflowv2"},{"name":"Document Store","summary":"Centralised data layer for upload, split, prepare and vector-store upsert.","maps_to_pattern":"agentic-rag","url":"https://docs.flowiseai.com/using-flowise/document-stores"},{"name":"Multi-Agent (Supervisor / Worker)","summary":"AgentFlow V1 multi-agent shape: Supervisor decomposes and delegates tasks; Workers execute specialised functions.","maps_to_pattern":"supervisor","url":"https://docs.flowiseai.com/using-flowise/agentflowv1/multi-agents"}],"pattern_composition":"flowchart TD\n  U[User input] --> KIND{Flow type}\n  KIND -->|Chatflow| CHAT[Single-agent chain]\n  KIND -->|Assistant| ASS[Assistant — instructions + tools + files]\n  KIND -->|Agentflow V2| WF[Explicit workflow graph]\n  WF --> AGT[Agent node<br/>reasoning + tools + retrieval]\n  WF --> COND[Condition node]\n  WF --> ROUTE[LLM Router — Scenarios + Instructions]\n  WF --> TOOL[Tool node]\n  AGT --> DS[(Document Stores<br/>ingest + split + upsert)]\n  DS --> VS[(Vector store)]\n  KIND -->|Agentflow V1 Multi-Agent| SUP[Supervisor]\n  SUP -.delegate.-> W1[Worker 1]\n  SUP -.delegate.-> W2[Worker 2]\n  W1 -.return.-> SUP\n  W2 -.return.-> SUP\n  AGT --> OUT[Answer]\n  SUP --> OUT","members":[{"pattern":"agentic-rag","role":"first-class","note":"Document Stores are a centralised RAG layer; AgentFlow V2 Agent nodes query Document Stores by semantic similarity, and Document Stores explicitly support upload, split, vector-store upsert and similarity search.","evidence":[{"type":"doc","title":"Flowise docs — Document Stores","url":"https://docs.flowiseai.com/using-flowise/document-stores","quote":"Document Stores offer a versatile approach to data management, enabling you to upload, split, and prepare your dataset and upsert it in a single location.","accessed":"2026-05-20"},{"type":"doc","title":"Flowise docs — Document Stores similarity search","url":"https://docs.flowiseai.com/using-flowise/document-stores","quote":"The LLM executes a similarity search against the Vector Store to retrieve the most semantically relevant chunks.","accessed":"2026-05-20"},{"type":"doc","title":"Flowise docs — AgentFlow V2 Document Store retrieval","url":"https://docs.flowiseai.com/using-flowise/agentflowv2","quote":"Queries one or more specified Document Stores, fetching relevant document chunks based on semantic similarity","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"AgentFlow V2 Agent nodes are authorised to use a set of Flowise Tools or MCP tools; the LLM decides a sequence of actions and which tools to invoke.","evidence":[{"type":"doc","title":"Flowise docs — AgentFlow V2 Tools","url":"https://docs.flowiseai.com/using-flowise/agentflowv2","quote":"Specify which pre-defined Flowise Tools the agent is authorized to use to achieve its goals","accessed":"2026-05-20"},{"type":"doc","title":"Flowise docs — AgentFlow V2 Agent tool selection","url":"https://docs.flowiseai.com/using-flowise/agentflowv2","quote":"The LLM dynamically decides a sequence of actions. Based on the user's goal, it can choose to use available Tools or query Document Stores","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"supervisor","role":"first-class","note":"AgentFlow V1 Multi-Agent ships an explicit Supervisor/Worker abstraction — the Supervisor orchestrates workflow, decomposes requests, and delegates one sub-task at a time to specialised Workers.","evidence":[{"type":"doc","title":"Flowise docs — Multi-Agent Supervisor","url":"https://docs.flowiseai.com/using-flowise/agentflowv1/multi-agents","quote":"The Supervisor acts as the system's orchestrator, overseeing the entire workflow.","accessed":"2026-05-20"},{"type":"doc","title":"Flowise docs — Supervisor delegation","url":"https://docs.flowiseai.com/using-flowise/agentflowv1/multi-agents","quote":"It analyzes user requests, decomposes them into sub-tasks, and assigns these to specialized worker agents.","accessed":"2026-05-20"},{"type":"doc","title":"Flowise docs — AgentFlow V2 Supervisor","url":"https://docs.flowiseai.com/using-flowise/agentflowv2","quote":"A Supervisor agent can formulate and delegate tasks to multiple Worker agents, with outputs from the Worker agents subsequently returned","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"visual-workflow-graph","role":"first-class","note":"AgentFlow V2 explicitly frames itself as a paradigm shift toward 'explicit workflow orchestration' on a visual node canvas; Agentflow is documented as superset of Chatflow + Assistant.","evidence":[{"type":"doc","title":"Flowise docs — AgentFlow V2 paradigm","url":"https://docs.flowiseai.com/using-flowise/agentflowv2","quote":"AgentFlow V2 represents a significant architectural evolution, introducing a new paradigm in Flowise that focuses on explicit workflow orchestration","accessed":"2026-05-20"},{"type":"doc","title":"Flowise docs — Agentflow","url":"https://docs.flowiseai.com/","quote":"Agentflow is the superset of Chatflow & Assistant. It can be used to create chat assistant, single-agent system, multi-agent systems, and complex workflow orchestration.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"Flowise ships dedicated ReAct Agent LLM and ReAct Agent Chat integrations under LangChain agents, each explicitly using 'Reasoning and Acting' logic to decide what action to take.","evidence":[{"type":"doc","title":"Flowise docs — ReAct Agent LLM","url":"https://docs.flowiseai.com/integrations/langchain/agents/react-agent-llm","quote":"Agent that uses the ReAct (Reasoning and Acting) logic to decide what action to take, optimized to be used with Non Chat Models.","accessed":"2026-05-24"},{"type":"doc","title":"Flowise docs — ReAct Agent Chat","url":"https://docs.flowiseai.com/integrations/langchain/agents/react-agent-chat","quote":"Agent that uses the ReAct (Reasoning and Acting) logic to decide what action to take, optimized to be used with Chat Models.","accessed":"2026-05-24"},{"type":"doc","title":"Flowise docs — Agents as reasoning engine","url":"https://docs.flowiseai.com/integrations/langchain/agents","quote":"Agents are systems that use an LLM as a reasoning engine to determine which actions to take and what the inputs to those actions should be.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"structured-output","role":"first-class","note":"AgentFlow V2's LLM Node provides a JSON Structured Output parameter that instructs the LLM to format output according to a specified JSON schema. Standalone Structured Output Parser node also exists in LangChain integrations.","evidence":[{"type":"doc","title":"Flowise docs — AgentFlow V2 JSON Structured Output","url":"https://docs.flowiseai.com/using-flowise/agentflowv2","quote":"Instructs the LLM to format its output according to a specific JSON schema — including keys, data types, and descriptions","accessed":"2026-05-24"},{"type":"doc","title":"Flowise docs — Output Parsers landing","url":"https://docs.flowiseai.com/integrations/langchain/output-parsers","quote":"Output Parser nodes are responsible for taking the output of a model and transforming it to a more suitable format for downstream tasks.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"fallback-chain","role":"limited","note":"Original stub claimed fallback-chain as first-class. Re-verified 2026-05-24: the Flowise docs path for multi-model fallback returns a 404, no AgentFlow V2 page names a fallback-chain or automatic alternative-model primitive, and search results surface only deployment-side (database) fallback and workflow Loop-node retries. Honest gap retained.","evidence":[],"evidence_status":"none"}],"alternatives":[{"composition":"langflow","relation":"similar-shape","note":"Python equivalent — visual LangChain builder."},{"composition":"n8n","relation":"similar-shape","note":"General-purpose workflow automation with embedded AI Agent nodes; broader workflow scope, less LLM-app-focused."},{"composition":"dify","relation":"similar-shape","note":"Open-source LLM-app platform with deeper RAG product surface."},{"composition":"coze","relation":"similar-shape","note":"Open-source visual LangChain/LLM flow builder."},{"composition":"sim","relation":"similar-shape","note":"Both target drag-and-drop construction of LLM/agent flows."}],"references":[{"type":"repo","title":"FlowiseAI/Flowise README","url":"https://github.com/FlowiseAI/Flowise","quote":"Source code in this repository is made available under the Apache License Version 2.0","accessed":"2026-05-20"},{"type":"doc","title":"Flowise docs landing","url":"https://docs.flowiseai.com/","quote":"Agentflow is the superset of Chatflow & Assistant. It can be used to create chat assistant, single-agent system, multi-agent systems, and complex workflow orchestration.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","apache-2","self-hostable","visual","langchain-js","agent-platform"]},{"id":"langflow","name":"Langflow","kind":"framework","category":"low-code-platform","build_surface":"low-code","vendor":"Langflow / DataStax (IBM)","language":"Python (visual)","license":"MIT","status":"active","url":"https://www.langflow.org/","repo":"https://github.com/langflow-ai/langflow","docs_url":"https://docs.langflow.org","intent":"Provide a Python-based, MIT-licensed visual builder for AI agents and workflows where components plug into a flow on a drag-and-drop canvas, with first-class agent-tool wiring, multi-agent orchestration, and one-click MCP-server publication.","description":"Langflow is an open-source (MIT) visual platform for building AI agents and workflows. Its README states: 'Langflow is a powerful platform for building and deploying AI-powered agents and workflows.' Flows are 'functional representations of application workflows' composed of components where 'each component is a single step in the workflow.' The Agent component 'is critical for building agent flows' and 'provides everything you need to create an agent, including multiple Large Language Model (LLM) providers, tool calling, and custom instructions.' Multi-agent flows are realised by setting an Agent component to Tool Mode and attaching it as a tool to another Agent. Langflow flows can be deployed as MCP servers, turning them into tools for MCP clients.","primary_use_cases":["visual building of agent flows with multiple LLM providers and tool calling","multi-agent orchestration by attaching agents as tools to other agents","RAG flows over vector store components","deploying flows as MCP servers consumed by external MCP clients","deploying flows as APIs or exporting them to Python apps"],"agent_loop_shape":"A Langflow flow is a directed graph of components on a visual canvas. The Agent component contains an LLM, tools, and custom instructions, and 'uses LLMs as a reasoning engine to process input, determine which actions to take to address the query, and then generate a response.' Tools are attached by connecting any component's Tool output to the Agent's Tools input. Multi-agent shape: any Agent component can be flipped into Tool Mode and used as a tool by another Agent — Langflow's documented mechanism for multi-agent flows. Flows can also be Run-Flow-attached to an agent or published as MCP servers; observability is wired through LangSmith / LangFuse.","key_concepts":[{"name":"Flow","summary":"Functional representation of an application workflow on a visual canvas.","maps_to_pattern":"visual-workflow-graph","url":"https://docs.langflow.org/concepts-overview"},{"name":"Component","summary":"A single step in the workflow; configurable and connectable to other components.","url":"https://docs.langflow.org/concepts-overview"},{"name":"Agent component","summary":"Reasoning + tool-calling unit with LLM, tools and custom instructions.","maps_to_pattern":"tool-use","url":"https://docs.langflow.org/agents"},{"name":"Tool / Tool Mode","summary":"Any component (including another Agent) can be used as a tool by an Agent.","url":"https://docs.langflow.org/agents-tools"},{"name":"MCP server publication","summary":"Deploy a flow as an MCP server so MCP clients can call it as a tool.","maps_to_pattern":"mcp","url":"https://github.com/langflow-ai/langflow"}],"pattern_composition":"flowchart TD\n  USR[Input / API / MCP client] --> FLOW[Flow on visual canvas]\n  FLOW --> AGT[Agent component<br/>LLM + tools + instructions]\n  AGT --> TOOLS[Tools — attached components]\n  TOOLS --> AGT2[Another Agent in Tool Mode<br/>= multi-agent]\n  TOOLS --> VS[(Vector store / retrieval)]\n  TOOLS --> RF[Run Flow component<br/>= sub-flow as tool]\n  FLOW -.deploy.-> API[(API endpoint)]\n  FLOW -.deploy.-> MCP[(MCP server)]\n  MCP -.consume.-> EXT[External MCP client<br/>Claude Desktop / Cursor]\n  AGT --> OBS[(LangSmith / LangFuse)]\n  AGT --> OUT[Answer]","members":[{"pattern":"agentic-rag","role":"first-class","note":"Vector-store and embedding components are documented as a 'common' shape in the flow alongside language model components; multi-agent orchestration capability explicitly includes 'conversation management and retrieval.'","evidence":[{"type":"repo","title":"Langflow README — multi-agent orchestration with retrieval","url":"https://github.com/langflow-ai/langflow","quote":"Multi-agent orchestration with conversation management and retrieval.","accessed":"2026-05-20"},{"type":"doc","title":"Langflow docs — language + vector + embedding components","url":"https://docs.langflow.org/components-models","quote":"Vector store, language model, and embedding model components are often used together in the same flow.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Agent component integrates 'tool calling' and 'custom instructions' as core capability; any Langflow component can be attached as a tool via the Agent's Tools input.","evidence":[{"type":"doc","title":"Langflow docs — Agent component","url":"https://docs.langflow.org/agents","quote":"The Agent component is critical for building agent flows. This component provides everything you need to create an agent, including multiple Large Language Model (LLM) providers, tool calling, and custom instructions.","accessed":"2026-05-20"},{"type":"doc","title":"Langflow docs — attaching tools","url":"https://docs.langflow.org/agents-tools","quote":"To attach a tool to an agent, you connect any component's Tool output to the Agent component's Tools input.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"supervisor","role":"supported","note":"Multi-agent in Langflow is realised by setting another Agent component to Tool Mode and attaching it as a tool. README also references 'Multi-agent orchestration.' Not framed as a named Supervisor abstraction, so downgraded from first-class to supported.","evidence":[{"type":"doc","title":"Langflow docs — multi-agent via Tool Mode","url":"https://docs.langflow.org/agents-tools","quote":"To create multi-agent flows, you can set another Agent component to Tool Mode, and then attach that agent as a tool.","accessed":"2026-05-20"},{"type":"repo","title":"Langflow README — multi-agent orchestration","url":"https://github.com/langflow-ai/langflow","quote":"Multi-agent orchestration with conversation management and retrieval.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"Langflow flows can be deployed as MCP servers and consumed as tools by MCP clients (e.g. Claude Desktop, Cursor). MCP Tools component also exists for the reverse direction.","evidence":[{"type":"repo","title":"Langflow README — MCP server deployment","url":"https://github.com/langflow-ai/langflow","quote":"Deploy as an MCP server and turn your flows into tools for MCP clients.","accessed":"2026-05-20"},{"type":"doc","title":"Langflow docs — connect MCP servers as tools","url":"https://docs.langflow.org/agents-tools","quote":"You can connect any Langflow component as a tool, including other Agent components and MCP servers through the MCP Tools component.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"visual-workflow-graph","role":"first-class","note":"Flows are explicitly 'functional representations of application workflows' composed of connected components on a visual canvas.","evidence":[{"type":"doc","title":"Langflow docs — Flow definition","url":"https://docs.langflow.org/concepts-overview","quote":"Flows, which are functional representations of application workflows.","accessed":"2026-05-20"},{"type":"doc","title":"Langflow docs — Component definition","url":"https://docs.langflow.org/concepts-overview","quote":"Each component is a single step in the workflow.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"limited","note":"Original stub claimed ReAct as first-class. Langflow Agent component docs describe an LLM-as-reasoning-engine loop but do not name ReAct explicitly. Downgraded.","evidence":[{"type":"doc","title":"Langflow docs — agent reasoning loop","url":"https://docs.langflow.org/agents","quote":"Agents use LLMs as a reasoning engine to process input, determine which actions to take to address the query, and then generate a response.","accessed":"2026-05-20"}],"evidence_status":"limited"},{"pattern":"structured-output","role":"first-class","note":"Langflow ships a dedicated Structured Output component that uses an LLM plus an output schema to transform input into JSON or Table data, with field types str/int/float/bool/dict.","evidence":[{"type":"doc","title":"Langflow docs — Structured Output overview","url":"https://docs.langflow.org/structured-output","quote":"The Structured Output component uses an LLM to transform any input into structured data (JSON or Table) using natural language formatting instructions and an output schema definition.","accessed":"2026-05-24"},{"type":"doc","title":"Langflow docs — Structured Output extraction","url":"https://docs.langflow.org/structured-output","quote":"you can specify the information you want to extract, even if that data isn't explicitly labeled or an exact keyword match.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"fallback-chain","role":"limited","note":"Original stub claimed fallback-chain as first-class. Langflow language-model docs mention provider/model alternatives via Bundles but do not document a named fallback-chain primitive. Verified 2026-05-24 against components-models page: no error-handling/fallback/retry discussion present. Honest gap retained.","evidence":[{"type":"doc","title":"Langflow docs — provider alternatives","url":"https://docs.langflow.org/components-models","quote":"If your provider or model isn't supported by the Language Model core component, additional language model components are available in Bundles.","accessed":"2026-05-20"}],"evidence_status":"limited"}],"alternatives":[{"composition":"flowise","relation":"similar-shape","note":"TypeScript equivalent — visual LangChain-JS builder."},{"composition":"n8n","relation":"similar-shape","note":"Workflow automation with embedded AI Agent nodes; broader scope than Langflow."},{"composition":"dify","relation":"similar-shape","note":"Open-source LLM-app platform with deeper RAG product surface."},{"composition":"coze","relation":"similar-shape","note":"Open-source visual LangChain flow builder (DataStax-owned)."},{"composition":"sim","relation":"similar-shape","note":"Both are visual, node-based agent-workflow builders."}],"references":[{"type":"repo","title":"langflow-ai/langflow README","url":"https://github.com/langflow-ai/langflow","quote":"Langflow is a powerful platform for building and deploying AI-powered agents and workflows.","accessed":"2026-05-20"},{"type":"doc","title":"Langflow docs — concepts overview","url":"https://docs.langflow.org/concepts-overview","quote":"Flows, which are functional representations of application workflows.","accessed":"2026-05-20"},{"type":"repo","title":"langflow-ai/langflow LICENSE","url":"https://github.com/langflow-ai/langflow/blob/main/LICENSE","quote":"MIT License","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","mit","self-hostable","visual","python","datastax","agent-platform","mcp"]},{"id":"n8n","name":"n8n","kind":"framework","category":"low-code-platform","build_surface":"low-code","vendor":"n8n GmbH (Berlin, Germany)","language":"TypeScript","license":"source-available (Sustainable Use License)","status":"active","first_released":"2019","url":"https://n8n.io","repo":"https://github.com/n8n-io/n8n","docs_url":"https://docs.n8n.io","intent":"Provide a fair-code, source-available workflow automation platform that builds event-, schedule-, or webhook-triggered workflows on a visual node canvas, with first-class AI Agent and LangChain-JS nodes for embedding LLM reasoning, RAG, and tool use inside automations.","description":"n8n is the fair-code workflow automation platform built by n8n GmbH in Berlin. Its README states: 'n8n is a workflow automation platform that gives technical teams the flexibility of code with the speed of no-code. With 400+ integrations, native AI capabilities, and a fair-code license, n8n lets you build powerful automations while maintaining full control over your data and deployments.' AI workflows are built around the AI Agent node, which the docs call 'the core of adding AI to your workflows', and the framework's AI capability is delivered via a documented LangChain-JS integration ('This feature is n8n's implementation of LangChain's JavaScript framework').","primary_use_cases":["visual workflow automation with embedded AI Agent nodes","event-, schedule-, and webhook-triggered AI workflows","RAG against multiple vector stores","exposing workflows as MCP tools to external clients"],"agent_loop_shape":"Workflows are directed graphs of typed nodes laid out in a visual editor. Each workflow starts with a trigger node (Chat, Webhook, Schedule, Form, Error, MCP Server) and pushes typed items down connections to subsequent nodes. The AI Agent root node is a cluster node requiring attached sub-nodes for chat model, optional memory, optional tools, and optional output parser. As of v1.82.0 the AI Agent always behaves as a Tools Agent (the prior 'agent type' setting was removed). Tools sub-nodes attached to the agent are looped over internally; the surrounding workflow handles routing, retries, branching, persistence.","key_concepts":[{"name":"Workflow","summary":"Visual graph of nodes describing the automation; the unit of execution.","url":"https://docs.n8n.io/workflows/"},{"name":"Trigger node","summary":"Node that starts a workflow on an event, schedule, chat message, form submit, webhook, or upstream error.","maps_to_pattern":"event-driven-agent","url":"https://docs.n8n.io/integrations/builtin/core-nodes/n8n-nodes-base.scheduletrigger/"},{"name":"AI Agent node","summary":"Root cluster node implementing the LangChain Tools-Agent loop; requires a chat-model sub-node and at least one tool sub-node.","maps_to_pattern":"react","url":"https://docs.n8n.io/integrations/builtin/cluster-nodes/root-nodes/n8n-nodes-langchain.agent/"},{"name":"Chain node","summary":"Root cluster node for non-agent LangChain chains (Basic LLM, Retrieval Q&A, Summarization, Text Classifier, Sentiment).","url":"https://docs.n8n.io/advanced-ai/langchain/langchain-n8n/"},{"name":"Tool sub-node","summary":"Sub-node attached to an AI Agent providing a callable tool (Calculator, HTTP Request, Custom Code, Workflow Tool, Vector Store Q&A, etc.).","maps_to_pattern":"tool-use","url":"https://docs.n8n.io/integrations/builtin/cluster-nodes/sub-nodes/"},{"name":"MCP Server Trigger","summary":"Trigger node that exposes n8n workflows as MCP tools to external clients over SSE or streamable HTTP.","maps_to_pattern":"mcp","url":"https://docs.n8n.io/integrations/builtin/core-nodes/n8n-nodes-langchain.mcptrigger/"}],"pattern_composition":"flowchart TD\n  T[Trigger node<br/>schedule/webhook/chat/form] --> A[AI Agent root node]\n  M[(Chat model sub-node)] --> A\n  MEM[(Memory sub-node)] --> A\n  TL1[Tool sub-nodes<br/>Calculator/HTTP/Workflow/VS Q&A] --> A\n  P[Output parser sub-node] --> A\n  A -->|tool calls loop| TL1\n  A --> HITL{Human-in-the-loop<br/>tool requires approval?}\n  HITL -- approve --> EXEC[Execute tool action]\n  HITL -- deny --> AGAIN[Return to agent]\n  EXEC --> OUT[Downstream workflow nodes]\n  A --> OUT","members":[{"pattern":"agentic-rag","role":"first-class","note":"RAG is documented end-to-end via a vector store as a tool attached to the AI Agent; vector store integrations include Simple Vector Store, PGVector, Pinecone, Qdrant, Supabase, Zep.","evidence":[{"type":"doc","title":"n8n docs — RAG in n8n","url":"https://docs.n8n.io/advanced-ai/rag-in-n8n/","quote":"Retrieval-Augmented Generation (RAG) is a technique that improves AI responses by combining language models with external data sources.","accessed":"2026-05-20"},{"type":"doc","title":"n8n docs — RAG query strategies","url":"https://docs.n8n.io/advanced-ai/rag-in-n8n/","quote":"You can query the data in two main ways: using an agent or directly through a node.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"approval-queue","role":"first-class","note":"Human-in-the-Loop middleware can require approval per tool; workflow pauses on tool call and waits for Approve/Deny on Slack/Telegram/n8n Chat.","evidence":[{"type":"doc","title":"n8n docs — Human-in-the-loop tools","url":"https://docs.n8n.io/advanced-ai/human-in-the-loop-tools/","quote":"You can require human approval before an AI Agent executes a specific tool. When a tool requires human review, the workflow pauses and waits for a person to take one of the following actions: Approve... Deny...","accessed":"2026-05-20"},{"type":"doc","title":"n8n docs — HITL granularity","url":"https://docs.n8n.io/advanced-ai/human-in-the-loop-tools/","quote":"HITL can be applied to all tools connected to an AI Agent node, or just selected individual tools, offering more precise control than general output gating.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"event-driven-agent","role":"first-class","note":"Workflows start at Trigger nodes — Webhook, Chat, Form, RSS, SSE, MCP Server, Error, and many app-specific triggers.","evidence":[{"type":"doc","title":"n8n docs — Webhook trigger","url":"https://docs.n8n.io/integrations/builtin/core-nodes/n8n-nodes-base.webhook/","quote":"Use the Webhook node to create webhooks, which can receive data from apps and services when an event occurs. It's a trigger node, which means it can start an n8n workflow.","accessed":"2026-05-20"},{"type":"doc","title":"n8n docs — Trigger nodes","url":"https://docs.n8n.io/advanced-ai/intro-tutorial/","quote":"Every workflow needs somewhere to start. In n8n these are called 'trigger nodes'.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"fallback-chain","role":"supported","note":"Two indirect mechanisms: the Auto-fixing Output Parser wraps another parser and falls back to a recovery LLM if the first fails; per-workflow Error Trigger runs an error workflow when an execution fails. Downgraded from first-class because upstream does not name 'fallback chain' explicitly.","evidence":[{"type":"doc","title":"n8n docs — Auto-fixing Output Parser","url":"https://docs.n8n.io/integrations/builtin/cluster-nodes/sub-nodes/n8n-nodes-langchain.outputparserautofixing/","quote":"The Auto-fixing Output Parser node wraps another output parser. If the first one fails, it calls out to another LLM to fix any errors.","accessed":"2026-05-20"},{"type":"doc","title":"n8n docs — Error workflows","url":"https://docs.n8n.io/flow-logic/error-handling/","quote":"For each workflow, you can set an error workflow in Workflow Settings. It runs if an execution fails.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"MCP Server Trigger exposes n8n workflows as MCP tools to external clients (SSE + streamable HTTP, no stdio). Companion MCP Client node also exists.","evidence":[{"type":"doc","title":"n8n docs — MCP Server Trigger","url":"https://docs.n8n.io/integrations/builtin/core-nodes/n8n-nodes-langchain.mcptrigger/","quote":"Use the MCP Server Trigger node to allow n8n to act as a Model Context Protocol (MCP) server, making n8n tools and workflows available to MCP clients.","accessed":"2026-05-20"},{"type":"doc","title":"n8n docs — MCP transports","url":"https://docs.n8n.io/integrations/builtin/core-nodes/n8n-nodes-langchain.mcptrigger/","quote":"The MCP Server Trigger node supports both Server-Sent Events (SSE)... and streamable HTTP for connections between clients and the server. It currently doesn't support standard input/output (stdio) transport.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"scheduled-agent","role":"first-class","note":"Schedule Trigger runs workflows on intervals (Seconds/Minutes/Hours/Days/Weeks/Months/Custom Cron). Documented per-interval semantics with quote-verified parameters.","evidence":[{"type":"doc","title":"n8n docs — Schedule trigger","url":"https://docs.n8n.io/integrations/builtin/core-nodes/n8n-nodes-base.scheduletrigger/","quote":"Use the Schedule Trigger node to run workflows at fixed intervals and times. This works in a similar way to the Cron software utility in Unix-like systems.","accessed":"2026-05-20"},{"type":"doc","title":"n8n docs — Schedule trigger custom cron","url":"https://docs.n8n.io/integrations/builtin/core-nodes/n8n-nodes-base.scheduletrigger/","quote":"Enter a custom cron **Expression** to set the schedule for the trigger.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"structured-output","role":"first-class","note":"Structured Output Parser sub-node accepts a JSON schema and returns matched fields; Auto-fixing variant wraps another parser and falls back to a recovery LLM if it fails.","evidence":[{"type":"doc","title":"n8n docs — Structured Output Parser","url":"https://docs.n8n.io/integrations/builtin/cluster-nodes/sub-nodes/n8n-nodes-langchain.outputparserstructured/","quote":"Use the Structured Output Parser node to return fields based on a JSON Schema.","accessed":"2026-05-20"},{"type":"doc","title":"n8n docs — Auto-fixing Output Parser wraps Structured","url":"https://docs.n8n.io/integrations/builtin/cluster-nodes/sub-nodes/n8n-nodes-langchain.outputparserautofixing/","quote":"The Auto-fixing Output Parser node wraps another output parser. If the first one fails, it calls out to another LLM to fix any errors.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"supervisor","role":"first-class","note":"n8n ships a dedicated AI Agent Tool node that lets a root-level agent call other agents as tools — the docs explicitly call it 'multi-agent orchestration' with primary-agent supervision and delegation, plus nestable hierarchies.","evidence":[{"type":"doc","title":"n8n docs — AI Agent Tool (multi-agent)","url":"https://docs.n8n.io/integrations/builtin/cluster-nodes/sub-nodes/n8n-nodes-langchain.toolaiagent/","quote":"The AI Agent Tool node allows a root-level agent in your workflow to call other agents as tools to simplify multi-agent orchestration.","accessed":"2026-05-24"},{"type":"doc","title":"n8n docs — AI Agent Tool supervisor / delegation","url":"https://docs.n8n.io/integrations/builtin/cluster-nodes/sub-nodes/n8n-nodes-langchain.toolaiagent/","quote":"The primary agent can supervise and delegate work to AI Agent Tool nodes that specialize in different tasks and knowledge.","accessed":"2026-05-24"},{"type":"doc","title":"n8n docs — Call n8n Workflow Tool","url":"https://docs.n8n.io/integrations/builtin/cluster-nodes/sub-nodes/n8n-nodes-langchain.toolworkflow/","quote":"The Call n8n Workflow Tool node is a tool that allows an agent to run another n8n workflow and fetch its output data.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"AI Agent requires at least one tool sub-node; dedicated tool sub-nodes ship for Calculator, Custom Code, HTTP Request, Workflow, SerpApi, Wikipedia, Wolfram|Alpha, Vector Store Q&A.","evidence":[{"type":"doc","title":"n8n docs — AI Agent (tool selection)","url":"https://docs.n8n.io/integrations/builtin/cluster-nodes/root-nodes/n8n-nodes-langchain.agent/","quote":"This agent uses external tools and APIs to perform actions and retrieve information. It can understand the capabilities of different tools and determine which tool to use depending on the task.","accessed":"2026-05-20"},{"type":"doc","title":"n8n docs — AI Agent (tool requirement)","url":"https://docs.n8n.io/integrations/builtin/cluster-nodes/root-nodes/n8n-nodes-langchain.agent/","quote":"You must connect at least one tool sub-node to an AI Agent node.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"iteration-node","role":"first-class","note":"The Loop Over Items node divides an incoming list into batches, runs the connected nodes once per batch iteration, and combines all processed data on the 'done' output.","evidence":[{"type":"doc","title":"Loop Over Items (Split in Batches) | n8n Docs","url":"https://docs.n8n.io/integrations/builtin/core-nodes/n8n-nodes-base.splitinbatches/","quote":"The Loop Over Items node helps you loop through data ... with each iteration, [it] returns a predefined amount of data through the loop output. When the node execution completes, it combines all of the processed data and returns it through the done output.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"mcp-bidirectional-bridge","role":"supported","note":"n8n exposes workflows as MCP servers via the MCP Server Trigger node and consumes MCP via the MCP Client node.","evidence":[{"type":"doc","url":"https://docs.n8n.io/integrations/builtin/core-nodes/n8n-nodes-langchain.mcptrigger/","quote":"n8n exposes workflows as MCP servers via the MCP Server Trigger node and consumes MCP via the MCP Client node.","accessed":"2026-06-19"}],"evidence_status":"full"}],"alternatives":[{"composition":"flowise","relation":"similar-shape","note":"Visual LangChain-JS builder; same 'visual nodes over LangChain' shape but focused on chatbot/agent flows."},{"composition":"langflow","relation":"similar-shape","note":"Visual LangChain builder (Python)."},{"composition":"dify","relation":"similar-shape","note":"Visual LLM-app builder; Python-based and product-oriented vs n8n's general workflow scope."}],"references":[{"type":"repo","title":"n8n-io/n8n README","url":"https://github.com/n8n-io/n8n/blob/master/README.md","quote":"n8n is a workflow automation platform that gives technical teams the flexibility of code with the speed of no-code.","accessed":"2026-05-20"},{"type":"doc","title":"n8n docs — Advanced AI intro tutorial","url":"https://docs.n8n.io/advanced-ai/intro-tutorial/","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-14","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","fair-code","german-vendor","visual","workflow-automation"]},{"id":"relevance-ai","name":"Relevance AI","kind":"framework","category":"low-code-platform","build_surface":"low-code","vendor":"Relevance AI","language":"Web (visual)","license":"proprietary (hosted)","status":"active","url":"https://relevanceai.com/","docs_url":"https://relevanceai.com/docs","intent":"Provide a hosted, enterprise visual platform on which domain experts assemble individual AI Agents (with tools, knowledge and sub-agents) and compose them into a Workforce — a multi-agent team — gated by approvals, escalations and an oversight dashboard for safe autonomous execution.","description":"Relevance AI is a hosted enterprise agent platform. The homepage frames it as 'The Enterprise Platform for Agents You Can Trust at Scale' where 'your domain experts and AI agents achieve results together, safely, at enterprise scale.' Core building blocks are Agents ('AI entities that autonomously complete tasks like human employees'), Tools ('no-code workflow builder for integrations and automations'), Knowledge ('RAG solution giving agents access to specific information beyond pre-trained knowledge'), and Workforce ('multi-agent teams where specialized AI agents collaborate on complex tasks'). Approvals and escalations are first-class: 'Approvals and escalations are essential components of Workforce that ensure your AI agents operate with appropriate oversight.'","primary_use_cases":["hosted enterprise AI agent building with no-code tools","multi-agent Workforce orchestration across domain-specific agents","human approval workflows for high-risk agent actions","RAG via Knowledge bases with semantic search","scaling repetitive knowledge-work tasks 24/7"],"agent_loop_shape":"An Agent is configured with an LLM core, an instruction set, a set of Tools (no-code workflow tools that wrap integrations), one or more Knowledge bases (RAG), and optionally Sub-agents. The agent autonomously selects tools to call. Sub-agents can be delegated to with Auto-Run for routine tasks or Approval Required for sensitive operations. Workforce is the multi-agent layer: it 'link[s] multiple agents together on a visual canvas to handle complex, multi-step workflows,' adds approvals and escalations as oversight gates, and routes tasks across specialised agents. The 'Workforce Task View serves as your central dashboard for monitoring and managing approval requests.'","key_concepts":[{"name":"Agent","summary":"Autonomous AI entity with LLM, tools, knowledge and optional sub-agents.","url":"https://relevanceai.com/docs/get-started/core-concepts/agents"},{"name":"Tool","summary":"No-code workflow that wraps an integration or automation and is callable by an agent.","maps_to_pattern":"tool-use","url":"https://relevanceai.com/docs/get-started/core-concepts/tools"},{"name":"Knowledge","summary":"RAG layer — knowledge bases with semantic search agents query to ground answers.","maps_to_pattern":"agentic-rag","url":"https://relevanceai.com/docs/get-started/core-concepts/knowledge"},{"name":"Sub-agent","summary":"Agent attached to a parent agent as a delegated task executor; Auto Run or Approval Required mode.","url":"https://relevanceai.com/docs/build/agents/build-your-agent/agent-settings/subagents"},{"name":"Workforce","summary":"Multi-agent team on a visual canvas with approvals, escalations and a task dashboard.","maps_to_pattern":"supervisor","url":"https://relevanceai.com/docs/get-started/core-concepts/workforces"},{"name":"Approval & Escalation","summary":"Oversight gates — agents draft actions and wait for human approval; escalate when uncertain.","maps_to_pattern":"approval-queue","url":"https://relevanceai.com/docs/build/workforces/workforce-features/approvals-and-escalations"}],"pattern_composition":"flowchart TD\n  USR[User / trigger] --> WF[Workforce — visual canvas]\n  WF --> AGT1[Agent A — LLM + tools + knowledge]\n  WF --> AGT2[Agent B — LLM + tools + knowledge]\n  AGT1 -.delegate.-> SUB[Sub-agent<br/>Auto Run / Approval Required]\n  AGT1 --> TOOLS[Tools — 1000+ connectors / MCP / custom]\n  AGT1 --> KN[(Knowledge — RAG with semantic search)]\n  AGT1 --> APP{Approval gate}\n  APP -- approve --> EXEC[Execute action]\n  APP -- deny --> AGT1\n  AGT1 -.escalate.-> HUM[(Human reviewer<br/>Workforce Task View)]\n  AGT2 --> APP\n  EXEC --> OUT[Result]","members":[{"pattern":"agentic-rag","role":"first-class","note":"Knowledge is the platform's named RAG layer with semantic search; agents query knowledge bases to ground answers in proprietary data. Two configuration modes: add-all-to-prompt or allow-agent-to-search.","evidence":[{"type":"doc","title":"Relevance AI docs — Knowledge / RAG","url":"https://relevanceai.com/docs/get-started/core-concepts/knowledge","quote":"Knowledge is Relevance AI's retrieval augmented generation (RAG) system. It lets you give your agents and tools access to specific information","accessed":"2026-05-20"},{"type":"doc","title":"Relevance AI docs — Knowledge usage modes","url":"https://relevanceai.com/docs/get-started/core-concepts/knowledge","quote":"When you add a knowledge base to an agent, you choose how the agent uses it: Add all to prompt or Allow agent to search","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"approval-queue","role":"first-class","note":"Approvals are documented as essential workforce primitives — agents draft actions and wait for human approval; the Workforce Task View serves as the central approval-management dashboard.","evidence":[{"type":"doc","title":"Relevance AI docs — Approvals purpose","url":"https://relevanceai.com/docs/build/workforces/workforce-features/approvals-and-escalations","quote":"Approvals and escalations are essential components of Workforce that ensure your AI agents operate with appropriate oversight.","accessed":"2026-05-20"},{"type":"doc","title":"Relevance AI docs — approval flow","url":"https://relevanceai.com/docs/build/workforces/workforce-features/approvals-and-escalations","quote":"The agent will draft its intended action and wait for human approval before proceeding.","accessed":"2026-05-20"},{"type":"doc","title":"Relevance AI docs — Workforce Task View","url":"https://relevanceai.com/docs/build/workforces/workforce-features/approvals-and-escalations","quote":"The Workforce Task View serves as your central dashboard for monitoring and managing approval requests.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"supervisor","role":"first-class","note":"Workforce is a documented multi-agent layer in which specialised agents collaborate on a visual canvas with explicit task routing, approvals and escalations.","evidence":[{"type":"doc","title":"Relevance AI docs — Workforce as multi-agent team","url":"https://relevanceai.com/docs","quote":"Multi-agent teams where specialized AI agents collaborate on complex tasks","accessed":"2026-05-20"},{"type":"doc","title":"Relevance AI docs — multi-agent canvas","url":"https://relevanceai.com/docs","quote":"Link multiple agents together on a visual canvas to handle complex, multi-step workflows","accessed":"2026-05-20"},{"type":"doc","title":"Relevance AI docs — Sub-agents delegation","url":"https://relevanceai.com/docs/build/agents/build-your-agent/agent-settings/subagents","quote":"Write clear directions for when and how the parent Agent should use this Sub-agent.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Tools are a named core concept — no-code workflows that wrap integrations and automations; 1,000+ connectors plus MCP support plus custom connectors are advertised on the homepage.","evidence":[{"type":"doc","title":"Relevance AI docs — Tools core concept","url":"https://relevanceai.com/docs","quote":"No-code workflow builder for integrations and automations","accessed":"2026-05-20"},{"type":"doc","title":"Relevance AI homepage — 1000+ connectors","url":"https://relevanceai.com/","quote":"Connect to 1,000+ apps","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"human-in-the-loop","role":"first-class","note":"Beyond approval gates, agents 'request approval when facing uncertainty' and escalations occur 'when an agent encounters a situation it cannot handle independently' — both surface specific human-in-the-loop primitives.","evidence":[{"type":"doc","title":"Relevance AI docs — agent uncertainty handling","url":"https://relevanceai.com/docs/build/workforces/workforce-features/approvals-and-escalations","quote":"Complete tasks autonomously when it has sufficient context and confidence","accessed":"2026-05-20"},{"type":"doc","title":"Relevance AI docs — escalation","url":"https://relevanceai.com/docs/build/workforces/workforce-features/approvals-and-escalations","quote":"Escalations occur when an agent encounters a situation it cannot handle independently.","accessed":"2026-05-20"}],"evidence_status":"full"}],"alternatives":[{"composition":"stackai","relation":"competes-with","note":"Enterprise visual AI agent platform with governance and approvals."},{"composition":"dify","relation":"similar-shape","note":"Open-source LLM-app platform with comparable RAG + tools + agent shape."},{"composition":"coze","relation":"similar-shape","note":"ByteDance hosted no-code peer with multi-agent mode."}],"references":[{"type":"doc","title":"Relevance AI homepage","url":"https://relevanceai.com/","quote":"The Enterprise Platform for Agents You Can Trust at Scale","accessed":"2026-05-20"},{"type":"doc","title":"Relevance AI docs — core concepts","url":"https://relevanceai.com/docs","quote":"AI entities that autonomously complete tasks like human employees","accessed":"2026-05-20"},{"type":"doc","title":"Relevance AI docs — approvals and escalations","url":"https://relevanceai.com/docs/build/workforces/workforce-features/approvals-and-escalations","quote":"Approvals and escalations are essential components of Workforce that ensure your AI agents operate with appropriate oversight.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-20","verification_status":"verified","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"mature","tags":["hosted","low-code","enterprise","multi-agent","approvals","rag"]},{"id":"stackai","name":"Stack AI","kind":"framework","category":"low-code-platform","build_surface":"low-code","vendor":"Stack AI","language":"Web (visual)","license":"proprietary (hosted)","status":"active","url":"https://www.stackai.com/","docs_url":"https://docs.stackai.com","intent":"Provide an enterprise visual platform for building, deploying and governing AI agents on a workflow canvas — orchestrating an AI Agent node over knowledge bases, OpenAPI-described tools, sub-flow tools and multi-project routing, with governance and observability wrapped around every deployment.","description":"Stack AI's documentation describes the product as 'an enterprise platform for building, deploying, and governing AI agents.' Builders 'Build AI workflows with a visual builder' and 'Deploy agents to chat, forms, APIs, and internal teams' while operating 'safely with governance, access controls, and observability.' Core building blocks live in the Core Nodes layer — 'the primary building blocks of every StackAI workflow — the AI reasoning layer, the knowledge retrieval layer, and the platform utility layer.' The AI Agent Node is the central reasoning unit and 'is the heartbeat (or heartbeats!) or your project.' Subflow Tools enable orchestration across a fleet of agents; Knowledge Bases provide RAG; OpenAPI-described tools let the LLM call external APIs.","primary_use_cases":["enterprise visual building of AI agents with governance and observability","RAG over indexed Knowledge Bases or live Search-Connected Apps","OpenAPI-driven tool integration where each endpoint becomes a callable tool","multi-agent orchestration via Subflow Tools and cross-project routing","deploying agents as chat, forms or APIs to internal teams"],"agent_loop_shape":"A Stack AI workflow is a graph of Core Nodes — AI Agent (reasoning), Knowledge (retrieval) and platform utility nodes. The AI Agent Node is configured with a provider/model, a system prompt, knowledge sources (indexed Knowledge Base or real-time Search Connected Apps), and tools. Tools are grouped under Providers; each OpenAPI endpoint becomes a distinct tool, and 'the LLM intelligently determines when and how to call these tools based on the context of the conversation and user inputs.' Multi-agent shape is realised through Subflow Tools — 'orchestrations across a fleet of AI agents' that can 'run in parallel and pass results back to the main Agent' — and AI Routing across StackAI Projects. Deployment surfaces include chat, forms, APIs and internal teams.","key_concepts":[{"name":"AI Agent Node","summary":"Core reasoning unit; configures LLM, tools, knowledge sources and agentic behaviour.","maps_to_pattern":"tool-use","url":"https://docs.stackai.com/workflow-builder/core-nodes"},{"name":"Knowledge Bases","summary":"Indexed content store queried at run time via semantic search; alternative to live Search Connected Apps.","maps_to_pattern":"agentic-rag","url":"https://docs.stackai.com/workflow-builder/core-nodes/ai-agent-node/knowledge-sources"},{"name":"Tools (OpenAPI / built-in)","summary":"Tools grouped under Providers; each OpenAPI endpoint becomes a distinct tool the LLM can call.","url":"https://docs.stackai.com/workflow-builder/core-nodes/ai-agent-node/tools"},{"name":"Subflow Tools","summary":"Sub-workflows attached to an AI Agent as callable tools — used for multi-agent orchestration.","maps_to_pattern":"supervisor","url":"https://docs.stackai.com/workflow-builder/core-nodes/ai-agent-node/subflow-tools"},{"name":"Workflow builder","summary":"Visual canvas where Core Nodes are wired into a project; outputs deploy to chat, forms, APIs.","maps_to_pattern":"visual-workflow-graph","url":"https://docs.stackai.com/welcome-to-stackai/overview"}],"pattern_composition":"flowchart TD\n  USR[User / API / form / chat] --> WF[Workflow canvas]\n  WF --> AGT[AI Agent Node<br/>provider + model + prompt]\n  AGT --> KB[Knowledge Bases — indexed RAG]\n  AGT --> SCA[Search Connected Apps — live search]\n  AGT --> TOOLS[Tools — OpenAPI endpoints + built-ins]\n  AGT --> SUB[Subflow Tools]\n  SUB --> SUB_A[Sub-agent A — parallel]\n  SUB --> SUB_B[Sub-agent B — parallel]\n  SUB_A -.results.-> AGT\n  SUB_B -.results.-> AGT\n  WF --> OUT[Output node — chat / form / API]\n  GOV[(Governance / access controls / observability)] --> WF","members":[{"pattern":"agentic-rag","role":"first-class","note":"Knowledge Bases are a documented core node: indexed content queried via semantic search at run time. Search Connected Apps provides a real-time alternative; both are first-class knowledge sources for the AI Agent.","evidence":[{"type":"doc","title":"Stack AI docs — Knowledge Sources (indexed RAG)","url":"https://docs.stackai.com/workflow-builder/core-nodes/ai-agent-node/knowledge-sources","quote":"A Knowledge Base is an indexed copy of your content stored inside Stack AI. You upload or connect your files, Stack AI ingests and indexes them, and the LLM node queries that index at run time using semantic search.","accessed":"2026-05-20"},{"type":"doc","title":"Stack AI docs — Knowledge Bases / RAG core node","url":"https://docs.stackai.com/workflow-builder/core-nodes","quote":"Connect document stores and structured data for retrieval-augmented generation.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Tools are first-class on the AI Agent Node — grouped under Providers, each OpenAPI endpoint becomes a distinct callable tool, and the LLM decides when to call which tool.","evidence":[{"type":"doc","title":"Stack AI docs — Tool autonomy","url":"https://docs.stackai.com/workflow-builder/core-nodes/ai-agent-node/tools","quote":"The LLM intelligently determines when and how to call these tools based on the context of the conversation and user inputs.","accessed":"2026-05-20"},{"type":"doc","title":"Stack AI docs — OpenAPI tool granularity","url":"https://docs.stackai.com/workflow-builder/core-nodes/ai-agent-node/tools","quote":"Each API endpoint becomes a distinct tool that the LLM can utilize.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"supervisor","role":"first-class","note":"Subflow Tools are the documented mechanism for multi-agent orchestration — sub-flows attached to a parent AI Agent that can run in parallel and pass results back. Replaces the original 'limited' supervisor role on the stub.","evidence":[{"type":"doc","title":"Stack AI docs — Subflow Tools purpose","url":"https://docs.stackai.com/workflow-builder/core-nodes/ai-agent-node/subflow-tools","quote":"Subflow Tools enables orchestrations across a fleet of AI agents.","accessed":"2026-05-20"},{"type":"doc","title":"Stack AI docs — Orchestrating AI Agents","url":"https://docs.stackai.com/guides-and-tips/stackai-hacks/orchestrating-ai-agents","quote":"Subflow tools can run in parallel and pass results back to the main Agent","accessed":"2026-05-20"},{"type":"doc","title":"Stack AI docs — multi-agent collaboration","url":"https://docs.stackai.com/guides-and-tips/stackai-hacks/orchestrating-ai-agents","quote":"Multiple large language models (LLMs) collaborate and complete a business process","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"visual-workflow-graph","role":"first-class","note":"Stack AI's primary surface is a visual workflow builder; Core Nodes are 'the primary building blocks of every StackAI workflow.'","evidence":[{"type":"doc","title":"Stack AI docs — Overview / Build AI workflows","url":"https://docs.stackai.com/welcome-to-stackai/overview","quote":"Build AI workflows with a visual builder.","accessed":"2026-05-20"},{"type":"doc","title":"Stack AI docs — Core Nodes","url":"https://docs.stackai.com/workflow-builder/core-nodes","quote":"Core nodes are the primary building blocks of every StackAI workflow — the AI reasoning layer, the knowledge retrieval layer, and the platform utility layer.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"structured-output","role":"first-class","note":"Stack AI's AI Agent Node exposes a Response Format setting — Text by default or JSON object — and supports a strict JSON-Schema mode where 'properties' define output fields and 'strict: true' pins the model to the schema.","evidence":[{"type":"doc","title":"Stack AI docs — AI Agent Node JSON response format","url":"https://docs.stackai.com/workflow-builder/core-nodes/ai-agent-node/main-settings","quote":"Text is the default response format. You can also choose to have the AI return a response formatted as a JSON object","accessed":"2026-05-24"},{"type":"doc","title":"Stack AI docs — AI Agent Node JSON schema enforcement","url":"https://docs.stackai.com/workflow-builder/core-nodes/ai-agent-node/main-settings","quote":"To have an LLM output a JSON object according to a provided schema, you must provide the schema in the following format","accessed":"2026-05-24"},{"type":"doc","title":"Stack AI docs — AI Agent Node strict schema flag","url":"https://docs.stackai.com/workflow-builder/core-nodes/ai-agent-node/main-settings","quote":"\"strict\": true if you want to enforce exactly this output schema","accessed":"2026-05-24"}],"evidence_status":"full"}],"alternatives":[{"composition":"relevance-ai","relation":"competes-with","note":"Other enterprise hosted agent platform with multi-agent canvas and approvals."},{"composition":"dify","relation":"similar-shape","note":"Open-source LLM-app platform; comparable workflow + agent + RAG shape."},{"composition":"coze","relation":"similar-shape","note":"ByteDance hosted no-code peer; bot + plugin + knowledge model."}],"references":[{"type":"doc","title":"Stack AI docs — Overview","url":"https://docs.stackai.com/welcome-to-stackai/overview","quote":"An enterprise platform for building, deploying, and governing AI agents.","accessed":"2026-05-20"},{"type":"doc","title":"Stack AI homepage","url":"https://www.stackai.com/","quote":"From process to AI agent, in minutes","accessed":"2026-05-20"},{"type":"doc","title":"Stack AI docs — Core Nodes","url":"https://docs.stackai.com/workflow-builder/core-nodes","quote":"Core nodes are the primary building blocks of every StackAI workflow","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"emerging","tags":["hosted","low-code","enterprise","governance","multi-agent","rag"]},{"id":"sim","name":"Sim","kind":"framework","category":"low-code-platform","build_surface":"low-code","vendor":"Sim Studio","language":"TypeScript","license":"Apache-2.0","status":"active","first_released":"2025","url":"https://sim.ai","repo":"https://github.com/simstudioai/sim","intent":"Provide a drag-and-drop canvas for building and deploying multi-agent workflows, where blocks for models, tools, and logic are wired into an executable graph without writing orchestration code.","description":"Sim (Sim Studio) is an open-source, drag-and-drop platform for building agentic workflows as visual graphs. Builders wire together blocks — model calls, tools, API connectors, conditionals, and loops — on a canvas that compiles to an executable workflow, with code escape hatches for custom logic. It is positioned as a 2026-native alternative to Langflow and Flowise for teams that want visual orchestration with deployment built in.","primary_use_cases":["visual agent workflows","internal automations","RAG and tool pipelines","rapid prototyping"],"agent_loop_shape":"A visual graph of typed blocks compiled to an executable workflow; control flow is the wired canvas rather than imperative code.","members":[{"pattern":"orchestrator-workers","role":"core","note":"The canvas compiles wired blocks into an orchestrated workflow graph that drives the agents.","evidence":[{"type":"repo","title":"simstudioai/sim","url":"https://github.com/simstudioai/sim","accessed":"2026-06-02"}],"evidence_status":"none"},{"pattern":"tool-use","role":"first-class","note":"Tool and API connector blocks let agents call external services from the canvas.","evidence_status":"none"},{"pattern":"human-in-the-loop","role":"supported","note":"Approval/manual blocks can pause a workflow for human input.","evidence_status":"none"}],"alternatives":[{"composition":"langflow","relation":"similar-shape","note":"Both are visual, node-based agent-workflow builders."},{"composition":"flowise","relation":"similar-shape","note":"Both target drag-and-drop construction of LLM/agent flows."}],"references":[{"type":"doc","title":"Sim — visual agent workflow builder","url":"https://sim.ai","accessed":"2026-06-02"},{"type":"repo","title":"simstudioai/sim","url":"https://github.com/simstudioai/sim","accessed":"2026-06-02"}],"last_analyzed":"2026-06-02","verification_status":"stub","last_updated":"2026-06-17","date_added":"2026-06-02","status_in_practice":"emerging","pattern_composition":"flowchart TD\n  fw[\"Sim\"]\n  fw --> p1[\"Orchestrator-Workers<br/>(core)\"]\n  fw --> p2[\"Tool Use<br/>(first-class)\"]\n  fw --> p3[\"Human-in-the-Loop<br/>(supported)\"]","tags":["low-code-platform","multi-agent","tool-use-environment","safety-control","low-code"],"key_concepts":[{"name":"Orchestrator-Workers","summary":"The canvas compiles wired blocks into an orchestrated workflow graph that drives the agents.","maps_to_pattern":"orchestrator-workers"},{"name":"Tool Use","summary":"Tool and API connector blocks let agents call external services from the canvas.","maps_to_pattern":"tool-use"},{"name":"Human-in-the-Loop","summary":"Approval/manual blocks can pause a workflow for human input.","maps_to_pattern":"human-in-the-loop"}]},{"id":"mem0","name":"mem0","kind":"framework","category":"memory-store","build_surface":"full-code","vendor":"Mem0","language":"Python, TypeScript","license":"Apache-2.0","status":"active","url":"https://mem0.ai/","repo":"https://github.com/mem0ai/mem0","docs_url":"https://docs.mem0.ai","intent":"Drop-in memory layer that extracts salient facts from agent conversations and serves them back as personalized context across sessions, users, and agents.","description":"Mem0 is a managed and open-source memory layer for AI agents that turns ongoing conversations into a stored set of user facts, preferences, and episodes which the agent can search on subsequent turns. The agent sends raw messages to `add`; an LLM extracts salient items, deduplicates them against prior memory, and writes them to a vector store scoped by `user_id`, `agent_id`, or `run_id`. On later turns the agent calls `search` to retrieve the most relevant memories instead of replaying the full chat history. The project ships Python and TypeScript SDKs plus a hosted platform, and is described in the paper 'Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory'.","primary_use_cases":["persistent user memory across sessions","personalized assistants that learn preferences over time","reducing prompt bloat and token cost on long conversations","cross-agent shared memory in multi-agent systems"],"agent_loop_shape":"Sidecar memory loop wrapped around an external agent. On every user turn the agent calls `mem0.search(query, user_id)` to retrieve relevant prior memories and injects them into the LLM prompt; after the turn the agent calls `mem0.add(messages, user_id)` which sends the messages through an extraction LLM that derives facts, resolves conflicts against existing memories, and writes the survivors to managed vector storage. Memory operations are add / search / update / delete; identity is keyed by user_id, agent_id, or run_id.","key_concepts":[{"name":"add / search / update / delete","summary":"Four operations on the memory store; `add` extracts facts via LLM, `search` returns relevant memories.","url":"https://docs.mem0.ai/core-concepts/memory-operations"},{"name":"user_id / agent_id / run_id","summary":"Scoping identifiers that key memories to a person, an agent, or a single run.","maps_to_pattern":"session-isolation","url":"https://docs.mem0.ai/core-concepts/memory-operations"},{"name":"Memory extraction (infer=True)","summary":"An LLM pulls out key facts, decisions, or preferences from raw messages and stores those rather than the verbatim turns.","url":"https://docs.mem0.ai/core-concepts/memory-operations"},{"name":"Short-term vs long-term memory","summary":"Short-term covers conversation history, working memory, attention; long-term covers factual, episodic, and semantic memory.","maps_to_pattern":"episodic-summaries","url":"https://docs.mem0.ai/core-concepts/memory-types"},{"name":"Vector store backend","summary":"Managed vector storage holds extracted memories; an entity-linking pass replaces the older optional graph store.","maps_to_pattern":"vector-memory","url":"https://docs.mem0.ai/core-concepts/memory-operations"}],"pattern_composition":"flowchart TD\n  user[User turn] --> search[mem0.search user_id]\n  search --> store[(Vector store)]\n  store --> ctx[Relevant memories]\n  ctx --> llm[Agent LLM]\n  llm --> reply[Reply to user]\n  reply --> add[mem0.add messages user_id]\n  add --> extract[Extraction LLM]\n  extract --> dedupe[Dedupe vs existing]\n  dedupe --> store","members":[{"pattern":"cross-session-memory","role":"first-class","note":"Core value prop: memories scoped by user_id persist across sessions and across agents, so the next conversation does not start blank.","evidence":[{"type":"doc","title":"Mem0 homepage","url":"https://mem0.ai/","quote":"AI memory that persists across sessions and agents","accessed":"2026-05-20"},{"type":"doc","title":"Mem0 docs — overview","url":"https://docs.mem0.ai/overview","quote":"Personalized replies: Memories persist across users and agents, cutting prompt bloat and repeat questions.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"vector-memory","role":"first-class","note":"Extracted memories land in managed vector storage; search returns by semantic similarity.","evidence":[{"type":"doc","title":"Mem0 docs — memory operations","url":"https://docs.mem0.ai/core-concepts/memory-operations","quote":"The resulting memories land in managed vector storage (and optional graph storage) so future searches return them quickly.","accessed":"2026-05-20"},{"type":"doc","title":"Mem0 docs — overview","url":"https://docs.mem0.ai/overview","quote":"Mem0 runs the vector store, graph services, and rerankers—no provisioning, tuning, or maintenance.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"session-isolation","role":"first-class","note":"Every read and write carries a user_id / agent_id / run_id; memories are keyed by identity for retrieval.","evidence":[{"type":"doc","title":"Mem0 docs — memory operations","url":"https://docs.mem0.ai/core-concepts/memory-operations","quote":"user_id, agent_id, or run_id that scope the memory for future searches.","accessed":"2026-05-20"},{"type":"repo","title":"Mem0 README","url":"https://github.com/mem0ai/mem0","quote":"Multi-Level Memory: Seamlessly retains User, Session, and Agent state","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"episodic-summaries","role":"supported","note":"Mem0's long-term memory taxonomy explicitly names episodic memory as compressed summaries of past interactions; the add pipeline produces these via LLM extraction rather than verbatim logs.","evidence":[{"type":"doc","title":"Mem0 docs — memory types","url":"https://docs.mem0.ai/core-concepts/memory-types","quote":"Episodic Memory: summaries of past interactions or completed tasks","accessed":"2026-05-20"},{"type":"doc","title":"Mem0 docs — memory operations","url":"https://docs.mem0.ai/core-concepts/memory-operations","quote":"Mem0 sends the messages through an LLM that pulls out key facts, decisions, or preferences to remember.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"knowledge-graph-memory","role":"limited","note":"Graph memory backends (Neo4j, Memgraph, Kuzu, AGE, Neptune) were a first-class option in the older SDK but have been removed from the open-source SDK and replaced by built-in entity linking; managed platform still offers graph services.","evidence":[{"type":"doc","title":"Mem0 docs — graph memory (deprecation note)","url":"https://docs.mem0.ai/open-source/graph_memory/overview","quote":"Graph store support has been removed from the open-source SDK. It is replaced by built-in entity linking, which runs natively with no external dependencies.","accessed":"2026-05-20"},{"type":"doc","title":"Mem0 docs — overview (managed)","url":"https://docs.mem0.ai/overview","quote":"Mem0 runs the vector store, graph services, and rerankers—no provisioning, tuning, or maintenance.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"memgpt-paging","role":"limited","note":"Honest downgrade — mem0 is positioned as an alternative to MemGPT-style paging rather than an implementation of it. The mem0 paper explicitly contrasts itself with MemGPT's hierarchical paging pipeline and substitutes LLM-extracted facts in a vector store.","evidence":[{"type":"paper","title":"Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory","url":"https://arxiv.org/abs/2504.19413","quote":"dynamically extracting, consolidating, and retrieving salient information from ongoing conversations","accessed":"2026-05-20"},{"type":"paper","title":"Mem0 paper (HTML) — comparison to MemGPT's hierarchical memory pipeline","url":"https://arxiv.org/html/2504.19413v1","quote":"MemGPT employs a sophisticated memory management pipeline consisting of three key components: a hierarchical memory system, self-directed memory operations, and an event-based control flow mechanism","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"agentic-memory","role":"first-class","note":"Mem0 implements memory-ops-as-tools: at write time the LLM picks ADD/UPDATE/DELETE/NOOP against retrieved memories via a function-calling tool interface, and the Mem0 MCP server exposes add/search/update/delete as MCP tools to other agents.","evidence":[{"type":"paper","title":"Mem0 paper (HTML) — LLM-driven memory operations","url":"https://arxiv.org/html/2504.19413v1","quote":"The LLM itself determines which of four distinct operations to execute: ADD for creation of new memories when no semantically equivalent memory exists; UPDATE for augmentation of existing memories with complementary information; DELETE for removal of memories contradicted by new information; and NOOP when the candidate fact requires no modification to the knowledge base.","accessed":"2026-05-24"},{"type":"doc","title":"Mem0 MCP server — memory tools exposed to agents","url":"https://docs.mem0.ai/platform/mem0-mcp","quote":"Mem0 MCP Server exposes Mem0's memory capabilities as MCP tools, letting AI agents decide when to save, search, or update information.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"adaptive-memory-decay","role":"first-class","note":"Mem0 ships a memory-decay feature that scores each memory by recency and access frequency so recently used items are boosted and stale ones are dampened toward deletion.","evidence":[{"type":"blog","title":"Introducing Memory Decay in Mem0","url":"https://mem0.ai/blog/introducing-memory-decay-in-mem0","quote":"A memory accessed today can pick up a `1.5×` boost. One that's been sitting untouched for weeks gets dampened toward `0.3×`.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"co-located-memory-surfacing","role":"supported","note":"Mem0 fuses semantic, keyword, and entity-matching retrieval and links entities across memories to surface co-located context.","evidence":[{"type":"doc","url":"https://github.com/mem0ai/mem0/blob/main/README.md","quote":"Multi-signal retrieval -- semantic, BM25 keyword, and entity matching scored in parallel and fused.","accessed":"2026-06-19"}],"evidence_status":"limited"}],"references":[{"type":"paper","title":"Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory","url":"https://arxiv.org/abs/2504.19413","quote":"Mem0 achieves 26% relative improvements in the LLM-as-a-Judge metric over OpenAI ... 91% lower p95 latency and saves more than 90% token cost","accessed":"2026-05-20"},{"type":"repo","title":"mem0ai/mem0 on GitHub","url":"https://github.com/mem0ai/mem0","quote":"enhances AI assistants and agents with an intelligent memory layer, enabling personalized AI interactions","accessed":"2026-05-20"},{"type":"doc","title":"Mem0 documentation — overview","url":"https://docs.mem0.ai/overview","quote":"The memory engine that keeps conversations contextual so users never repeat themselves and your agents respond with continuity.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-06-19","date_added":"2026-04-30","status_in_practice":"mature","tags":["memory-store","memory","full-code"],"anti_patterns_avoided":[{"pattern":"memory-extraction-attack","note":"Every memory operation is scoped by user_id/agent_id and isolated at the storage layer, so one principal's memories cannot be retrieved into another's context.","evidence":[{"type":"doc","title":"Mem0 — memory security best practices","url":"https://mem0.ai/blog/ai-memory-security-best-practices","quote":"Every memory operation in Mem0 is scoped by user_id and optionally by agent_id. Memories are isolated at the storage level, meaning one user's memories are never retrieved in another user's context.","accessed":"2026-06-19"}]},{"pattern":"memory-poisoning","note":"New memories are extracted by an LLM and checked against existing memories for duplicates and contradictions before being written, so false or conflicting facts do not silently poison the store.","evidence":[{"type":"doc","title":"Mem0 — memory operations","url":"https://docs.mem0.ai/core-concepts/memory-operations","quote":"Existing memories are checked for duplicates or contradictions so the latest truth wins.","accessed":"2026-06-19"}]}]},{"id":"zep","name":"Zep","kind":"framework","category":"memory-store","build_surface":"full-code","vendor":"Zep AI","language":"Python, TypeScript, Go","license":"Apache-2.0","status":"active","url":"https://www.getzep.com/","repo":"https://github.com/getzep/zep","docs_url":"https://help.getzep.com/","intent":"Context engineering platform that builds a per-user temporal knowledge graph from chat messages and business data, then assembles low-latency context for agent turns.","description":"Zep is a context engineering platform built around Graphiti, an open-source temporal knowledge graph engine that ingests conversational messages and structured business data and turns them into a graph of entities and facts with bi-temporal validity windows. Each user has their own User Graph; threads under that user feed messages into the graph via `thread.add_messages`, and agents retrieve assembled context with `thread.get_user_context` or low-level `graph.search`. When new data contradicts a prior fact, Zep stores the invalidation time on the fact's edge rather than deleting it, so agents can reason about how preferences and relationships changed. SDKs are available in Python, TypeScript, and Go.","primary_use_cases":["long-term agent memory with temporal reasoning","voice and video agents needing sub-200ms context retrieval","personalized assistants that track changing user state","Graph RAG over conversational and business data"],"agent_loop_shape":"Sidecar memory service backed by a temporal knowledge graph. The application creates a User and a Thread; on each user turn it calls `thread.add_messages` which ingests messages into that user's graph, where Graphiti extracts entities and facts as edges with valid_at and invalid_at timestamps. Before generating a reply the agent calls `thread.get_user_context` (high-level) or `graph.search` (low-level) to retrieve assembled context. Fact invalidation is handled automatically: when new data contradicts an existing fact, Zep stamps the old edge as invalidated rather than deleting it.","key_concepts":[{"name":"User Graph","summary":"Per-user knowledge graph; threads belonging to that user feed into it.","maps_to_pattern":"session-isolation","url":"https://help.getzep.com/concepts"},{"name":"Threads","summary":"A conversation under a user; thread.add_messages ingests messages into the user-level graph.","url":"https://help.getzep.com/threads"},{"name":"Graphiti (temporal knowledge graph)","summary":"Open-source engine that builds the graph; entities are nodes, facts are edges with valid_at/invalid_at.","maps_to_pattern":"knowledge-graph-memory","url":"https://github.com/getzep/graphiti"},{"name":"Fact invalidation","summary":"When new data contradicts a fact, the edge is stamped invalidated rather than deleted.","url":"https://help.getzep.com/concepts"},{"name":"thread.get_user_context","summary":"High-level retrieval call returning assembled context (facts, entities, summaries) for the agent turn.","maps_to_pattern":"cross-session-memory","url":"https://help.getzep.com/concepts"},{"name":"Episodes / thread summaries / user summary","summary":"Distinct primitives Zep produces from a user graph: facts, entities, episodes, thread summaries, observations, user summary.","maps_to_pattern":"episodic-summaries","url":"https://help.getzep.com/concepts"}],"pattern_composition":"flowchart TD\n  user[User turn] --> thread[Thread under User]\n  thread --> add[thread.add_messages]\n  add --> graphiti[Graphiti extractor]\n  graphiti --> entities[Entities as nodes]\n  graphiti --> facts[Facts as edges]\n  facts --> valid[valid_at / invalid_at]\n  newdata[New contradicting fact] --> invalidate[Stamp prior edge invalid]\n  invalidate --> facts\n  agent[Agent before reply] --> get[thread.get_user_context]\n  get --> usergraph[(User Graph)]\n  entities --> usergraph\n  facts --> usergraph\n  usergraph --> ctx[Assembled context]\n  ctx --> llm[Agent LLM]","members":[{"pattern":"cross-session-memory","role":"first-class","note":"The User Graph persists across threads; messages added to any thread of that user are ingested into that user's graph, so prior sessions inform later ones.","evidence":[{"type":"doc","title":"Zep docs — concepts (threads)","url":"https://help.getzep.com/concepts","quote":"Conversation threads of a user. By default, all messages added to any thread of that user are ingested into that user's graph.","accessed":"2026-05-20"},{"type":"doc","title":"Zep homepage","url":"https://www.getzep.com/","quote":"Zep assembles the right context from chat history, business data, and user behavior.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"knowledge-graph-memory","role":"first-class","note":"Graphiti is the core engine; nodes are entities, edges are facts, and the whole store is positioned as a unified knowledge graph rather than a vector index.","evidence":[{"type":"doc","title":"Zep docs — concepts","url":"https://help.getzep.com/concepts","quote":"Zep's unified knowledge store for agents. Nodes represent entities, edges represent facts/relationships.","accessed":"2026-05-20"},{"type":"repo","title":"Graphiti README","url":"https://github.com/getzep/graphiti","quote":"Triplets (Entity → Relationship → Entity) with temporal validity windows","accessed":"2026-05-20"},{"type":"repo","title":"Zep README","url":"https://github.com/getzep/zep","quote":"Graphiti autonomously builds and maintains knowledge graphs while reasoning about state changes over time.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"session-isolation","role":"first-class","note":"Every thread belongs to a user; the per-user User Graph is the isolation boundary, and threadIds map to your business objects for further scoping.","evidence":[{"type":"doc","title":"Zep docs — concepts (user graph)","url":"https://help.getzep.com/concepts","quote":"Special type of graph for storing personalized context for a user of your application.","accessed":"2026-05-20"},{"type":"doc","title":"Zep docs — threads","url":"https://help.getzep.com/threads","quote":"Threads represent a conversation. Each User can have multiple threads, and each thread is a sequence of chat messages.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"episodic-summaries","role":"first-class","note":"Zep produces explicit episodic primitives: episodes (raw ingested data), thread summaries, and a user summary, alongside extracted facts and entities.","evidence":[{"type":"doc","title":"Zep docs — concepts (primitives)","url":"https://help.getzep.com/concepts","quote":"The distinct primitives Zep produces from a user graph: facts, entities, episodes, thread summaries, observations, and user summary.","accessed":"2026-05-20"},{"type":"repo","title":"Graphiti README — episodes","url":"https://github.com/getzep/graphiti","quote":"Episodes — the raw data that produced it","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"five-tier-memory-cascade","role":"limited","note":"Honest downgrade — Zep's Graphiti uses three hierarchical subgraph tiers (episode, semantic entity, community) and distinguishes episodic vs semantic memory, but does not formalise the five-tier sensory/working/short-term/episodic/long-term cognitive cascade. The runtime user graph produces six primitives (facts, entities, episodes, thread summaries, observations, user summary).","evidence":[{"type":"doc","title":"Zep docs — concepts","url":"https://help.getzep.com/concepts","quote":"The distinct primitives Zep produces from a user graph: facts, entities, episodes, thread summaries, observations, and user summary.","accessed":"2026-05-20"},{"type":"paper","title":"Zep: A Temporal Knowledge Graph Architecture for Agent Memory (HTML)","url":"https://arxiv.org/html/2501.13956v1","quote":"This graph comprises three hierarchical tiers of subgraphs: an episode subgraph, a semantic entity subgraph, and a community subgraph.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"world-model-graph-memory","role":"core","note":"Zep's Graphiti engine decomposes each ingested episode into entities (nodes) and relationships (edges) with temporal validity, building a typed knowledge graph used as the agent's evolving model of the world.","evidence":[{"type":"doc","title":"Zep Documentation — Graphiti Getting Started Overview","url":"https://help.getzep.com/graphiti/getting-started/overview","quote":"A Context Graph is a temporal knowledge graph — a graph of entities, relationships, and facts, such as \"Kendra loves Adidas shoes.\"","accessed":"2026-06-17"}],"evidence_status":"limited"}],"references":[{"type":"paper","title":"Zep: A Temporal Knowledge Graph Architecture for Agent Memory","url":"https://arxiv.org/abs/2501.13956","quote":"a novel memory layer service for AI agents that outperforms the current state-of-the-art system, MemGPT","accessed":"2026-05-20"},{"type":"repo","title":"getzep/zep on GitHub","url":"https://github.com/getzep/zep","quote":"Zep is an end-to-end context engineering platform that delivers the right information at the right time with sub-200ms latency.","accessed":"2026-05-20"},{"type":"repo","title":"getzep/graphiti on GitHub","url":"https://github.com/getzep/graphiti","quote":"When information changes, old facts are invalidated — not deleted.","accessed":"2026-05-20"},{"type":"doc","title":"Zep documentation — concepts","url":"https://help.getzep.com/concepts","quote":"When new data invalidates a prior fact, the time the fact became invalid is stored on that fact's edge in the knowledge graph.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["memory-store","memory","full-code"]},{"id":"langmem","name":"LangMem","kind":"framework","category":"memory-store","build_surface":"full-code","vendor":"LangChain","language":"Python","license":"MIT","status":"active","url":"https://langchain-ai.github.io/langmem/","repo":"https://github.com/langchain-ai/langmem","docs_url":"https://langchain-ai.github.io/langmem/","intent":"LangChain's long-term-memory SDK that lets agents store, search, and update semantic, episodic, and procedural memories outside the prompt window.","description":"LangMem is a Python library from the LangChain team that adds long-term memory primitives to agent applications. Its conceptual guide names three memory types — semantic (facts that ground responses), episodic (successful past interactions used as learning examples), and procedural (how the agent should behave and respond) — and exposes typed extraction, storage, and prompt-optimization APIs over each. LangMem integrates with LangGraph's BaseStore so writes can be managed automatically and surfaced as tools the agent calls directly, rather than only as a sidecar service.","primary_use_cases":["persistent agent memory across LangGraph runs","episodic recall of prior successful interactions","semantic fact stores that agents update over time","procedural memory for evolving agent system prompts"],"agent_loop_shape":"Library invoked from inside an agent's tool-calling loop, optionally backed by LangGraph's BaseStore. The agent (or a background manager) calls LangMem extractors that classify writes into semantic, episodic, and procedural buckets; semantic and episodic memories are persisted as searchable items, while procedural memory is materialised as updated system-prompt instructions via a prompt-optimizer (metaprompt / gradient / prompt_memory). Retrieved memories are injected back into the next prompt.","key_concepts":[{"name":"Semantic memory","summary":"Facts and relationships that ground the agent's responses; extracted from conversations.","maps_to_pattern":"semantic-memory","url":"https://langchain-ai.github.io/langmem/concepts/conceptual_guide/"},{"name":"Episodic memory","summary":"Successful past interactions kept as learning examples that guide future behavior.","maps_to_pattern":"episodic-memory","url":"https://langchain-ai.github.io/langmem/concepts/conceptual_guide/"},{"name":"Procedural memory","summary":"How the agent should behave and respond; materialised by updating the system prompt via an optimizer.","maps_to_pattern":"procedural-memory","url":"https://langchain-ai.github.io/langmem/concepts/conceptual_guide/"},{"name":"Prompt optimizer (metaprompt / gradient / prompt_memory)","summary":"Three optimizer strategies that update system prompts from past conversations and feedback.","url":"https://langchain-ai.github.io/langmem/guides/optimize_memory_prompt/"},{"name":"LangGraph BaseStore integration","summary":"Backing store used to persist extracted memories automatically across agent runs.","url":"https://langchain-ai.github.io/langmem/guides/extract_semantic_memories/"}],"members":[{"pattern":"episodic-memory","role":"first-class","note":"LangMem's conceptual guide defines episodic memory as preserving successful interactions as learning examples, and there is a dedicated extract_episodic_memories guide.","evidence":[{"type":"doc","title":"LangMem — Long-term Memory in LLM Applications","url":"https://langchain-ai.github.io/langmem/concepts/conceptual_guide/","quote":"Episodic memory preserves successful interactions as learning examples that guide future behavior.","accessed":"2026-05-24"},{"type":"doc","title":"LangMem — How to Extract Episodic Memories","url":"https://langchain-ai.github.io/langmem/guides/extract_episodic_memories/","quote":"experience replay—capturing not just what happened, but the complete chain of thought that led to success","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"semantic-memory","role":"first-class","note":"LangMem's conceptual guide names semantic memory as one of three memory types and ships an extract_semantic_memories guide for the collection pattern.","evidence":[{"type":"doc","title":"LangMem — Long-term Memory in LLM Applications","url":"https://langchain-ai.github.io/langmem/concepts/conceptual_guide/","quote":"Semantic memory stores the essential facts and other information that ground an agent's responses.","accessed":"2026-05-24"},{"type":"doc","title":"LangMem — How to Extract Semantic Memories","url":"https://langchain-ai.github.io/langmem/guides/extract_semantic_memories/","quote":"Semantic memories help agents learn from conversations. They extract and store meaningful information that might be useful in future interactions.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"procedural-memory","role":"supported","note":"Procedural memory is the third memory type in LangMem's taxonomy and is materialised by the prompt-optimizer, which updates system prompts from conversation logs and feedback.","evidence":[{"type":"doc","title":"LangMem — Long-term Memory in LLM Applications","url":"https://langchain-ai.github.io/langmem/concepts/conceptual_guide/","quote":"Procedural memory encodes how an agent should behave and respond.","accessed":"2026-05-24"},{"type":"doc","title":"LangMem — How to Optimize a Prompt","url":"https://langchain-ai.github.io/langmem/guides/optimize_memory_prompt/","quote":"Update system prompts based on conversation logs or agent trajectories. Feed past conversations and feedback into an optimizer to fix common issues like missed requirements or poor response structure.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"guardrail-erosion-through-compaction","role":"supported","note":"LangMem's summarize_messages / SummarizationNode periodically replaces old messages with a running summary while keeping recent messages, the mechanism that erodes a guardrail left in the summarised span; separating message and summary state keys is the way to keep load-bearing wording out of the rewritten span.","evidence":[{"type":"doc","title":"How to Manage Long Context with Summarization (LangMem)","url":"https://langchain-ai.github.io/langmem/guides/summarization/","quote":"We also recommend using separate LangGraph state keys for the full message history (e.g., \"messages\") and summarization results (e.g., \"summary\")","accessed":"2026-06-14"}],"evidence_status":"limited"}],"references":[{"type":"doc","title":"LangMem — Long-term Memory in LLM Applications (conceptual guide)","url":"https://langchain-ai.github.io/langmem/concepts/conceptual_guide/","quote":"Semantic memory stores the essential facts and other information that ground an agent's responses.","accessed":"2026-05-24"},{"type":"doc","title":"LangMem — How to Optimize a Prompt","url":"https://langchain-ai.github.io/langmem/guides/optimize_memory_prompt/","quote":"Update system prompts based on conversation logs or agent trajectories. Feed past conversations and feedback into an optimizer to fix common issues like missed requirements or poor response structure.","accessed":"2026-05-24"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-05-24","status_in_practice":"emerging","pattern_composition":"flowchart TD\n  fw[\"LangMem\"]\n  fw --> p1[\"Episodic Memory<br/>(first-class)\"]\n  fw --> p2[\"Semantic Memory<br/>(first-class)\"]\n  fw --> p3[\"Procedural Memory<br/>(supported)\"]\n  fw --> p4[\"Guardrail Erosion Through Compaction<br/>(supported)\"]","tags":["memory-store","memory","anti-patterns","full-code"]},{"id":"cognee","name":"Cognee","kind":"framework","category":"memory-store","build_surface":"full-code","vendor":"Topoteretes","language":"Python","license":"Apache-2.0","status":"active","url":"https://cognee.ai/","repo":"https://github.com/topoteretes/cognee","docs_url":"https://docs.cognee.ai/","intent":"Knowledge-graph-backed memory control plane that turns raw documents, conversations, and structured data into a queryable graph of entities and relationships, paired with a vector store for semantic similarity.","description":"Cognee is an open-source memory framework that ingests heterogeneous sources (text, code, conversations, structured records) and runs a `.cognify` pipeline that converts plain text into chunks, embeddings, summaries, nodes, and edges. The graph store captures entities and relationships in a knowledge graph, and a parallel vector store holds embeddings for semantic similarity, so agents can search by meaning and by explicit relationships from a single query API. Cognee positions itself as a memory control plane combining embeddings, graphs, and cognitive-science approaches.","primary_use_cases":["knowledge-graph memory for agents","hybrid graph + vector retrieval","long-term semantic memory over heterogeneous sources","entity-relation extraction for agent context"],"agent_loop_shape":"Pipeline-style ingestion plus query API. The application calls cognee.add() on raw sources and cognee.cognify() to run a six-task extraction pipeline; chunks, embeddings, summaries, nodes, and edges are persisted into a graph store and a parallel vector store. At agent turn time the application calls a search API that can combine vector similarity with graph traversal, and injects results into the prompt.","key_concepts":[{"name":".cognify pipeline","summary":"Six ordered tasks that turn ingested plain text into chunks, embeddings, summaries, nodes, and edges in Cognee's vector and graph stores.","url":"https://docs.cognee.ai/core-concepts/graph-generation"},{"name":"Graph store","summary":"Captures entities and relationships in a knowledge graph (nodes and edges that show connections between concepts).","maps_to_pattern":"knowledge-graph-memory","url":"https://docs.cognee.ai/core-concepts"},{"name":"Vector store","summary":"Holds embeddings for semantic similarity; numerical representations that find conceptually related content.","maps_to_pattern":"vector-memory","url":"https://docs.cognee.ai/core-concepts"},{"name":"Dual storage architecture","summary":"Vector + graph storage gives both semantic search and structural reasoning over the same data.","maps_to_pattern":"semantic-memory","url":"https://docs.cognee.ai/core-concepts"},{"name":"Triplet extraction","summary":"Cognee asks the LLM to turn each chunk into graph nodes and edges as (start_node, relationship_name, end_node) triplets, with names inferred from content.","url":"https://docs.cognee.ai/core-concepts/graph-generation"}],"members":[{"pattern":"knowledge-graph-memory","role":"first-class","note":"Cognee's core artefact is a typed knowledge graph built by the `.cognify` pipeline: nodes are entities, edges are relationships, and they are committed to a graph database.","evidence":[{"type":"doc","title":"Cognee docs — core concepts (Architecture)","url":"https://docs.cognee.ai/core-concepts","quote":"Graph store — Captures entities and relationships in a knowledge graph (nodes and edges that show connections between concepts)","accessed":"2026-05-24"},{"type":"doc","title":"Cognee docs — graph generation","url":"https://docs.cognee.ai/core-concepts/graph-generation","quote":"The .cognify operation takes the ingested data with Add and turns plain text into structured knowledge: chunks, embeddings, summaries, nodes, and edges that live in Cognee's vector and graph stores.","accessed":"2026-05-24"},{"type":"repo","title":"topoteretes/cognee README — Why Use Cognee","url":"https://github.com/topoteretes/cognee","quote":"Knowledge infrastructure — unified ingestion, graph/vector search, runs locally, ontology grounding, multimodal","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"semantic-memory","role":"first-class","note":"Cognee's dual storage architecture exposes both semantic search and structural reasoning; summaries and embeddings are indexed in the vector database for semantic search, while the graph holds entities and relationships.","evidence":[{"type":"doc","title":"Cognee docs — core concepts","url":"https://docs.cognee.ai/core-concepts","quote":"Dual storage architecture gives you both semantic search and structural reasoning","accessed":"2026-05-24"},{"type":"doc","title":"Cognee docs — graph generation (After cognify finishes)","url":"https://docs.cognee.ai/core-concepts/graph-generation","quote":"Summaries are stored and indexed in the vector database for semantic search","accessed":"2026-05-24"},{"type":"repo","title":"topoteretes/cognee README — About","url":"https://github.com/topoteretes/cognee","quote":"It combines embeddings, graphs and cognitive science approaches to make your documents both searchable by meaning and connected by relationships as they change and evolve.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"vector-memory","role":"supported","note":"Cognee maintains a parallel vector store of embeddings produced by the `.cognify` pipeline, so hybrid graph+vector retrieval is possible from one query call.","evidence":[{"type":"doc","title":"Cognee docs — core concepts (Architecture)","url":"https://docs.cognee.ai/core-concepts","quote":"Vector store — Holds embeddings for semantic similarity (numerical representations that find conceptually related content)","accessed":"2026-05-24"},{"type":"doc","title":"Cognee docs — graph generation (Final outcome)","url":"https://docs.cognee.ai/core-concepts/graph-generation","quote":"Vector database contains embeddings for summaries and nodes","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"world-model-graph-memory","role":"core","note":"Cognee's Cognify pipeline extracts entities and their typed relationships into a self-hosted knowledge graph that serves as the agent's persistent, queryable structured memory for reasoning across sessions.","evidence":[{"type":"repo","title":"topoteretes/cognee - GitHub","url":"https://github.com/topoteretes/cognee","quote":"Cognee continuously builds a self-hosted knowledge graph that gives your agents persistent long-term memory across sessions.","accessed":"2026-06-17"},{"type":"blog","title":"AI Memory Architecture: How Cognee Builds AI Memory for Agents","url":"https://www.cognee.ai/blog/fundamentals/how-cognee-builds-ai-memory","quote":"Graph store for entities, relationships, and structural traversal.","accessed":"2026-06-17"}],"evidence_status":"full"}],"references":[{"type":"doc","title":"Cognee documentation — core concepts","url":"https://docs.cognee.ai/core-concepts","quote":"Dual storage architecture gives you both semantic search and structural reasoning","accessed":"2026-05-24"},{"type":"doc","title":"Cognee documentation — graph generation","url":"https://docs.cognee.ai/core-concepts/graph-generation","quote":"The .cognify pipeline is made of six ordered Tasks. Each task takes the output of the previous one and moves your data closer to becoming a searchable knowledge graph.","accessed":"2026-05-24"},{"type":"repo","title":"topoteretes/cognee on GitHub","url":"https://github.com/topoteretes/cognee","quote":"It combines embeddings, graphs and cognitive science approaches to make your documents both searchable by meaning and connected by relationships as they change and evolve.","accessed":"2026-05-24"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-05-24","status_in_practice":"emerging","pattern_composition":"flowchart TD\n  fw[\"Cognee\"]\n  fw --> p1[\"World-Model Graph Memory<br/>(core)\"]\n  fw --> p2[\"Knowledge Graph Memory<br/>(first-class)\"]\n  fw --> p3[\"Semantic Memory<br/>(first-class)\"]\n  fw --> p4[\"Vector Memory<br/>(supported)\"]","tags":["memory-store","memory","full-code"]},{"id":"anchord-mcp","name":"Anchord MCP","kind":"framework","category":"memory-store","vendor":"Anchord","language":"TypeScript","license":"proprietary","status":"active","first_released":"2025-01-01","url":"https://glama.ai/mcp/servers/nolenation04/anchord-mcp","docs_url":"https://glama.ai/mcp/servers/nolenation04/anchord-mcp","intent":"Anchord MCP is an MCP server that gives agents access to the Anchord identity-resolution API so they resolve companies and people to canonical AnchorIDs and run a pre-write safety check before any record write.","description":"Anchord MCP is an MCP server that gives AI agents access to the Anchord identity-resolution API as a set of tools. The resolve tools map companies and people to canonical AnchorIDs against Anchord's identity graph and return needs_review when multiple plausible matches exist. The guard_write tools provide an evaluation-only pre-write safety check that returns allowed true or false with block codes before any record is written. The server is a thin proxy to the hosted Anchord SaaS platform, carrying no business logic of its own.","primary_use_cases":["resolving companies and people to canonical AnchorIDs","pre-write safety checks before mutating records","exporting merged golden records for an entity","grounding agent-emitted identifiers against a system of record"],"agent_loop_shape":"An agent calls resolve_company or resolve_person to map an entity to a canonical AnchorID; if the server returns needs_review the agent stops and surfaces the candidates rather than writing. Before a mutating write the agent calls guard_write, which evaluates the proposed write and returns allowed true or false with block codes. The agent can fetch the merged golden record for an AnchorID at any point. All processing happens server-side; the local MCP server is a thin proxy.","build_surface":"full-code","members":[{"pattern":"canonical-entity-grounding","role":"core","note":"Anchord's MCP server makes agents resolve companies and people to a canonical AnchorID against its identity graph (system of record) instead of emitting identifiers themselves, returning needs_review on ambiguity.","evidence":[{"type":"doc","title":"Anchord MCP (Glama listing)","url":"https://glama.ai/mcp/servers/nolenation04/anchord-mcp","quote":"Resolve companies and people to canonical AnchorIDs, run pre-write safety checks, and export golden records — through the standard MCP tool interface.","accessed":"2026-06-17"},{"type":"doc","title":"Anchord MCP (Glama listing)","url":"https://glama.ai/mcp/servers/nolenation04/anchord-mcp","quote":"Only `resolve_*` returns `needs_review`. It means Anchord found multiple plausible matches and cannot auto-resolve with confidence.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"dry-run-harness","role":"first-class","note":"The guard_write tools run an evaluation-only pre-write safety check that returns allowed true or false with block codes, so the agent learns whether a proposed write would be accepted before committing it.","evidence":[{"type":"doc","title":"Anchord MCP (Glama listing)","url":"https://glama.ai/mcp/servers/nolenation04/anchord-mcp","quote":"Evaluate the proposed write → allowed: true | false (with block codes)","accessed":"2026-06-17"},{"type":"doc","title":"Anchord MCP (Glama listing)","url":"https://glama.ai/mcp/servers/nolenation04/anchord-mcp","quote":"IF needs_review STOP. Do not write. Surface candidates to the user.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"mcp","role":"core","note":"Anchord exposes its identity-resolution API to agents through the standard Model Context Protocol tool interface, so any MCP-capable client can call resolve, guard_write, and golden-record tools without bespoke integration.","evidence":[{"type":"doc","title":"Anchord MCP (Glama listing)","url":"https://glama.ai/mcp/servers/nolenation04/anchord-mcp","quote":"Resolve companies and people to canonical AnchorIDs, run pre-write safety checks, and export golden records — through the standard MCP tool interface.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"AnchorID","summary":"The canonical identifier Anchord assigns to a resolved company or person in its identity graph, used as the system-of-record key an agent grounds against instead of inventing its own identifiers.","maps_to_pattern":"canonical-entity-grounding","url":"https://glama.ai/mcp/servers/nolenation04/anchord-mcp"},{"name":"needs_review","summary":"The status the resolve tools return when multiple plausible matches exist and Anchord cannot auto-resolve with confidence; the agent must stop and surface candidates rather than write.","maps_to_pattern":"canonical-entity-grounding","url":"https://glama.ai/mcp/servers/nolenation04/anchord-mcp"},{"name":"guard_write","summary":"An evaluation-only pre-write tool that judges a proposed record write and returns allowed true or false plus machine-readable block codes before anything is committed.","maps_to_pattern":"dry-run-harness","url":"https://glama.ai/mcp/servers/nolenation04/anchord-mcp"},{"name":"Golden record","summary":"The merged, canonical view of an entity that Anchord exports for a given AnchorID, consolidating attributes from across sources into one authoritative record.","url":"https://glama.ai/mcp/servers/nolenation04/anchord-mcp"}],"references":[{"type":"doc","title":"Anchord MCP — server listing (Glama)","url":"https://glama.ai/mcp/servers/nolenation04/anchord-mcp","accessed":"2026-06-17"}],"alternatives":[{"composition":"k2view-genai-data-fusion","relation":"complements","note":"K2view assembles a unified per-entity data view for agents; Anchord focuses on resolving entities to canonical IDs and guarding writes, a narrower slice of the same master-data concern."},{"composition":"stainless-mcp","relation":"similar-shape","note":"Both wrap a hosted API behind an MCP server as a thin proxy so agents call it as standard tools; Stainless generates MCP servers from API specs while Anchord exposes one specific identity-resolution API."}],"verification_status":"partial","status_in_practice":"experimental","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Anchord MCP\"]\n  fw --> p1[\"Canonical-Entity Grounding<br/>(core)\"]\n  fw --> p2[\"Model Context Protocol<br/>(core)\"]\n  fw --> p3[\"Dry-Run Harness<br/>(first-class)\"]","tags":["memory-store","tool-use-environment","safety-control","full-code"]},{"id":"qdrant","name":"Qdrant","kind":"framework","category":"memory-store","vendor":"Qdrant","language":"Rust","license":"Apache-2.0","status":"active","first_released":"2021-05-01","url":"https://qdrant.tech/","repo":"https://github.com/qdrant/qdrant","docs_url":"https://qdrant.tech/documentation/","intent":"Qdrant is a vector database that stores embeddings as points and retrieves the most similar ones at query time, with payload-based partitioning so one shared instance can isolate each user's or tenant's vectors.","description":"Qdrant is an open-source vector search engine written in Rust. It stores vectors (embeddings) together with JSON payloads and returns the nearest neighbours for a query vector, which is how agents and RAG pipelines retrieve semantically similar memories or documents. Search can be filtered by payload conditions and combined across dense and sparse vectors. For multi-user deployments, Qdrant supports payload-based partitioning within a single collection so each tenant can only access their own vectors.","primary_use_cases":["storing embeddings for semantic retrieval","nearest-neighbour vector search for RAG and agent memory","payload-filtered and hybrid search","multi-tenant vector isolation"],"agent_loop_shape":"Qdrant is the retrieval backend an agent calls, not an agent loop. Documents or memories are embedded and upserted as points with payloads; at query time the agent embeds its query and asks Qdrant for the top-k nearest points, optionally constrained by a payload filter. In multi-tenant use the query carries a tenant identifier filter so the search only ranges over that tenant's vectors, and the returned items are fed back into the agent's context.","build_surface":"full-code","members":[{"pattern":"vector-memory","role":"core","note":"Qdrant stores embeddings as the core unit of search and returns the most similar points for a query vector, which is the mechanism agents use to retrieve semantically relevant memories and documents.","evidence":[{"type":"doc","title":"Vectors | Qdrant documentation","url":"https://qdrant.tech/documentation/concepts/vectors/","quote":"Vectors (or embeddings) are the core concept of the Qdrant Vector Search engine.","accessed":"2026-06-17"},{"type":"doc","title":"Vectors | Qdrant documentation","url":"https://qdrant.tech/documentation/concepts/vectors/","quote":"Vectors define the similarity between objects in the vector space. If a pair of vectors are similar in vector space, it means that the objects they represent are similar in some way.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"tenant-scoped-tool-binding","role":"supported","note":"In a shared instance Qdrant partitions vectors by a tenant identifier in the payload and filters every search on it, so each user or tenant can only reach their own vectors and cannot see another tenant's data.","evidence":[{"type":"doc","title":"Multitenancy | Qdrant documentation","url":"https://qdrant.tech/documentation/manage-data/multitenancy/","quote":"When an instance is shared between multiple users, you may need to partition vectors by user. This is done so that each user can only access their own vectors and can't see the vectors of other users.","accessed":"2026-06-17"},{"type":"doc","title":"Multitenancy | Qdrant documentation","url":"https://qdrant.tech/documentation/manage-data/multitenancy/","quote":"Use a filter along with `group_id` to filter vectors for each user.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"hybrid-search","role":"first-class","note":"Qdrant's hybrid query fuses results from dense and sparse vectors, combining semantic understanding from dense embeddings with precise keyword matching from sparse vectors, using fusion methods such as Reciprocal Rank Fusion.","evidence":[{"type":"doc","title":"Hybrid Queries | Qdrant documentation","url":"https://qdrant.tech/documentation/concepts/hybrid-queries/","quote":"it is often useful to combine dense and sparse vectors to get the best of both worlds: semantic understanding from dense vectors and precise word matching from sparse vectors.","accessed":"2026-06-17"},{"type":"doc","title":"Hybrid Queries | Qdrant documentation","url":"https://qdrant.tech/documentation/concepts/hybrid-queries/","quote":"considers the positions of results within each query and boosts those that appear closer to the top in multiple sets of results.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"cross-encoder-reranking","role":"supported","note":"Qdrant supports multi-stage retrieval where a cheap representation fetches a large candidate set and a larger, more accurate representation (e.g. a ColBERT multi-vector model) re-scores the candidates to produce the final ranking; it provides the rescoring mechanism rather than bundling a specific cross-encoder, hence supported.","evidence":[{"type":"doc","title":"Hybrid Queries | Qdrant documentation","url":"https://qdrant.tech/documentation/concepts/hybrid-queries/","quote":"First, use a smaller and cheaper representation to get a large list of candidates. Then, re-score the candidates using the larger and more accurate representation.","accessed":"2026-06-17"},{"type":"doc","title":"Hybrid Queries | Qdrant documentation","url":"https://qdrant.tech/documentation/concepts/hybrid-queries/","quote":"Fetch 100 results using the default vector, then re-score them using a multi-vector to get the top 10.","accessed":"2026-06-17"}],"evidence_status":"full"}],"key_concepts":[{"name":"Point","summary":"Qdrant's stored unit: a vector (embedding) together with an optional JSON payload and id, which is what nearest-neighbour search ranks and returns.","maps_to_pattern":"vector-memory","url":"https://qdrant.tech/documentation/concepts/vectors/"},{"name":"Payload filter","summary":"Conditions (must / should / must_not) on payload fields applied alongside vector similarity so search ranges only over points matching the metadata constraints.","url":"https://qdrant.tech/documentation/concepts/filtering/"},{"name":"Hybrid query","summary":"A query combining dense and sparse vector results, fused via Reciprocal Rank Fusion or Distribution-Based Score Fusion, to get both semantic and exact-keyword relevance.","maps_to_pattern":"hybrid-search","url":"https://qdrant.tech/documentation/concepts/hybrid-queries/"},{"name":"Prefetch","summary":"A nested sub-query whose results the outer query re-scores, enabling multi-stage retrieval where a cheap vector fetches candidates and a richer representation reranks them.","maps_to_pattern":"cross-encoder-reranking","url":"https://qdrant.tech/documentation/concepts/hybrid-queries/"}],"references":[{"type":"doc","title":"Qdrant Documentation","url":"https://qdrant.tech/documentation/","accessed":"2026-06-17"},{"type":"doc","title":"Filtering | Qdrant documentation","url":"https://qdrant.tech/documentation/concepts/filtering/","quote":"With Qdrant, you can set conditions when searching or retrieving points. For example, you can impose conditions on both the payload and the `id` of the point.","accessed":"2026-06-17"},{"type":"repo","title":"qdrant/qdrant — Vector Search Engine (GitHub)","url":"https://github.com/qdrant/qdrant","accessed":"2026-06-17"}],"alternatives":[{"composition":"zep","relation":"complements","note":"Zep is an agent memory service that layers session and knowledge-graph memory on top of a vector store; Qdrant is the lower-level vector index such a memory layer can sit on."},{"composition":"mem0","relation":"complements","note":"mem0 provides an agent-memory API and can use a vector database as its backing store; Qdrant supplies the raw nearest-neighbour search underneath."},{"composition":"cognee","relation":"competes-with","note":"Cognee is another memory-store option for agents; both back semantic retrieval, though Cognee adds graph structuring on top while Qdrant stays a pure vector engine."}],"verification_status":"needs-verification","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-19","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Qdrant\"]\n  fw --> p1[\"Vector Memory<br/>(core)\"]\n  fw --> p2[\"Hybrid Search<br/>(first-class)\"]\n  fw --> p3[\"Tenant-Scoped Tool Binding<br/>(supported)\"]\n  fw --> p4[\"Cross-Encoder Reranking<br/>(supported)\"]","tags":["memory-store","memory","safety-control","retrieval","full-code"],"anti_patterns_avoided":[{"pattern":"race-conditions-shared-tool-resources","note":"Optimistic concurrency via a version field on points lets a writer reject a stale overwrite, preventing lost updates when clients share a resource.","evidence":[{"type":"doc","title":"Qdrant — manage points","url":"https://qdrant.tech/documentation/manage-data/points/","quote":"To prevent such situations, Client B can use conditional updates. For this, we would need to introduce an additional field in the payload, e.g. version, which would be incremented on each update.","accessed":"2026-06-19"}]}]},{"id":"cohere-command","name":"Cohere Command R+ / Command A Agents","kind":"framework","category":"model-vendor-agent","build_surface":"full-code","vendor":"Cohere","language":"API (Python, TypeScript, REST)","license":"proprietary","status":"active","url":"https://cohere.com/command","docs_url":"https://docs.cohere.com/v2/docs/tool-use-overview","intent":"Enterprise-grade Cohere model family (Command R/R+/A) built so multi-step tool use, RAG with inline citations, and JSON-schema-constrained outputs are first-class API behaviours rather than client-side workarounds.","description":"Cohere markets the Command family as 'AI purpose-built for real-world agentic applications anchored in your data'. The Chat API directly supports multi-step (a.k.a. agentic) tool use: the model can decide to chain tool calls and feed each result into the next step. RAG is a documented mode on the same endpoint, with citations emitted alongside generated text. Structured outputs are constrained either by JSON mode or by a caller-supplied JSON Schema. Cohere also packages Command into the North platform for enterprise agent workflows.","primary_use_cases":["enterprise RAG with grounded, inline-cited answers","multi-step tool/function calling agents","JSON-schema-constrained structured extraction","regulated-industry chat backends and copilots"],"agent_loop_shape":"Stateless HTTP request/response Chat API. The caller drives a tool-use loop: send user message + tools schema, receive either a final answer or a list of tool_calls, execute the tools, post the tool_results back as a new turn, repeat until the model returns a normal assistant message. RAG mode uses a documents[] parameter on the same endpoint and emits citations[] alongside text. Same loop shape supports JSON mode and JSON-Schema-constrained outputs via response_format.","key_concepts":[{"name":"Tool use API","summary":"Chat-API mechanism to connect Command models to external tools (search, APIs, functions, databases).","maps_to_pattern":"tool-use","url":"https://docs.cohere.com/v2/docs/tool-use-overview"},{"name":"Multi-step tool use","summary":"Model can chain tool calls; the workflow loop runs steps 2 and 3 multiple times until done.","maps_to_pattern":"react","url":"https://docs.cohere.com/v2/docs/tool-use-overview"},{"name":"Parallel tool calls","summary":"Model can return more than one tool call per step — same tool repeated or different tools.","maps_to_pattern":"parallel-tool-calls","url":"https://docs.cohere.com/v2/docs/tool-use-overview"},{"name":"RAG with citations","summary":"Pass documents in the documents[] parameter; model returns text with inline fine-grained citations.","maps_to_pattern":"citation-streaming","url":"https://docs.cohere.com/v2/docs/retrieval-augmented-generation-rag"},{"name":"JSON mode / JSON Schema mode","summary":"response_format='json_object' guarantees valid JSON; passing a schema constrains the model to it.","maps_to_pattern":"structured-output","url":"https://docs.cohere.com/docs/structured-outputs"},{"name":"North","summary":"Cohere's enterprise agent platform powered by Command models; used in regulated industries.","url":"https://cohere.com/command"}],"pattern_composition":"flowchart TD\n  USER[User request] --> CHAT[Cohere Chat API]\n  CHAT --> MODE{Mode}\n  MODE -->|tool use| TOOL[Model emits tool_calls]\n  TOOL --> PARALLEL[Parallel / sequential tool calls]\n  PARALLEL --> EXEC[Caller executes tools]\n  EXEC -->|tool_results turn| CHAT\n  MODE -->|RAG| DOCS[documents parameter]\n  DOCS --> GROUND[Grounded answer + inline citations]\n  MODE -->|structured| JSON[response_format json_object or JSON Schema]\n  JSON --> VALID[Schema-constrained output]\n  CHAT --> ANS[Assistant reply]","members":[{"pattern":"agentic-rag","role":"first-class","note":"RAG is a documented mode on the Chat API; the model generates answers grounded on documents[] and emits inline citations out-of-the-box.","evidence":[{"type":"doc","title":"Cohere RAG — grounded generation","url":"https://docs.cohere.com/v2/docs/retrieval-augmented-generation-rag","quote":"generate text that is grounded on supplementary documents, thus minimizing hallucinations","accessed":"2026-05-20"},{"type":"doc","title":"Cohere RAG — documents parameter","url":"https://docs.cohere.com/v2/docs/retrieval-augmented-generation-rag","quote":"The Chat endpoint supports a few different options for structuring documents in the `documents` parameter","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"citation-streaming","role":"first-class","note":"Cohere advertises fine-grained citations as a built-in output of the RAG pipeline.","evidence":[{"type":"doc","title":"Cohere RAG — inline citations","url":"https://docs.cohere.com/v2/docs/retrieval-augmented-generation-rag","quote":"Cohere models are trained to generate fine-grained citations, out-of-the-box, alongside their text output","accessed":"2026-05-20"},{"type":"doc","title":"Cohere RAG — example","url":"https://docs.cohere.com/v2/docs/retrieval-augmented-generation-rag","quote":"The response also includes **inline citations** that reference the first two documents","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"Multi-step tool use loop — model decides on a sequence of tool calls, executes each, feeds the observation back. This is the ReAct shape even though Cohere's docs don't use the academic name.","evidence":[{"type":"doc","title":"Cohere multi-step tool use","url":"https://docs.cohere.com/v2/docs/tool-use-overview","quote":"the model might decide to do a sequence of tool calls in order to answer the user request. This means that steps 2 and 3 will run multiple times in loop.","accessed":"2026-05-20"},{"type":"doc","title":"Cohere tool use overview — multi-step name","url":"https://docs.cohere.com/v2/docs/tool-use-overview","quote":"It is called multi-step tool use","accessed":"2026-05-24"},{"type":"blog","title":"Cohere Command page — agentic positioning","url":"https://cohere.com/command","quote":"High-performance models for agentic, multimodal, multilingual AI","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"structured-output","role":"first-class","note":"JSON mode guarantees a valid JSON object; JSON Schema mode constrains output to a caller-supplied schema.","evidence":[{"type":"doc","title":"Cohere structured outputs — JSON mode","url":"https://docs.cohere.com/docs/structured-outputs","quote":"By setting the `response_format` type to `\"json_object\"` in the Chat API, the output of the model is guaranteed to be a valid JSON object.","accessed":"2026-05-20"},{"type":"doc","title":"Cohere structured outputs — JSON Schema mode","url":"https://docs.cohere.com/docs/structured-outputs","quote":"This forces the LLM to stick to this schema, thus giving you greater control over the output.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Tool use is a top-level Chat API feature, advertised as the way to connect Command models to external systems.","evidence":[{"type":"doc","title":"Cohere tool use overview","url":"https://docs.cohere.com/v2/docs/tool-use-overview","quote":"Tool use is a technique which allows developers to connect Cohere's Command family of models to external tools like search engines, APIs, functions, databases, etc.","accessed":"2026-05-20"},{"type":"doc","title":"Cohere Command product page — Tool Use API","url":"https://cohere.com/command","quote":"Connect Command to our Tool Use API to build powerful agents and research tools across in-house and cloud applications","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"parallel-tool-calls","role":"first-class","note":"Model can return more than one tool_call per assistant turn — same tool repeated or different tools.","evidence":[{"type":"doc","title":"Cohere tool use — multiple calls per step","url":"https://docs.cohere.com/v2/docs/tool-use-overview","quote":"The model can determine that more than one tool call is required. This can be calling the same tool multiple times or different tools for any number of calls.","accessed":"2026-05-20"},{"type":"doc","title":"Cohere tool use overview — repeated calls","url":"https://docs.cohere.com/v2/docs/tool-use-overview","quote":"This can be calling the same tool multiple times or different tools for any number of calls.","accessed":"2026-05-24"}],"evidence_status":"full"}],"references":[{"type":"doc","title":"Cohere tool use overview","url":"https://docs.cohere.com/v2/docs/tool-use-overview","accessed":"2026-05-20"},{"type":"doc","title":"Cohere RAG","url":"https://docs.cohere.com/v2/docs/retrieval-augmented-generation-rag","accessed":"2026-05-20"},{"type":"doc","title":"Cohere structured outputs","url":"https://docs.cohere.com/docs/structured-outputs","accessed":"2026-05-20"},{"type":"blog","title":"Command A product page","url":"https://cohere.com/command","quote":"Our most efficient and performant model to date, specializing in agentic AI, multilingual, and human evaluations for real-life use cases","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"mature","tags":["enterprise","rag","citations","canada","model-vendor"]},{"id":"deepseek-agent","name":"DeepSeek Agent","kind":"framework","category":"model-vendor-agent","build_surface":"full-code","vendor":"DeepSeek","language":"API","license":"proprietary","status":"active","url":"https://www.deepseek.com","docs_url":"https://api-docs.deepseek.com","intent":"OpenAI/Anthropic-compatible Chinese model API (deepseek-chat, deepseek-reasoner) that exposes function calling with a strict JSON-Schema mode, separate reasoning_content channel, and is positioned primarily as a drop-in backend for third-party agent harnesses (Claude Code, Copilot CLI, OpenCode).","description":"DeepSeek does not ship a hosted agent product; instead the API mirrors OpenAI's Chat Completions shape (and is also Anthropic-compatible) so any existing agent harness can swap DeepSeek in as the model. Function calling is supported with up to 128 tools, a strict mode that pins outputs to a JSON Schema, and tool_choice (none/auto/required). The reasoning model deepseek-reasoner emits a separate reasoning_content stream alongside the answer. DeepSeek's own listed agent integrations include Claude Code, GitHub Copilot, GitHub Copilot CLI and others — the company is the model, the agents are the third parties.","primary_use_cases":["drop-in backend model for Claude Code / Copilot CLI / OpenCode style agents","schema-strict function calling for tool-using agents","reasoning agents that need a separate chain-of-thought channel","JSON-output extraction pipelines"],"agent_loop_shape":"Stateless OpenAI/Anthropic-compatible Chat Completions endpoint. The caller drives a standard tools-loop: post messages + tools[], receive either content or tool_calls, execute tools, append tool messages, repeat. For deepseek-reasoner the model also emits a reasoning_content field at the same level as content, which clients are expected to surface or discard. DeepSeek does not host the agent runtime — the harness lives in the caller (Claude Code, etc.).","key_concepts":[{"name":"OpenAI/Anthropic-compatible API","summary":"DeepSeek's API mirrors OpenAI's and Anthropic's shapes so SDKs and agent harnesses can switch backends by changing config only.","url":"https://api-docs.deepseek.com"},{"name":"Function Calling","summary":"Tool-call API with up to 128 functions and tool_choice control (none/auto/required).","maps_to_pattern":"tool-use","url":"https://api-docs.deepseek.com/guides/function_calling"},{"name":"Strict mode (JSON Schema)","summary":"Forces tool-call arguments to match the function's JSON Schema; supports object, string, number, integer, boolean, array, enum, anyOf and $ref/$def.","maps_to_pattern":"structured-output","url":"https://api-docs.deepseek.com/guides/function_calling"},{"name":"JSON output mode","summary":"response_format={'type':'json_object'} returns valid JSON; requires the word 'json' in the prompt plus an example schema.","maps_to_pattern":"structured-output","url":"https://api-docs.deepseek.com/guides/json_mode"},{"name":"deepseek-reasoner reasoning_content","summary":"Reasoning model surfaces a CoT in reasoning_content at the same level as content, separately consumable by clients.","maps_to_pattern":"extended-thinking","url":"https://api-docs.deepseek.com/guides/reasoning_model"}],"pattern_composition":"flowchart TD\n  USER[User request] --> HARNESS[Third-party agent harness<br/>Claude Code / Copilot CLI / OpenCode]\n  HARNESS -->|OpenAI- or Anthropic-shaped/| API[DeepSeek Chat API]\n  API --> MODEL{Model}\n  MODEL -->|deepseek-chat| ANSWER[content]\n  MODEL -->|deepseek-reasoner| COT[reasoning_content + content]\n  MODEL -->|tools enabled| CALL[tool_calls<br/>strict-JSON-schema args]\n  CALL --> HARNESS\n  HARNESS -->|tool results turn| API","members":[{"pattern":"structured-output","role":"first-class","note":"Two complementary mechanisms: response_format json_object for free-form JSON, and strict mode for schema-pinned function-call arguments.","evidence":[{"type":"doc","title":"DeepSeek JSON mode","url":"https://api-docs.deepseek.com/guides/json_mode","quote":"Set the `response_format` parameter to `{'type': 'json_object'}`.","accessed":"2026-05-20"},{"type":"doc","title":"DeepSeek function calling — strict mode","url":"https://api-docs.deepseek.com/guides/function_calling","quote":"In `strict` mode, the model strictly adheres to the format requirements of the Function's JSON schema when outputting a Function call","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Function calling supports up to 128 tools with tool_choice none/auto/required; the user is responsible for executing the function — DeepSeek only emits the call.","evidence":[{"type":"doc","title":"DeepSeek function calling — overview","url":"https://api-docs.deepseek.com/guides/function_calling","quote":"Function Calling allows the model to call external tools to enhance its capabilities.","accessed":"2026-05-20"},{"type":"doc","title":"DeepSeek function calling — model does not execute","url":"https://api-docs.deepseek.com/guides/function_calling","quote":"The functionality of the `get_weather` function needs to be provided by the user. The model itself does not execute specific functions.","accessed":"2026-05-20"},{"type":"api-ref","title":"DeepSeek chat completion — tool_choice","url":"https://api-docs.deepseek.com/api/create-chat-completion","quote":"`auto` means the model can pick between generating a message or calling one or more tools.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"extended-thinking","role":"first-class","note":"deepseek-reasoner exposes the chain-of-thought in a separate reasoning_content field at the same level as content; clients can display or distill it.","evidence":[{"type":"doc","title":"DeepSeek reasoning model — CoT","url":"https://api-docs.deepseek.com/guides/reasoning_model","quote":"Before delivering the final answer, the model first generates a Chain of Thought (CoT) to enhance the accuracy of its responses.","accessed":"2026-05-20"},{"type":"doc","title":"DeepSeek reasoning model — reasoning_content field","url":"https://api-docs.deepseek.com/guides/reasoning_model","quote":"`reasoning_content`: The content of the CoT, which is at the same level as `content` in the output structure.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"provider-string-routing","role":"supported","note":"DeepSeek positions itself as a drop-in backend behind agent harnesses (Claude Code, Copilot CLI, OpenCode) by exposing OpenAI- and Anthropic-compatible API shapes; routing is a config change in the harness.","evidence":[{"type":"doc","title":"DeepSeek API — OpenAI/Anthropic compatibility","url":"https://api-docs.deepseek.com","quote":"The DeepSeek API uses an API format compatible with OpenAI/Anthropic. By modifying the configuration, you can use the OpenAI/Anthropic SDK or softwares compatible with the OpenAI/Anthropic API to access the DeepSeek API.","accessed":"2026-05-20"},{"type":"doc","title":"DeepSeek API — agent integrations","url":"https://api-docs.deepseek.com","quote":"The DeepSeek API is supported by many popular AI agent and coding assistant tools. If you use tools like Claude Code, GitHub Copilot, or OpenCode, you can use DeepSeek as the backend model directly — no code required.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"large-reasoning-model-paradigm","role":"core","note":"deepseek-reasoner is a reasoning-tuned model that first emits a chain-of-thought (returned as reasoning_content) to deliberate before producing the final answer, trading inference time for accuracy.","evidence":[{"type":"api-ref","title":"Reasoning Model (deepseek-reasoner) — DeepSeek API Docs","url":"https://api-docs.deepseek.com/guides/reasoning_model","quote":"the model first generates a Chain of Thought (CoT) to enhance the accuracy of its responses","accessed":"2026-06-17"},{"type":"api-ref","title":"Reasoning Model (deepseek-reasoner) — DeepSeek API Docs","url":"https://api-docs.deepseek.com/guides/reasoning_model","quote":"In each round of the conversation, the model outputs the CoT (reasoning_content) and the final answer (content)","accessed":"2026-06-17"}],"evidence_status":"full"}],"references":[{"type":"doc","title":"DeepSeek API documentation home","url":"https://api-docs.deepseek.com","accessed":"2026-05-20"},{"type":"doc","title":"DeepSeek function calling","url":"https://api-docs.deepseek.com/guides/function_calling","accessed":"2026-05-20"},{"type":"doc","title":"DeepSeek reasoning model","url":"https://api-docs.deepseek.com/guides/reasoning_model","accessed":"2026-05-20"},{"type":"repo","title":"awesome-deepseek-agent (third-party harness directory)","url":"https://github.com/deepseek-ai/awesome-deepseek-agent","accessed":"2026-05-20"}],"last_analyzed":"2026-05-20","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["model-vendor","openai-compatible","anthropic-compatible","chinese-origin","reasoning"]},{"id":"doubao-agents","name":"Doubao Agents","kind":"framework","category":"model-vendor-agent","build_surface":"closed-application","vendor":"ByteDance","language":"Web product + API","license":"proprietary","status":"active","url":"https://www.doubao.com","docs_url":"https://www.volcengine.com/docs/82379","intent":"ByteDance's consumer Doubao chatbot and the Volcano Engine Ark API behind it, offering text/image/video/audio model invocation with documented tool-call building blocks: Function Calling, Web Search, Knowledge Base Search, image processing, and structured output (beta).","description":"Doubao is the consumer ByteDance assistant launched August 2023; by November 2024 it was China's most popular AI chatbot with around 60 million MAU and is powered by ByteDance's Volcano Engine cloud, which the company says processes around 120 trillion tokens per day. Developers reach the same Doubao models through Volcano Engine's Ark platform, whose docs catalogue tool-call building blocks (Function Calling, Web Search a.k.a. 联网内容插件, Knowledge Base Search, image processing) alongside deep thinking, streaming and structured output (beta) modes. The international consumer variant is sold as Cici/Dola and is reportedly powered by Microsoft Azure-hosted OpenAI GPT models rather than Doubao itself.","primary_use_cases":["consumer chat assistant (Chinese market, mobile + web)","tool-augmented Doubao model invocation via Ark API","web-search-grounded answers via 联网内容插件","private knowledge base search and document Q&A"],"agent_loop_shape":"Two surfaces. (1) Consumer Doubao chat app: ByteDance-hosted loop, opaque to users — conversation, search, tool invocations all happen server-side. (2) Volcano Engine Ark API: developer-driven loop similar to other model-vendor APIs — send chat completion request with optional tool definitions and built-in plugins (web search, knowledge base), receive content or tool_calls, execute, return, repeat. Ark also offers deep thinking, streaming, structured-output (beta), context caching and file inputs as request-level toggles.","key_concepts":[{"name":"Doubao chat app","summary":"ByteDance's consumer AI assistant; web, iOS, Android. ~60M MAU in China as of late 2024.","url":"https://www.doubao.com"},{"name":"Volcano Engine Ark (火山方舟)","summary":"Developer platform exposing Doubao models with managed inference, fine-tuning and deployment.","url":"https://www.volcengine.com/product/doubao"},{"name":"Function Calling (函数调用)","summary":"Tool-call API listed under 工具调用 in the Ark docs alongside web search and knowledge base search.","maps_to_pattern":"tool-use","url":"https://www.volcengine.com/docs/82379/1262342"},{"name":"Web Search (联网内容插件)","summary":"Built-in web-search plugin Ark exposes as a tool option for Doubao models.","maps_to_pattern":"agentic-rag","url":"https://www.volcengine.com/docs/82379"},{"name":"Knowledge Base Search (私域知识库搜索)","summary":"Built-in private knowledge-base retrieval tool the Ark platform offers alongside Doubao chat.","maps_to_pattern":"agentic-rag","url":"https://www.volcengine.com/docs/82379"},{"name":"Deep thinking + structured output (beta)","summary":"Ark request-level toggles for chain-of-thought style reasoning and JSON-shaped outputs.","maps_to_pattern":"structured-output","url":"https://www.volcengine.com/docs/82379"}],"pattern_composition":"flowchart TD\n  USER[User] --> SURFACE{Surface}\n  SURFACE -->|consumer| APP[Doubao chat app<br/>web / iOS / Android]\n  SURFACE -->|developer| ARK[Volcano Engine Ark API]\n  ARK --> MODEL[Doubao model]\n  MODEL --> TOOLS{Tool options}\n  TOOLS --> FC[Function Calling]\n  TOOLS --> WS[Web Search 联网内容插件]\n  TOOLS --> KB[Knowledge Base Search]\n  TOOLS --> IMG[Image processing]\n  MODEL --> MODES{Request toggles}\n  MODES --> DT[Deep thinking]\n  MODES --> ST[Streaming]\n  MODES --> SO[Structured output beta]\n  MODES --> CC[Context caching]","members":[{"pattern":"agentic-rag","role":"first-class","note":"Ark's 联网内容插件 (Web Search plugin) provides real-time internet retrieval to Doubao models; Knowledge Search (私域知识库搜索) covers private-corpus RAG. The Chinese plugin docs document this verbatim.","evidence":[{"type":"doc","title":"Volcano Engine Ark — 联网内容插件 definition (Chinese)","url":"https://www.volcengine.com/docs/82379/1338552?lang=zh","quote":"联网内容插件，支持实时搜索互联网公开域网页资源和字节系的图文、视频资源，可以帮助大语言模型获取最新的、专业的信息","accessed":"2026-05-24"},{"type":"doc","title":"Volcano Engine Ark — 联网内容插件 benefits (Chinese)","url":"https://www.volcengine.com/docs/82379/1338552?lang=zh","quote":"实时补充与更新联网信源，支持高时效问题回答，有效弥补大模型幻觉、知识更新不及时的缺陷","accessed":"2026-05-24"},{"type":"doc","title":"Volcano Engine Ark — tool palette index (Chinese)","url":"https://www.volcengine.com/docs/82379/1958524","quote":"私域知识库搜索 Knowledge Search","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Function Calling (函数调用) is one of the documented categories under 工具调用 (Tool Calling) in the Volcano Engine Ark docs alongside Web Search and Knowledge Base Search; the Web Search plugin docs explicitly note it competes with Function Calling per-request.","evidence":[{"type":"doc","title":"Volcano Engine Ark — Function Calling reference (Chinese)","url":"https://www.volcengine.com/docs/82379/1262342?lang=zh","quote":"Function Calling（函数调用）","accessed":"2026-05-20"},{"type":"doc","title":"Volcano Engine Ark — Function Calling vs Web Search (Chinese)","url":"https://www.volcengine.com/docs/82379/1338552?lang=zh","quote":"开启此插件时，Function Calling 功能不生效","accessed":"2026-05-24"},{"type":"doc","title":"Volcano Engine Ark — tool calling categories (Chinese)","url":"https://www.volcengine.com/docs/82379/1958524","quote":"Function Calling（函数调用）","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"structured-output","role":"supported","note":"Ark docs list 结构化输出 (structured output) as a beta feature under advanced usage; available via the Responses API. Detailed schema reference content was not directly visible to scrapers, but the feature is named explicitly in the navigation and feature lists.","evidence":[{"type":"doc","title":"Volcano Engine Ark — structured output beta (Chinese)","url":"https://www.volcengine.com/docs/82379/1958523","quote":"结构化输出(beta)","accessed":"2026-05-24"},{"type":"doc","title":"Volcano Engine Ark — structured output beta in nav (Chinese)","url":"https://www.volcengine.com/docs/82379/1568221?lang=zh","quote":"结构化输出(beta)","accessed":"2026-05-24"}],"evidence_status":"full"}],"references":[{"type":"doc","title":"Volcano Engine Doubao product page","url":"https://www.volcengine.com/product/doubao","accessed":"2026-05-20"},{"type":"doc","title":"Volcano Engine Ark documentation index","url":"https://www.volcengine.com/docs/82379","accessed":"2026-05-20"},{"type":"doc","title":"Wikipedia — Doubao","url":"https://en.wikipedia.org/wiki/Doubao","quote":"By November 2024, it had become China's most popular AI chatbot, with approximately 60 million monthly active users","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"mature","tags":["chinese-origin","bytedance","model-vendor","consumer-chat","volcano-engine"]},{"id":"genspark","name":"Genspark","kind":"framework","category":"model-vendor-agent","build_surface":"closed-application","vendor":"MainFunc","language":"Web product","license":"proprietary","status":"active","url":"https://www.genspark.ai","intent":"Consumer general-purpose 'super agent' / AI workspace blending multi-LLM routing, browser actions, code execution, generated 'Sparkpages' and downstream apps; built and operated by MainFunc.","description":"Genspark markets itself as an AI-powered workspace whose Super Agent autonomously completes user tasks — from research to running multi-step workflows. The company's GitHub publicly showcases Clawverse, claimed to be built by 100+ autonomous coding sprints of 17,500 lines with zero human-written code. Public, citable technical documentation for Genspark's internal agent loop is scarce: most claims about a mixture-of-agents (MoA) or specific tool counts circulate through press/marketing rather than verifiable docs. The product page and several deep technical pages return 403 to unauthenticated scrapers, so several claims are recorded here at evidence_status='none'.","primary_use_cases":["consumer research and task agent","generated 'Sparkpages' answer pages","AI-built apps / artifacts (e.g. Clawverse)","broad task automation across browser, search, code"],"agent_loop_shape":"Hosted consumer agent loop; not documented in citable upstream form. From public surfaces and the GitHub showcase: a user task is decomposed into sub-actions executed against multiple LLMs, a browser surface and code-execution capability; results are aggregated into long-form 'Sparkpages' or app artifacts. Internal details (which models, what routing, what step budget) are not published.","key_concepts":[{"name":"Super Agent","summary":"Marketed top-level autonomous agent that completes user-defined tasks; one Super Bowl ad described it as a tool that 'completes tasks for that day'.","url":"https://www.genspark.ai"},{"name":"Sparkpages","summary":"Generated long-form answer/report pages produced as agent output. Detailed product docs are not publicly fetchable.","url":"https://www.genspark.ai"},{"name":"Clawverse","summary":"Showcase project on Genspark's GitHub claimed to be built entirely by autonomous coding sprints.","url":"https://github.com/genspark-ai"}],"pattern_composition":"flowchart TD\n  USER[User task] --> SUPER[Super Agent]\n  SUPER --> ROUTE{Route work}\n  ROUTE --> LLMS[Multiple LLMs]\n  ROUTE --> BROWSER[Browser surface]\n  ROUTE --> CODE[Code execution]\n  LLMS --> AGG[Aggregate]\n  BROWSER --> AGG\n  CODE --> AGG\n  AGG --> OUT{Output}\n  OUT --> SPARKPAGE[Sparkpage]\n  OUT --> APP[App artifact e.g. Clawverse]","members":[{"pattern":"agentic-rag","role":"first-class","note":"Sparkpages and 'workspace' positioning imply tool-driven retrieval over the open web, but no upstream technical doc was found that describes the retrieval pipeline in detail.","evidence":[],"evidence_status":"none"},{"pattern":"browser-agent","role":"first-class","note":"Genspark publishes an Omaha-for-Genspark-browser repo on GitHub, indicating an in-house Chromium-based browser surface, but the technical model of how the agent drives it is not publicly documented.","evidence":[{"type":"repo","title":"Genspark AI GitHub — Omaha for Genspark browser","url":"https://github.com/genspark-ai","quote":"Omaha for Genspark browser","accessed":"2026-05-20"}],"evidence_status":"limited"},{"pattern":"code-execution","role":"first-class","note":"The Clawverse showcase claim implies the agent runs and iterates on generated code; details of the execution environment are not in the public docs we could fetch.","evidence":[{"type":"repo","title":"Genspark AI GitHub — Clawverse","url":"https://github.com/genspark-ai","quote":"Clawverse — Build your pixel island, visit friends, collect everything. Built entirely by Genspark AI — 100+ autonomous coding sprints, 17,500 lines of code, zero human-written code.","accessed":"2026-05-20"}],"evidence_status":"limited"},{"pattern":"mixture-of-experts-routing","role":"first-class","note":"Genspark Super Agent's headline architecture is a Mixture-of-Agents that orchestrates nine LLMs plus 80+ in-house tools, dynamically routing each subtask to the best-suited model. Genspark.ai also runs a primary blog post on the design ('Less Control, More Tools') and OpenAI features the launch on openai.com/index/genspark/.","evidence":[{"type":"blog","title":"MarkTechPost — Meet Genspark Super Agent (quoting Genspark's own architecture)","url":"https://www.marktechpost.com/2025/04/05/meet-genspark-super-agent-the-all-in-one-ai-agent-that-autonomously-think-plan-act-and-use-tools-to-handle-all-your-everyday-tasks/","quote":"GenSpark's unique architecture utilizes a 'Mixture-of-Agents' design, integrating nine distinct large language models (LLMs), over 80 in-house tools, and more than ten curated datasets.","accessed":"2026-05-24"},{"type":"blog","title":"Genspark blog — Less Control, More Tools (own engineering post)","url":"https://www.genspark.ai/blog/genspark-less-control-more-tools","quote":"Less Control, More Tools: How Genspark Built a Super Agent That Scales","accessed":"2026-05-24"},{"type":"blog","title":"OpenAI customer story — Genspark on GPT-4.1 + Realtime API","url":"https://openai.com/index/genspark/","quote":"Genspark ships no-code personal agents with GPT-4.1 and OpenAI Realtime API","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"multilingual-voice-agent","role":"first-class","note":"Claimed as part of Super Agent's surface area in marketing; no verifiable upstream technical doc reached.","evidence":[],"evidence_status":"none"},{"pattern":"orchestrator-workers","role":"first-class","note":"Implied by the Super Agent + sub-tools framing; not documented in citable upstream sources.","evidence":[],"evidence_status":"none"},{"pattern":"plan-and-execute","role":"first-class","note":"Implied by autonomous task completion; not documented in citable upstream sources.","evidence":[],"evidence_status":"none"},{"pattern":"supervisor","role":"first-class","note":"Implied by the 'super agent over sub-agents' framing; not documented in citable upstream sources.","evidence":[],"evidence_status":"none"},{"pattern":"tool-use","role":"first-class","note":"Tool use is central to the Super Agent claim; specific tool catalogue and invocation contract not publicly documented.","evidence":[],"evidence_status":"none"}],"alternatives":[{"composition":"manus","relation":"competes-with","note":"Another general autonomous-agent product launched 2024-2025."}],"references":[{"type":"repo","title":"Genspark AI GitHub org","url":"https://github.com/genspark-ai","accessed":"2026-05-20"},{"type":"doc","title":"Wikipedia — Super Bowl commercials (Genspark mention)","url":"https://en.wikipedia.org/wiki/List_of_Super_Bowl_commercials","quote":"The AI powered workspace tool Genspark aired a commercial starring Matthew Broderick","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"emerging","tags":["consumer-agent","general-agent","us-startup","thin-public-docs"]},{"id":"kimi-agent","name":"Kimi (Moonshot)","kind":"framework","category":"model-vendor-agent","build_surface":"closed-application","vendor":"Moonshot AI","language":"Web product + API","license":"proprietary (Kimi product) / open-weight MIT (Kimi K2)","status":"active","url":"https://kimi.com","docs_url":"https://platform.kimi.ai/docs","intent":"Moonshot's Kimi assistant and the open-weight Kimi K2 model family — a long-context (256K) MoE model explicitly trained for tool calling and agentic behaviour, surfaced through the Kimi consumer product line (Deep Research, Agent Swarm, Kimi Code, OK Computer).","description":"Kimi began in 2023 as Moonshot AI's chatbot known for 128K-token context; subsequent versions raised the window to 256K and added multimodal text/image/video input. Kimi K2 (released July 2025) is the open-weight 1T-total / 32B-active MoE model behind the product, explicitly trained for tool use and autonomous problem-solving and licensed permissively. The Kimi consumer surface bundles long-context chat with Deep Research, Agent Swarm, Slides/Docs/Sheets/Websites generation, and a sub-product called OK Computer that can build multi-page sites and editable slides from prompts and process up to 1M rows of input data. Public English documentation of the underlying API's tool-call schema redirects between several domains (platform.moonshot.cn → platform.kimi.com → platform.kimi.ai) and several pages were not directly fetchable; primary technical evidence here comes from the Kimi K2 model documentation.","primary_use_cases":["long-context document analysis and Q&A","agentic tool-calling deployments via Kimi K2 open weights","Deep Research and Agent Swarm consumer workflows","AI-built slides, websites, sheets and docs from prompts"],"agent_loop_shape":"Two surfaces. (1) Kimi consumer app: Moonshot-hosted assistant; user starts a chat or invokes a sub-product (Deep Research, Agent Swarm, OK Computer); model autonomously calls internal tools and produces multi-modal artifacts. (2) Kimi K2 API: standard chat-completions tool-call loop where the model autonomously decides when and how to invoke tools, designed for drop-in general chat and agentic experiences.","key_concepts":[{"name":"Kimi K2","summary":"Open-weight 1T total / 32B active MoE model (384 experts, 8 routed per token) trained on 15.5T tokens; designed for tool use and autonomous problem-solving.","maps_to_pattern":"tool-use","url":"https://github.com/MoonshotAI/Kimi-K2"},{"name":"Long-context window (256K)","summary":"Kimi K2 / Kimi 2.6 supports 256K tokens of context, with K2 documentation also listing 128K base.","maps_to_pattern":"context-window-packing","url":"https://platform.kimi.com/docs"},{"name":"Agent Swarm","summary":"Product feature in the Kimi consumer app advertised on the home page alongside Deep Research, Kimi Code and others.","url":"https://kimi.com"},{"name":"Deep Research","summary":"Kimi consumer research mode; the older Kimi Explore Edition reportedly deeply reads 500+ pages per search.","maps_to_pattern":"agentic-rag","url":"https://kimi.com"},{"name":"OK Computer","summary":"Kimi sub-product that builds multi-page websites and editable slides from prompts and processes up to 1M rows of input data.","url":"https://kimi.com"}],"pattern_composition":"flowchart TD\n  USER[User] --> SURFACE{Surface}\n  SURFACE -->|consumer| KIMI[Kimi chat app]\n  KIMI --> SUBS[Sub-products]\n  SUBS --> DR[Deep Research]\n  SUBS --> SWARM[Agent Swarm]\n  SUBS --> CODE[Kimi Code]\n  SUBS --> OK[OK Computer<br/>sites + slides]\n  SURFACE -->|API| K2[Kimi K2 open-weight MoE<br/>1T total / 32B active]\n  K2 --> CTX[256K context]\n  K2 --> TOOLS[Autonomous tool-calling]\n  TOOLS --> APP[Caller-side agent harness]","members":[{"pattern":"agentic-rag","role":"first-class","note":"Kimi Explore / Deep Research is positioned as autonomous AI-powered search. K2.5 is documented as equipped with search, code-interpreter and web-browsing tools and explicitly enables deep research and report generation.","evidence":[{"type":"doc","title":"Wikipedia — Kimi (chatbot), Explore Edition","url":"https://en.wikipedia.org/wiki/Kimi_(chatbot)","quote":"Kimi Explore Edition, featuring AI-powered autonomous search, went live globally","accessed":"2026-05-20"},{"type":"blog","title":"Kimi K2.5 tech blog — agentic tooling","url":"https://www.kimi.com/blog/kimi-k2-5","quote":"search, code-interpreter, and web-browsing tools","accessed":"2026-05-24"},{"type":"doc","title":"Kimi K2.5 model page — deep research","url":"https://www.kimi.com/ai-models/kimi-k2-5","quote":"Kimi K2.5 enables deep research and report generation for complex problems.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"browser-agent","role":"first-class","note":"Kimi K2.5 ships Kimi WebBridge — a browser extension for AI agents — and turns prompts into full websites. K2.5 is positioned as a visual agentic model with web-browsing tools.","evidence":[{"type":"doc","title":"Wikipedia — Kimi (chatbot), OK Computer","url":"https://en.wikipedia.org/wiki/Kimi_(chatbot)","quote":"OK Computer...is capable of creating multi-page websites and editable slides from simple user prompts and can process up to 1 million rows of input data at once.","accessed":"2026-05-20"},{"type":"doc","title":"Kimi K2.5 model page — WebBridge","url":"https://www.kimi.com/ai-models/kimi-k2-5","quote":"Kimi WebBridge: A browser extension for AI agents","accessed":"2026-05-24"},{"type":"doc","title":"Kimi K2.5 model page — websites","url":"https://www.kimi.com/ai-models/kimi-k2-5","quote":"From recreating complex page layouts to building full websites with animations","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"context-window-packing","role":"first-class","note":"Long context is Kimi's signature: initial 128K, later 256K.","evidence":[{"type":"doc","title":"Wikipedia — Kimi initial 128K context","url":"https://en.wikipedia.org/wiki/Kimi_(chatbot)","quote":"Its first version, released in 2023, was known for supporting up to 128,000 tokens of context.","accessed":"2026-05-20"},{"type":"doc","title":"Wikipedia — Kimi 256K context","url":"https://en.wikipedia.org/wiki/Kimi_(chatbot)","quote":"increased its context window from 128K tokens to 256K tokens.","accessed":"2026-05-20"},{"type":"doc","title":"Kimi platform docs (Chinese)","url":"https://platform.kimi.com/docs","quote":"支持256K上下文长度","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Kimi K2 is explicitly designed for tool use; the model autonomously decides when and how to invoke tools.","evidence":[{"type":"repo","title":"Kimi K2 repo — tool calling","url":"https://github.com/MoonshotAI/Kimi-K2","quote":"Kimi-K2-Instruct has strong tool-calling capabilities.","accessed":"2026-05-20"},{"type":"repo","title":"Kimi K2 repo — autonomous tool selection","url":"https://github.com/MoonshotAI/Kimi-K2","quote":"the model will autonomously decide when and how to invoke them.","accessed":"2026-05-20"},{"type":"repo","title":"Kimi K2 repo — designed for agentic use","url":"https://github.com/MoonshotAI/Kimi-K2","quote":"Specifically designed for tool use, reasoning, and autonomous problem-solving.","accessed":"2026-05-20"}],"evidence_status":"full"}],"references":[{"type":"repo","title":"Kimi K2 GitHub README","url":"https://github.com/MoonshotAI/Kimi-K2","accessed":"2026-05-20"},{"type":"doc","title":"Kimi AI product page","url":"https://kimi.com","quote":"Kimi AI with K2.6 | Better Coding, Smarter Agents","accessed":"2026-05-20"},{"type":"doc","title":"Wikipedia — Kimi (chatbot)","url":"https://en.wikipedia.org/wiki/Kimi_(chatbot)","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"mature","tags":["chinese-origin","open-weight","long-context","moonshot","consumer-chat"]},{"id":"manus","name":"Manus","kind":"framework","category":"model-vendor-agent","build_surface":"closed-application","vendor":"Meta (formerly Butterfly Effect AI / Monica)","language":"N/A","license":"proprietary","status":"active","first_released":"2025-03-05","url":"https://manus.im/","docs_url":"https://manus.im/blog","intent":"Provide a general-purpose autonomous AI agent that completes end-to-end knowledge work in a dedicated cloud sandbox — browsing, executing code, editing files, tracking a todo.md plan — and fanning out into hundreds of parallel sub-agent clones when a task suits wide-rather-than-deep work.","description":"Manus, originally launched in March 2025 and acquired by Meta in December 2025, is a hosted general AI agent: every task is given its own isolated cloud VM in which the agent reads/writes a todo.md plan, drives a browser, runs Python/JavaScript/Bash, and operates a Linux filesystem. The product pioneered the Wide Research / clone fan-out pattern, in which a main Manus instance spawns up to ~100 fully-capable sub-agents — each a general-purpose Manus, each in its own context window — to process bulk similar items in parallel. The team has publicly documented their context engineering practice — KV-cache stability, file-system-as-context, masking tool logits instead of hot-swapping tool definitions — as the operational core of the loop. The company continues to operate from Singapore after joining Meta.","primary_use_cases":["general-purpose autonomous knowledge work in a hosted sandbox","Wide Research / clone fan-out tasks over hundreds of similar items","long-horizon agent runs needing browser + code + file tools"],"agent_loop_shape":"Imperative ReAct-style outer loop driving a long-lived cloud VM. Each task allocated its own Manus Sandbox (fully-isolated Ubuntu VM with networking, filesystem, headless Chromium, shell). One action per iteration, executed in-sandbox, observation appended, cycle repeats. Manus typically writes and maintains a todo.md plan inside the sandbox so the current goal stays at the recent end of the context. Typical task ~50 iterations. Sandboxes persist for hours, auto-sleep when idle, auto-wake on user return. Wide Research sub-protocol spawns N parallel sub-agents, each itself a general-purpose Manus in its own context.","key_concepts":[{"name":"Manus Sandbox","summary":"Fully-isolated cloud VM allocated per task — networking, filesystem, browser, software tools.","maps_to_pattern":"sandbox-isolation","url":"https://manus.im/blog/manus-sandbox"},{"name":"todo.md plan file","summary":"Persistent markdown plan the agent authors and updates step-by-step inside the sandbox; keeps the current goal near the end of context.","maps_to_pattern":"todo-list-driven-agent","url":"https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus"},{"name":"Wide Research / 副本扇出","summary":"Protocol to spawn dozens-to-hundreds of full-capability Manus clones in parallel; each clone gets its own context and depth equal to the main agent's.","maps_to_pattern":"clone-fan-out-research","url":"https://manus.im/blog/introducing-wide-research"},{"name":"Manus Browser Operator","summary":"Browser extension letting Manus operate directly inside the user's local browser environment.","maps_to_pattern":"browser-agent","url":"https://manus.im/blog/manus-browser-operator"},{"name":"File system as ultimate context","summary":"Manus treats the sandbox filesystem as unlimited, persistent, directly-operable context that the agent reads and writes instead of stuffing the prompt.","url":"https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus"},{"name":"Tool-masking state machine","summary":"Rather than hot-swapping tool definitions mid-task (which breaks KV cache), Manus masks tool-name token logits during decoding to constrain available actions.","url":"https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus"}],"pattern_composition":"flowchart TD\n  USER[User task] --> MAIN[Manus main instance]\n  MAIN --> SB[Manus Sandbox<br/>per-task cloud VM]\n  SB --> TODO[todo.md plan<br/>append + check-off]\n  SB --> BROWSER[Chromium browser]\n  SB --> SHELL[Bash / Python / JS]\n  SB --> FS[(Filesystem<br/>persistent context)]\n  MAIN -->|loop ~50 iters| ACT{Action selection<br/>tool-masking state machine}\n  ACT --> BROWSER\n  ACT --> SHELL\n  ACT --> FS\n  ACT -->|observation + failures kept| MAIN\n  SB -.->|auto-sleep/wake| SB\n  MAIN -.->|Wide Research fan-out| CLONES[N parallel Manus clones<br/>each full-capability, own context]\n  CLONES -.->|synthesise results| MAIN","members":[{"pattern":"agent-resumption","role":"first-class","note":"Sandboxes auto-sleep on inactivity and auto-wake on user return; files/data are preserved across the sleep/wake cycle. E2B sandboxes can run for hours in persistent sessions.","evidence":[{"type":"blog","title":"Manus Sandbox — sleep/wake","url":"https://manus.im/blog/manus-sandbox","quote":"When the Sandbox is inactive (no operations, file edits, etc.), it will automatically go to sleep. When you return to the task and Manus needs to operate the Sandbox, it will be automatically woken up. During the sleep/wake cycle, the files and data in the Sandbox remain unchanged.","accessed":"2026-05-20"},{"type":"blog","title":"E2B — Manus persistent sessions","url":"https://e2b.dev/blog/how-manus-uses-e2b-to-provide-agents-with-virtual-computers","quote":"E2B sandboxes can run for hours in persistent sessions while the agent decides in each iteration which action to perform.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"browser-agent","role":"first-class","note":"Browser Operator extension drives the user's local browser; Manus 1.5's integrated browser also runs end-to-end test/fix loops against what it builds.","evidence":[{"type":"blog","title":"Manus Browser Operator","url":"https://manus.im/blog/manus-browser-operator","quote":"a browser extension that enables Manus to operate directly within your local browser environment.","accessed":"2026-05-20"},{"type":"blog","title":"Manus 1.5 release — integrated browser","url":"https://manus.im/blog/manus-1.5-release","quote":"With its integrated browser, Manus can actively test what it builds — launching the application, interacting with it like a real user, detecting issues, and fixing them autonomously","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"computer-use","role":"first-class","note":"Manus Sandbox is a full cloud VM (networking, filesystem, browser, shell) supporting Python/JavaScript/Bash execution alongside browser control.","evidence":[{"type":"blog","title":"Manus Sandbox — fully isolated VM","url":"https://manus.im/blog/manus-sandbox","quote":"Manus Sandbox is a fully isolated cloud virtual machine that Manus allocates for each task.","accessed":"2026-05-20"},{"type":"blog","title":"Manus Sandbox — capabilities","url":"https://manus.im/blog/manus-sandbox","quote":"It has full capabilities: networking, file system, browser, various software tools.","accessed":"2026-05-20"},{"type":"blog","title":"E2B — Manus executors","url":"https://e2b.dev/blog/how-manus-uses-e2b-to-provide-agents-with-virtual-computers","quote":"agents can run Python, JavaScript, Bash, and more","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"Per-iteration single action chosen from a predefined action space, executed in the sandbox, observation appended; standard ReAct loop.","evidence":[{"type":"blog","title":"Context Engineering — ReAct loop","url":"https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus","quote":"After receiving a user input, the agent proceeds through a chain of tool uses to complete the task. In each iteration, the model selects an action from a predefined action space based on the current context. The action is then executed in the environment (e.g., Manus's virtual machine sandbox) to produce an observation.","accessed":"2026-05-20"},{"type":"blog","title":"Context Engineering — observation appended to context","url":"https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus","quote":"The action and observation are appended to the context, forming the input for the next iteration.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"step-budget","role":"supported","note":"Manus does not expose a user-facing hard step cap, but its loop is dimensioned in iterations — the published anchor is ~50 tool calls per typical task. Downgraded from first-class because there is no exposed knob.","evidence":[{"type":"blog","title":"Context Engineering — typical task length","url":"https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus","quote":"A typical task in Manus requires around 50 tool calls on average.","accessed":"2026-05-20"},{"type":"blog","title":"Context Engineering — long loop drift","url":"https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus","quote":"That's a long loop—and since Manus relies on LLMs for decision-making, it's vulnerable to drifting off-topic or forgetting earlier goals.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"subagent-isolation","role":"first-class","note":"Wide Research sub-agents are full-capability general-purpose Manus instances, each with its own dedicated context window. Sandboxes also isolate tasks from each other at the VM level.","evidence":[{"type":"blog","title":"Wide Research — full-capability sub-agents","url":"https://manus.im/blog/introducing-wide-research","quote":"every subagent in Wide Research is a fully capable, general-purpose Manus instance.","accessed":"2026-05-20"},{"type":"doc","title":"Wide Research 中文 docs — per-project context (Mandarin)","url":"https://manus.im/docs/zh-cn/features/wide-research","quote":"项目 #250 获得了与项目 #1 相同的分析深度，因为每个项目都有自己的专用 Agent 和完整的上下文窗口","accessed":"2026-05-20"},{"type":"blog","title":"Manus Sandbox — VM-level isolation","url":"https://manus.im/blog/manus-sandbox","quote":"Each Sandbox runs in its own environment, does not affect other tasks, and can execute in parallel.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Tool palette includes Chromium browser, terminal, filesystem; tool availability is regulated by a context-aware state machine that masks token logits rather than removing tool definitions.","evidence":[{"type":"blog","title":"Context Engineering — tool-masking state machine","url":"https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus","quote":"Manus uses a context-aware state machine to manage tool availability. Rather than removing tools, it masks the token logits during decoding to prevent (or enforce) the selection of certain actions based on the current context.","accessed":"2026-05-20"},{"type":"blog","title":"E2B — Manus tool palette","url":"https://e2b.dev/blog/how-manus-uses-e2b-to-provide-agents-with-virtual-computers","quote":"using the Chromium browser to visit URLs, save images, and scroll, through executing terminal commands to using the filesystem to create, edit, or delete files","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"clone-fan-out-research","role":"core","note":"Wide Research is Manus's signature feature: a system-level mechanism for parallel processing and an agent-to-agent collaboration protocol that runs ~20 sub-tasks (or hundreds) simultaneously.","evidence":[{"type":"blog","title":"Introducing Wide Research","url":"https://manus.im/blog/introducing-wide-research","quote":"Wide Research is a system-level mechanism for parallel processing, and a protocol for agent-to-agent collaboration.","accessed":"2026-05-20"},{"type":"doc","title":"Help Center — Wide Research","url":"https://help.manus.im/en/articles/11960169-what-is-wide-research","quote":"Wide Research can run 20 subtasks simultaneously, equivalent to having 20 Agents helping with one task.","accessed":"2026-05-20"},{"type":"doc","title":"Wide Research 中文 docs — parallel scale (Mandarin)","url":"https://manus.im/docs/zh-cn/features/wide-research","quote":"数百个并行工作的独立 Agent","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"todo-list-driven-agent","role":"first-class","note":"Manus authors a todo.md and updates it step-by-step; the file keeps the current goal at the recent end of context.","evidence":[{"type":"blog","title":"Context Engineering — todo.md plan","url":"https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus","quote":"when handling complex tasks, it tends to create a todo.md file—and update it step-by-step as the task progresses, checking off completed items.","accessed":"2026-05-20"},{"type":"blog","title":"Context Engineering — todo as recitation","url":"https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus","quote":"By constantly rewriting the todo list, Manus is reciting its objectives into the end of the context. This pushes the global plan into the model's recent attention span.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"filesystem-as-context","role":"core","note":"Manus treats the filesystem as the ultimate context — unlimited, persistent, and directly operable by the agent — and designs compression to be restorable from files.","evidence":[{"type":"blog","title":"Context Engineering for AI Agents: Lessons from Building Manus","url":"https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus","quote":"That's why we treat the file system as the ultimate context in Manus: unlimited in size, persistent by nature, and directly operable by the agent itself.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"tool-result-reinforcement","role":"supported","note":"Manus recites its goal back into the end of the context after actions and deliberately leaves failed actions and their observations in, so each return carries goal re-grounding and a failure steer.","evidence":[{"type":"doc","title":"Context Engineering for AI Agents: Lessons from Building Manus","url":"https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus","quote":"By constantly rewriting the todo list, Manus is reciting its objectives into the end of the context.","accessed":"2026-06-14"},{"type":"doc","title":"Context Engineering for AI Agents: Lessons from Building Manus","url":"https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus","quote":"One of the most effective ways to improve agent behavior is deceptively simple: leave the wrong turns in the context.","accessed":"2026-06-14"}],"evidence_status":"full"},{"pattern":"standing-state-injection","role":"core","note":"Manus rewrites a todo.md each step so the current objectives are recited into the end of the context, pushing the global plan into recent attention to avoid lost-in-the-middle drift across ~50 tool calls.","evidence":[{"type":"blog","title":"Context Engineering for AI Agents: Lessons from Building Manus","url":"https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus","quote":"By constantly rewriting the todo list, Manus is reciting its objectives into the end of the context.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"instantiates":["long-running-autonomous-agent","browser-computer-use-stack"],"anti_patterns_avoided":[{"pattern":"errors-swept-under-the-rug","note":"Manus deliberately leaves failed actions and stack traces in the context so the model can adapt; erasing failure removes the evidence the model needs to learn from.","evidence":[{"type":"blog","title":"Context Engineering — keep failures in context","url":"https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus","quote":"When the model sees a failed action—and the resulting observation or stack trace—it implicitly updates its internal beliefs.","accessed":"2026-05-20"}]},{"pattern":"role-typed-subagents","note":"Wide Research sub-agents are deliberately NOT typed as manager/coder/designer; each is a full-capability general-purpose Manus.","evidence":[{"type":"blog","title":"Wide Research — vs role-typed multi-agent","url":"https://manus.im/blog/introducing-wide-research","quote":"predefined roles (like 'manager', 'coder', or 'designer')","accessed":"2026-05-20"}]},{"pattern":"tool-loadout-hotswap","note":"Manus masks token logits instead of adding/removing tools mid-iteration, avoiding the cache-invalidation and confusion of hot-swapping the tool loadout.","evidence":[{"type":"doc","title":"Manus — context engineering","url":"https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus","quote":"unless absolutely necessary, avoid dynamically adding or removing tools mid-iteration ... Rather than removing tools, it masks the token logits during decoding to prevent (or enforce) the selection of certain actions based on the current context","accessed":"2026-06-19"}]}],"alternatives":[{"composition":"openmanus","relation":"similar-shape","note":"Open-source clone reproducing Manus's browser + sandbox + plan-file loop."},{"composition":"genspark","relation":"competes-with","note":"Another general autonomous-agent product launched 2024-2025."},{"composition":"devin","relation":"competes-with","note":"Cognition's autonomous AI engineer with a sandbox + browser + plan loop; closest US competitor pre-Manus."},{"composition":"anthropic-computer-use","relation":"competes-with","note":"Anthropic's computer-use loop targets the same hosted-VM-plus-browser shape; Manus 1.5 explicitly integrates the Agent Skills standard."}],"references":[{"type":"blog","title":"Context Engineering for AI Agents — Lessons from Building Manus","url":"https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus","accessed":"2026-05-20"},{"type":"blog","title":"Manus joins Meta for next era of innovation","url":"https://manus.im/blog/manus-joins-meta-for-next-era-of-innovation","quote":"Manus is joining Meta","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-06-19","date_added":"2026-04-30","status_in_practice":"emerging","tags":["hosted","general-agent","wide-research","chinese-origin","singapore"]},{"id":"minimax-agent","name":"MiniMax Agent","kind":"framework","category":"model-vendor-agent","build_surface":"full-code","vendor":"MiniMax","language":"API + Web product","license":"proprietary (product) / open-weight (MiniMax-M1, M2)","status":"active","url":"https://www.minimax.io","docs_url":"https://platform.minimax.io","intent":"Shanghai-based MiniMax bundles a foundation-model line (ABAB → MiniMax-01 → M1 → M2.x) with a consumer MiniMax Agent that evaluates tasks, assembles an 'Agent Team' for them, and learns user-specific habits into custom skills.","description":"MiniMax was founded in 2021 by SenseTime alumni and listed in Hong Kong in January 2026. It develops a series of multimodal foundation models with strong code and Agent capabilities; the company markets MiniMax-M1 as the world's first open-source, large-scale, hybrid-attention reasoning model with an 80K-token reasoning output and tool-use benchmark leadership among open-weights. MiniMax Agent is the consumer surface: an assistant that evaluates a task, builds an agent team, learns user habits, and turns repetitive workflows into custom skills. The API exposes function calling for tool use; verifiable English platform docs were patchy at access time (several pages 404 or redirect repeatedly between intl.minimaxi.com / platform.minimax.io).","primary_use_cases":["consumer multi-skill assistant (work and life tasks)","agent-team task decomposition over arbitrary user goals","open-weight reasoning model for self-hosted tool-use agents","long-output reasoning (80K token thinking budget)"],"agent_loop_shape":"Two surfaces. (1) MiniMax Agent product: hosted loop that evaluates the user task, assembles an Agent Team, and dispatches sub-tasks; surfaces skills, memory and schedules in a single chat. (2) MiniMax API (M1 / M2.x via Chat Completions): standard tool-call loop; reasoning models expose a long 80K-token thinking budget and benchmark in TAU-bench style agentic tool-use scenarios.","key_concepts":[{"name":"MiniMax-M1 reasoning model","summary":"First open-source large-scale hybrid-attention reasoning model with 80K reasoning tokens and strong agentic tool-use benchmarks.","maps_to_pattern":"extended-thinking","url":"https://www.minimax.io/news/minimaxm1"},{"name":"MiniMax-M2.x line","summary":"Newer self-improvement-oriented reasoning + agent model line announced for production agent harnesses.","url":"https://www.minimax.io"},{"name":"Agent Team","summary":"Product feature where MiniMax Agent evaluates a task and assembles a team of agents to solve it.","maps_to_pattern":"orchestrator-workers","url":"https://www.minimax.io/agent"},{"name":"Custom Skills","summary":"Repetitive workflows the agent learns from the user's habits and turns into reusable skills.","maps_to_pattern":"skill-library","url":"https://www.minimax.io/agent"},{"name":"Function Calling","summary":"Tool-use API on the MiniMax platform; documented under the Function Calling endpoint reference.","maps_to_pattern":"tool-use","url":"https://platform.minimax.io"}],"pattern_composition":"flowchart TD\n  USER[User task] --> AGENT[MiniMax Agent chat]\n  AGENT --> EVAL[Evaluate task]\n  EVAL --> TEAM[Assemble Agent Team]\n  TEAM --> WORKERS[Sub-agents / skills]\n  WORKERS --> RESULT[Result]\n  AGENT --> SKILLS[(Custom Skills<br/>learnt from habits)]\n  AGENT --> MODEL{Backing model}\n  MODEL -->|reasoning| M1[MiniMax-M1<br/>80K reasoning tokens]\n  MODEL -->|production agent| M2[MiniMax-M2.x]\n  MODEL --> TOOLS[Function calling]\n  TOOLS -->|tool_calls + results| AGENT","members":[{"pattern":"structured-output","role":"first-class","note":"Function calling on MiniMax-M2.7 returns tool_calls with JSON-format arguments; the platform documents the schema of tools, tool_calls and function.arguments fields directly.","evidence":[{"type":"doc","title":"MiniMax M2 function call — tool_calls field","url":"https://platform.minimax.io/docs/guides/text-m2-function-call","quote":"`tool_calls`: Contains information about functions the model has decided to invoke","accessed":"2026-05-24"},{"type":"doc","title":"MiniMax M2 function call — JSON arguments","url":"https://platform.minimax.io/docs/guides/text-m2-function-call","quote":"`function.arguments`: Function call parameters (JSON string format)","accessed":"2026-05-24"},{"type":"doc","title":"MiniMax M2 function call — tools schema","url":"https://platform.minimax.io/docs/guides/text-m2-function-call","quote":"`tools`: Defines the list of callable functions, including function names, descriptions, and parameter schemas","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Documented as a top-level API capability ('agent tool use scenarios' in M1 release) and as a benchmarked competence of the open-weight models.","evidence":[{"type":"blog","title":"MiniMax-M1 release — agentic tool use","url":"https://www.minimax.io/news/minimaxm1","quote":"In agent tool use scenarios (TAU-bench), MiniMax-M1-40k also leads all open-weight models and outperforms Gemini-2.5 Pro","accessed":"2026-05-20"},{"type":"doc","title":"MiniMax company overview — agent capabilities","url":"https://www.minimax.io/platform_overview","quote":"independently developed a series of multimodal foundation models with powerful code and Agent capabilities","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"orchestrator-workers","role":"first-class","note":"MiniMax Agent's headline product behaviour is 'Build Agent Team: Evaluate tasks and assemble teams to solve problems' — an orchestrator-over-workers shape. The M2 launch blog also documents coordinated tool-calling and an AI-native organisation of multiple agents.","evidence":[{"type":"doc","title":"MiniMax Agent product page — Build Agent Team","url":"https://www.minimax.io/agent","quote":"Build Agent Team","accessed":"2026-05-20"},{"type":"doc","title":"MiniMax Agent product page — assemble teams","url":"https://www.minimax.io/agent","quote":"Evaluate tasks and assemble teams to solve problems.","accessed":"2026-05-24"},{"type":"blog","title":"MiniMax M2 launch — coordinated tool calling","url":"https://www.minimax.io/news/minimax-m2","quote":"outstanding planning and stable execution of complex, long-chain tool-calling tasks, coordinating calls to the Shell, Browser, Python code interpreter, and various MCP tools","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"skill-library","role":"first-class","note":"'Know Your Habits' turns repetitive tasks into custom skills accessible from the same chat surface.","evidence":[{"type":"doc","title":"MiniMax Agent — Know Your Habits heading","url":"https://www.minimax.io/agent","quote":"Know Your Habits","accessed":"2026-05-24"},{"type":"doc","title":"MiniMax Agent — custom skills body","url":"https://www.minimax.io/agent","quote":"Learn how you work and turn repetitive tasks into custom skills.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"extended-thinking","role":"first-class","note":"MiniMax-M1 surfaces an 80K-token reasoning output budget, claimed to be an industry leader for reasoning length.","evidence":[{"type":"blog","title":"MiniMax-M1 release — 80K reasoning output","url":"https://www.minimax.io/news/minimaxm1","quote":"includes an industry-leading 80,000 token reasoning output","accessed":"2026-05-20"},{"type":"blog","title":"MiniMax-M1 release — hybrid attention reasoning model","url":"https://www.minimax.io/news/minimaxm1","quote":"the world's first open-source, large-scale, hybrid-attention reasoning model","accessed":"2026-05-20"}],"evidence_status":"full"}],"references":[{"type":"blog","title":"MiniMax-M1 release notes","url":"https://www.minimax.io/news/minimaxm1","accessed":"2026-05-20"},{"type":"doc","title":"MiniMax platform overview","url":"https://www.minimax.io/platform_overview","accessed":"2026-05-20"},{"type":"doc","title":"MiniMax Agent product page","url":"https://www.minimax.io/agent","accessed":"2026-05-20"},{"type":"doc","title":"Wikipedia — MiniMax (company)","url":"https://en.wikipedia.org/wiki/MiniMax_(company)","quote":"MiniMax Group Inc. is an artificial intelligence (AI) company based in Shanghai, China.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"emerging","tags":["chinese-origin","shanghai","open-weight","reasoning","agent-team"]},{"id":"trae","name":"Trae","kind":"framework","category":"model-vendor-agent","build_surface":"full-code","vendor":"ByteDance","language":"TypeScript (Electron IDE) + Python (Trae Agent CLI)","license":"proprietary (IDE) / open-source (Trae Agent CLI)","status":"active","url":"https://trae.ai/","docs_url":"https://docs.trae.ai","intent":"ByteDance's AI IDE family: the Trae IDE with a unified Chat-Builder interface and @Agent + MCP multi-agent system, the autonomous TRAE SOLO coding agent, and the open-source Trae Agent CLI that topped SWE-bench Verified.","description":"Trae is ByteDance's AI coding stack. The IDE provides a unified Chat-Builder interface and an @Agent system with MCP for multi-agent collaboration. TRAE SOLO is positioned as a more autonomous responsive coding agent for real-world software development, with a mobile variant added in 2026. Separately, the open-source Trae Agent CLI (github.com/bytedance/trae-agent) ranked #1 on SWE-bench Verified with Claude 4 and provides a transparent, modular architecture researchers can extend; it supports MCP servers and a toolkit of file editing, bash execution and 'sequential thinking'.","primary_use_cases":["AI-assisted coding inside the Trae IDE","autonomous coding sessions via TRAE SOLO","research and benchmarking via the open-source Trae Agent CLI","MCP-driven multi-agent code workflows"],"agent_loop_shape":"Three surfaces. (1) Trae IDE Chat-Builder: user task in chat is routed to @Agent which composes tool calls and (optionally) sub-agents over MCP. (2) TRAE SOLO: more autonomous loop that executes longer, higher-trust workflows responsively. (3) Trae Agent CLI: a Python ReAct-style CLI taking a natural-language task, running tools (file editing, bash execution, sequential thinking) and optional MCP servers, until the task is done.","key_concepts":[{"name":"Chat-Builder unified interface","summary":"The Trae IDE's single surface combining conversational chat with a builder mode that produces concrete changes.","url":"https://www.trae.ai"},{"name":"@Agent system","summary":"Mention-based agent invocation that lets the user (or another agent) call a specific agent and pass context.","maps_to_pattern":"agent-as-tool-embedding","url":"https://www.trae.ai/blog"},{"name":"MCP multi-agent collaboration","summary":"MCP is the protocol Trae uses to compose multiple agents and external tool servers.","maps_to_pattern":"mcp","url":"https://github.com/bytedance/trae-agent"},{"name":"TRAE SOLO","summary":"Autonomous coding agent variant positioned for real-world software work; mobile variant added 2026.","url":"https://www.trae.ai/blog"},{"name":"Trae Agent CLI","summary":"Open-source Python CLI for general software-engineering tasks; tops SWE-bench Verified.","url":"https://github.com/bytedance/trae-agent"},{"name":"Sequential thinking tool","summary":"Built-in 'thinking' tool inside the Trae Agent toolkit alongside file editing and bash execution.","maps_to_pattern":"extended-thinking","url":"https://github.com/bytedance/trae-agent"}],"pattern_composition":"flowchart TD\n  USER[User] --> SURFACE{Surface}\n  SURFACE -->|IDE| IDE[Trae IDE<br/>Chat-Builder]\n  SURFACE -->|autonomous| SOLO[TRAE SOLO]\n  SURFACE -->|open-source| CLI[Trae Agent CLI]\n  IDE --> AGENTSYS[@Agent system]\n  AGENTSYS --> MCP[MCP servers / sub-agents]\n  CLI --> LOOP[ReAct loop]\n  LOOP --> TOOLS[file edit / bash / sequential thinking]\n  LOOP --> MCP\n  TOOLS --> LOOP\n  MCP --> LOOP","members":[{"pattern":"agent-computer-interface","role":"first-class","note":"Trae IDE is an Electron-based agent-aware IDE; the CLI variant provides a transparent agent-computer interface for software-engineering tasks.","evidence":[{"type":"repo","title":"Trae Agent README — CLI for software engineering","url":"https://github.com/bytedance/trae-agent","quote":"Trae Agent is an LLM-based agent for general purpose software engineering tasks. It provides a powerful CLI interface","accessed":"2026-05-20"},{"type":"blog","title":"Trae blog — Chat-Builder","url":"https://www.trae.ai/blog","quote":"unveils major updates: a unified Chat-Builder interface, @Agent system with MCP for multi-agent collaboration","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"MCP is documented both as the IDE's protocol for multi-agent collaboration and as a configuration-level integration point in the open-source CLI.","evidence":[{"type":"repo","title":"Trae Agent README — MCP config","url":"https://github.com/bytedance/trae-agent","quote":"To enable Model Context Protocol (MCP) services, add an `mcp_servers` section to your configuration","accessed":"2026-05-20"},{"type":"blog","title":"Trae blog — @Agent + MCP","url":"https://www.trae.ai/blog","quote":"@Agent system with MCP for multi-agent collaboration, and smarter context handling","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"Trae Agent CLI executes complex software-engineering workflows via an explicit tool-call loop with reasoning and structured thinking; same shape underlies the IDE's @Agent.","evidence":[{"type":"repo","title":"Trae Agent README — workflow execution","url":"https://github.com/bytedance/trae-agent","quote":"It provides a powerful CLI interface that can understand natural language instructions and execute complex software engineering workflows","accessed":"2026-05-20"},{"type":"repo","title":"Trae Agent README — tool ecosystem","url":"https://github.com/bytedance/trae-agent","quote":"Rich Tool Ecosystem: File editing, bash execution, sequential thinking, and more","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"subagent-isolation","role":"limited","note":"Trae IDE markets @Agent + MCP as multi-agent collaboration, with a dedicated @Agent blog ('Rules + Tools to Define Your Future Agents'). Specific isolation guarantees per sub-agent are not spelled out in the public blog/index text we could verify; the IDE's per-agent rule/tool restriction is the closest documented surface.","evidence":[{"type":"blog","title":"Trae blog — multi-agent collaboration","url":"https://www.trae.ai/blog","quote":"@Agent system with MCP for multi-agent collaboration","accessed":"2026-05-20"},{"type":"blog","title":"Trae blog — @Agent: Rules + Tools","url":"https://www.trae.ai/blog","quote":"@Agent: Rules + Tools to Define Your Future Agents","accessed":"2026-05-24"},{"type":"blog","title":"Trae blog — deep dive into @Agent","url":"https://www.trae.ai/blog","quote":"Deep dive into @Agent and how we built it.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Documented tool palette includes file editing, bash execution, and sequential thinking, plus any MCP servers added to the configuration. Configuration example lists exact tool names.","evidence":[{"type":"repo","title":"Trae Agent README — toolkit","url":"https://github.com/bytedance/trae-agent","quote":"Trae Agent provides a comprehensive toolkit for software engineering tasks including file editing, bash execution, structured thinking","accessed":"2026-05-20"},{"type":"repo","title":"Trae Agent README — Rich Tool Ecosystem","url":"https://github.com/bytedance/trae-agent","quote":"Rich Tool Ecosystem: File editing, bash execution, sequential thinking, and more","accessed":"2026-05-24"},{"type":"repo","title":"Trae Agent README — configured tool names","url":"https://github.com/bytedance/trae-agent","quote":"tools:  # tools used with Trae Agent\n      - bash\n      - str_replace_based_edit_tool\n      - sequentialthinking\n      - task_done","accessed":"2026-05-24"}],"evidence_status":"full"}],"alternatives":[{"composition":"marscode","relation":"same-vendor","note":"Both are ByteDance coding products on Doubao; Trae is the standalone IDE, MarsCode the extension + cloud IDE."}],"references":[{"type":"repo","title":"Trae Agent open-source CLI","url":"https://github.com/bytedance/trae-agent","accessed":"2026-05-20"},{"type":"blog","title":"Trae blog index","url":"https://www.trae.ai/blog","accessed":"2026-05-20"},{"type":"blog","title":"Trae Agent #1 on SWE-bench Verified","url":"https://www.trae.ai/blog","quote":"Trae Agent reached #1 on SWE-bench Verified, with Claude 4 — and it's open-sourced.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"emerging","tags":["bytedance","ide","coding-agent","mcp","open-source-cli"]},{"id":"zhipu-glm-agent","name":"Zhipu GLM Agent","kind":"framework","category":"model-vendor-agent","build_surface":"full-code","vendor":"Zhipu AI (Z.ai)","language":"API","license":"proprietary (hosted API) / MIT (GLM open weights since July 2025)","status":"active","url":"https://open.bigmodel.cn","docs_url":"https://docs.bigmodel.cn","intent":"Tsinghua-spinoff Zhipu (rebranded internationally as Z.ai in July 2025) ships the GLM model line through the BigModel platform, with function calling, web search, knowledge-base retrieval and JSON output all available as tool/response options on the same chat-completion API; AutoGLM is the consumer agent built on top.","description":"Zhipu (Z.ai) is one of China's three biggest LLM providers and IPO'd on the Hong Kong Stock Exchange on 8 January 2026. Its GLM model line has been MIT-open-source since July 2025 (GLM-4.5 onward), with the latest GLM-5/5.1 and GLM-5V-Turbo positioned for agentic engineering, coding and long-chain execution. The BigModel chat-completion API supports tools: function calling, knowledge-base retrieval and web search are listed together as tool options, and structured output is selectable via response_format json_object. AutoGLM is the consumer agent application that uses voice commands to complete tasks on a phone and can self-execute operations exceeding 50 steps across applications.","primary_use_cases":["function-calling agents over GLM hosted API","RAG via knowledge-base retrieval and web search as built-in tools","JSON-structured extraction via response_format","consumer voice agent (AutoGLM) on smartphones"],"agent_loop_shape":"Two surfaces. (1) BigModel chat-completion API: standard tools-loop where tools[] can include functions, knowledge-base retrieval, or web search; the model emits tool_calls, the caller executes, results return as tool messages; response_format json_object pins shape. (2) AutoGLM: hosted consumer agent that takes voice commands, plans multi-step actions, and self-executes >50-step workflows across smartphone apps.","key_concepts":[{"name":"GLM-4.6 / GLM-5 / GLM-5.1","summary":"Flagship Zhipu models; GLM-4.6 raises context to 200K, GLM-5/5.1 positioned as base models for agentic engineering with open-source SOTA-level coding.","url":"https://docs.bigmodel.cn/cn/guide/start/model-overview"},{"name":"Tools parameter (function/retrieval/web search)","summary":"Single tools parameter accepts function calls, knowledge-base retrieval, and web search; model emits a JSON-typed call the caller handles.","maps_to_pattern":"tool-use","url":"https://docs.bigmodel.cn/cn/api-reference/llm/function-calling"},{"name":"response_format json_object","summary":"Forces the model to return structured JSON; documented as suitable for extraction and structured API responses.","maps_to_pattern":"structured-output","url":"https://docs.bigmodel.cn"},{"name":"AutoGLM","summary":"Consumer agent application that uses voice commands to complete tasks; can self-execute operations exceeding 50 steps across smartphone apps.","maps_to_pattern":"mobile-ui-agent","url":"https://www.zhipuai.cn/en/"},{"name":"GLM-5-Turbo for agent harnesses","summary":"Trained at the training level for core agent capabilities — tool calling, instruction following, long-chain execution — targeted at agent runtimes ('Claw').","url":"https://www.zhipuai.cn/en/"}],"pattern_composition":"flowchart TD\n  USER[User] --> SURFACE{Surface}\n  SURFACE -->|API| API[BigModel chat completion]\n  API --> TOOLS[tools parameter]\n  TOOLS --> FC[function call]\n  TOOLS --> KB[knowledge base retrieval]\n  TOOLS --> WS[web search]\n  API --> FMT[response_format json_object]\n  API --> GLM[GLM-4.x / 5.x / 5V-Turbo]\n  SURFACE -->|consumer| AUTO[AutoGLM]\n  AUTO --> VOICE[voice commands]\n  AUTO --> STEPS[>50-step cross-app execution]","members":[{"pattern":"agentic-rag","role":"first-class","note":"Web search and knowledge-base retrieval are listed together with function calling as values the tools[] parameter accepts — retrieval is model-callable, not a separate API. Knowledge-base retrieval is configured per-tool with knowledge_id + prompt_template; web search adds a dedicated Web Search API tuned for LLMs.","evidence":[{"type":"doc","title":"BigModel chat-completion docs — tools palette (Chinese)","url":"https://docs.bigmodel.cn","quote":"模型可以调用的工具列表。支持函数调用、知识库检索和网络搜索。使用此参数提供模型可以生成 `JSON` 输入的函数列表","accessed":"2026-05-20"},{"type":"doc","title":"BigModel docs — Knowledge base retrieval tool (Chinese)","url":"https://docs.bigmodel.cn/cn/guide/tools/knowledge/retrieval","quote":"通过在大语言模型生成答案之前，先从知识库中检索相关知识，然后将相关知识作为背景信息输入给大模型","accessed":"2026-05-24"},{"type":"doc","title":"BigModel docs — Web Search API (Chinese)","url":"https://docs.bigmodel.cn/cn/guide/tools/web-search","quote":"Web Search API 是一个专给大模型用的搜索引擎，在传统搜索引擎网页读取、排序的能力基础上，增强了意图识别能力","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"structured-output","role":"first-class","note":"response_format json_object is a top-level toggle that constrains the model to valid JSON; BigModel also documents JSON-Schema-based structured output on GLM-4.6 / GLM-4.7 with explicit Schema design guidance.","evidence":[{"type":"doc","title":"BigModel chat-completion docs — json_object (Chinese)","url":"https://docs.bigmodel.cn","quote":"{ \"type\": \"json_object\" } 表示`JSON`输出模式，模型会返回有效的`JSON`格式数据，适用于结构化数据提取、`API`响应生成等场景","accessed":"2026-05-20"},{"type":"doc","title":"BigModel docs — Structured output overview (Chinese)","url":"https://docs.bigmodel.cn/cn/guide/capabilities/struct-output","quote":"结构化输出（JSON 模式）可以确保 AI 返回符合预定义格式的 JSON 数据，为程序化处理 AI 输出提供可靠保障。","accessed":"2026-05-24"},{"type":"doc","title":"BigModel docs — response_format json_object parameter (Chinese)","url":"https://docs.bigmodel.cn/cn/guide/capabilities/struct-output","quote":"`response_format`: 指定响应格式，设置为 `{\"type\": \"json_object\"}` 启用 JSON 模式","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Function calling is a documented chat-completion capability; the model emits a function name + JSON arguments for the caller to execute.","evidence":[{"type":"doc","title":"BigModel chat-completion docs — tool calls (Chinese)","url":"https://docs.bigmodel.cn","quote":"生成的应该被调用的工具信息，流式返回时会逐步生成","accessed":"2026-05-20"},{"type":"doc","title":"BigModel chat-completion docs — function call output (Chinese)","url":"https://docs.bigmodel.cn","quote":"生成的函数名称和 `JSON` 格式参数","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mobile-ui-agent","role":"supported","note":"AutoGLM is positioned as a voice-driven smartphone agent that self-executes >50-step cross-application workflows; this is a separate consumer surface from the BigModel API.","evidence":[{"type":"doc","title":"Z.ai — AutoGLM","url":"https://www.zhipuai.cn/en/","quote":"An agent model capable of autonomous planning, reasoning, and execution, addressing core challenges in task planning, data scarcity, and policy optimization","accessed":"2026-05-20"},{"type":"doc","title":"Z.ai — AutoGLM step count","url":"https://www.zhipuai.cn/en/","quote":"AutoGLM can self-execute operations exceeding 50 steps and execute tasks across applications","accessed":"2026-05-20"},{"type":"doc","title":"Wikipedia — Z.ai / AutoGLM","url":"https://en.wikipedia.org/wiki/Zhipu","quote":"an AI agent application that uses voice commands to complete tasks within a smartphone","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"rumination-agent","role":"core","note":"A deep-reasoning 'rumination' model runs an extended think-search-verify loop over hundreds of steps, integrating search tools during deep thought to autonomously tackle open-ended research tasks.","evidence":[{"type":"doc","title":"GLM-Z1-Rumination-32B-0414 model card","url":"https://huggingface.co/zai-org/GLM-Z1-Rumination-32B-0414","quote":"GLM-Z1-Rumination-32B-0414 is a deep reasoning model with rumination capabilities ... the rumination model employs longer periods of deep thought to solve more open-ended and complex problems ... [and] integrates search tools during its deep thinking process to handle complex tasks.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"references":[{"type":"doc","title":"BigModel platform documentation","url":"https://docs.bigmodel.cn","accessed":"2026-05-20"},{"type":"doc","title":"Z.ai product page — GLM-5 / AutoGLM","url":"https://www.zhipuai.cn/en/","accessed":"2026-05-20"},{"type":"doc","title":"Wikipedia — Z.ai (Zhipu)","url":"https://en.wikipedia.org/wiki/Zhipu","quote":"released under the free and open-source MIT License since July 2025","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"emerging","tags":["chinese-origin","tsinghua","open-weight-mit","model-vendor","voice-agent"]},{"id":"llama-guard","name":"Meta Llama Guard 3","kind":"framework","category":"model-vendor-agent","vendor":"Meta","language":"N/A","license":"Llama 3.1 Community License","status":"active","first_released":"2024-07-23","url":"https://huggingface.co/meta-llama/Llama-Guard-3-8B","repo":"https://github.com/meta-llama/PurpleLlama","docs_url":"https://github.com/meta-llama/PurpleLlama/blob/main/Llama-Guard3/8B/MODEL_CARD.md","intent":"Llama Guard 3 is a fine-tuned Llama 3.1 8B safety classifier that labels prompts and responses as safe or unsafe against a fixed hazard taxonomy.","description":"Meta Llama Guard 3 is an open-weight content-safety classifier from Meta, built by fine-tuning the Llama-3.1-8B pretrained model for safety classification. It takes a prompt or a model response and generates a safe/unsafe label plus, when unsafe, the violated hazard categories. The classification can be applied to both LLM inputs and LLM outputs, giving the host application a signal to refuse out-of-policy content.","primary_use_cases":["classifying prompts and responses as safe or unsafe","input and output content moderation for LLM apps","emitting machine-readable hazard category codes","gating tool-call and code-interpreter abuse"],"agent_loop_shape":"Llama Guard is a single-pass classifier rather than an agent loop: it takes a conversation prompt or response and emits a safe/unsafe label with violated categories, and the calling application decides whether to allow, block, or refuse based on that output.","build_surface":"full-code","members":[{"pattern":"multimodal-guardrails","role":"core","note":"A natively multimodal safety classifier evaluates mixed text-and-image prompts and responses together, classifying each as safe or unsafe across modalities rather than text only.","evidence":[{"type":"doc","title":"meta-llama/Llama-Guard-4-12B model card","url":"https://huggingface.co/meta-llama/Llama-Guard-4-12B","quote":"Llama Guard 4 is a natively multimodal safety classifier with 12 billion parameters trained jointly on text and multiple images.","accessed":"2026-06-17"},{"type":"doc","title":"meta-llama/Llama-Guard-4-12B model card","url":"https://huggingface.co/meta-llama/Llama-Guard-4-12B","quote":"Unlike Llama Guard 3-11B-vision, Llama Guard 4 now supports safety classification when multiple images are given in the prompt as input.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"refusal","role":"core","note":"A safety classifier evaluates each prompt and response against a hazard taxonomy and emits safe/unsafe plus the violated categories, giving the host application the signal to refuse out-of-policy requests.","evidence":[{"type":"doc","title":"meta-llama/Llama-Guard-3-8B model card","url":"https://huggingface.co/meta-llama/Llama-Guard-3-8B","quote":"it generates text in its output that indicates whether a given prompt or response is safe or unsafe, and if unsafe, it also lists the content categories violated.","accessed":"2026-06-17"},{"type":"doc","title":"meta-llama/Llama-Guard-3-8B model card","url":"https://huggingface.co/meta-llama/Llama-Guard-3-8B","quote":"can be used to classify content in both LLM inputs (prompt classification) and in LLM responses (response classification).","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"typed-refusal-codes","role":"core","note":"Defines a single fixed taxonomy of machine-readable hazard codes (S1-S14, aligned to the MLCommons taxonomy) that the guard model emits as the refusal/block reason, so unsafe classifications are triaged by code rather than by parsing human-readable text.","evidence":[{"type":"doc","title":"Llama Guard 3-8B Model Card","url":"https://github.com/meta-llama/PurpleLlama/blob/main/Llama-Guard3/8B/MODEL_CARD.md","quote":"The model is trained to predict safety labels on the 14 categories shown below, based on the MLCommons taxonomy of 13 hazards, as well as an additional category for Code Interpreter Abuse for tool calls use cases.","accessed":"2026-06-17"},{"type":"doc","title":"Llama Guard 3-8B Model Card - Hazard categories","url":"https://github.com/meta-llama/PurpleLlama/blob/main/Llama-Guard3/8B/MODEL_CARD.md","quote":"S1: Violent Crimes S2: Non-Violent Crimes S3: Sex-Related Crimes S4: Child Sexual Exploitation S5: Defamation S6: Specialized Advice S7: Privacy S8: Intellectual Property S9: Indiscriminate Weapons S10: Hate S11: Suicide & Self-Harm S12: Sexual Content S13: Elections S14: Code Interpreter Abuse","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"input-output-guardrails","role":"core","note":"The same classifier model is applied to both the prompt (input guardrail) and the model response (output guardrail), classifying content on both sides of the LLM call.","evidence":[{"type":"doc","title":"meta-llama/Llama-Guard-3-8B model card","url":"https://huggingface.co/meta-llama/Llama-Guard-3-8B","quote":"Similar to previous versions, it can be used to classify content in both LLM inputs (prompt classification) and in LLM responses (response classification).","accessed":"2026-06-17"},{"type":"doc","title":"Llama Guard 3-8B Model Card","url":"https://github.com/meta-llama/PurpleLlama/blob/main/Llama-Guard3/8B/MODEL_CARD.md","quote":"Llama Guard 3-8B is a Llama-3.1-8B pretrained model, fine-tuned for content safety classification.","accessed":"2026-06-17"}],"evidence_status":"full"}],"key_concepts":[{"name":"Hazard taxonomy (S1-S14)","summary":"The fixed set of 14 content-safety categories, aligned to the MLCommons hazard taxonomy plus a Code Interpreter Abuse category, that the model emits as the machine-readable violation reason.","maps_to_pattern":"typed-refusal-codes","url":"https://github.com/meta-llama/PurpleLlama/blob/main/Llama-Guard3/8B/MODEL_CARD.md"},{"name":"Prompt vs response classification","summary":"The two modes the same classifier runs in — classifying the user prompt before the model answers, and classifying the model's response after — so it can guard both sides of a turn.","maps_to_pattern":"input-output-guardrails","url":"https://huggingface.co/meta-llama/Llama-Guard-3-8B"},{"name":"Safe / unsafe label","summary":"The single-token binary verdict the model generates for an input, followed by the violated category codes when the verdict is unsafe, which the host app uses as a refusal signal.","maps_to_pattern":"refusal","url":"https://huggingface.co/meta-llama/Llama-Guard-3-8B"}],"references":[{"type":"doc","title":"Llama Guard 3-8B Model Card (PurpleLlama)","url":"https://github.com/meta-llama/PurpleLlama/blob/main/Llama-Guard3/8B/MODEL_CARD.md","accessed":"2026-06-17"},{"type":"doc","title":"meta-llama/Llama-Guard-3-8B (Hugging Face)","url":"https://huggingface.co/meta-llama/Llama-Guard-3-8B","accessed":"2026-06-17"},{"type":"repo","title":"meta-llama/PurpleLlama (GitHub)","url":"https://github.com/meta-llama/PurpleLlama","accessed":"2026-06-17"}],"alternatives":[{"composition":"nemo-guardrails","relation":"competes-with","note":"NeMo Guardrails is a programmable rail framework for LLM apps; Llama Guard is a single classifier model that such a framework (or any host app) can call as its content-safety check."},{"composition":"llamafirewall","relation":"same-vendor","note":"Both are Meta safety components; LlamaFirewall composes guardrails including alignment checks, and Llama Guard is the content-safety classifier it can use as one layer."},{"composition":"guardrails-ai","relation":"competes-with","note":"Guardrails AI validates and constrains LLM I/O via validators; Llama Guard provides the model-based safety verdict, an overlapping concern approached as a classifier rather than a validation framework."}],"verification_status":"partial","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Meta Llama Guard 3\"]\n  fw --> p1[\"Multimodal Guardrails<br/>(core)\"]\n  fw --> p2[\"Refusal<br/>(core)\"]\n  fw --> p3[\"Typed Refusal Codes<br/>(core)\"]\n  fw --> p4[\"Input/Output Guardrails<br/>(core)\"]","tags":["model-vendor-agent","safety-control","full-code"]},{"id":"ollama","name":"Ollama","kind":"framework","category":"model-vendor-agent","vendor":"Ollama","language":"Go","license":"MIT","status":"active","first_released":"2023-07-07","url":"https://ollama.com","repo":"https://github.com/ollama/ollama","docs_url":"https://docs.ollama.com/","intent":"Ollama downloads and runs open-weight language models on the operator's own machine and serves them over a local API, including an OpenAI-compatible endpoint, so applications can run inference without sending prompts to a third party.","description":"Ollama is an open-source runtime for running large language models locally. It pulls model weights from a library, runs inference on the local machine, and exposes a local REST API on port 11434. It also provides an OpenAI-compatible endpoint so existing OpenAI client code can be pointed at the local instance by changing the base URL. Because inference runs locally, prompts and outputs do not leave the operator's machine unless cloud models are explicitly used.","primary_use_cases":["running open-weight LLMs locally","serving local inference over a REST API","OpenAI-compatible drop-in for existing clients","keeping prompts and data on the operator's own machine"],"agent_loop_shape":"Ollama is the inference engine an agent loop calls rather than the loop itself. A model is pulled into the local library and loaded on demand; the application sends a request naming the model to the local API (native or the OpenAI-compatible /v1 path) and receives a completion. The same endpoint serves any pulled model, so the caller selects which model answers by passing its name, and all computation stays on the local host.","build_surface":"full-code","members":[{"pattern":"sovereign-inference-stack","role":"core","note":"Ollama downloads and runs model weights and inference entirely on the operator's own machine and exposes a local API, so prompts and outputs never cross into a third-party API.","evidence":[{"type":"doc","title":"Ollama FAQ","url":"https://docs.ollama.com/faq","quote":"Ollama runs locally. We don't see your prompts or data when you run locally.","accessed":"2026-06-17"},{"type":"doc","title":"Ollama documentation — overview","url":"https://docs.ollama.com/","quote":"Ollama is the easiest way to get up and running with large language models such as gpt-oss, Gemma 4, DeepSeek-R1, Qwen3 and more.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"provider-string-routing","role":"supported","note":"Ollama exposes an OpenAI-compatible API so existing clients reach a local model by setting the base URL and naming the model, letting callers select which served model answers via the standard provider/model interface instead of provider-specific code.","evidence":[{"type":"doc","title":"OpenAI compatibility | Ollama","url":"https://docs.ollama.com/openai","quote":"Ollama provides compatibility with parts of the OpenAI API to help connect existing applications to Ollama.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"tool-use","role":"supported","note":"Ollama supports tool (function) calling: the caller passes tool definitions with name, description, and parameter schema, the model decides when to emit a typed tool call, and the tool result is fed back so the model can continue; availability depends on the served model supporting tools.","evidence":[{"type":"doc","title":"Tool calling | Ollama","url":"https://docs.ollama.com/capabilities/tool-calling","quote":"Ollama supports tool calling (also known as function calling) which allows a model to invoke tools and incorporate their results into its replies.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"structured-output","role":"supported","note":"Ollama lets the caller enforce a JSON schema on model responses via the format field so replies conform to a typed shape for reliable downstream extraction; constraint is applied by the runtime rather than a separate validation layer.","evidence":[{"type":"doc","title":"Structured outputs | Ollama","url":"https://docs.ollama.com/capabilities/structured-outputs","quote":"Structured outputs let you enforce a JSON schema on model responses so you can reliably extract structured data, describe images, or keep every reply consistent.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"Model library / pull","summary":"A registry of open-weight models that `ollama pull` downloads to the local machine; `ollama ls` lists the local catalog the runtime can load on demand.","url":"https://docs.ollama.com/cli"},{"name":"Local API (port 11434)","summary":"The REST endpoint Ollama serves on the local host for generate/chat/embeddings requests, so applications send a model name and receive completions without leaving the machine.","maps_to_pattern":"sovereign-inference-stack","url":"https://docs.ollama.com/"},{"name":"OpenAI compatibility","summary":"A parallel /v1 endpoint that mimics the OpenAI API so existing OpenAI client code reaches a local model just by changing the base URL and model name.","maps_to_pattern":"provider-string-routing","url":"https://docs.ollama.com/openai"},{"name":"Modelfile","summary":"A declarative file specifying a base model plus parameters, system prompt, and template, used to build customized local models with `ollama create`.","url":"https://docs.ollama.com/"}],"references":[{"type":"doc","title":"Ollama documentation","url":"https://docs.ollama.com/","accessed":"2026-06-17"},{"type":"repo","title":"ollama/ollama (GitHub)","url":"https://github.com/ollama/ollama","accessed":"2026-06-17"},{"type":"doc","title":"OpenAI compatibility | Ollama","url":"https://docs.ollama.com/openai","accessed":"2026-06-17"}],"alternatives":[{"composition":"vllm","relation":"competes-with","note":"vLLM is a high-throughput self-hosted inference server aimed at production serving; Ollama targets easy single-machine local inference, both keeping weights and prompts on the operator's hardware."}],"verification_status":"partial","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Ollama\"]\n  fw --> p1[\"Sovereign Inference Stack<br/>(core)\"]\n  fw --> p2[\"Provider-String Routing<br/>(supported)\"]\n  fw --> p3[\"Tool Use<br/>(supported)\"]\n  fw --> p4[\"Structured Output<br/>(supported)\"]","tags":["model-vendor-agent","safety-control","routing-composition","tool-use-environment","structure-data","full-code"]},{"id":"openai-model-spec","name":"OpenAI Model Spec","kind":"framework","category":"model-vendor-agent","vendor":"OpenAI","language":"N/A","license":"CC0-1.0","status":"active","first_released":"2024-05-08","url":"https://model-spec.openai.com/","repo":"https://github.com/openai/model_spec","docs_url":"https://model-spec.openai.com/","intent":"The OpenAI Model Spec is a public document that specifies the intended behaviour of OpenAI's models, defining an authority ordering across instruction sources and the rules for when the model should comply, refuse, or partially answer.","description":"The Model Spec is a human-authored specification of desired model behaviour for OpenAI's models in the API and ChatGPT. It defines a chain of command that assigns each instruction source a default authority level, so Platform and System instructions override Developer, which override User, which override Guideline-level defaults. It also sets out when the assistant should refuse a request, decline prohibited help while still assisting with a permissible goal, or provide only neutral factual information. The document is released into the public domain under CC0.","primary_use_cases":["specifying intended model behaviour","defining instruction authority and conflict resolution","rules for refusal and partial compliance","shared reference for model alignment and evaluation"],"agent_loop_shape":"The Model Spec is not a runtime but a behavioural contract the model is trained and evaluated against. On each request the model treats incoming instructions according to their position in the chain of command, applying higher-authority instructions over lower ones, and follows the spec's rules for borderline requests: comply when permissible, refuse disallowed content, or offer a constrained partial answer. The spec thus governs how a model resolves conflicting instructions before any tool or response is emitted.","build_surface":"closed-application","members":[{"pattern":"priority-matrix-conflict-resolution","role":"first-class","note":"A human-authored chain of command assigns each instruction source a default authority level (Platform/System > Developer > User > Guideline) and resolves conflicts deterministically by that ordering rather than leaving precedence to model discretion.","evidence":[{"type":"spec","title":"OpenAI Model Spec (2025/09/12) - Chain of command","url":"https://model-spec.openai.com/2025-09-12.html","quote":"Each section of the spec, and message role in the input conversation, is designated with a default authority level","accessed":"2026-06-17"},{"type":"spec","title":"OpenAI Model Spec (2025/09/12) - Chain of command","url":"https://model-spec.openai.com/2025-09-12.html","quote":"Instructions with higher authority override those with lower authority.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"refusal","role":"first-class","note":"The spec defines when the assistant must refuse rather than comply — declining to facilitate illicit behaviour — and prescribes refusing the prohibited part while still helping with any permissible goal, so refusal is a specified behaviour, not left to the model.","evidence":[{"type":"spec","title":"OpenAI Model Spec (2025/09/12) - Refusals","url":"https://model-spec.openai.com/2025-09-12.html","quote":"If the user or developer asks the assistant to facilitate illicit behavior, the assistant should refuse to help.","accessed":"2026-06-17"},{"type":"spec","title":"OpenAI Model Spec (2025/09/12) - Refusals","url":"https://model-spec.openai.com/2025-09-12.html","quote":"If the user asks for prohibited help to accomplish a permissible goal, the assistant should politely decline to offer the prohibited help and suggest other ways it could help with the permissible goal.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"constitutional-charter","role":"first-class","note":"The spec defines a Root authority level of fundamental rules that no system message, developer, or user can override, encoding inviolable boundaries that sit above the rest of the chain of command.","evidence":[{"type":"spec","title":"OpenAI Model Spec (2025/09/12) - Instructions and levels of authority","url":"https://model-spec.openai.com/2025-09-12.html","quote":"Root: Fundamental root rules that cannot be overridden by system messages, developers or users.","accessed":"2026-06-17"},{"type":"spec","title":"OpenAI Model Spec (2025/09/12) - Root rules","url":"https://model-spec.openai.com/2025-09-12.html","quote":"Root-level instructions are mostly prohibitive, requiring models to avoid behaviors that could contribute to catastrophic risks, cause direct physical harm to people, violate laws, or undermine the chain of command.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"mandatory-red-flag-escalation","role":"supported","note":"The spec prescribes a high-risk trigger class — an identified dangerous situation or imminent real-world harm — on which the assistant must proactively act to prevent harm rather than answer the request as posed.","evidence":[{"type":"spec","title":"OpenAI Model Spec (2025/09/12) - Try to prevent imminent real-world harm","url":"https://model-spec.openai.com/2025-09-12.html","quote":"The assistant should proactively try to prevent imminent real-world harm when it identifies a dangerous situation","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"Chain of command","summary":"An ordering of instruction sources by default authority level — Root, then Platform/System, Developer, User, and Guideline — that the model uses to resolve conflicting instructions deterministically.","maps_to_pattern":"priority-matrix-conflict-resolution","url":"https://model-spec.openai.com/2025-09-12.html"},{"name":"Levels of authority","summary":"Each section of the spec and each message role is tagged with one of the authority levels (root, system, developer, user, guideline), and higher-authority content overrides lower-authority content.","maps_to_pattern":"constitutional-charter","url":"https://model-spec.openai.com/2025-09-12.html"},{"name":"Seek the truth together","summary":"A behavioural principle directing the assistant to be honest and not sycophantic, pushing back when a request conflicts with established principles or the user's interests rather than agreeing to please.","maps_to_pattern":"sycophancy"},{"name":"Refusals and partial compliance","summary":"Rules for borderline requests: refuse disallowed content, decline the prohibited part of a request while still helping with a permissible goal, or give only neutral factual information.","maps_to_pattern":"refusal","url":"https://model-spec.openai.com/2025-09-12.html"}],"references":[{"type":"spec","title":"OpenAI Model Spec (2025/09/12)","url":"https://model-spec.openai.com/2025-09-12.html","quote":"Root: Fundamental root rules that cannot be overridden by system messages, developers or users.","accessed":"2026-06-17"},{"type":"repo","title":"openai/model_spec — GitHub","url":"https://github.com/openai/model_spec","accessed":"2026-06-17"}],"alternatives":[{"composition":"llama-guard","relation":"complements","note":"Llama Guard is a deployed input/output safety classifier; the Model Spec is the upstream behavioural contract a model is trained and evaluated against rather than a runtime filter."}],"verification_status":"needs-verification","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-19","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"OpenAI Model Spec\"]\n  fw --> p1[\"Priority Matrix (Conflict Resolution)<br/>(first-class)\"]\n  fw --> p2[\"Refusal<br/>(first-class)\"]\n  fw --> p3[\"Constitutional Charter<br/>(first-class)\"]\n  fw --> p4[\"Mandatory Red-Flag Escalation<br/>(supported)\"]","tags":["model-vendor-agent","safety-control","closed-application"],"anti_patterns_avoided":[{"pattern":"over-helpfulness","note":"The model spec instructs the assistant not to overstep or autonomously pursue goals beyond what was asked, bounding over-helpfulness.","evidence":[{"type":"doc","title":"OpenAI Model Spec","url":"https://model-spec.openai.com/2025-10-27.html","quote":"the assistant should never overstep and attempt to autonomously pursue goals in ways that aren't directly stated or logically dictated by the instructions","accessed":"2026-06-19"}]},{"pattern":"sycophancy","note":"The model spec instructs the assistant not to be sycophantic and not to simply agree or flatter the user.","evidence":[{"type":"doc","title":"OpenAI Model Spec","url":"https://model-spec.openai.com/2025-04-11.html","quote":"Do not be sycophantic ... it should not just say yes to everything (like a sycophant). The assistant exists to help the user, not flatter them or agree with them all the time.","accessed":"2026-06-19"}]}]},{"id":"salesforce-xlam","name":"Salesforce xLAM","kind":"framework","category":"model-vendor-agent","vendor":"Salesforce","language":"N/A","license":"cc-by-nc-4.0","status":"active","first_released":"2024-07-18","url":"https://huggingface.co/Salesforce/xLAM-7b-fc-r","repo":"https://github.com/SalesforceAIResearch/xLAM","docs_url":"https://huggingface.co/Salesforce/xLAM-7b-fc-r","intent":"Salesforce xLAM is a family of Large Action Models tuned to plan and execute tasks by emitting structured function calls against available APIs rather than producing free-form text.","description":"xLAM is a family of Large Action Models from Salesforce AI Research designed to act as the action engine of AI agents. The models translate user intentions into executable actions and autonomously plan and execute tasks. The function-calling (fc) variants are optimised to return structured, JSON-formatted function calls based on the input query and the available APIs, comparable to the function-calling mode of hosted chat models. They are released for research under a CC BY-NC 4.0 license.","primary_use_cases":["function calling for AI agents","translating user intent into executable API actions","structured JSON tool-call generation","autonomous task planning and execution"],"agent_loop_shape":"xLAM is the model at the core of an agent rather than a full runtime. Given a user query and a set of available APIs, an fc-series model produces a structured function call selecting the API and arguments to invoke; the surrounding harness executes the call and can feed the result back for the next step. The model is tuned so its action output follows a JSON function-calling format rather than free text, so downstream code can parse and dispatch the call deterministically.","build_surface":"full-code","members":[{"pattern":"large-action-models","role":"core","note":"xLAM is a family of Large Action Models optimised for function calling and autonomous task execution rather than text generation, intended as the action engine of AI agents.","evidence":[{"type":"doc","title":"Salesforce/xLAM-7b-fc-r model card","url":"https://huggingface.co/Salesforce/xLAM-7b-fc-r","quote":"Large Action Models (LAMs) are advanced large language models designed to enhance decision-making and translate user intentions into executable actions that interact with the world. LAMs autonomously plan and execute tasks to achieve specific goals, serving as the brains of AI agents.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"structured-output","role":"first-class","note":"The fc-series models are tuned to return structured, JSON-formatted function calls based on the query and available APIs, so their action output conforms to a parseable schema rather than free text.","evidence":[{"type":"doc","title":"Salesforce/xLAM-7b-fc-r model card — function calling","url":"https://huggingface.co/Salesforce/xLAM-7b-fc-r","quote":"The `fc` series of models are optimized for function-calling capability, providing fast, accurate, and structured responses based on input queries and available APIs.","accessed":"2026-06-17"},{"type":"doc","title":"Salesforce/xLAM-7b-fc-r model card — usage","url":"https://huggingface.co/Salesforce/xLAM-7b-fc-r","quote":"It allows us to extract JSON output that is similar to the function-calling mode of ChatGPT","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"tool-use","role":"core","note":"The models are trained for function-calling/tool use: given a natural-language query and a set of available functions, they produce the typed function/tool call(s) to execute rather than free-form text.","evidence":[{"type":"doc","title":"Salesforce/xLAM-7b-fc-r model card — function calling","url":"https://huggingface.co/Salesforce/xLAM-7b-fc-r","quote":"Function-calling, or tool use, is one of the key capabilities for AI agents.","accessed":"2026-06-17"},{"type":"doc","title":"Salesforce/xLAM-7b-fc-r model card — function calling","url":"https://huggingface.co/Salesforce/xLAM-7b-fc-r","quote":"It requires the model not only understand and generate human-like text but also to execute functional API calls based on natural language instructions.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"plan-and-execute","role":"supported","note":"Beyond single calls, the LAMs are described as autonomously planning and executing multi-step tasks to reach a goal, with the prompt format inviting one or more function calls to achieve the purpose.","evidence":[{"type":"doc","title":"Salesforce/xLAM-7b-fc-r model card","url":"https://huggingface.co/Salesforce/xLAM-7b-fc-r","quote":"LAMs autonomously plan and execute tasks to achieve specific goals, serving as the brains of AI agents.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"Large Action Model (LAM)","summary":"An LLM tuned to translate user intentions into executable actions in the world, acting as the decision-making brain of an agent rather than a pure text generator.","maps_to_pattern":"large-action-models","url":"https://huggingface.co/Salesforce/xLAM-7b-fc-r"},{"name":"fc (function-calling) series","summary":"Variants of xLAM optimised specifically for function-calling, returning fast, accurate, structured JSON tool calls selected from the query and the available APIs.","maps_to_pattern":"tool-use","url":"https://huggingface.co/Salesforce/xLAM-7b-fc-r"},{"name":"JSON function-call output","summary":"The action output format: parseable JSON that mirrors the function-calling mode of hosted chat models, so a harness can dispatch the call deterministically.","maps_to_pattern":"structured-output","url":"https://huggingface.co/Salesforce/xLAM-7b-fc-r"}],"references":[{"type":"doc","title":"Salesforce/xLAM-7b-fc-r — model card","url":"https://huggingface.co/Salesforce/xLAM-7b-fc-r","quote":"The `fc` series of models are optimized for function-calling capability, providing fast, accurate, and structured responses based on input queries and available APIs.","accessed":"2026-06-17"},{"type":"repo","title":"SalesforceAIResearch/xLAM — GitHub","url":"https://github.com/SalesforceAIResearch/xLAM","accessed":"2026-06-17"}],"alternatives":[{"composition":"cohere-command","relation":"similar-shape","note":"Both are vendor models tuned for tool/function-calling as the action engine of an agent; Cohere Command R+ targets the same agentic tool-call role through a hosted model."}],"verification_status":"partial","status_in_practice":"emerging","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Salesforce xLAM\"]\n  fw --> p1[\"Large Action Models (LAMs)<br/>(core)\"]\n  fw --> p2[\"Tool Use<br/>(core)\"]\n  fw --> p3[\"Structured Output<br/>(first-class)\"]\n  fw --> p4[\"Plan-and-Execute<br/>(supported)\"]","tags":["model-vendor-agent","tool-use-environment","structure-data","planning-control-flow","full-code"]},{"id":"anthropic-citations","name":"Anthropic Citations API","kind":"framework","category":"model-vendor-agent","vendor":"Anthropic","language":"N/A (HTTP API)","license":"proprietary","status":"active","first_released":"2025-01-23","url":"https://platform.claude.com/docs/en/build-with-claude/citations","docs_url":"https://platform.claude.com/docs/en/build-with-claude/citations","intent":"The Citations API has Claude ground its answer in supplied source documents and return, for each claim, the exact source span that supports it as a verifiable pointer.","description":"Citations is a feature of the Anthropic Messages API that lets a caller attach source documents to a request and enable citations on them. Claude chunks the documents, and its response is split into text blocks where each block carries a list of citations pointing at specific locations in the source documents (character ranges for plain text, page numbers for PDFs, content-block indices for custom content). The cited_text in each citation is extracted from the document rather than generated, so it is guaranteed to point at real source text. Citations are also emitted incrementally over the streaming API via citations_delta events.","primary_use_cases":["grounded question answering over supplied documents","attaching verifiable source spans to model claims","citation rendering in RAG applications","document-grounded summarisation with provenance"],"agent_loop_shape":"The caller includes one or more documents in a Messages request and sets citations.enabled on each. Claude chunks the documents into citable units, answers the question, and returns multiple text blocks where each block pairs a claim with the citations that support it. Each citation references a specific document location and includes the exact cited_text extracted from the source. When streaming, citations arrive as citations_delta events appended to the current text block.","build_surface":"full-code","members":[{"pattern":"verify-before-cite-gate","role":"core","note":"Citations are parsed and resolved against the provided source documents so every returned cited_text is a valid pointer into a real source, preventing fabricated references.","evidence":[{"type":"doc","title":"Citations - Claude API Docs","url":"https://platform.claude.com/docs/en/build-with-claude/citations","quote":"Better citation reliability: Because citations are parsed into the respective response formats mentioned above and cited_text is extracted, citations are guaranteed to contain valid pointers to the provided documents.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"citation-attribution","role":"core","note":"The response is split into text blocks where each block carries the claim and the list of citations that support it, and every citation references a specific character range, page, or content-block index in a named source document, so the binding from answer span to source location is explicit and machine-readable.","evidence":[{"type":"doc","title":"Citations - Claude API Docs","url":"https://platform.claude.com/docs/en/build-with-claude/citations","quote":"Responses may now include multiple text blocks where each text block can contain a claim that Claude is making and a list of citations that support the claim.","accessed":"2026-06-17"},{"type":"doc","title":"Citations - Claude API Docs","url":"https://platform.claude.com/docs/en/build-with-claude/citations","quote":"Citations reference specific locations in source documents. The format of these citations are dependent on the type of document being cited from.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"citation-streaming","role":"supported","note":"Over the streaming API, citations are delivered incrementally through citations_delta events, each adding a single citation to the citations list on the current text content block so a UI can render source links in place as the answer streams.","evidence":[{"type":"doc","title":"Citations - Claude API Docs","url":"https://platform.claude.com/docs/en/build-with-claude/citations","quote":"For streaming responses, a citations_delta type is included that contains a single citation to be added to the citations list on the current text content block.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"Document chunking","summary":"Supplied documents are chunked to define the minimum granularity of possible citations; plain text and PDFs are chunked into sentences, while custom-content blocks are used as-is, so Claude can cite a single sentence or chain several into a paragraph.","url":"https://platform.claude.com/docs/en/build-with-claude/citations"},{"name":"cited_text","summary":"The exact source span backing a claim, extracted from the document rather than generated, returned in each citation and not counted toward output tokens.","maps_to_pattern":"verify-before-cite-gate","url":"https://platform.claude.com/docs/en/build-with-claude/citations"},{"name":"Citation location types","summary":"Three location formats keyed to document type: char_location (character range, plain text), page_location (page numbers, PDFs), and content_block_location (block indices, custom content).","maps_to_pattern":"citation-attribution","url":"https://platform.claude.com/docs/en/build-with-claude/citations"}],"references":[{"type":"doc","title":"Citations - Claude API Docs","url":"https://platform.claude.com/docs/en/build-with-claude/citations","quote":"Claude is capable of providing detailed citations when answering questions about documents, helping you track and verify information sources in responses.","accessed":"2026-06-17"},{"type":"doc","title":"Citations - Claude API Docs (citation reliability)","url":"https://platform.claude.com/docs/en/build-with-claude/citations","quote":"Because citations are parsed into the respective response formats mentioned above and cited_text is extracted, citations are guaranteed to contain valid pointers to the provided documents.","accessed":"2026-06-17"}],"alternatives":[{"composition":"claude-agent-sdk","relation":"same-vendor","note":"Same vendor (Anthropic); the Agent SDK is the broader agent harness, while the Citations API is a single Messages-API feature for grounded, source-pinned answers."},{"composition":"pageindex","relation":"complements","note":"PageIndex is a document-retrieval/reasoning layer; pairing retrieval that selects source passages with the Citations API that pins answer spans to those passages covers both halves of grounded QA."}],"verification_status":"needs-verification","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-19","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Anthropic Citations API\"]\n  fw --> p1[\"Verify-Before-Cite Resolution Gate<br/>(core)\"]\n  fw --> p2[\"Citation Attribution<br/>(core)\"]\n  fw --> p3[\"Citation Streaming<br/>(supported)\"]","tags":["model-vendor-agent","verification-reflection","retrieval","streaming-ux","full-code"],"anti_patterns_avoided":[{"pattern":"hallucinated-citations","note":"Citations make cited text reference the actual source documents, minimising fabricated references.","evidence":[{"type":"doc","title":"Anthropic — Citations API","url":"https://claude.com/blog/introducing-citations-api","quote":"Cited text will reference source documents to minimize hallucinations.","accessed":"2026-06-19"}]}]},{"id":"agentscope","name":"AgentScope","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"Alibaba ModelScope","language":"Python","license":"Apache-2.0","status":"active","url":"https://github.com/agentscope-ai/agentscope","repo":"https://github.com/agentscope-ai/agentscope","docs_url":"https://doc.agentscope.io","intent":"Provide a production-ready agent framework with built-in ReAct agent, MsgHub for multi-agent message routing, MCP/A2A integration, real-time steering via hooks, structured output, and short/long-term memory.","description":"AgentScope is Alibaba ModelScope's Apache-2.0 Python framework billed as 'Build and run agents you can see, understand and trust.' It ships a ReActAgent out of the box (sync/async tools, streaming, hooks, structured output, configurable long-term memory), a Toolkit registry, MCP client transports (stdio / SSE / streamable HTTP), an MsgHub for multi-agent message routing, and OpenTelemetry-based observability for local, serverless, or Kubernetes deployment.","primary_use_cases":["ReAct-style single-agent assistants with tool use","multi-agent orchestration via MsgHub message routing","MCP-integrated agents over stdio / SSE / streamable HTTP","human-in-the-loop steering through real-time user interrupts","OTel-instrumented production deployment"],"agent_loop_shape":"ReAct loop wrapped with hook points. A ReActAgent assembles formatted prompt + memory + registered tool schemas, the model emits reasoning plus tool calls or a final reply, tools are dispatched through a Toolkit (in-process functions or MCP-backed via stdio/HTTP/SSE clients), and results are appended to memory. Hooks fire pre/post each step, enabling interrupts and structured-output validation. Multi-agent topologies are expressed by chaining agents through an MsgHub that routes messages.","key_concepts":[{"name":"ReActAgent","summary":"Out-of-the-box ReAct agent supporting sync/async tools, streaming responses, hooks, structured output, and configurable long-term memory.","maps_to_pattern":"react","url":"https://doc.agentscope.io/tutorial/quickstart_agent.html"},{"name":"Toolkit","summary":"Registry that registers tool functions for an agent; tools can be backed by MCP clients.","maps_to_pattern":"tool-use","url":"https://doc.agentscope.io/tutorial/task_tool.html"},{"name":"MCP clients","summary":"HttpStatefulClient / StdIOStatefulClient / HttpStatelessClient transports; stateless clients create per-call sessions.","maps_to_pattern":"mcp","url":"https://doc.agentscope.io/tutorial/task_mcp.html"},{"name":"MsgHub","summary":"Message hub for flexible multi-agent orchestration and workflows.","maps_to_pattern":"orchestrator-workers"},{"name":"Long-term memory modes","summary":"agent_control / static_control / both — determine whether the agent self-manages memory or runtime auto-records.","maps_to_pattern":"cross-session-memory"},{"name":"OTel observability","summary":"Built-in OpenTelemetry instrumentation for local, serverless, or Kubernetes deployment."}],"pattern_composition":"flowchart TD\n  USER[User input] --> AGENT[ReActAgent]\n  AGENT --> FMT[Formatter<br/>e.g. DashScopeChatFormatter]\n  FMT --> MODEL[(LLM)]\n  MODEL -->|tool calls| TK[Toolkit]\n  TK --> FN[In-process tool]\n  TK --> MCP_C[MCP client<br/>stdio / HTTP / SSE]\n  MCP_C --> MCP_S[(MCP server)]\n  FN --> AGENT\n  MCP_S --> AGENT\n  AGENT -.hooks.-> HIL[User interrupt / steering]\n  AGENT -.structured output.-> SO[(Pydantic schema)]\n  AGENT -.long_term_memory_mode.-> MEM[(Long-term memory)]\n  AGENT --> HUB[MsgHub]\n  HUB --> AGENT2[Another agent]\n  HUB --> AGENT3[Another agent]\n  AGENT -.OTel.-> OBS[Observability]","members":[{"pattern":"react","role":"first-class","note":"ReActAgent is shipped as a top-level class; users register tools on a Toolkit and pass it to the agent.","evidence":[{"type":"repo","title":"agentscope-ai/agentscope README","url":"https://github.com/agentscope-ai/agentscope","quote":"start building your agents in 5 minutes with built-in ReAct agent, tools, skills, human-in-the-loop steering, memory, planning, realtime voice, evaluation and model finetuning","accessed":"2026-05-20"},{"type":"doc","title":"AgentScope — Create ReAct Agent","url":"https://doc.agentscope.io/tutorial/quickstart_agent.html","quote":"AgentScope provides out-of-the-box ReAct agent ReActAgent under agentscope.agent that can be used directly.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Tools are registered on a Toolkit instance via register_tool_function(); supports sync and async tools.","evidence":[{"type":"doc","title":"AgentScope — Toolkit","url":"https://doc.agentscope.io/tutorial/quickstart_agent.html","quote":"toolkit = Toolkit(); toolkit.register_tool_function(execute_python_code)","accessed":"2026-05-20"},{"type":"doc","title":"AgentScope — Toolkit class purpose","url":"https://doc.agentscope.io/tutorial/task_tool.html","quote":"The Toolkit class is designed to manage tool functions, extracting their JSON Schema from docstrings and providing a unified interface for tool execution.","accessed":"2026-05-24"},{"type":"doc","title":"AgentScope — sync/async tools","url":"https://doc.agentscope.io/tutorial/task_tool.html","quote":"Support both synchronous and asynchronous tool functions","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"Stateful and stateless MCP clients for HTTP / streamable HTTP / SSE / stdio transports; register an entire server or individual functions.","evidence":[{"type":"doc","title":"AgentScope — MCP transports","url":"https://doc.agentscope.io/tutorial/task_mcp.html","quote":"HttpStatefulClient for HTTP/SSE servers; StdIOStatefulClient for StdIO servers; HttpStatelessClient for HTTP/SSE servers","accessed":"2026-05-20"},{"type":"repo","title":"AgentScope README — MCP/A2A","url":"https://github.com/agentscope-ai/agentscope","quote":"built-in support for MCP and A2A; message hub for flexible multi-agent orchestration and workflows","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"human-in-the-loop","role":"first-class","note":"Hooks expose pre/post step points and the ReActAgent supports real-time steering with user interrupts.","evidence":[{"type":"repo","title":"AgentScope README — features","url":"https://github.com/agentscope-ai/agentscope","quote":"built-in ReAct agent, tools, skills, human-in-the-loop steering, memory, planning","accessed":"2026-05-20"},{"type":"repo","title":"AgentScope README — realtime interruption","url":"https://github.com/agentscope-ai/agentscope","quote":"Support realtime interruption in ReActAgent: conversation can be interrupted via cancellation in realtime and resumed seamlessly via robust memory preservation.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"orchestrator-workers","role":"first-class","note":"MsgHub routes messages between multiple agents for flexible multi-agent orchestration.","evidence":[{"type":"repo","title":"AgentScope README — MsgHub","url":"https://github.com/agentscope-ai/agentscope","quote":"message hub for flexible multi-agent orchestration and workflows","accessed":"2026-05-20"},{"type":"repo","title":"AgentScope README — Multi-Agent Workflows","url":"https://github.com/agentscope-ai/agentscope","quote":"AgentScope provides ``MsgHub`` and pipelines to streamline multi-agent conversations, offering efficient message routing and seamless information sharing","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"structured-output","role":"first-class","note":"ReActAgent supports structured output as a documented capability of the agent.","evidence":[{"type":"doc","title":"AgentScope — ReActAgent capabilities","url":"https://doc.agentscope.io/tutorial/quickstart_agent.html","quote":"supports multiple capabilities including hooks, structured output, real-time steering with user interrupts, synchronous/asynchronous tools, streaming responses, and memory management","accessed":"2026-05-20"},{"type":"doc","title":"AgentScope — Support structured output","url":"https://doc.agentscope.io/tutorial/quickstart_agent.html","quote":"Support structured output","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"cross-session-memory","role":"first-class","note":"long_term_memory_mode = agent_control | static_control | both — the agent persists memory across sessions either by self-management or runtime auto-record.","evidence":[{"type":"doc","title":"AgentScope — long-term memory modes","url":"https://doc.agentscope.io/tutorial/quickstart_agent.html","quote":"long_term_memory_mode parameter offers three options: agent_control (agent manages memory itself), static_control (automatic retrieval/recording at reply start/end), and both","accessed":"2026-05-20"},{"type":"doc","title":"AgentScope — agent_control mode","url":"https://doc.agentscope.io/tutorial/quickstart_agent.html","quote":"agent_control: allow agent to control the long-term memory by itself","accessed":"2026-05-24"},{"type":"doc","title":"AgentScope — static_control mode","url":"https://doc.agentscope.io/tutorial/quickstart_agent.html","quote":"static_control: retrieving and recording from/to long-term memory will happen in the beginning/end of each reply","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"browser-agent","role":"limited","note":"AgentScope ships a Browser-use Agent example linked from the README; not a first-class core abstraction.","evidence":[{"type":"repo","title":"AgentScope README — Browser-use Agent example","url":"https://github.com/agentscope-ai/agentscope","quote":"[Browser-use Agent](https://github.com/agentscope-ai/agentscope/tree/main/examples/agent/browser_agent)","accessed":"2026-05-24"},{"type":"repo","title":"AgentScope README — Meta Planner Agent","url":"https://github.com/agentscope-ai/agentscope","quote":"[Meta Planner Agent](https://github.com/agentscope-ai/agentscope/tree/main/examples/agent/meta_planner_agent)","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"agentic-rag","role":"limited","note":"RAG and Long-Term Memory are shipped as functionality examples linked from the README; no first-class RAG-Agent class documented in the quickstart.","evidence":[{"type":"repo","title":"AgentScope README — RAG example","url":"https://github.com/agentscope-ai/agentscope","quote":"[RAG](https://github.com/agentscope-ai/agentscope/tree/main/examples/functionality/rag)","accessed":"2026-05-24"},{"type":"repo","title":"AgentScope README — Long-Term Memory example","url":"https://github.com/agentscope-ai/agentscope","quote":"[Long-Term Memory](https://github.com/agentscope-ai/agentscope/tree/main/examples/functionality/long_term_memory)","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"supervisor","role":"limited","note":"The Meta Planner Agent example documents a planner that decomposes complex tasks and orchestrates sub-agents — AgentScope's closest analogue to a built-in supervisor.","evidence":[{"type":"doc","title":"AgentScope — Meta Planner Agent example","url":"https://github.com/agentscope-ai/agentscope/tree/main/examples/agent/meta_planner_agent","quote":"how to build a planner agent that can decompose complex task into manageable subtasks and orchestrates sub-agents to complete them","accessed":"2026-05-24"},{"type":"doc","title":"AgentScope — Meta Planner Agent example","url":"https://github.com/agentscope-ai/agentscope/tree/main/examples/agent/meta_planner_agent","quote":"equipped with a tool function named `create_worker` to create sub-agents dynamically and finish the assigned subtask","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"plan-and-execute","role":"limited","note":"Planning is listed among built-in capabilities; a Plan example and a Meta Planner Agent example ship from the README.","evidence":[{"type":"repo","title":"AgentScope README — planning capability","url":"https://github.com/agentscope-ai/agentscope","quote":"built-in ReAct agent, tools, skills, human-in-the-loop steering, memory, planning, realtime voice, evaluation and model finetuning","accessed":"2026-05-20"},{"type":"repo","title":"AgentScope README — Plan functionality example","url":"https://github.com/agentscope-ai/agentscope","quote":"[Plan](https://github.com/agentscope-ai/agentscope/tree/main/examples/functionality/plan)","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"autogen-conversational","role":"limited","note":"Multi-agent message routing via MsgHub structurally resembles conversational multi-agent systems; AgentScope does not adopt the AutoGen ConversableAgent vocabulary. No verbatim upstream quote uses the 'conversational multi-agent' framing.","evidence":[],"evidence_status":"none"},{"pattern":"debate","role":"limited","note":"Multi-agent debate is shipped as a workflow example simulating a multi-turn discussion between several solvers and an aggregator.","evidence":[{"type":"doc","title":"AgentScope — Multi-agent debate workflow","url":"https://github.com/agentscope-ai/agentscope/tree/main/examples/workflows/multiagent_debate","quote":"Debate workflow simulates a multi-turn discussion between different agents, mostly several solvers and an aggregator.","accessed":"2026-05-24"},{"type":"doc","title":"AgentScope — Multi-agent debate workflow","url":"https://github.com/agentscope-ai/agentscope/tree/main/examples/workflows/multiagent_debate","quote":"two debater agents will discuss a topic in a fixed order, and express their arguments based on the previous debate history.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"agent-as-tool-embedding","role":"limited","note":"Agent-as-Tool is a documented multi-agent abstraction: agents can be registered as tools callable by other agents via the Toolkit class.","evidence":[{"type":"doc","title":"AgentScope — Agent as Tool","url":"https://java.agentscope.io/en/multi-agent/agent-as-tool.html","quote":"Agent as Tool allows registering an agent as a tool that can be called by other agents, which is useful for building hierarchical or collaborative multi-agent systems.","accessed":"2026-05-24"},{"type":"doc","title":"AgentScope — Tool","url":"https://docs.agentscope.io/basic-concepts/tool","quote":"wrapping any Python function as a tool using the `Toolkit` class","accessed":"2026-05-24"}],"evidence_status":"full"}],"alternatives":[{"composition":"qwen-agent","relation":"same-vendor","note":"Alibaba DAMO Academy framework with a more multi-agent/conversational shape; not Qwen-coupled."}],"references":[{"type":"repo","title":"agentscope-ai/agentscope","url":"https://github.com/agentscope-ai/agentscope","quote":"A production-ready, easy-to-use agent framework with essential abstractions that work with rising model capability and built-in support for finetuning.","accessed":"2026-05-20"},{"type":"doc","title":"AgentScope documentation","url":"https://doc.agentscope.io","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"emerging","tags":["open-source","python","react-agent","mcp","msghub","modelscope"]},{"id":"agentverse","name":"AgentVerse","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"OpenBMB / Tsinghua","language":"Python","license":"Apache-2.0","status":"active","url":"https://github.com/OpenBMB/AgentVerse","repo":"https://github.com/OpenBMB/AgentVerse","intent":"Multi-agent framework with two distinct modes: task-solving (collaborative agents work toward a shared goal) and simulation (autonomous agents interact in an environment to study emergent social behaviour).","description":"AgentVerse is OpenBMB's multi-agent framework with a deliberate dual-mode design. In task-solving mode, several role-specialised agents coordinate to solve a single user task (similar in spirit to AutoGen GroupChat). In simulation mode, autonomous agents interact in a configured environment (e.g. a simulated software-development office, a research-paper writing committee, a virtual classroom) to study emergent collective behaviour. The framework provides shared environment abstractions (agents, environments, rules, recorders) that both modes plug into.","primary_use_cases":["multi-agent task-solving with role specialisation","social-simulation research (agent societies, emergent behaviour)","education: configurable multi-agent classrooms and debate environments","studying communication protocols between non-cooperative agents"],"agent_loop_shape":"Environment-loop. The environment ticks; on each tick, agents perceive shared state, act (speak, use tools, move), and the environment updates. Task-solving mode runs a small fixed agent team until task termination. Simulation mode runs N autonomous agents indefinitely with no shared goal, logging interactions for offline analysis.","key_concepts":[{"name":"Task-solving mode","summary":"Collaborative multi-agent coordination toward a shared goal.","maps_to_pattern":"group-chat-manager"},{"name":"Simulation mode","summary":"Autonomous agent society for studying emergent behaviour.","maps_to_pattern":"actor-model-agents"},{"name":"Environment / agent / rule abstraction","summary":"Pluggable triad that both modes share.","maps_to_pattern":"blackboard"}],"pattern_composition":"flowchart TD\n  CONFIG[Environment config] --> MODE{Mode}\n  MODE -->|task-solving| TASK[Task team<br/>role-specialised agents]\n  MODE -->|simulation| SIM[Agent society<br/>N autonomous agents]\n  TASK --> ENV[(Environment<br/>shared state)]\n  SIM --> ENV\n  ENV --> RULES[Rules / order]\n  RULES --> TICK[Tick: agents perceive → act → state updates]\n  TICK --> ENV\n  ENV --> RECORDER[Recorder<br/>log interactions]\n  TASK -.terminate on goal.-> END[Result]\n  SIM -.run indefinitely.-> RECORDER","members":[{"pattern":"role-assignment","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"group-chat-manager","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"camel-role-playing","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"orchestrator-workers","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"inter-agent-communication","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"actor-model-agents","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"blackboard","role":"supported","note":"Shared environment state.","evidence":[],"evidence_status":"none"},{"pattern":"tool-use","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"debate","role":"supported","note":"Simulation mode supports debate environments.","evidence":[],"evidence_status":"none"},{"pattern":"dynamic-expert-recruitment","role":"core","note":"A recruiter stage generates expert role descriptions at run time from the goal and the loop re-adjusts the group composition based on ongoing problem-solving progress.","evidence":[{"type":"paper","title":"AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors (arXiv:2308.10848)","url":"https://ar5iv.labs.arxiv.org/html/2308.10848","quote":"Determine and adjust the agent group's composition based on the ongoing problem-solving progression.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"alternatives":[{"composition":"autogen","relation":"competes-with","note":"Both are research multi-agent frameworks; AutoGen has stronger production traction."},{"composition":"camel-ai","relation":"competes-with","note":"CAMEL also offers role-playing simulation; AgentVerse pairs that with task-solving mode."},{"composition":"crewai","relation":"competes-with","note":"CrewAI is the more product-oriented peer."},{"composition":"metagpt","relation":"competes-with"}],"last_analyzed":"2026-05-22","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-05-22","status_in_practice":"emerging","tags":["orchestration-framework","multi-agent","tool-use-environment","full-code"],"references":[{"type":"doc","title":"AgentVerse — Homepage","url":"https://github.com/OpenBMB/AgentVerse"}]},{"id":"agno","name":"Agno","aliases":["PhiData"],"kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"Agno","language":"Python","license":"Apache-2.0","status":"active","first_released":"2022-05-04","url":"https://agno.com","repo":"https://github.com/agno-agi/agno","docs_url":"https://docs.agno.com","intent":"Provide an SDK and runtime for building agent platforms — role-defined Agents, multi-mode Teams (coordinate / route / broadcast / sequential), Workflows, persistent memory/storage/knowledge, MCP toolkits, and AgentOS with cron scheduling, RBAC, OpenTelemetry tracing, and human-approval gates.","description":"Agno (formerly PhiData) is an Apache-2.0 Python SDK that frames the world as Agents — 'AI programs that use tools to accomplish tasks'. On top of agents it adds Teams with coordination modes (Coordinate / Route / Broadcast / Sequential), step-based Workflows, persistent Memory with a MemoryManager for LLM-driven user-memory extraction, Agentic-RAG via the Knowledge module, and 100+ integration Toolkits including MCPTools (stdio / SSE / streamable HTTP). The companion AgentOS platform exposes the runtime and a control plane for scheduled jobs, RBAC, OTel tracing, and human-approval gates.","primary_use_cases":["multi-agent teams with Coordinate / Route / Broadcast / Sequential modes","Agentic RAG via the Knowledge module","persistent user memory across runs and sessions","production agent platform via AgentOS (cron + RBAC + OTel)","MCP-integrated agents with 100+ toolkit integrations"],"agent_loop_shape":"Stateful tool-calling loop. Each Agent run assembles instructions, knowledge (RAG), memory, and tool schemas; the model emits tool calls or a final reply; tool results are appended and the loop iterates until the model returns text or hits a cap. Team wraps this loop with a leader+members topology selected via mode: Coordinate (leader picks members, crafts tasks, synthesises), Route (leader routes to one specialist), Broadcast (same task to all members), Sequential (chain agent responses). With persistent memory enabled the agent stores user memories across runs, sessions, and agents.","key_concepts":[{"name":"Agent","summary":"AI program that uses tools to accomplish tasks; the primary unit.","url":"https://docs.agno.com/agents/building-agents.md"},{"name":"Team modes","summary":"Coordinate / Route / Broadcast / Sequential — four documented multi-agent orchestration shapes.","maps_to_pattern":"orchestrator-workers","url":"https://docs.agno.com/teams/overview.md"},{"name":"Workflow","summary":"Step-based sequential and parallel execution wrapping agents, with conditional branches.","url":"https://docs.agno.com/workflows/overview.md"},{"name":"Memory + MemoryManager","summary":"User preferences and history persisted across conversations; MemoryManager drives LLM-based memory extraction.","maps_to_pattern":"cross-session-memory","url":"https://docs.agno.com/agents/usage/agent-with-memory.md"},{"name":"Knowledge / Agentic RAG","summary":"Searchable knowledge bases that the agent can autonomously retrieve from.","maps_to_pattern":"agentic-rag","url":"https://docs.agno.com/examples/agents/knowledge/agentic-rag.md"},{"name":"MCPTools","summary":"Toolkit that integrates MCP servers via stdio / SSE / streamable HTTP.","maps_to_pattern":"mcp","url":"https://docs.agno.com/agent-os/mcp/tools.md"},{"name":"AgentOS","summary":"The runtime and control plane that runs your agent platform.","url":"https://docs.agno.com/agent-os/introduction.md"}],"pattern_composition":"flowchart TD\n  USER[User input] --> AGENT[Agent<br/>stateful loop]\n  AGENT --> MODEL[(LLM)]\n  MODEL -->|tool calls| TOOLS[Toolkits<br/>100+ integrations]\n  TOOLS -->|MCPTools / MultiMCPTools| MCP[(MCP servers<br/>stdio / SSE / HTTP)]\n  TOOLS --> MODEL\n  AGENT -.memory.-> MEM[(MemoryManager)]\n  AGENT -.knowledge.-> KB[(Knowledge / Agentic RAG)]\n  AGENT --> TEAM{Team mode}\n  TEAM -->|Coordinate| WORKERS[Member agents]\n  TEAM -->|Route| SPEC[Specialist]\n  TEAM -->|Broadcast| ALL[All members]\n  TEAM -->|Sequential| CHAIN[Chained agents]\n  AGENT --> WF[Workflow steps]\n  AGENT -.runs on.-> OS[AgentOS<br/>control plane]\n  OS --> CRON[Scheduled jobs]\n  OS --> RBAC[RBAC + OTel]\n  OS --> HIL[Human-approval gates]","members":[{"pattern":"agentic-rag","role":"first-class","note":"Knowledge module powers autonomous retrieval; documented as 'Agentic RAG'.","evidence":[{"type":"doc","title":"Agno docs index — Agentic RAG","url":"https://docs.agno.com/examples/agents/knowledge/agentic-rag.md","quote":"Agentic RAG: Autonomous retrieval pipeline","accessed":"2026-05-20"},{"type":"doc","title":"Agno docs index — Knowledge","url":"https://docs.agno.com/agents/usage/agent-with-knowledge.md","quote":"Agent with Knowledge: Implements searchable knowledge bases ... 'Agentic RAG' implementation","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"cross-session-memory","role":"first-class","note":"MemoryManager persists user memories across runs/sessions/agents.","evidence":[{"type":"doc","title":"Agno — Agent with Memory","url":"https://docs.agno.com/agents/usage/agent-with-memory.md","quote":"Store user preferences that persist across conversations.","accessed":"2026-05-20"},{"type":"doc","title":"Agno docs index — Memory Manager","url":"https://docs.agno.com/examples/agents/memory-and-learning/memory-manager.md","quote":"Memory Manager: Cross-session memory","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"MCPTools and MultiMCPTools speak stdio / SSE / streamable HTTP; AgentOS can also be exposed as an MCP server.","evidence":[{"type":"doc","title":"Agno — MCPTools overview","url":"https://docs.agno.com/agent-os/mcp/tools.md","quote":"MCPTools Overview ... Use MCPTools in agents","accessed":"2026-05-20"},{"type":"doc","title":"Agno — AgentOS as MCP server","url":"https://docs.agno.com/agent-os/mcp/mcp.md","quote":"AgentOS as MCP Server ... Expose AgentOS via MCP","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"orchestrator-workers","role":"first-class","note":"Team modes include Coordinate (leader picks members, crafts tasks, synthesises) and Broadcast (same task to all members, synthesise).","evidence":[{"type":"doc","title":"Agno docs index — Teams overview","url":"https://docs.agno.com/teams/overview.md","quote":"Teams Overview: Coordinate multiple agents","accessed":"2026-05-20"},{"type":"doc","title":"Agno docs index — Broadcast Pattern","url":"https://docs.agno.com/teams/broadcast.md","quote":"Broadcast Pattern: Send tasks to all members","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"supervisor","role":"first-class","note":"Router pattern: leader directs tasks to a specific specialist; Sequential pattern chains member responses.","evidence":[{"type":"doc","title":"Agno docs index — Router Pattern","url":"https://docs.agno.com/teams/router.md","quote":"Router Pattern: Direct tasks to specific agents","accessed":"2026-05-20"},{"type":"doc","title":"Agno docs index — Sequential Pattern","url":"https://docs.agno.com/teams/sequential.md","quote":"Sequential Pattern: Chain agent responses","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"Agent run loop iterates model + tool calls until a final message without tool calls is produced.","evidence":[{"type":"doc","title":"Agno — Agent Run Loop","url":"https://docs.agno.com/agents/run.md","quote":"If the model makes a tool call, the agent executes it and returns results to the model. The model processes the updated context, repeating this loop until it produces a final message without tool calls.","accessed":"2026-05-24"},{"type":"doc","title":"Agno — Tools overview (LLM loop)","url":"https://docs.agno.com/concepts/tools/overview.md","quote":"The heart of Agent execution is the LLM loop.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Tools are functions agents call to interact with external systems; 120+ pre-built toolkits ship with the framework.","evidence":[{"type":"doc","title":"Agno — Tools overview","url":"https://docs.agno.com/concepts/tools/overview.md","quote":"Tools are functions Agents call to interact with external systems.","accessed":"2026-05-24"},{"type":"doc","title":"Agno — Tools overview (120+ toolkits)","url":"https://docs.agno.com/concepts/tools/overview.md","quote":"Agno comes with 120+ pre-built toolkits, which you can use to give your Agents all kind of abilities.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"structured-output","role":"first-class","note":"Agents accept Pydantic models via output_schema and return validated typed objects rather than text.","evidence":[{"type":"doc","title":"Agno — Structured Outputs","url":"https://docs.agno.com/agents/structured-output.md","quote":"Use Pydantic models to validate what goes in and what comes back.","accessed":"2026-05-24"},{"type":"doc","title":"Agno — Structured Outputs (validated objects)","url":"https://docs.agno.com/agents/structured-output.md","quote":"Get validated Pydantic objects instead of text.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"human-in-the-loop","role":"first-class","note":"@tool(requires_confirmation=True) marks tools that pause the run for user approval; AgentOS surfaces pending tool calls for admin approval.","evidence":[{"type":"doc","title":"Agno — AgentOS HITL","url":"https://docs.agno.com/agent-os/usage/hitl.md","quote":"When an agent needs to execute a tool that requires confirmation, the run pauses and waits for user approval before proceeding.","accessed":"2026-05-24"},{"type":"doc","title":"Agno — AgentOS Control Plane (approvals)","url":"https://docs.agno.com/agent-os/control-plane.md","quote":"Pending tool executions can require human confirmation. Approvals pause runs until an admin resolves the request.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"scheduled-agent","role":"first-class","note":"AgentOS Scheduler runs agents, teams, and workflows on cron schedules with configurable intervals, retries, and timezones.","evidence":[{"type":"doc","title":"Agno — AgentOS Control Plane (cron)","url":"https://docs.agno.com/agent-os/control-plane.md","quote":"Automate agents, teams, and workflows via AgentOS cron schedules. Configure custom intervals, retry logic, and timezones.","accessed":"2026-05-24"},{"type":"doc","title":"Agno — AgentOS Scheduler overview","url":"https://docs.agno.com/agent-os/scheduler/overview.md","quote":"Deploy and manage scheduled execution for agents and workflows via AgentOS cron jobs.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"code-execution","role":"limited","note":"PythonTools and ShellTools are pre-built toolkits that let agents execute Python and shell commands; save_to_file_and_run / run_python_code are explicit code-execution functions.","evidence":[{"type":"doc","title":"Agno — Toolkits (Python)","url":"https://docs.agno.com/examples/tools/python-tools","quote":"PythonTools is a pre-built toolkit imported when building Agno agents.","accessed":"2026-05-24"},{"type":"repo","title":"agno/tools/python.py","url":"https://github.com/agno-agi/agno/blob/main/libs/agno/agno/tools/python.py","quote":"This function saves Python code to a file called `file_name` and then runs it.","accessed":"2026-05-24"}],"evidence_status":"full"}],"instantiates":["memory-architecture","multi-agent-coordination","long-running-autonomous-agent"],"alternatives":[{"composition":"crewai","relation":"competes-with","note":"Role-defined multi-agent teams with broadly similar surface area."},{"composition":"langchain","relation":"competes-with","note":"Wider integration set, less of a runtime/platform."}],"references":[{"type":"repo","title":"agno-agi/agno","url":"https://github.com/agno-agi/agno","accessed":"2026-05-20"},{"type":"doc","title":"Agno docs","url":"https://docs.agno.com","quote":"An agent platform for building, running, and managing a fleet of agents.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","python","agent-platform","agentos","team-modes","mcp"]},{"id":"atomic-agents","name":"Atomic Agents","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"BrainBlend AI","language":"Python","license":"MIT","status":"active","url":"https://github.com/BrainBlend-AI/atomic-agents","repo":"https://github.com/BrainBlend-AI/atomic-agents","intent":"Provide a lightweight, schema-driven framework for building Agentic AI pipelines as composable LEGO-style blocks — each AtomicAgent or tool has a Pydantic input schema, output schema, system prompt, history, and context providers, so components chain by aligning schemas.","description":"Atomic Agents (BrainBlend AI, MIT) is 'designed around the concept of atomicity to be an extremely lightweight and modular framework for building Agentic AI pipelines and applications.' Components are 'LEGO blocks': each AtomicAgent or BaseTool is single-purpose, reusable, and predictable, with explicit Pydantic input and output schemas. The framework is built on Instructor for provider portability (OpenAI, Anthropic, Groq, Ollama, Mistral, Cohere, Gemini), exposes ChatHistory and BaseDynamicContextProvider for runtime context injection, and ships an Atomic Forge CLI with 13+ pre-built tools. v2.0 renamed BaseAgent to AtomicAgent and added run_stream() / run_async_stream().","primary_use_cases":["schema-driven single-agent pipelines","multi-step chains where components align by Pydantic schema","provider-portable agents via Instructor","custom tools registered through Atomic Forge CLI"],"agent_loop_shape":"Single-agent IPO (Input → Process → Output) cycle. An AtomicAgent owns a Pydantic input schema, system prompt, ChatHistory, output schema, and an Instructor-backed LLM client. Each run() call validates the input, assembles prompt + history + dynamic context, calls the model under structured-output mode via Instructor, validates the output, and returns it. Tools are separate BaseTool blocks with their own schemas; chaining is purely by matching one block's output schema to the next's input schema — there is no built-in supervisor or orchestrator runtime.","key_concepts":[{"name":"AtomicAgent","summary":"Core agent class (renamed from BaseAgent in v2.0) with input/output schemas, system prompt, history, and context providers.","url":"https://github.com/BrainBlend-AI/atomic-agents"},{"name":"Pydantic schemas","summary":"Each block declares an input schema and output schema; chaining is by schema alignment.","maps_to_pattern":"structured-output"},{"name":"Instructor integration","summary":"All LLM calls go through Instructor, giving multi-provider support (OpenAI, Anthropic, Groq, Ollama, Mistral, Cohere, Gemini).","maps_to_pattern":"schema-extensibility"},{"name":"ChatHistory","summary":"Conversation history component attached to an agent."},{"name":"BaseDynamicContextProvider","summary":"Injects dynamic runtime information into the system prompt."},{"name":"Atomic Forge CLI","summary":"CLI tool ('Atomic Assembler') that manages 13+ pre-built tools.","maps_to_pattern":"tool-use"}],"pattern_composition":"flowchart TD\n  IN[Input<br/>Pydantic schema] --> AG[AtomicAgent]\n  AG --> SP[System prompt]\n  AG --> HIST[ChatHistory]\n  AG --> CTX[Dynamic Context Providers]\n  AG --> INST[Instructor-backed LLM client]\n  INST --> MODEL[(OpenAI / Anthropic / Groq / Ollama / ...)]\n  MODEL --> OUT[Output<br/>Pydantic schema]\n  OUT -.chain by schema.-> AG2[Next AtomicAgent or BaseTool]\n  AG -.run_stream.-> STREAM[Streamed deltas]\n  CLI[Atomic Forge CLI] --> TOOLS[13+ pre-built BaseTool blocks]","members":[{"pattern":"structured-output","role":"first-class","note":"Every AtomicAgent and BaseTool declares Pydantic input/output schemas; Instructor enforces structured output on the model.","evidence":[{"type":"repo","title":"Atomic Agents README — schema-driven","url":"https://github.com/BrainBlend-AI/atomic-agents","quote":"Built on Instructor and Pydantic, it enables you to create AI applications with the same software engineering principles.","accessed":"2026-05-20"},{"type":"repo","title":"Atomic Agents README — LEGO blocks","url":"https://github.com/BrainBlend-AI/atomic-agents","quote":"Think of it like building AI applications with LEGO blocks - each component (agent, tool, context provider) is: Single-purpose, Reusable, Composable, Predictable.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"schema-extensibility","role":"first-class","note":"Schema-aligned chaining is the framework's core compositional mechanism; provider portability via Instructor enables swapping models without touching schemas.","evidence":[{"type":"repo","title":"Atomic Agents README — Instructor providers","url":"https://github.com/BrainBlend-AI/atomic-agents","quote":"Atomic Agents depends on the Instructor package ... any other API supported by Instructor can also be used — such as Ollama, Groq, Mistral, Cohere, Anthropic, Gemini.","accessed":"2026-05-20"},{"type":"repo","title":"Atomic Agents README — Extensibility","url":"https://github.com/BrainBlend-AI/atomic-agents","quote":"Extensibility: Easily swap out components or integrate new ones without disrupting the entire system.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"BaseTool blocks chain into agents by matching schemas; Atomic Forge CLI manages 13+ pre-built tools.","evidence":[{"type":"repo","title":"Atomic Agents README — Atomic Forge","url":"https://github.com/BrainBlend-AI/atomic-agents","quote":"Atomic Forge CLI: Tool management system with 13+ pre-built tools (search, calculator, PDF reader, weather, etc.)","accessed":"2026-05-20"},{"type":"repo","title":"Atomic Agents README — Atomic Assembler","url":"https://github.com/BrainBlend-AI/atomic-agents","quote":"This also installs the CLI Atomic Assembler, which can be used to download Tools (and soon also Agents and Pipelines).","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"prompt-chaining","role":"first-class","note":"Pipelines are built by chaining components whose output schema matches the next component's input schema.","evidence":[{"type":"repo","title":"Atomic Agents README — chaining by schema alignment","url":"https://github.com/BrainBlend-AI/atomic-agents","quote":"Atomic Agents makes it easy to chain agents and tools together by aligning their input and output schemas.","accessed":"2026-05-24"},{"type":"repo","title":"Atomic Agents README — schema chaining example","url":"https://github.com/BrainBlend-AI/atomic-agents","quote":"By aligning the agent's output schema with the input schema of the search tool, you can easily chain them together or switch between different search providers.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"chain-of-thought","role":"limited","note":"Output schemas can model intermediate reasoning fields, but no explicit CoT helper is documented in core surveyed pages.","evidence":[],"evidence_status":"none"},{"pattern":"react","role":"limited","note":"ReAct-style tool loops can be built by chaining agent and tool blocks; not a first-class abstraction.","evidence":[],"evidence_status":"none"},{"pattern":"mcp","role":"limited","note":"MCP is not discussed in the main README. No verbatim upstream quote supporting first-class MCP integration found in surveyed v2.x README.","evidence":[],"evidence_status":"none"},{"pattern":"agentic-rag","role":"limited","note":"RAG is demonstrated as an example application (RAG Chatbot) using retrieval tools chained to an agent; not a first-class abstraction in the core framework.","evidence":[{"type":"doc","title":"Atomic Agents — RAG Chatbot example","url":"https://github.com/BrainBlend-AI/atomic-agents/blob/main/atomic-examples/rag-chatbot/README.md","quote":"RAG (Retrieval-Augmented Generation) Chatbot example for the Atomic Agents project.","accessed":"2026-05-24"},{"type":"doc","title":"Atomic Agents — RAG Chatbot example","url":"https://github.com/BrainBlend-AI/atomic-agents/blob/main/atomic-examples/rag-chatbot/README.md","quote":"For each user question: The Query Agent generates an optimized semantic search query","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"orchestrator-workers","role":"limited","note":"An orchestrator is user-implemented (Orchestration Agent example); the framework intentionally contrasts itself with autonomous multi-agent systems.","evidence":[{"type":"repo","title":"Atomic Agents README — autonomous multi-agent contrast","url":"https://github.com/BrainBlend-AI/atomic-agents","quote":"While existing frameworks for agentic AI focus on building autonomous multi-agent systems, they often lack the control and predictability required for real-world applications.","accessed":"2026-05-24"},{"type":"repo","title":"Atomic Agents README — Orchestration Agent example","url":"https://github.com/BrainBlend-AI/atomic-agents","quote":"Orchestration Agent: Shows how to create an Orchestrator Agent that intelligently decides between using different tools (search or calculator) based on user input.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"supervisor","role":"limited","note":"Orchestration Agent is shown as an example: an Orchestrator Agent decides between tools based on user input; not a built-in supervisor runtime, but the closest analogue documented.","evidence":[{"type":"doc","title":"Atomic Agents — Orchestration Agent example","url":"https://github.com/BrainBlend-AI/atomic-agents/blob/main/atomic-examples/orchestration-agent/README.md","quote":"Orchestrator Agent that intelligently decides between using a search tool or a calculator tool based on user input","accessed":"2026-05-24"},{"type":"doc","title":"Atomic Agents — README","url":"https://github.com/BrainBlend-AI/atomic-agents","quote":"Shows how to create an Orchestrator Agent that intelligently decides between using different tools (search or calculator) based on user input.","accessed":"2026-05-24"}],"evidence_status":"full"}],"references":[{"type":"repo","title":"BrainBlend-AI/atomic-agents","url":"https://github.com/BrainBlend-AI/atomic-agents","quote":"designed around the concept of atomicity to be an extremely lightweight and modular framework for building Agentic AI pipelines","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"emerging","tags":["open-source","python","schema-driven","instructor","pydantic","lightweight"]},{"id":"autoagent","name":"AutoAgent","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"HKU Data Intelligence Lab","language":"Python","license":"MIT","status":"active","first_released":"2025-02-10","url":"https://github.com/HKUDS/AutoAgent","repo":"https://github.com/HKUDS/AutoAgent","intent":"Allow non-coders to build and run LLM agents through natural-language dialogue — the framework profiles agents, generates tools and workflows, and runs them in Docker-isolated environments, with architecture drawn from OpenAI Swarm and Magentic-One.","description":"AutoAgent (HKUDS, formerly MetaChain, MIT) is a 'Fully-Automated & Zero-Code LLM Agent Framework'. It exposes three modes: User Mode (a ready-to-use multi-agent research assistant), Agent Editor (natural-language single-agent construction with automated profiling and tool generation), and Workflow Editor (natural-language multi-agent workflow construction without tool creation). Code generation and execution run inside Docker containers, with browser environment integration; design influences are explicitly OpenAI Swarm and Microsoft's Magentic-One three-agent pattern. Evaluated on GAIA and MultiHopRAG.","primary_use_cases":["zero-code single-agent construction via natural language","natural-language multi-agent workflow construction","ready-to-use multi-agent research assistant (User Mode)","GAIA-benchmark style autonomous problem solving"],"agent_loop_shape":"Three-mode shell. In User Mode a ready-built multi-agent research assistant runs over the user's task; in Agent Editor the user describes an agent in natural language and AutoAgent profiles it, generates required tools, and emits a runnable agent; in Workflow Editor the user describes a multi-agent workflow and AutoAgent synthesises an executable workflow without authoring new tools. Inside each agent the loop is tool-calling: model picks a function, the function runs inside a Docker container (file system, code execution, browser), the result is fed back. Architectural inspiration is OpenAI Swarm (handoffs) and Magentic-One (lead orchestrator + workers).","key_concepts":[{"name":"User Mode","summary":"Ready-to-use multi-agent system for research tasks.","url":"https://github.com/HKUDS/AutoAgent"},{"name":"Agent Editor","summary":"Natural-language single-agent construction: profiling + tool generation."},{"name":"Workflow Editor","summary":"Natural-language multi-agent workflow construction (no tool creation)."},{"name":"Self-managing workflow generation","summary":"Dynamically creates, optimises, and adapts agent workflows from high-level task descriptions.","maps_to_pattern":"automatic-workflow-search"},{"name":"Docker sandbox","summary":"Code generation and execution run inside Docker containers.","maps_to_pattern":"code-execution"},{"name":"Swarm + Magentic-One inspiration","summary":"Acknowledged architectural influences for handoff-based and orchestrator-worker shapes."}],"pattern_composition":"flowchart TD\n  USER[User in natural language] --> MODE{Mode}\n  MODE -->|User Mode| RES[Multi-agent research assistant]\n  MODE -->|Agent Editor| AE[Natural-language agent builder<br/>profiles + generates tools]\n  MODE -->|Workflow Editor| WE[Natural-language workflow builder<br/>no tool creation]\n  AE --> AGENT[Generated agent]\n  WE --> WF[Generated multi-agent workflow]\n  AGENT --> LOOP[Tool-calling loop]\n  WF --> LOOP\n  LOOP --> DOCKER[(Docker sandbox<br/>code + files + browser)]\n  DOCKER --> LOOP\n  LOOP -.handoffs.-> WORKER[Other agent]\n  RES -.Magentic-One.-> ORCH[Lead orchestrator + workers]","members":[{"pattern":"automatic-workflow-search","role":"first-class","note":"Self-managing workflow generation: AutoAgent dynamically creates and adapts agent workflows from high-level task descriptions.","evidence":[{"type":"repo","title":"AutoAgent README — self-managing workflow","url":"https://github.com/HKUDS/AutoAgent","quote":"Dynamically creates, optimizes and adapts agent workflows based on high-level task descriptions.","accessed":"2026-05-20"},{"type":"repo","title":"AutoAgent README — zero-code","url":"https://github.com/HKUDS/AutoAgent","quote":"democratizes AI development by allowing anyone, regardless of coding experience, to create and customize their own agents, tools, and workflows using natural language alone.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"agent-as-tool-embedding","role":"first-class","note":"Agent Editor produces agents that are themselves invoked as units inside larger workflows; tools are also LLM-generated artefacts.","evidence":[{"type":"repo","title":"AutoAgent README — natural language agent building","url":"https://github.com/HKUDS/AutoAgent","quote":"Automatically constructs and orchestrates collaborative agent systems purely through natural dialogue, eliminating the need for manual coding or technical configuration.","accessed":"2026-05-24"},{"type":"repo","title":"AutoAgent README — Intelligent Resource Orchestration","url":"https://github.com/HKUDS/AutoAgent","quote":"Enables controlled code generation for creating tools, agents, and workflows through iterative self-improvement, supporting both single agent creation and multi-agent workflow generation.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"code-execution","role":"first-class","note":"Code generation and execution via Docker containerization.","evidence":[{"type":"repo","title":"AutoAgent README — Docker containerization","url":"https://github.com/HKUDS/AutoAgent","quote":"We use Docker to containerize the agent-interactive environment. So please install Docker first.","accessed":"2026-05-24"},{"type":"repo","title":"AutoAgent README — controlled code generation","url":"https://github.com/HKUDS/AutoAgent","quote":"Enables controlled code generation for creating tools, agents, and workflows through iterative self-improvement","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"supervisor","role":"first-class","note":"Architectural inspiration explicitly includes Magentic-One's three-agent (lead-orchestrator + workers) design.","evidence":[{"type":"repo","title":"AutoAgent README — Magentic-one inspiration","url":"https://github.com/HKUDS/AutoAgent","quote":"Our framework architecture draws inspiration from OpenAI Swarm, while our user mode's three-agent design benefits from Magentic-one's insights.","accessed":"2026-05-24"},{"type":"repo","title":"AutoAgent README — User Mode multi-agent","url":"https://github.com/HKUDS/AutoAgent","quote":"AutoAgent features a ready-to-use multi-agent system accessible through user mode on the start page. This system serves as a comprehensive AI research assistant","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Function-calling tool use; LLM-generated tools are wired into agents at construction time; supports third-party tool platforms like RapidAPI.","evidence":[{"type":"repo","title":"AutoAgent README — natural-language tool creation","url":"https://github.com/HKUDS/AutoAgent","quote":"AutoAgent allows you to create tools, agents, and workflows using natural language alone.","accessed":"2026-05-24"},{"type":"repo","title":"AutoAgent README — function calling option","url":"https://github.com/HKUDS/AutoAgent","quote":"FN_CALL: Enable function calling (default: None). Most of time, you could ignore this option because we have already set the default value based on the model name.","accessed":"2026-05-24"},{"type":"repo","title":"AutoAgent README — third-party tool platforms","url":"https://github.com/HKUDS/AutoAgent","quote":"If you want to create tools from the third-party tool platforms, such as RapidAPI, you should subscribe tools from the platform and add your own API keys","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"browser-agent","role":"limited","note":"Browser environment integration is part of the Docker sandbox setup; full GUI Agent / Computer-Use support is on the roadmap, not yet first-class.","evidence":[{"type":"repo","title":"AutoAgent README — browser cookies","url":"https://github.com/HKUDS/AutoAgent","quote":"You can import the browser cookies to the browser environment to let the agent better access some specific websites.","accessed":"2026-05-24"},{"type":"repo","title":"AutoAgent README — GUI Agent roadmap","url":"https://github.com/HKUDS/AutoAgent","quote":"GUI Agent: Supporting Computer-Use agents with GUI interaction","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"agentic-rag","role":"limited","note":"Evaluated on MultiHopRAG via an 'Agentic-RAG Task' evaluation script; agentic retrieval is a benchmark capability rather than a core API.","evidence":[{"type":"repo","title":"AutoAgent README — Agentic-RAG inference","url":"https://github.com/HKUDS/AutoAgent","quote":"For the Agentic-RAG task, you can run the following command to run the inference.","accessed":"2026-05-24"},{"type":"repo","title":"AutoAgent README — MultiHopRAG dataset","url":"https://github.com/HKUDS/AutoAgent","quote":"Turn to this page and download it. Save them to your datapath.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"plan-and-execute","role":"limited","note":"Workflow generation produces an executable plan that subsequent agents execute; the workflow editor mode supports natural-language plan synthesis. AutoAgent does not use the 'plan-and-execute' label verbatim — closest mapping.","evidence":[{"type":"doc","title":"AutoAgent — README (workflow creation)","url":"https://github.com/HKUDS/AutoAgent","quote":"Dynamically creates, optimizes and adapts agent workflows based on high-level task descriptions, even when users cannot fully specify implementation details.","accessed":"2026-05-24"},{"type":"doc","title":"AutoAgent — README (workflow editor)","url":"https://github.com/HKUDS/AutoAgent","quote":"You can also create the agent workflows using natural language description with the `workflow editor` mode","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"structured-output","role":"limited","note":"Agent profiling and workflow synthesis rely on structured outputs internally — automated profiling steps emit agent/workflow profiles. Not exposed as a user-facing schema API.","evidence":[{"type":"doc","title":"AutoAgent — Automated agent profiling","url":"https://github.com/HKUDS/AutoAgent","quote":"Automated agent profiling","accessed":"2026-05-24"},{"type":"doc","title":"AutoAgent — Controlled code generation","url":"https://github.com/HKUDS/AutoAgent","quote":"Enables controlled code generation for creating tools, agents, and workflows through iterative self-improvement","accessed":"2026-05-24"}],"evidence_status":"full"}],"references":[{"type":"repo","title":"HKUDS/AutoAgent","url":"https://github.com/HKUDS/AutoAgent","quote":"Fully-Automated & Zero-Code LLM Agent Framework","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"experimental","tags":["open-source","python","zero-code","natural-language-agent-builder","docker-sandbox","research-prototype"]},{"id":"autogen","name":"AutoGen","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"Microsoft","language":"Python, .NET","license":"MIT (code packages); CC-BY-4.0 (top-level docs)","status":"active","first_released":"2023-08-18","url":"https://microsoft.github.io/autogen/","repo":"https://github.com/microsoft/autogen","docs_url":"https://microsoft.github.io/autogen/stable/","intent":"Build scalable, event-driven multi-agent AI applications in which conversable agents exchange asynchronous messages, optionally execute code, and coordinate through group chats or actor-style pub/sub.","description":"AutoGen is Microsoft's open-source programming framework for agentic AI. It is layered: AutoGen Core is a low-level event-driven runtime that builds event-driven, distributed, scalable agent systems on the Actor model; AutoGen AgentChat is a higher-level conversational API on Core; AutoGen Studio is a low-code web UI. The v0.4 release (2025-01-10) was a from-the-ground-up rewrite introducing the actor / event-driven foundation; v0.2 (the original ConversableAgent / GroupChatManager design) is still maintained on the 0.2 branch.","primary_use_cases":["multi-agent conversational systems with role-defined assistants","code-executing agents in Docker sandboxes","actor-style distributed agent systems with pub/sub","Magentic-One style orchestrated worker teams"],"agent_loop_shape":"Multi-agent conversational model. In AgentChat one or more AssistantAgents (LLM + tools) and an optional UserProxyAgent participate in a Team — RoundRobinGroupChat, SelectorGroupChat, Swarm, or MagenticOneGroupChat — that drives speaker turns until a termination condition fires. In Core, agents are independent actors that exchange asynchronous messages and can subscribe to topics for pub/sub routing. Tools are passed to AssistantAgent and executed in the same run() call; code via CodeExecutorAgent (typically Docker).","key_concepts":[{"name":"AssistantAgent","summary":"Built-in agent that uses an LLM and can use tools.","maps_to_pattern":"tool-use","url":"https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/agents.html"},{"name":"UserProxyAgent","summary":"Special built-in agent that acts as a proxy for a human user to provide feedback to the team.","maps_to_pattern":"human-in-the-loop","url":"https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/human-in-the-loop.html"},{"name":"RoundRobinGroupChat / SelectorGroupChat / Swarm / MagenticOneGroupChat","summary":"Team topologies — round-robin turns, LLM-selected next speaker, HandoffMessage-based, or Magentic-One orchestrated.","maps_to_pattern":"supervisor","url":"https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/teams.html"},{"name":"CodeExecutorAgent","summary":"Experimental agent that generates and executes code snippets; Docker recommended for isolation.","maps_to_pattern":"code-execution","url":"https://microsoft.github.io/autogen/stable/reference/python/autogen_agentchat.agents.html"},{"name":"Actor model + topic pub/sub (Core)","summary":"Agents are actors; asynchronous messages enable event-driven and request/response communication.","maps_to_pattern":"event-driven-agent","url":"https://microsoft.github.io/autogen/stable/user-guide/core-user-guide/index.html"},{"name":"TerminationCondition","summary":"Mandatory stop predicate for Teams; MaxMessageTermination, TextMentionTermination, etc.","maps_to_pattern":"step-budget","url":"https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/termination.html"}],"pattern_composition":"flowchart TD\n  USER[User input] --> UPA[UserProxyAgent<br/>human-in-the-loop]\n  UPA --> TEAM{Team<br/>RoundRobin / Selector / Swarm / MagenticOne}\n  TEAM -->|round-robin turns| AA1[AssistantAgent #1<br/>system prompt + tools]\n  TEAM -->|model-selected speaker| AA2[AssistantAgent #2<br/>system prompt + tools]\n  TEAM -->|HandoffMessage| AAH[AssistantAgent<br/>handoff target]\n  AA1 -->|tool call in same run| TOOLS[FunctionTool / PythonCodeExecutionTool]\n  AA2 -->|tool call| TOOLS\n  TEAM -->|delegates code| CEA[CodeExecutorAgent<br/>DockerCommandLineCodeExecutor]\n  CEA --> TEAM\n  TOOLS --> AA1\n  AA1 -->|output_content_type| SO[(StructuredMessage<br/>Pydantic BaseModel)]\n  TEAM --> TERM{Termination condition}\n  TERM -->|stop| OUT[TaskResult]\n  TERM -->|continue| TEAM\n  TEAM -.save_state/load_state.-> STATE[(State dict)]\n  CORE_AGENT1[Actor agent] -->|publish| TOPIC[(Topic / Subscription)]\n  TOPIC -->|subscribe| CORE_AGENT2[Actor agent]\n  TEAM -.built on.- CORE_AGENT1","members":[{"pattern":"agent-resumption","role":"limited","note":"save_state() / load_state() serialise team/agent state to dicts; no automatic per-step checkpointing — caller invokes save/load.","evidence":[{"type":"doc","title":"AutoGen — State management","url":"https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/state.html","quote":"In many cases, it is useful to save the state of these components to disk and load them back later.","accessed":"2026-05-20"},{"type":"doc","title":"AutoGen — save/load_state API","url":"https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/state.html","quote":"We can get the state of a team by calling save_state method on the team and load it back by calling load_state method.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"autogen-conversational","role":"first-class","note":"ConversableAgent (v0.2) and RoundRobinGroupChat (v0.4+) are the canonical conversational shapes; AutoGen named the pattern in the catalog.","evidence":[{"type":"doc","title":"AutoGen v0.2 — ConversableAgent","url":"https://microsoft.github.io/autogen/0.2/docs/Use-Cases/agent_chat","quote":"a generic ConversableAgent class for Agents that are capable of conversing with each other through the exchange of messages to jointly finish a task","accessed":"2026-05-20"},{"type":"doc","title":"AutoGen — Teams","url":"https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/teams.html","quote":"A team that runs a group chat with participants taking turns in a round-robin fashion","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"camel-role-playing","role":"limited","note":"AutoGen's group-chat selector explicitly uses 'role play game' framing — agents are assigned named roles (assistant, user_proxy) and the system selects which role plays next. Structurally identical to CAMEL's two-agent dialectic; AutoGen does not use the 'CAMEL' label.","evidence":[{"type":"doc","title":"AutoGen — All About Agent Descriptions","url":"https://microsoft.github.io/autogen/0.2/blog/2023/12/29/AgentDescriptions/","quote":"You are in a role play game. The following roles are available","accessed":"2026-05-24"},{"type":"doc","title":"AutoGen — All About Agent Descriptions","url":"https://microsoft.github.io/autogen/0.2/blog/2023/12/29/AgentDescriptions/","quote":"user_proxy: A user that can run Python code or input command line commands at a Linux terminal and report back the execution results.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"code-as-action","role":"first-class","note":"CodeExecutorAgent generates and executes code; an alternative is AssistantAgent + PythonCodeExecutionTool.","evidence":[{"type":"api-ref","title":"AutoGen — CodeExecutorAgent","url":"https://microsoft.github.io/autogen/stable/reference/python/autogen_agentchat.agents.html","quote":"(Experimental) An agent that generates and executes code snippets based on user instructions.","accessed":"2026-05-20"},{"type":"api-ref","title":"AutoGen — code-as-action alternative","url":"https://microsoft.github.io/autogen/stable/reference/python/autogen_agentchat.agents.html","quote":"Using AssistantAgent with PythonCodeExecutionTool is an alternative to this agent.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"code-execution","role":"first-class","note":"DockerCommandLineCodeExecutor is the recommended execution backend; sandboxed by container.","evidence":[{"type":"api-ref","title":"AutoGen — DockerCommandLineCodeExecutor","url":"https://microsoft.github.io/autogen/stable/reference/python/autogen_agentchat.agents.html","quote":"DockerCommandLineCodeExecutor(work_dir=\"coding\") to execute code snippets in a Docker container.","accessed":"2026-05-20"},{"type":"api-ref","title":"AutoGen — Docker recommended","url":"https://microsoft.github.io/autogen/stable/reference/python/autogen_agentchat.agents.html","quote":"It is recommended that the CodeExecutorAgent agent uses a Docker container to execute code.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"event-driven-agent","role":"first-class","note":"Core uses the Actor model; agents communicate via asynchronous messages with pub/sub Topic subscriptions.","evidence":[{"type":"doc","title":"AutoGen — landing page","url":"https://microsoft.github.io/autogen/stable/","quote":"An event-driven programming framework for building scalable multi-agent AI systems.","accessed":"2026-05-20"},{"type":"doc","title":"AutoGen Core — Actor model","url":"https://microsoft.github.io/autogen/stable/user-guide/core-user-guide/index.html","quote":"Agents are developed by using the Actor model.","accessed":"2026-05-20"},{"type":"doc","title":"AutoGen Core — async messaging","url":"https://microsoft.github.io/autogen/stable/user-guide/core-user-guide/index.html","quote":"Agents communicate through asynchronous messages, enabling event-driven and request/response communication models.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"handoff","role":"first-class","note":"Swarm topology — agents emit HandoffMessage indicating which other agent to hand off to; speaker selected from the most recent HandoffMessage.","evidence":[{"type":"doc","title":"AutoGen — Swarm","url":"https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/swarm.html","quote":"Swarm implements a team in which agents can hand off task to other agents based on their capabilities.","accessed":"2026-05-20"},{"type":"doc","title":"AutoGen — HandoffMessage","url":"https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/swarm.html","quote":"Each agent has the ability to generate HandoffMessage to signal which other agents it can hand off to.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"orchestrator-workers","role":"first-class","note":"MagenticOneGroupChat is a lead-agent-directs-workers topology; the orchestrator plans tasks, gathers facts, and assigns subtasks to other agents.","evidence":[{"type":"doc","title":"AutoGen — Magentic-One orchestrator","url":"https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/magentic-one.html","quote":"the lead agent responsible for task decomposition and planning, directing other agents in executing subtasks","accessed":"2026-05-20"},{"type":"doc","title":"AutoGen — orchestrator plans and gathers facts","url":"https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/magentic-one.html","quote":"the Orchestrator begins by creating a plan to tackle the task, gathering needed facts","accessed":"2026-05-24"},{"type":"doc","title":"AutoGen — orchestrator assigns subtasks","url":"https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/magentic-one.html","quote":"it assigns one of Magentic-One other agents a subtask to complete","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"step-budget","role":"first-class","note":"Termination conditions are MANDATORY on Teams. MaxMessageTermination caps messages; TextMentionTermination stops on a sentinel string.","evidence":[{"type":"doc","title":"AutoGen — MaxMessageTermination","url":"https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/termination.html","quote":"MaxMessageTermination: Stops after a specified number of messages have been produced, including both agent and task messages.","accessed":"2026-05-20"},{"type":"doc","title":"AutoGen — termination rationale","url":"https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/termination.html","quote":"However, a run can go on forever, and in many cases, we need to know when to stop them. This is the role of the termination condition.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"structured-output","role":"first-class","note":"AssistantAgent's output_content_type accepts a Pydantic BaseModel; replies arrive as StructuredMessage instead of TextMessage.","evidence":[{"type":"api-ref","title":"AutoGen — StructuredMessage output","url":"https://microsoft.github.io/autogen/stable/reference/python/autogen_agentchat.agents.html","quote":"The output content type for StructuredMessage response as a Pydantic model.","accessed":"2026-05-20"},{"type":"api-ref","title":"AutoGen — StructuredMessage reply","url":"https://microsoft.github.io/autogen/stable/reference/python/autogen_agentchat.agents.html","quote":"the agent will respond with a `StructuredMessage` instead of a `TextMessage`","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"subagent-isolation","role":"limited","note":"Implicit via Actor model in Core — each agent runs inside an agent runtime that manages lifecycles and enforces security boundaries. No first-class context-isolation API specifically named.","evidence":[{"type":"doc","title":"AutoGen Core — agent runtime","url":"https://microsoft.github.io/autogen/stable/user-guide/core-user-guide/framework/agent-and-agent-runtime.html","quote":"An agent runtime is the execution environment for agents in AutoGen.","accessed":"2026-05-24"},{"type":"doc","title":"AutoGen Core — runtime enforces boundaries","url":"https://microsoft.github.io/autogen/stable/user-guide/core-user-guide/framework/agent-and-agent-runtime.html","quote":"An agent runtime provides the necessary infrastructure to facilitate communication between agents, manage agent lifecycles, enforce security boundaries, and support monitoring and debugging.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"supervisor","role":"first-class","note":"SelectorGroupChat uses an LLM to pick the next speaker; Magentic-One orchestrator does high-level planning and tracking.","evidence":[{"type":"doc","title":"AutoGen — SelectorGroupChat","url":"https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/teams.html","quote":"A team that selects the next speaker using a ChatCompletion model after each message","accessed":"2026-05-20"},{"type":"doc","title":"AutoGen — Magentic-One orchestrator","url":"https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/magentic-one.html","quote":"responsible for high-level planning, directing other agents and tracking task progress","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"AssistantAgent uses tools in the same run() call (v0.4 change from v0.2 where tools were executed by the calling agent).","evidence":[{"type":"doc","title":"AutoGen — AssistantAgent tools","url":"https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/agents.html","quote":"AssistantAgent is a built-in agent that uses a language model and has the ability to use tools.","accessed":"2026-05-20"},{"type":"doc","title":"AutoGen — v0.4 tool execution","url":"https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/agents.html","quote":"Unlike in v0.2 AgentChat, the tools are executed by the same agent directly within the same call to run().","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"composable-termination-conditions","role":"first-class","note":"TerminationCondition base + 11 built-in primitives (MaxMessageTermination, TextMentionTermination, TokenUsageTermination, TimeoutTermination, HandoffTermination, etc.) composed with `&` (AND) and `|` (OR). The pattern's vocabulary was inspired here.","evidence_status":"full","evidence":[{"type":"doc","title":"AutoGen — Termination","url":"https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/termination.html","quote":"A callable that takes a sequence of BaseAgentEvent or BaseChatMessage objects and returns a StopMessage if conversation should terminate, or None otherwise.","accessed":"2026-05-23"},{"type":"doc","title":"AutoGen — Termination conditions list","url":"https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/termination.html","quote":"AutoGen provides 11 built-in implementations: MaxMessageTermination, TextMentionTermination, TokenUsageTermination, TimeoutTermination, plus others; conditions combine with AND and OR operators.","accessed":"2026-05-23"}]},{"pattern":"agent-adapter","role":"supported","note":"AutoGen's LangChainToolAdapter wraps a foreign LangChain tool, extracting its name, description, and schema and re-presenting it through AutoGen's own tool interface.","evidence":[{"type":"api-ref","title":"autogen_ext.tools.langchain._langchain_adapter","url":"https://microsoft.github.io/autogen/dev//_modules/autogen_ext/tools/langchain/_langchain_adapter.html","quote":"Allows you to wrap a LangChain tool and make it available to AutoGen.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"magentic-one-generalist","role":"core","note":"Ships the exact architecture: a lead Orchestrator agent directs four specialist agents (WebSurfer, FileSurfer, Coder, ComputerTerminal), released open-source on AutoGen.","evidence":[{"type":"doc","title":"Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks","url":"https://www.microsoft.com/en-us/research/articles/magentic-one-a-generalist-multi-agent-system-for-solving-complex-tasks/","quote":"Magentic-One employs a multi-agent architecture where a lead agent, the Orchestrator, directs four other agents to solve tasks.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"voting-based-cooperation","role":"supported","note":"The multi-agent debate design pattern has an aggregator agent tally solver agents' answers by majority vote to produce the final decision.","evidence":[{"type":"doc","title":"Multi-Agent Debate — AutoGen","url":"https://microsoft.github.io/autogen/stable//user-guide/core-user-guide/design-patterns/multi-agent-debate.html","quote":"The aggregator agent uses majority voting to aggregate the final responses from all solver agents to get a final answer","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"generator-critic-separation","role":"core","note":"AutoGen's documented Reflection pattern pairs a coder agent that only generates artifacts with a separate reviewer agent that only critiques them, enforcing strict generator/critic role separation.","evidence":[{"type":"doc","title":"Reflection — AutoGen (core design patterns)","url":"https://microsoft.github.io/autogen/stable//user-guide/core-user-guide/design-patterns/reflection.html","quote":"There will be two agents: a coder agent and a reviewer agent, the coder agent will generate a code snippet, and the reviewer agent will generate a critique of the code snippet.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"anti_patterns_avoided":[{"pattern":"infinite-debate","note":"Termination conditions are mandatory; framework explicitly anti-recommends unbounded multi-agent loops.","evidence":[{"type":"doc","title":"AutoGen — termination rationale","url":"https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/termination.html","quote":"However, a run can go on forever, and in many cases, we need to know when to stop them.","accessed":"2026-05-20"}]}],"instantiates":["multi-agent-coordination","multi-agent-debate"],"alternatives":[{"composition":"langgraph","relation":"competes-with","note":"Graph-of-nodes vs event-driven actors + group chat."},{"composition":"crewai","relation":"competes-with","note":"Conversational multi-agent framework, role-driven."},{"composition":"openai-swarm","relation":"similar-shape","note":"Handoff-based swarm pattern."},{"composition":"semantic-kernel","relation":"same-vendor","note":"Microsoft's other agent framework."},{"composition":"microsoft-agent-framework","relation":"successor","note":"AutoGen's multi-agent orchestration was merged into this framework."},{"composition":"agentverse","relation":"competes-with","note":"Both are research multi-agent frameworks; AutoGen has stronger production traction."},{"composition":"chatdev","relation":"competes-with","note":"AutoGen GroupChat is the Western analogue for role-based multi-agent collaboration."},{"composition":"openagents","relation":"similar-shape","note":"Both are research platforms shipping agents in the wild."},{"composition":"smolagents","relation":"similar-shape","note":"Already marked code-as-action / code-execution first-class in this catalog."},{"composition":"taskweaver","relation":"competes-with","note":"Same MSR family; AutoGen is conversation-first, TaskWeaver is code-first."},{"composition":"xagent","relation":"similar-shape","note":"AutoGen offers similar multi-agent shape without the outer-inner planner split."},{"composition":"deerflow-v1","relation":"similar-shape","note":"Both orchestrate multiple specialised agents, but AutoGen is conversation-driven and general-purpose whereas DeerFlow is a fixed plan-and-execute graph specialised for deep research."}],"references":[{"type":"repo","title":"microsoft/autogen","url":"https://github.com/microsoft/autogen","quote":"A programming framework for agentic AI","accessed":"2026-05-20"},{"type":"doc","title":"AutoGen v0.4 migration guide","url":"https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/migration-guide.html","quote":"AutoGen v0.4, a from-the-ground-up rewrite adopting an asynchronous, event-driven architecture to address issues such as observability, flexibility, interactive control, and scale.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","multi-agent","actor-model","event-driven","microsoft"]},{"id":"autogpt","name":"AutoGPT","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"Significant Gravitas","language":"Python, TypeScript","license":"MIT (classic/) + Polyform Shield (autogpt_platform/)","status":"active","first_released":"2023-03-16","url":"https://agpt.co/","repo":"https://github.com/Significant-Gravitas/AutoGPT","docs_url":"https://docs.agpt.co/","intent":"Provide a platform to create, deploy, and run continuous autonomous AI agents that chain LLM reasoning with a fixed Command/Tool catalogue, an episodic action history, file/web/code-exec components, and an optional human-approval gate.","description":"AutoGPT is the founding 'autonomous-agent' project — the prototype that turned ReAct + a fixed command palette + a goal into a viral category in March 2023. The current top-level repo ships two products: 'AutoGPT Platform' (the new low-code block builder in autogpt_platform/) and 'AutoGPT Classic' (classic/original_autogpt/), the original ReAct loop. The classic loop composes Agent + Forge components: ActionHistory (EpisodicActionHistory), Watchdog, CodeExecutor (Docker-sandboxed), WebPlaywright, WebSearch, FileManager, Todo, Skill (SKILL.md), and UserInteraction.","primary_use_cases":["continuous autonomous agents over a fixed Command catalogue","low-code block-graph agent builder (autogpt_platform)","Docker-sandboxed code execution + web automation","todo-driven goal pursuit with episodic history"],"agent_loop_shape":"ReAct-style cycle: each step the prompt strategy assembles system prompt + episodic action history + watchdog hints + context, sends to LLM, parses ActionProposal, optionally pauses for user approval before executing the chosen Command. EpisodicActionHistory records every proposal-result pair for replay/resume. WatchdogComponent guards against runaway loops; --continuous-limit caps total iterations. TodoComponent maintains an explicit task list. The new platform reframes this as a directed graph of blocks edited in the Agent Builder UI.","key_concepts":[{"name":"Agent + Forge components","summary":"Agent(BaseAgent) composed of pluggable components (Action History, Watchdog, Code Executor, Web, File Manager, Todo, Skill, User Interaction).","maps_to_pattern":"react","url":"https://github.com/Significant-Gravitas/AutoGPT/blob/master/classic/original_autogpt/autogpt/agents/agent.py"},{"name":"EpisodicActionHistory","summary":"Every (proposal, result) tuple persists for resume/replay.","maps_to_pattern":"agent-resumption"},{"name":"Continuous mode + continuous_limit","summary":"Opt-in autopilot with iteration cap; watchdog watches for runaway loops.","maps_to_pattern":"step-budget"},{"name":"CodeExecutorComponent","summary":"Per-agent Docker sandbox (agent_id_sandbox).","maps_to_pattern":"code-execution"},{"name":"TodoComponent","summary":"Explicit task-list driver.","maps_to_pattern":"todo-list-driven-agent"},{"name":"AutoGPT Platform blocks","summary":"Graph builder: each block performs a single action.","maps_to_pattern":"event-driven-agent","url":"https://github.com/Significant-Gravitas/AutoGPT"}],"pattern_composition":"flowchart TD\n  USER[goal + objectives] --> AGT[Agent<br/>OneShotAgentPromptStrategy]\n  AGT -->|prompt assembly| HIST[(EpisodicActionHistory)]\n  AGT -->|ActionProposal| AP[Action proposal]\n  AP --> WD{WatchdogComponent<br/>step-budget}\n  WD -->|cap reached| END[stop]\n  WD -->|ok| UI{UserInteractionComponent<br/>continuous?}\n  UI -->|approve / continuous| EXEC[Execute Command]\n  EXEC --> CMD[Command catalogue]\n  CMD --> CODE[CodeExecutorComponent<br/>Docker sandbox]\n  CMD --> WEB[WebPlaywright + WebSearch]\n  CMD --> FILE[FileManagerComponent]\n  CMD --> TODO[TodoComponent]\n  CMD --> SKILL[SkillComponent<br/>SKILL.md]\n  EXEC -->|result| HIST\n  HIST --> AGT\n  PLAT[AutoGPT Platform<br/>graph of blocks] -.alternative.- AGT","members":[{"pattern":"agent-resumption","role":"limited","note":"EpisodicActionHistory persists proposal-result tuples; resume requires reloading the history.","evidence":[{"type":"repo","title":"AutoGPT — agent.py history field","url":"https://github.com/Significant-Gravitas/AutoGPT/blob/master/classic/original_autogpt/autogpt/agents/agent.py","quote":"history: EpisodicActionHistory[AnyActionProposal] = Field(default_factory=EpisodicActionHistory[AnyActionProposal])","accessed":"2026-05-20"},{"type":"repo","title":"AutoGPT README — continuous operation","url":"https://github.com/Significant-Gravitas/AutoGPT","quote":"Once deployed, agents can be triggered by external sources and can operate continuously.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"code-execution","role":"first-class","note":"CodeExecutorComponent with per-agent Docker sandbox.","evidence":[{"type":"repo","title":"AutoGPT — agent.py sandbox name","url":"https://github.com/Significant-Gravitas/AutoGPT/blob/master/classic/original_autogpt/autogpt/agents/agent.py","quote":"docker_container_name=f\"{settings.agent_id}_sandbox\"","accessed":"2026-05-24"},{"type":"repo","title":"AutoGPT README — Docker required","url":"https://github.com/Significant-Gravitas/AutoGPT","quote":"Docker Engine (20.10.0 or newer); Docker Compose (2.0.0 or newer)","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"event-driven-agent","role":"first-class","note":"AutoGPT Platform expresses agents as directed graphs of blocks, each performing a single action; blocks can be triggered by external sources.","evidence":[{"type":"repo","title":"AutoGPT README — blocks graph","url":"https://github.com/Significant-Gravitas/AutoGPT","quote":"You build your agent by connecting blocks, where each block performs a single action.","accessed":"2026-05-20"},{"type":"repo","title":"AutoGPT README — external triggers","url":"https://github.com/Significant-Gravitas/AutoGPT","quote":"Once deployed, agents can be triggered by external sources and can operate continuously.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"OneShotAgentPromptStrategy is the canonical ReAct strategy.","evidence":[{"type":"repo","title":"AutoGPT — OneShotAgentPromptStrategy import","url":"https://github.com/Significant-Gravitas/AutoGPT/blob/master/classic/original_autogpt/autogpt/agents/agent.py","quote":"from .prompt_strategies.one_shot import (OneShotAgentActionProposal, OneShotAgentPromptStrategy,)","accessed":"2026-05-20"},{"type":"repo","title":"AutoGPT README — create continuous AI agents","url":"https://github.com/Significant-Gravitas/AutoGPT","quote":"AutoGPT is a powerful platform that allows you to create, deploy, and manage continuous AI agents that automate complex workflows.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"step-budget","role":"first-class","note":"--continuous-limit caps iterations; WatchdogComponent watches for runaway loops.","evidence":[{"type":"repo","title":"AutoGPT — continuous mode warning","url":"https://github.com/Significant-Gravitas/AutoGPT/blob/master/classic/original_autogpt/autogpt/app/configurator.py","quote":"Continuous mode is not recommended. It is potentially dangerous and may cause your AI to run forever or carry out actions you would not usually authorise.","accessed":"2026-05-20"},{"type":"repo","title":"AutoGPT — continuous-limit flag","url":"https://github.com/Significant-Gravitas/AutoGPT/blob/master/classic/original_autogpt/autogpt/app/configurator.py","quote":"--continuous-limit can only be used with --continuous","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"todo-list-driven-agent","role":"first-class","note":"TodoComponent is a first-class component on the Agent.","evidence":[{"type":"repo","title":"AutoGPT — TodoComponent constructor","url":"https://github.com/Significant-Gravitas/AutoGPT/blob/master/classic/original_autogpt/autogpt/agents/agent.py","quote":"self.todo = TodoComponent(llm_provider=llm_provider, smart_llm=str(app_config.smart_llm),)","accessed":"2026-05-20"},{"type":"repo","title":"AutoGPT — WatchdogComponent watchdog","url":"https://github.com/Significant-Gravitas/AutoGPT/blob/master/classic/original_autogpt/autogpt/agents/agent.py","quote":"self.watchdog = WatchdogComponent(settings.config, settings.history).run_after(ContextComponent)","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Command system; function_specs_from_commands exports OpenAI function specs.","evidence":[{"type":"repo","title":"AutoGPT — function_specs_from_commands import","url":"https://github.com/Significant-Gravitas/AutoGPT/blob/master/classic/original_autogpt/autogpt/agents/agent.py","quote":"from forge.llm.providers.utils import function_specs_from_commands","accessed":"2026-05-20"},{"type":"repo","title":"AutoGPT README — automate complex workflows","url":"https://github.com/Significant-Gravitas/AutoGPT","quote":"AutoGPT is a powerful platform that allows you to create, deploy, and manage continuous AI agents that automate complex workflows.","accessed":"2026-05-24"}],"evidence_status":"full"}],"instantiates":["long-running-autonomous-agent"],"alternatives":[{"composition":"babyagi","relation":"similar-shape","note":"Sibling viral 2023 task-list agent."},{"composition":"openmanus","relation":"similar-shape","note":"General-computer-using descendant."},{"composition":"crewai","relation":"competes-with","note":"Richer multi-role descendant."},{"composition":"xagent","relation":"competes-with","note":"Both are autonomous LLM agents; XAgent adds Docker-isolated ToolServer and explicit Planner/Actor/Dispatcher split."}],"references":[{"type":"repo","title":"Significant-Gravitas/AutoGPT","url":"https://github.com/Significant-Gravitas/AutoGPT","quote":"AutoGPT is a powerful platform that allows you to create, deploy, and manage continuous AI agents that automate complex workflows","accessed":"2026-05-20"},{"type":"doc","title":"AutoGPT docs","url":"https://docs.agpt.co/","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","autonomous-agent","viral-2023","command-catalogue","block-graph-platform"]},{"id":"babyagi","name":"BabyAGI","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"Yohei Nakajima","language":"Python","license":"MIT","status":"maintenance","first_released":"2023-03-28","url":"https://github.com/yoheinakajima/babyagi","repo":"https://github.com/yoheinakajima/babyagi","intent":"Explore the self-building autonomous-agent direction via the functionz function framework — a database-backed store of named functions with dependency tracking that experimental agents like process_user_input and self_build can extend at runtime. The original March-2023 task-list agent has been archived.","description":"BabyAGI is Yohei Nakajima's experimental autonomous-agent project. The original March 2023 BabyAGI — the viral task-creation / prioritisation / execution loop that defined the early 'autonomous agent' category — has been archived to babyagi_archive (September 2024 snapshot). The current repository is a complete rewrite around functionz: a Python function framework that stores, manages, and executes functions from a database with graph-based dependency tracking, automatic loading, logging, and a web dashboard. Experimental agents (process_user_input, self_build) generate new functions from user requests, decomposing them into 'smaller reusable components'. Yohei is the sole maintainer ('nights and weekends'); PR throughput is slow.","primary_use_cases":["self-building agent research prototype","function-store / function-graph experimentation","historical reference for the 2023 task-list agent shape (via babyagi_archive)"],"agent_loop_shape":"Self-building function-graph loop. The functionz store holds named Python functions with declared dependencies. Self-building agents (process_user_input, self_build) receive a user request, break it into 'smaller reusable components', generate the required functions, register them in the database, and execute them — logging every step and surfacing the graph in a web dashboard. The original task-list shape (task creation → prioritisation → execution → enqueue) lives in the archived repo, not the current code.","key_concepts":[{"name":"functionz","summary":"Function framework for storing, managing, and executing functions from a database; graph-based dependency tracking, automatic loading, logging."},{"name":"self-building agents","summary":"process_user_input and self_build generate new functions from user requests, breaking tasks into reusable components.","maps_to_pattern":"todo-list-driven-agent"},{"name":"Web dashboard","summary":"Built-in dashboard for managing functions and viewing logs."},{"name":"babyagi_archive","summary":"September-2024 snapshot of the original task-list BabyAGI."}],"pattern_composition":"flowchart TD\n  USER[User request] --> AGT[process_user_input / self_build]\n  AGT -->|decompose| COMP[Reusable components]\n  COMP --> FNGEN[Generate functions]\n  FNGEN --> STORE[(functionz database<br/>graph + dependencies)]\n  STORE --> EXEC[Execute functions]\n  EXEC --> LOG[(Log + dashboard)]\n  LOG -.->|new requests| AGT\n  STORE -.archived March-2023 task-list loop.-> ARCHIVE[babyagi_archive]","members":[{"pattern":"todo-list-driven-agent","role":"first-class","note":"Self-building agents break user requests into 'smaller reusable components' and generate functions for them — a function-graph variant of the task-list shape that BabyAGI originally popularised.","evidence":[{"type":"repo","title":"BabyAGI README — self-building","url":"https://github.com/yoheinakajima/babyagi","quote":"experimental agents like process_user_input and self_build that can generate new functions autonomously based on user requests","accessed":"2026-05-20"},{"type":"repo","title":"BabyAGI README — archive note","url":"https://github.com/yoheinakajima/babyagi","quote":"The original BabyAGI from March 2023 introduced task planning as a method for developing autonomous agents. This project has been archived and moved to the babyagi_archive repo (September 2024 snapshot).","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"functionz is literally a tool/function store with dependency graph; agents execute functions from it.","evidence":[{"type":"repo","title":"BabyAGI README — functionz core","url":"https://github.com/yoheinakajima/babyagi","quote":"The core is a new function framework (functionz) for storing, managing, and executing functions from a database. It offers a graph-based structure for tracking imports, dependent functions, and authentication secrets, with automatic loading and comprehensive logging capabilities.","accessed":"2026-05-24"},{"type":"repo","title":"BabyAGI README — process_user_input","url":"https://github.com/yoheinakajima/babyagi","quote":"This function first determines whether to use an existing function or generate new ones. If new functions are needed, it breaks them down into smaller reusable components and combines them into a final function.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"self-archaeology","role":"limited","note":"self_build generates candidate tasks then process_user_input extends the function store at runtime, inspecting whether existing functions suffice before generating new ones — a partial fit to self-archaeology since the system primarily extends rather than mines its own history.","evidence":[{"type":"repo","title":"BabyAGI README — self_build","url":"https://github.com/yoheinakajima/babyagi","quote":"This function takes a user description and generates X distinct tasks that a user might ask an AI assistant. Each task is processed by process_user_input, creating new functions if no existing ones suffice.","accessed":"2026-05-24"},{"type":"repo","title":"BabyAGI README — process_user_input decision","url":"https://github.com/yoheinakajima/babyagi","quote":"This function first determines whether to use an existing function or generate new ones. If new functions are needed, it breaks them down into smaller reusable components and combines them into a final function.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"react","role":"limited","note":"Function generation and execution use LLM reasoning but the loop is not the ReAct interleaved thought-action-observation cycle. No mention of ReAct, reason+act, or interleaved thought/action in the current repo's README.","evidence":[],"evidence_status":"none"},{"pattern":"scheduled-agent","role":"limited","note":"The original BabyAGI ran a continuous infinite loop rather than cron-style scheduled execution; the archive README explicitly frames it as run-continuously.","evidence":[{"type":"doc","title":"BabyAGI archive — README (infinite loop)","url":"https://github.com/yoheinakajima/babyagi_archive","quote":"The script works by running an infinite loop that does the following steps:","accessed":"2026-05-24"},{"type":"doc","title":"BabyAGI archive — README (continuous run)","url":"https://github.com/yoheinakajima/babyagi_archive","quote":"This script is designed to be run continuously as part of a task management system.","accessed":"2026-05-24"}],"evidence_status":"full"}],"alternatives":[{"composition":"autogpt","relation":"similar-shape","note":"Sibling viral 2023 task-list agent."},{"composition":"xagent","relation":"competes-with","note":"BabyAGI is the simpler ancestor of the autonomous-agent shape."}],"references":[{"type":"repo","title":"yoheinakajima/babyagi","url":"https://github.com/yoheinakajima/babyagi","quote":"An experimental framework for a self-building autonomous agent.","accessed":"2026-05-20"},{"type":"repo","title":"BabyAGI maintainer note","url":"https://github.com/yoheinakajima/babyagi","quote":"I have not been great at managing PRs. Please be patient as things will move slow while I am working on this alone (on nights and weekends).","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"experimental","tags":["open-source","python","self-building","viral-2023-archived","functionz","research-prototype"]},{"id":"bee-agent","name":"BeeAI Framework","aliases":["Bee Agent Framework"],"kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"Linux Foundation AI & Data (originally IBM)","language":"Python, TypeScript","license":"Apache-2.0","status":"active","url":"https://github.com/i-am-bee/beeai-framework","repo":"https://github.com/i-am-bee/beeai-framework","docs_url":"https://framework.beeai.dev","intent":"Provide a multi-language (Python + TypeScript) framework for production-ready multi-agent systems with a RequirementAgent that enforces declared rules across LLMs, multi-agent workflows, event-driven observability, agent state serialisation, MCP/A2A protocol support, and a unified backend over multiple LLM providers.","description":"BeeAI (formerly the IBM Bee Agent Framework) is the Apache-2.0 multi-agent framework now governed under the Linux Foundation AI & Data program. Tagline: 'Build production-ready multi-agent systems in Python or TypeScript.' Its headline abstraction is the RequirementAgent — a rule-driven agent that produces 'predictable, controlled behavior across different LLMs by setting rules the agent must follow'. The framework ships unified Backend connectors, built-in tools (web search, weather, code execution) plus custom tool support, RAG with vector stores, conversation Memory, multi-agent Workflows, event-based observability, agent serialisation for resumption, and protocol support for MCP and A2A.","primary_use_cases":["rule-bound agents that behave consistently across LLM providers","multi-agent workflows with complex execution flows","agents that persist state across sessions (serialisation)","MCP- and A2A-integrated agents","event-driven observability of agent behaviour"],"agent_loop_shape":"Rule-driven tool-calling loop. A RequirementAgent runs an LLM-backed loop in which each step is checked against declared requirements (rules) before tools are called; the unified Backend abstracts the underlying provider so the same rules apply across LLMs. Multi-agent topologies are built on top via Workflows that orchestrate several agents. Event emitters fire on every step for logging, tracing, and error handling. Agent state is serialised via Save/load for cross-session persistence; MCP and A2A protocols expose and consume external agents and tools.","key_concepts":[{"name":"RequirementAgent","summary":"Rule-driven agent producing predictable behaviour across LLMs.","url":"https://framework.beeai.dev"},{"name":"Workflows","summary":"Orchestrate multi-agent systems with complex execution flows.","maps_to_pattern":"orchestrator-workers"},{"name":"Backend","summary":"Unified interface connecting to various LLM providers."},{"name":"Memory","summary":"Conversation history management with multiple strategies."},{"name":"Serialization","summary":"Save and load agent state for persistence across sessions.","maps_to_pattern":"agent-resumption"},{"name":"Observability events","summary":"Event-based monitoring, logging, and error handling."},{"name":"MCP + A2A","summary":"Host agents in servers with support for multiple protocols including MCP and A2A.","maps_to_pattern":"mcp"}],"pattern_composition":"flowchart TD\n  USER[User input] --> RA[RequirementAgent<br/>declared rules]\n  RA --> BACK[Backend<br/>unified LLM connector]\n  BACK --> MODEL[(LLM)]\n  MODEL -->|tool calls| TOOLS[Built-in + custom tools<br/>web / weather / code-exec]\n  TOOLS --> CODE[Code execution sandbox]\n  TOOLS --> RAG[(RAG + vector stores)]\n  RA -.memory.-> MEM[(Conversation memory)]\n  RA -.serialization.-> STATE[(Saved agent state)]\n  RA -.events.-> OBS[Observability events]\n  RA --> WF[Workflows<br/>multi-agent orchestration]\n  WF --> RA2[Other agent]\n  RA -.MCP / A2A.-> EXT[External MCP / A2A peers]","members":[{"pattern":"agent-resumption","role":"first-class","note":"Serialization saves and loads agent state across sessions.","evidence":[{"type":"repo","title":"BeeAI Framework README — serialization","url":"https://github.com/i-am-bee/beeai-framework","quote":"Save and load agent state for persistence across sessions","accessed":"2026-05-24"},{"type":"repo","title":"BeeAI Framework README — monitoring","url":"https://github.com/i-am-bee/beeai-framework","quote":"Monitor agent behavior with events, logging, and robust error handling","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"agentic-rag","role":"first-class","note":"Retrieval-augmented generation systems with vector stores are a documented capability.","evidence":[{"type":"repo","title":"BeeAI Framework README — RAG","url":"https://github.com/i-am-bee/beeai-framework","quote":"Build retrieval-augmented generation systems with vector stores and document processing","accessed":"2026-05-24"},{"type":"repo","title":"BeeAI Framework README — tools","url":"https://github.com/i-am-bee/beeai-framework","quote":"Extend agents with built in tools (web search, weather, code execution, and more)","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"code-execution","role":"first-class","note":"Code execution is a built-in tool alongside web search and weather.","evidence":[{"type":"repo","title":"BeeAI Framework README — tools","url":"https://github.com/i-am-bee/beeai-framework","quote":"Extend agents with built in tools (web search, weather, code execution, and more)","accessed":"2026-05-24"},{"type":"repo","title":"BeeAI Framework README — RequirementAgent","url":"https://github.com/i-am-bee/beeai-framework","quote":"Create predictable, controlled behavior across different LLMs by setting rules the agent must follow","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"event-driven-agent","role":"first-class","note":"Event-based monitoring, logging, and error handling are the documented observability model.","evidence":[{"type":"repo","title":"BeeAI Framework README — observability","url":"https://github.com/i-am-bee/beeai-framework","quote":"Monitor agent behavior with events, logging, and robust error handling","accessed":"2026-05-24"},{"type":"repo","title":"BeeAI Framework README — workflows","url":"https://github.com/i-am-bee/beeai-framework","quote":"Orchestrate multi-agent systems with complex execution flows","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"Hosting agents in servers with support for multiple protocols such as A2A and MCP.","evidence":[{"type":"repo","title":"BeeAI Framework README — protocols","url":"https://github.com/i-am-bee/beeai-framework","quote":"Host agents in servers with support for multiple protocols such as A2A and MCP","accessed":"2026-05-24"},{"type":"doc","title":"BeeAI Framework — Python README MCP","url":"https://github.com/i-am-bee/beeai-framework/blob/main/python/README.md","quote":"Host agents in servers with support for multiple protocols such as A2A and MCP","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"orchestrator-workers","role":"first-class","note":"Workflows orchestrate multi-agent systems with complex execution flows.","evidence":[{"type":"repo","title":"BeeAI Framework README — workflows","url":"https://github.com/i-am-bee/beeai-framework","quote":"Orchestrate multi-agent systems with complex execution flows","accessed":"2026-05-24"},{"type":"doc","title":"BeeAI Framework docs — agents","url":"https://framework.beeai.dev/modules/agents","quote":"Create a team of specialized agents that can collaborate","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"Agents 'capable of reasoning, acting, and adapting' (the ReAct pattern) sit alongside the rule-bound RequirementAgent.","evidence":[{"type":"doc","title":"BeeAI Framework docs — ReActAgent","url":"https://framework.beeai.dev/modules/agents","quote":"The ReActAgent implements the ReAct (Reasoning and Acting) pattern, which structures agent behavior into a cyclical process of reasoning, action, and observation.","accessed":"2026-05-24"},{"type":"doc","title":"BeeAI Framework docs — ToolCallingAgent","url":"https://framework.beeai.dev/modules/agents","quote":"The ToolCallingAgent is optimized for scenarios where tool usage is the primary focus. It handles tool calls more efficiently and can execute multiple tools in parallel.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Built-in plus custom tools are a top-level capability.","evidence":[{"type":"repo","title":"BeeAI Framework README — tools","url":"https://github.com/i-am-bee/beeai-framework","quote":"Extend agents with built in tools (web search, weather, code execution, and more)","accessed":"2026-05-24"},{"type":"doc","title":"BeeAI Framework docs — ToolCallingAgent","url":"https://framework.beeai.dev/modules/agents","quote":"The ToolCallingAgent is optimized for scenarios where tool usage is the primary focus. It handles tool calls more efficiently and can execute multiple tools in parallel.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"structured-output","role":"first-class","note":"RequirementAgent rules constrain outputs and expected_output accepts Pydantic models or JSON Schema; output_structured is exposed on responses.","evidence":[{"type":"doc","title":"BeeAI Framework docs — expected_output","url":"https://framework.beeai.dev/modules/agents","quote":"expected_output can also receive a Pydantic Model or a JSON Schema","accessed":"2026-05-24"},{"type":"doc","title":"BeeAI Framework docs — output_structured","url":"https://framework.beeai.dev/modules/agents","quote":"output_structured is defined only when expected_output is a Pydantic model or a JSON schema","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"step-budget","role":"first-class","note":"RequirementAgent and base agents expose max_iterations and total_max_retries for explicit step and retry caps.","evidence":[{"type":"doc","title":"BeeAI Framework docs — max_iterations","url":"https://framework.beeai.dev/modules/agents","quote":"max_iterations: Sets the maximum number of reasoning cycles the agent can perform","accessed":"2026-05-24"},{"type":"doc","title":"BeeAI Framework docs — total_max_retries","url":"https://framework.beeai.dev/modules/agents","quote":"total_max_retries: Controls the total number of retry attempts across the entire agent execution","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"handoff","role":"first-class","note":"HandoffTool delegates between specialized agents in multi-agent setups (e.g. KnowledgeLookup / Weather Agent handoffs).","evidence":[{"type":"doc","title":"BeeAI Framework docs — HandoffTool","url":"https://framework.beeai.dev/modules/agents","quote":"HandoffTool(knowledge_agent, name=\"KnowledgeLookup\", description=\"Consult the Knowledge Agent for general questions.\")","accessed":"2026-05-24"},{"type":"doc","title":"BeeAI Framework docs — multi-agent collaboration","url":"https://framework.beeai.dev/modules/agents","quote":"Create a team of specialized agents that can collaborate","accessed":"2026-05-24"}],"evidence_status":"full"}],"references":[{"type":"repo","title":"i-am-bee/beeai-framework","url":"https://github.com/i-am-bee/beeai-framework","quote":"Build production-ready multi-agent systems in Python or TypeScript.","accessed":"2026-05-20"},{"type":"repo","title":"BeeAI Framework — governance","url":"https://github.com/i-am-bee/beeai-framework","quote":"Part of the Linux Foundation AI & Data program, following open, collaborative, and community-driven practices.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"emerging","tags":["open-source","python","typescript","multi-agent","linux-foundation","ibm-origin","requirement-agent","mcp","a2a"]},{"id":"burr","name":"Burr","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"Apache Software Foundation (originally DAGWorks)","language":"Python","license":"Apache-2.0","status":"active","url":"https://burr.dagworks.io","repo":"https://github.com/apache/burr","docs_url":"https://burr.dagworks.io","intent":"Build stateful decision-making applications (chatbots, agents, simulations) as an explicit state machine of @action-decorated functions with pluggable persisters, a Tracker UI for inspection, OpenTelemetry integration, streaming actions, and fork-and-replay for debugging and evaluation.","description":"Burr is a framework for developing stateful AI applications, donated by DAGWorks to the Apache Software Foundation and incubating there. An Application is a state machine in which @action-decorated functions read from and write to a typed State; transitions between actions are explicit. Built-in features: streaming actions for low time-to-first-token; pluggable Persisters (e.g. SQLLitePersister) for save/load and resume-from-checkpoint; fork and replay (initialize_from with fork_from_app_id) for debugging and evaluation; a tracking UI; and OpenTelemetry integration. Framework-agnostic — works with any LLM library or non-LLM use case.","primary_use_cases":["explicit-state-machine chatbots and agents","RAG and decision pipelines that must persist and resume","debugging-by-replay via fork-from-app","OpenTelemetry-instrumented production decision flows"],"agent_loop_shape":"Explicit state machine of actions over typed state. Each step the runtime selects the next @action function based on declared transitions and current state, executes it, applies its state mutations, fires hooks, and records to the configured Persister. Streaming actions return a StreamingResultContainer for incremental consumption. Resumption uses initialize_from(persister, app_id=..., resume_at_next_action=True) to rebuild State from a persisted checkpoint. Fork-and-replay rewinds to any past step via fork_from_app_id, supporting debugging and eval. OpenTelemetry traces and the Tracker UI provide observability.","key_concepts":[{"name":"Action","summary":"Unit of work decorated with @action; reads/writes the state.","url":"https://burr.dagworks.io"},{"name":"Application","summary":"State machine orchestrating actions and transitions."},{"name":"State + Transitions","summary":"Typed State plus explicit transitions between actions."},{"name":"Persister","summary":"Pluggable state checkpoint store (e.g. SQLLitePersister); custom persisters implement BaseStatePersister.","maps_to_pattern":"agent-resumption","url":"https://burr.dagworks.io/concepts/state-persistence/"},{"name":"Fork & replay","summary":"initialize_from(fork_from_app_id=...) rewinds and forks state from a previous application.","maps_to_pattern":"replay-time-travel"},{"name":"Tracker UI + OTel","summary":"Open-source telemetry UI for real-time tracing; OpenTelemetry integration for export."},{"name":"Streaming actions","summary":"stream_result() / astream_result() yield a StreamingResultContainer cached iterator."}],"pattern_composition":"flowchart TD\n  USER[Input] --> APP[Application<br/>state machine]\n  APP --> SEL{Select next action<br/>transitions over state}\n  SEL --> ACT[@action function]\n  ACT --> STATE[(Typed State)]\n  ACT -.stream.-> STREAM[StreamingResultContainer]\n  APP -.every step.-> PERS[(Persister<br/>SQLLitePersister / custom)]\n  PERS -.initialize_from + resume_at_next_action.-> APP\n  PERS -.fork_from_app_id.-> FORK[Forked Application]\n  APP -.hooks + OTel.-> TRACK[Tracker UI + traces]","members":[{"pattern":"agent-resumption","role":"first-class","note":"Pluggable Persister saves and loads State; initialize_from(persister, resume_at_next_action=True) resumes execution from the last checkpoint.","evidence":[{"type":"doc","title":"Burr — state persistence","url":"https://burr.dagworks.io/concepts/state-persistence/","quote":"Burr comes with a core set of APIs that enable state persistence – the ability to save and load state automatically from a database.","accessed":"2026-05-20"},{"type":"doc","title":"Burr — resume_at_next_action","url":"https://burr.dagworks.io/concepts/state-persistence/","quote":"The initialize_from() method includes a resume_at_next_action parameter that determines whether to start where you left off, or go back to the default_entrypoint.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"replay-time-travel","role":"first-class","note":"Forking state from a previous application via fork_from_app_id; documented as 'state snapshots and debugging capabilities'.","evidence":[{"type":"doc","title":"Burr — fork state","url":"https://burr.dagworks.io/concepts/state-persistence/","quote":"Burr enables forking state from a previous application using fork_from_app_id and related parameters — useful for debugging or rewinding to specific points.","accessed":"2026-05-20"},{"type":"doc","title":"Burr testimonial — snapshots and replay","url":"https://github.com/apache/burr","quote":"Burr's state management part is really helpful for creating state snapshots and build debugging, replaying and even building evaluation cases around that","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"decision-log","role":"first-class","note":"Every action, state mutation, and transition is recorded; Tracker UI provides real-time inspection.","evidence":[{"type":"repo","title":"Burr README — tagline","url":"https://github.com/apache/burr","quote":"Build applications that make decisions (chatbots, agents, simulations, etc...). Monitor, trace, persist, and execute on your own infrastructure.","accessed":"2026-05-20"},{"type":"repo","title":"Burr — telemetry UI","url":"https://github.com/apache/burr","quote":"Open-source telemetry UI for real-time tracing; Execution introspection and visualization","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"lineage-tracking","role":"first-class","note":"State snapshots plus fork-from-app give per-run lineage usable for debugging and evaluation.","evidence":[{"type":"repo","title":"Burr README — telemetry UI","url":"https://github.com/apache/burr","quote":"A UI you can use view execution telemetry for introspection and debugging","accessed":"2026-05-24"},{"type":"repo","title":"Burr README — snapshots and replay","url":"https://github.com/apache/burr","quote":"Burr's state management part is really helpful for creating state snapshots and build debugging, replaying and even building evaluation cases around that","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"event-driven-agent","role":"first-class","note":"Transitions between actions are condition-driven; hooks fire on lifecycle events; integrates with OpenTelemetry.","evidence":[{"type":"doc","title":"Burr docs — transitions","url":"https://burr.dagworks.io/concepts/transitions/","quote":"Transitions define explicitly how actions are connected and which action is available next for any given state.","accessed":"2026-05-24"},{"type":"doc","title":"Burr docs — transition conditions","url":"https://burr.dagworks.io/concepts/transitions/","quote":"Conditions are evaluated in the order they are specified, and the first one that evaluates to True will be the transition that is selected.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"approval-queue","role":"limited","note":"Hooks and lifecycle adapters enable approval-style gates around node execution; not a dedicated 'approval queue' API.","evidence":[{"type":"doc","title":"Burr — Hooks","url":"https://burr.dagworks.io/concepts/hooks/","quote":"Hooks allow you to customize every aspect of Burr's execution, plugging in whatever tooling, observability framework, or debugging tool you need.","accessed":"2026-05-24"},{"type":"doc","title":"Burr — Hooks (lifecycle adapters)","url":"https://burr.dagworks.io/concepts/hooks/","quote":"Burr has a system of lifecycle adapters (adapted from the similar Hamilton concept), which allow you to run tooling before and after various places in a node's execution.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"cost-observability","role":"limited","note":"OpenTelemetry integration surfaces cost via custom spans; not a first-class cost panel.","evidence":[{"type":"doc","title":"Burr docs — OpenTelemetry integration","url":"https://burr.dagworks.io/concepts/additional-visibility/","quote":"Burr comes with the ability to see inside your actions. This is a very pluggable framework that comes with the default tracking client, but can also be hooked up to tools such as OpenTelemetry (OTel)","accessed":"2026-05-24"},{"type":"doc","title":"Burr docs — OTel spans","url":"https://burr.dagworks.io/concepts/additional-visibility/","quote":"Burr can capture OpenTelemetry traces/spans that are logged from within a Burr step.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"react","role":"limited","note":"ReAct-style agents are buildable on top of Burr's action/state model; Burr does not ship a ReAct agent class. No ReAct primitive surfaced in README or surveyed concept docs.","evidence":[],"evidence_status":"none"},{"pattern":"tool-use","role":"limited","note":"Tools are user-defined @action functions calling external systems; framework is LLM- and tool-agnostic with no dedicated tool abstraction.","evidence":[{"type":"doc","title":"Burr — Actions","url":"https://burr.dagworks.io/concepts/actions/","quote":"You can define actions by decorating a function with the `@action` decorator.","accessed":"2026-05-24"},{"type":"doc","title":"Burr — README","url":"https://github.com/apache/burr","quote":"@action(reads=[], writes=[\"prompt\", \"chat_history\"])","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"step-budget","role":"limited","note":"Step budget is user-encoded as state and transition guards; no built-in step-cap primitive in surveyed README or concept docs.","evidence":[],"evidence_status":"none"},{"pattern":"spec-driven-loop","role":"limited","note":"Transitions encode the spec for valid runs; not framed in the docs as 'spec-driven'.","evidence":[],"evidence_status":"none"},{"pattern":"stateless-reducer-agent","role":"core","note":"Models the application as a state machine where actions are pure (state)->(new state); state is entirely immutable, so any prior state can be serialized, stored, and reloaded for replay/time-travel debugging.","evidence":[{"type":"doc","title":"State - Apache Burr","url":"https://burr.dagworks.io/concepts/state/","quote":"It is entirely immutable, meaning that you can only create new states from old ones, not modify them in place. ... This is useful for reloading state from a previous invocation (for debugging or as part of the application), or for storing state in a database.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"alternatives":[{"composition":"hamilton","relation":"complements","note":"Same vendor (DAGWorks). Hamilton handles DAG transformations; Burr handles agent loops and state machines."},{"composition":"langgraph","relation":"similar-shape","note":"Both model agents as graphs over a typed state object with checkpointing."}],"references":[{"type":"repo","title":"apache/burr","url":"https://github.com/apache/burr","quote":"Build applications that make decisions (chatbots, agents, simulations, etc...). Monitor, trace, persist, and execute on your own infrastructure.","accessed":"2026-05-20"},{"type":"doc","title":"Burr documentation","url":"https://burr.dagworks.io","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"emerging","tags":["open-source","python","state-machine","apache-incubating","tracker-ui","opentelemetry"]},{"id":"camel-ai","name":"CAMEL-AI","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"CAMEL-AI.org","language":"Python","license":"Apache-2.0","status":"active","url":"https://www.camel-ai.org","repo":"https://github.com/camel-ai/camel","docs_url":"https://docs.camel-ai.org","intent":"Study agent scaling laws by providing a multi-agent framework whose core building blocks are ChatAgent (tool-calling LLM agent), RolePlaying (AI-assistant + AI-user dialectic), Workforce (managed multi-agent collaboration), MCP-backed toolkits, RAG pipelines, code interpreters, and large-scale simulated societies.","description":"CAMEL ('Communicative Agents for \"Mind\" Exploration of Large Language Model Society') is an Apache-2.0 Python multi-agent framework whose stated mission is 'finding the scaling laws of agents'. Building blocks: ChatAgent (atomic LLM-driven reasoning unit with tool use), RolePlaying (two-agent cooperative paradigm where an AI-assistant and an AI-user collaborate on a task), Workforce (manager-led multi-agent collaboration in camel/societies/workforce), 20+ toolkits including SearchToolkit, MCP support, RAG retrievers, code interpreters (Python/shell/browser), and OWL agent for reasoning. The framework is used at scale (up to ~1M agents in OASIS-style simulations).","primary_use_cases":["two-agent role-playing task solving (CAMEL paradigm)","managed multi-agent collaboration via Workforce","ChatAgent-based tool-using assistants","MCP-backed agentic workflows over external infrastructure","large-scale social simulation (e.g. OASIS, ~1M agents)"],"agent_loop_shape":"ChatAgent inner loop is tool-calling: model emits messages and tool calls, tools (Python interpreter, shell, browser, MCP, search) execute, results return as messages. RolePlaying sits on top: a Society pairs an AI-assistant and an AI-user who exchange messages until the task converges or a step cap is hit. Workforce wraps this with a manager that decomposes tasks, assigns to workers, and synthesises results. Memory persists across turns; RAG pipelines combine chunking, retrieval, and generation; OASIS-style simulations scale this to large agent populations.","key_concepts":[{"name":"ChatAgent","summary":"Atomic reasoning unit driven by an LLM, capable of tool calls and decision-making.","url":"https://docs.camel-ai.org"},{"name":"RolePlaying / Society","summary":"Two-agent cooperative paradigm (AI-assistant + AI-user) for autonomous task solving.","maps_to_pattern":"camel-role-playing"},{"name":"Workforce","summary":"Managed multi-agent collaboration in camel/societies/workforce (e.g. 'Hackathon Judge Committee with Workforce').","maps_to_pattern":"supervisor"},{"name":"Toolkits + MCP","summary":"20+ toolkits (search, web, code), with MCP support (ACI MCP, Cloudflare MCP CAMEL).","maps_to_pattern":"mcp"},{"name":"Interpreters","summary":"Execution backends (Python, shell, browsers) for live code evaluation and automation.","maps_to_pattern":"code-execution"},{"name":"RAG pipelines","summary":"Combine chunking, retrieval, and generation for grounded responses.","maps_to_pattern":"agentic-rag"}],"pattern_composition":"flowchart TD\n  USER[User task] --> RP[RolePlaying / Society]\n  RP --> AIASSIST[AI-assistant<br/>ChatAgent]\n  RP --> AIUSER[AI-user<br/>ChatAgent]\n  AIASSIST <-->|messages| AIUSER\n  AIASSIST -->|tool calls| TK[Toolkits]\n  TK --> INTERP[Python / shell / browser interpreters]\n  TK --> MCP[(MCP servers)]\n  TK --> RAG[(RAG retrievers)]\n  WF[Workforce<br/>manager + workers] --> AIASSIST\n  WF --> WORKERS[Worker ChatAgents]\n  AIASSIST -.memory.-> MEM[(Memory / storage)]\n  WF -.OASIS.-> SIM[Large-scale simulation]","members":[{"pattern":"camel-role-playing","role":"first-class","note":"RolePlaying class (camel/societies/role_playing.py) implements the AI-assistant + AI-user paradigm that the framework was originally named for.","evidence":[{"type":"repo","title":"CAMEL README — role-playing","url":"https://github.com/camel-ai/camel","quote":"Role-Playing Agents via the RolePlaying class for agent collaboration","accessed":"2026-05-20"},{"type":"repo","title":"CAMEL README — scaling laws","url":"https://github.com/camel-ai/camel/blob/master/README.md","quote":"CAMEL is an open-source community dedicated to finding the scaling laws of agents.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"supervisor","role":"first-class","note":"Workforce in camel/societies/workforce realises the manager + workers pattern; e.g. 'Hackathon Judge Committee with Workforce'.","evidence":[{"type":"repo","title":"CAMEL README — Workforce","url":"https://github.com/camel-ai/camel","quote":"Workforce/Supervisor patterns for managed agent coordination","accessed":"2026-05-20"},{"type":"doc","title":"CAMEL docs — Societies","url":"https://docs.camel-ai.org","quote":"Societies: Coordinator layers managing multi-agent collaboration with role assignment and task delegation.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"ChatAgent accepts tools (e.g. SearchToolkit().search_duckduckgo); 20+ toolkits ship.","evidence":[{"type":"repo","title":"CAMEL README — ChatAgent + tools","url":"https://github.com/camel-ai/camel/blob/master/README.md","quote":"from camel.agents import ChatAgent from camel.toolkits import SearchToolkit search_tool = SearchToolkit().search_duckduckgo","accessed":"2026-05-24"},{"type":"doc","title":"CAMEL docs — Interpreters tool capability","url":"https://docs.camel-ai.org/key_modules/interpreters","quote":"Interpreters empower agents to run code dynamically—enabling evaluation, testing, task automation, and rich feedback loops.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"MCP support is documented with real integrations (ACI MCP, Cloudflare MCP CAMEL).","evidence":[{"type":"repo","title":"CAMEL README — Cloudflare MCP","url":"https://github.com/camel-ai/camel","quote":"Intelligent agents manage Cloudflare resources dynamically","accessed":"2026-05-24"},{"type":"repo","title":"CAMEL README — real-world usecases","url":"https://github.com/camel-ai/camel","quote":"Real-world usecases demonstrating how CAMEL's multi-agent framework enables real business value","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"agentic-rag","role":"first-class","note":"Dedicated RAG pipelines documented; combine chunking, retrieval, and generation.","evidence":[{"type":"doc","title":"CAMEL docs — RAG pipelines","url":"https://docs.camel-ai.org","quote":"Orchestrate Retrieval‐Augmented Generation (RAG) pipelines","accessed":"2026-05-24"},{"type":"doc","title":"CAMEL docs — RAG chunking","url":"https://docs.camel-ai.org","quote":"Combine chunking, retrieval, and generation for grounded, accurate responses","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"code-execution","role":"first-class","note":"Execution backends include Python, shell, and browser interpreters for live code evaluation.","evidence":[{"type":"doc","title":"CAMEL docs — Python interpreter","url":"https://docs.camel-ai.org/key_modules/interpreters","quote":"Fast, local, and safe—executes trusted Python code directly within the CAMEL agent process.","accessed":"2026-05-24"},{"type":"doc","title":"CAMEL docs — Shell interpreter","url":"https://docs.camel-ai.org/key_modules/interpreters","quote":"Execute shell commands or scripts in a separate process for isolation and flexibility.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"cross-session-memory","role":"first-class","note":"Memory and storage layers persist chat history and tool outputs.","evidence":[{"type":"doc","title":"CAMEL docs — Memory for long-horizon context","url":"https://docs.camel-ai.org","quote":"Manage Memory for long‐horizon context and learning","accessed":"2026-05-24"},{"type":"doc","title":"CAMEL docs — persistent context layers","url":"https://docs.camel-ai.org","quote":"Persistent context layers for chat history, tool outputs, and learned knowledge","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"browser-agent","role":"limited","note":"Browser interpreter is one of the execution backends; not framed as a dedicated 'browser agent' class.","evidence":[{"type":"doc","title":"CAMEL docs — browser backend","url":"https://docs.camel-ai.org","quote":"Execution backends (Python, shell, browsers) for live code evaluation and automation","accessed":"2026-05-24"},{"type":"doc","title":"CAMEL docs — browser tools","url":"https://docs.camel-ai.org","quote":"tools like browsers, code interpreters, and multimodal models","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"computer-use","role":"limited","note":"The Hybrid Browser Toolkit provides GUI-level browser control; can be exposed via MCP to give AI assistants direct browser control.","evidence":[{"type":"doc","title":"CAMEL-AI — Hybrid Browser Toolkit blog","url":"https://www.camel-ai.org/blogs/camel-browser-toolkit-blog","quote":"The toolkit can be accessed through MCP (Model Context Protocol), allowing AI assistants like Claude to control browsers directly.","accessed":"2026-05-24"},{"type":"doc","title":"CAMEL-AI — Hybrid Browser Toolkit blog","url":"https://www.camel-ai.org/blogs/camel-browser-toolkit-blog","quote":"Instead of treating the browser as a black box that we interact with through screenshots, the new HybridBrowserToolkit speaks the browser's native language through TypeScript.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"code-as-action","role":"limited","note":"CodeExecutionToolkit supports executing model-emitted code across sandboxes; the framework treats execute_code as a first-class action. Does not adopt the CodeAct label.","evidence":[{"type":"doc","title":"CAMEL-AI — CodeExecutionToolkit","url":"https://docs.camel-ai.org/reference/camel.toolkits.code_execution","quote":"A toolkit for code execution.","accessed":"2026-05-24"},{"type":"doc","title":"CAMEL-AI — CodeExecutionToolkit","url":"https://docs.camel-ai.org/reference/camel.toolkits.code_execution","quote":"The environment type used to execute code. (default: `subprocess`)","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"agent-as-judge","role":"first-class","note":"Workforce examples include a 'Hackathon Judge Committee' — a judging society over candidate solutions.","evidence":[{"type":"repo","title":"CAMEL README — judge committee","url":"https://github.com/camel-ai/camel","quote":"Create A Hackathon Judge Committee with Workforce","accessed":"2026-05-24"},{"type":"repo","title":"CAMEL README — collaborative judging","url":"https://github.com/camel-ai/camel","quote":"Building a team of agents for collaborative judging.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"world-model-separation","role":"limited","note":"OASIS-style large-scale simulation distinguishes simulated environment from agent reasoning, but the framework does not document an explicit world-model API.","evidence":[{"type":"doc","title":"CAMEL docs — OASIS large-scale simulation","url":"https://docs.camel-ai.org","quote":"Large‐scale social simulation environment: model Reddit, Twitter, and user interactions","accessed":"2026-05-24"},{"type":"doc","title":"CAMEL docs — multi-agent social simulations","url":"https://docs.camel-ai.org","quote":"Platforms like Oasis for large‐scale multi‐agent social simulations","accessed":"2026-05-24"}],"evidence_status":"full"}],"instantiates":["multi-agent-coordination"],"alternatives":[{"composition":"agentverse","relation":"competes-with","note":"CAMEL also offers role-playing simulation; AgentVerse pairs that with task-solving mode."},{"composition":"chatdev","relation":"wrapped-by","note":"ChatDev is built on CAMEL role-playing dialogues as its substrate."},{"composition":"crewai","relation":"similar-shape","note":"Both root themselves in role-playing autonomous agents; CAMEL emphasises the two-agent task-specifier/AI-assistant dialectic vs CrewAI's n-agent crew."}],"references":[{"type":"repo","title":"camel-ai/camel","url":"https://github.com/camel-ai/camel","quote":"🐫 CAMEL: The first and the best multi-agent framework. Finding the Scaling Law of Agents.","accessed":"2026-05-20"},{"type":"doc","title":"CAMEL docs","url":"https://docs.camel-ai.org","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","python","role-playing","multi-agent","scaling-laws","workforce","mcp"]},{"id":"chatdev","name":"ChatDev","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"OpenBMB / Tsinghua KEG","language":"Python","license":"Apache-2.0","status":"active","url":"https://chatdev.toscl.com/","repo":"https://github.com/OpenBMB/ChatDev","intent":"Communicative multi-agent framework for software development that runs a waterfall SDLC (design → coding → testing → documentation) through role-specialised LLM agents (CEO, CPO, CTO, Programmer, Reviewer, Tester) chained via natural-language conversations.","description":"ChatDev (2.0 now branded DevAll) is OpenBMB's multi-agent framework that simulates an entire virtual software company. Each phase of the waterfall — high-level design, requirements analysis, coding, testing, documentation — is delegated to a pair of agents in conversation (e.g. CTO ↔ Programmer for coding). The framework is built on CAMEL-style role-playing dialogues and produces a fully-runnable software project as output, complete with code, tests, and docs. ChatDev is one of the most-cited Chinese-origin multi-agent frameworks and the canonical reference for role-specialised SDLC pipelines.","primary_use_cases":["research on multi-agent software development with role specialisation","rapid generation of small-to-medium software projects from a single prompt","study of communicative-dehallucination protocols between paired LLM agents","educational demonstration of SOP-driven multi-agent collaboration"],"agent_loop_shape":"Waterfall pipeline. A configuration file enumerates phases (DemandAnalysis, LanguageChoose, Coding, CodeReviewComment, CodeReviewModification, Test, EnvironmentDoc, Manual) and assigns each phase a pair of role-playing agents. Each phase runs as a multi-turn role-playing dialogue between the two assigned agents (e.g. Tester ↔ Programmer) until a phase-specific termination criterion is met. Outputs (code, docs) accumulate in a shared workspace passed across phases.","key_concepts":[{"name":"Waterfall SDLC phases","summary":"Design → Coding → Testing → Documentation as a hard-coded pipeline.","maps_to_pattern":"plan-and-execute"},{"name":"Role-playing pairs","summary":"Each phase is a dialogue between two role-specialised agents.","maps_to_pattern":"camel-role-playing"},{"name":"Communicative dehallucination","summary":"The reviewer-programmer dialogue surfaces hallucinations through mutual cross-check.","maps_to_pattern":"communicative-dehallucination"}],"pattern_composition":"flowchart TD\n  REQ[One-line idea] --> CONFIG[Phase config<br/>ChatChainConfig.json]\n  CONFIG --> PHASE1[Demand Analysis<br/>CEO ↔ CPO]\n  PHASE1 --> PHASE2[Language Choose<br/>CEO ↔ CTO]\n  PHASE2 --> PHASE3[Coding<br/>CTO ↔ Programmer]\n  PHASE3 --> PHASE4[Code Review<br/>Reviewer ↔ Programmer]\n  PHASE4 --> PHASE5[Test<br/>Tester ↔ Programmer]\n  PHASE5 --> PHASE6[Documentation<br/>CCO ↔ CTO]\n  PHASE6 --> OUT[(Software project:<br/>code + tests + docs)]\n  PHASE3 -.communicative dehallucination.-> PHASE4\n  PHASE4 -.role-playing dialogue.-> PHASE5","members":[{"pattern":"camel-role-playing","role":"first-class","note":"ChatDev is built on CAMEL role-playing dialogues.","evidence":[],"evidence_status":"none"},{"pattern":"role-assignment","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"communicative-dehallucination","role":"first-class","note":"Reviewer ↔ Programmer cycle.","evidence":[],"evidence_status":"none"},{"pattern":"supervisor","role":"supported","note":"CEO role coordinates phase progress.","evidence":[],"evidence_status":"none"},{"pattern":"orchestrator-workers","role":"supported","note":"Phase-level orchestration over role agents.","evidence":[],"evidence_status":"none"},{"pattern":"plan-and-execute","role":"first-class","note":"Waterfall phases are a fixed plan.","evidence":[],"evidence_status":"none"},{"pattern":"inter-agent-communication","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"tool-use","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"code-as-action","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"chat-chain","role":"core","note":"Decomposes development into ordered phases, each phase split into atomic two-agent (instructor/assistant) role-playing chats whose artefact feeds the next phase.","evidence":[{"type":"paper","title":"ChatDev: Communicative Agents for Software Development","url":"https://ar5iv.labs.arxiv.org/html/2307.07924","quote":"The chat chain acts as a facilitator, breaking down each stage into atomic subtasks. [...] Each phase is further divided into atomic chats. These atomic chats involve task-oriented role-playing between two agents.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"alternatives":[{"composition":"metagpt","relation":"competes-with","note":"Both are Chinese multi-agent SDLC frameworks with role-specialised agents; MetaGPT emphasises SOP encoding."},{"composition":"openmanus","relation":"similar-shape","note":"OpenMANUS is from the same FoundationAgents lineage as MetaGPT."},{"composition":"autogen","relation":"competes-with","note":"AutoGen GroupChat is the Western analogue for role-based multi-agent collaboration."},{"composition":"crewai","relation":"competes-with","note":"CrewAI is the production-oriented multi-agent peer."},{"composition":"camel-ai","relation":"wraps","note":"ChatDev is built on CAMEL role-playing dialogues as its substrate."},{"composition":"codefuse","relation":"similar-shape","note":"Both target multi-agent SDLC; ChatDev is academic, CodeFuse is industrial (Ant Group)."}],"last_analyzed":"2026-05-22","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-05-22","status_in_practice":"emerging","tags":["orchestration-framework","multi-agent","planning-control-flow","tool-use-environment","full-code"],"references":[{"type":"doc","title":"ChatDev — Homepage","url":"https://chatdev.toscl.com/"},{"type":"repo","title":"ChatDev — Source repository","url":"https://github.com/OpenBMB/ChatDev"}]},{"id":"crewai","name":"CrewAI","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"CrewAI Inc.","language":"Python","license":"MIT","status":"active","first_released":"2023-10-27","url":"https://crewai.com","repo":"https://github.com/crewAIInc/crewAI","docs_url":"https://docs.crewai.com/en/introduction","intent":"Orchestrate teams of role-playing autonomous agents that collaborate on multi-step tasks under a declared Process (sequential or hierarchical), optionally driven by event-driven Flows.","description":"CrewAI is the MIT-licensed Python framework from CrewAI Inc. that models a multi-agent system as a Crew: a collaborative group of role-defined Agent objects assigned ordered Task objects and executed under a declared Process (sequential by default, or hierarchical with a manager agent). On top of crews, CrewAI exposes Flows, an event-driven orchestration layer built around @start, @listen, and @router decorators that manages typed state and routes execution based on emitted events. The framework ships a unified memory store (LanceDB by default at ./.crewai/memory), automatic delegation/ask-question collaboration tools, MCP integration via the mcps field and MCPServerAdapter, and structured outputs via Pydantic.","primary_use_cases":["teams of role-playing agents collaborating on a project","event-driven multi-step workflows that mix LLM calls and code","hierarchical workflows with a manager agent delegating to specialists"],"agent_loop_shape":"Crew/Process loop: a Crew binds Agents to ordered Tasks and runs them under Process.sequential (linear chaining, each task feeds the next) or Process.hierarchical (a manager_llm delegates to crew members and validates outcomes). Flows wrap Crews in an event-driven graph where @start() methods are entry points, @listen(other_method) fires when its upstream emits, and @router() emits labels that other listeners gate on. Per-agent budgets (max_iter default 25, max_rpm, max_execution_time) bound the inner ReAct/tool loop.","key_concepts":[{"name":"Crew","summary":"Collaborative group of agents working together to achieve a set of tasks.","url":"https://docs.crewai.com/en/concepts/crews"},{"name":"Agent","summary":"Autonomous unit defined by role / goal / backstory that performs tasks, makes decisions, and uses tools.","url":"https://docs.crewai.com/en/concepts/agents"},{"name":"Task","summary":"A specific assignment completed by an Agent.","url":"https://docs.crewai.com/en/concepts/tasks"},{"name":"Process","summary":"How Tasks are orchestrated inside a Crew. Process.sequential = linear order; Process.hierarchical = manager_llm delegates and validates.","url":"https://docs.crewai.com/en/concepts/processes"},{"name":"Flow","summary":"Event-driven workflow layer built on @start / @listen / @router decorators with managed typed state.","maps_to_pattern":"event-driven-agent","url":"https://docs.crewai.com/en/concepts/flows"},{"name":"Unified Memory","summary":"Single Memory class replacing separate short-term, long-term, entity, and external memory types; default LanceDB store at ./.crewai/memory.","maps_to_pattern":"cross-session-memory","url":"https://docs.crewai.com/en/concepts/memory"}],"pattern_composition":"flowchart TD\n  USER[User goal] --> CREW[Crew<br/>binds Agents + Tasks]\n  CREW -->|Process.sequential| SEQ[Linear chain<br/>task1 -> task2 -> ...]\n  CREW -->|Process.hierarchical| MGR[Manager Agent<br/>manager_llm]\n  MGR -->|delegate| WORKER1[Worker Agent A<br/>role/goal/backstory]\n  MGR -->|delegate| WORKER2[Worker Agent B]\n  MGR -->|validate outcomes| END[Final output]\n  SEQ --> END\n  WORKER1 -->|tool calls| TOOLS[(Tools + MCP servers)]\n  WORKER2 -->|tool calls| TOOLS\n  WORKER1 -.->|delegate via allow_delegation| WORKER2\n  CREW --> MEM[(Unified Memory<br/>LanceDB at ./.crewai/memory)]\n  CREW --> RAG[(Knowledge Base<br/>ChromaDB / Qdrant)]\n  USER -.->|Flow alternative| FLOW[Flow<br/>@start / @listen / @router]\n  FLOW -->|invoke| CREW","members":[{"pattern":"agent-resumption","role":"limited","note":"replay_from_task can replay from the most recent kickoff; no general checkpoint/resume durability comparable to LangGraph's Checkpointer.","evidence":[{"type":"doc","title":"Replay Tasks from Latest Crew Kickoff","url":"https://docs.crewai.com/en/learn/replay-tasks-from-latest-crew-kickoff","quote":"CrewAI provides the ability to replay from a task specified from the latest crew kickoff.","accessed":"2026-05-20"},{"type":"doc","title":"Replay Tasks — limitation","url":"https://docs.crewai.com/en/learn/replay-tasks-from-latest-crew-kickoff","quote":"Currently, only the latest kickoff is supported, so if you use kickoff_for_each, it will only allow you to replay from the most recent crew run.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"agentic-rag","role":"limited","note":"Knowledge sources expose a provider-neutral RAG client (default ChromaDB, Qdrant supported) with automatic query rewriting, but RAG is not the framework's headline shape.","evidence":[{"type":"doc","title":"Knowledge","url":"https://docs.crewai.com/en/concepts/knowledge","quote":"Knowledge in CrewAI is a powerful system that allows AI agents to access and utilize external information sources during their tasks.","accessed":"2026-05-20"},{"type":"doc","title":"Knowledge — RAG client","url":"https://docs.crewai.com/en/concepts/knowledge","quote":"CrewAI exposes a provider-neutral RAG client abstraction for vector stores. The default provider is ChromaDB, and Qdrant is supported as well.","accessed":"2026-05-20"},{"type":"doc","title":"Knowledge — query rewriting","url":"https://docs.crewai.com/en/concepts/knowledge","quote":"When an agent executes a task with knowledge sources available, the raw task prompt is automatically transformed into a more effective search query.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"camel-role-playing","role":"limited","note":"CrewAI promotes role/goal/backstory triples on every agent (verbatim 'role-playing' language in the repo description), but does not implement CAMEL's two-agent task-specifier/AI-assistant dialectic.","evidence":[{"type":"repo","title":"crewAIInc/crewAI","url":"https://github.com/crewAIInc/crewAI","quote":"Framework for orchestrating role-playing, autonomous AI agents.","accessed":"2026-05-20"},{"type":"doc","title":"Agents — role attribute","url":"https://docs.crewai.com/en/concepts/agents","quote":"Defines the agent's function and expertise within the crew.","accessed":"2026-05-20"},{"type":"doc","title":"Agents — backstory attribute","url":"https://docs.crewai.com/en/concepts/agents","quote":"Provides context and personality to the agent, enriching interactions.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"cross-session-memory","role":"first-class","note":"Unified Memory class with LLM-driven save-time analysis and adaptive-depth recall; default LanceDB store at ./.crewai/memory (or $CREWAI_STORAGE_DIR/memory).","evidence":[{"type":"doc","title":"Memory — unified system","url":"https://docs.crewai.com/en/concepts/memory","quote":"CrewAI provides a unified memory system -- a single Memory class that replaces separate short-term, long-term, entity, and external memory types","accessed":"2026-05-20"},{"type":"doc","title":"Memory — default store","url":"https://docs.crewai.com/en/concepts/memory","quote":"Default: LanceDB, stored under ./.crewai/memory (or $CREWAI_STORAGE_DIR/memory if the env var is set)","accessed":"2026-05-20"},{"type":"doc","title":"Memory — analysis & recall","url":"https://docs.crewai.com/en/concepts/memory","quote":"Memory uses an LLM to analyze content when saving (inferring scope, categories, and importance) and supports adaptive-depth recall","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"event-driven-agent","role":"first-class","note":"Flows wrap Crews in an event-driven graph; @start() entry points, @listen() listeners triggered by upstream emits, @router() labelled branches.","evidence":[{"type":"doc","title":"Flows — overview","url":"https://docs.crewai.com/en/concepts/flows","quote":"Flows allow you to create structured, event-driven workflows...providing a seamless way to connect multiple tasks, manage state, and control the flow of execution.","accessed":"2026-05-20"},{"type":"doc","title":"Flows — event model","url":"https://docs.crewai.com/en/concepts/flows","quote":"Built on an event-driven model, allowing for dynamic and responsive workflows","accessed":"2026-05-20"},{"type":"doc","title":"Flows — @listen decorator","url":"https://docs.crewai.com/en/concepts/flows","quote":"The @listen() decorator is used to mark a method as a listener for the output of another task in the Flow...executed when the specified task emits an output.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"handoff","role":"first-class","note":"When allow_delegation=True, agents automatically gain delegation and ask-question collaboration tools; the 'Delegate work to coworker' tool is the canonical handoff mechanism.","evidence":[{"type":"doc","title":"Collaboration","url":"https://docs.crewai.com/en/concepts/collaboration","quote":"Collaboration in CrewAI enables agents to work together as a team by delegating tasks and asking questions to leverage each other's expertise.","accessed":"2026-05-20"},{"type":"doc","title":"Collaboration — allow_delegation","url":"https://docs.crewai.com/en/concepts/collaboration","quote":"When `allow_delegation=True`, agents automatically gain access to powerful collaboration tools.","accessed":"2026-05-20"},{"type":"doc","title":"Collaboration — delegation tool","url":"https://docs.crewai.com/en/concepts/collaboration","quote":"Delegate work to coworker(task: str, context: str, coworker: str)","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"First-class MCP support: agents declare an `mcps` field for direct integration, or use MCPServerAdapter from crewai-tools for manual connection management.","evidence":[{"type":"doc","title":"MCP Overview","url":"https://docs.crewai.com/en/mcp/overview","quote":"The Model Context Protocol (MCP) provides a standardized way for AI agents to provide context to LLMs by communicating with external services, known as MCP Servers.","accessed":"2026-05-20"},{"type":"doc","title":"MCP — mcps field","url":"https://docs.crewai.com/en/mcp/overview","quote":"Use the `mcps` field directly on agents for seamless MCP tool integration.","accessed":"2026-05-20"},{"type":"doc","title":"MCP — MCPServerAdapter","url":"https://docs.crewai.com/en/mcp/overview","quote":"For complex scenarios requiring manual connection management, use the `MCPServerAdapter` class from `crewai-tools`.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"orchestrator-workers","role":"first-class","note":"Process.hierarchical pairs a manager_llm (or manager_agent) with crew members; the manager allocates tasks based on role and capability and validates outcomes.","evidence":[{"type":"doc","title":"Hierarchical Process — manager","url":"https://docs.crewai.com/en/learn/hierarchical-process","quote":"A manager agent coordinates the workflow, delegates tasks, and validates outcomes for streamlined and effective execution.","accessed":"2026-05-20"},{"type":"doc","title":"Hierarchical Process — allocation","url":"https://docs.crewai.com/en/learn/hierarchical-process","quote":"A manager agent allocates tasks among crew members based on their roles and capabilities.","accessed":"2026-05-20"},{"type":"doc","title":"Processes — manager_llm requirement","url":"https://docs.crewai.com/en/concepts/processes","quote":"A manager language model (`manager_llm`) or a custom manager agent (`manager_agent`) must be specified in the crew to enable the hierarchical process.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"role-assignment","role":"first-class","note":"Every Agent is defined by a role/goal/backstory triple; these fields directly drive the agent's behaviour and the framework markets itself as a 'role-playing' agent framework.","evidence":[{"type":"repo","title":"crewAIInc/crewAI repo description","url":"https://github.com/crewAIInc/crewAI","quote":"Framework for orchestrating role-playing, autonomous AI agents.","accessed":"2026-05-20"},{"type":"doc","title":"Agents — role","url":"https://docs.crewai.com/en/concepts/agents","quote":"Defines the agent's function and expertise within the crew.","accessed":"2026-05-20"},{"type":"doc","title":"Agents — goal","url":"https://docs.crewai.com/en/concepts/agents","quote":"The individual objective that guides the agent's decision-making.","accessed":"2026-05-20"},{"type":"doc","title":"Agents — backstory","url":"https://docs.crewai.com/en/concepts/agents","quote":"Provides context and personality to the agent, enriching interactions.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"step-budget","role":"first-class","note":"Per-agent budgets bound the inner ReAct/tool loop: max_iter (default 25), max_rpm, max_execution_time.","evidence":[{"type":"doc","title":"Customizing Agents — max_iter","url":"https://docs.crewai.com/en/learn/customizing-agents","quote":"Limits the maximum number of iterations (`max_iter`) for a task to prevent infinite loops, with a default of 25.","accessed":"2026-05-20"},{"type":"doc","title":"Customizing Agents — max_rpm","url":"https://docs.crewai.com/en/learn/customizing-agents","quote":"Sets the maximum requests per minute (`max_rpm`). Can be set to `None` for unlimited requests to external services.","accessed":"2026-05-20"},{"type":"doc","title":"Customizing Agents — max_execution_time","url":"https://docs.crewai.com/en/learn/customizing-agents","quote":"Sets the maximum time allowed for an agent to complete a task.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"structured-output","role":"first-class","note":"Tasks accept output_pydantic / output_json for structured outputs; the framework markets structured outputs via Pydantic.","evidence":[{"type":"doc","title":"Agents — structured outputs","url":"https://docs.crewai.com/en/concepts/agents","quote":"Compose agents with tools, memory, knowledge, and structured outputs using Pydantic.","accessed":"2026-05-20"},{"type":"doc","title":"CrewAI Tasks — output_pydantic","url":"https://docs.crewai.com/en/concepts/tasks","quote":"The `output_pydantic` property allows you to define a Pydantic model that the task output should conform to.","accessed":"2026-05-24"},{"type":"doc","title":"CrewAI Tasks — output_json","url":"https://docs.crewai.com/en/concepts/tasks","quote":"The `output_json` property allows you to define the expected output in JSON format.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"supervisor","role":"first-class","note":"Hierarchical Process with manager_llm IS the supervisor pattern in CrewAI: a central manager coordinates and validates worker agents.","evidence":[{"type":"doc","title":"Hierarchical Process — manager coordinates","url":"https://docs.crewai.com/en/learn/hierarchical-process","quote":"A manager agent coordinates the workflow, delegates tasks, and validates outcomes for streamlined and effective execution.","accessed":"2026-05-20"},{"type":"doc","title":"Hierarchical Process — validation","url":"https://docs.crewai.com/en/learn/hierarchical-process","quote":"The manager evaluates outcomes to ensure they meet the required standards.","accessed":"2026-05-20"},{"type":"doc","title":"Hierarchical Process — manager_llm","url":"https://docs.crewai.com/en/learn/hierarchical-process","quote":"Configuring the `manager_llm` parameter is crucial for the hierarchical process.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Tools are first-class: Agents declare callable Tools; async tools supported for non-blocking I/O.","evidence":[{"type":"doc","title":"Tools — definition","url":"https://docs.crewai.com/en/concepts/tools","quote":"A tool in CrewAI is a skill or function that agents can utilize to perform various actions.","accessed":"2026-05-20"},{"type":"doc","title":"Tools — callable functions","url":"https://docs.crewai.com/en/concepts/tools","quote":"Tools give agents callable functions to take action.","accessed":"2026-05-20"},{"type":"doc","title":"Tools — async tools","url":"https://docs.crewai.com/en/concepts/tools","quote":"CrewAI supports asynchronous tools, allowing you to implement tools that perform non-blocking operations like network requests, file I/O.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"workflow-success-business-invalid","role":"supported","note":"Task guardrails (incl. an LLM-based hallucination guardrail) score the deliverable against business criteria before the task is treated as done, splitting execution-success from business-validity exactly as the corrective requires.","evidence":[{"type":"doc","title":"Hallucination Guardrail - CrewAI","url":"https://docs.crewai.com/en/enterprise/features/hallucination-guardrail","quote":"When a guardrail is added to a task, it automatically validates the output before the task is marked as complete","accessed":"2026-06-14"}],"evidence_status":"full"},{"pattern":"one-tool-one-agent","role":"core","note":"Advocates a team of narrowly-scoped specialist agents over one general/super-agent, each focused on a single domain role.","evidence":[{"type":"doc","title":"Crafting Effective Agents - CrewAI","url":"https://docs.crewai.com/en/guides/agents/crafting-effective-agents","quote":"Agents perform significantly better when given specialized roles rather than general ones. [...] A highly focused agent delivers more precise, relevant outputs","accessed":"2026-06-17"},{"type":"doc","title":"Multi-agent - Docs by LangChain","url":"https://docs.langchain.com/oss/python/langchain/multi-agent","quote":"Multi-agent patterns are particularly valuable when a single agent has too many tools and makes poor decisions about which to use","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"goal-decomposition","role":"first-class","note":"A manager agent reads the goal, breaks it into subtasks, allocates each to crew members by role/capability, and validates the outcomes.","evidence":[{"type":"doc","title":"Hierarchical Process - CrewAI","url":"https://docs.crewai.com/en/learn/hierarchical-process","quote":"A manager agent allocates tasks among crew members based on their roles and capabilities. ... A 'manager' agent coordinates the workflow, delegates tasks, and validates outcomes for streamlined and effective execution.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"instantiates":["multi-agent-coordination"],"alternatives":[{"composition":"autogen","relation":"competes-with","note":"Microsoft's multi-agent conversational framework targets the same 'team of role-played agents collaborating on tasks' niche."},{"composition":"langgraph","relation":"similar-shape","note":"Both express orchestration as a declared topology over typed state (LangGraph: nodes + StateGraph; CrewAI: Crews under a Process or Flows with @listen/@router)."},{"composition":"metagpt","relation":"similar-shape","note":"MetaGPT's role-played software-company crew (PM, Architect, Engineer) is the closest sibling to CrewAI's Crew+Agent+Process model."},{"composition":"camel-ai","relation":"similar-shape","note":"Both root themselves in role-playing autonomous agents; CAMEL emphasises the two-agent task-specifier/AI-assistant dialectic vs CrewAI's n-agent crew."},{"composition":"llamaindex","relation":"competes-with","note":"LlamaIndex's AgentWorkflow targets similar multi-agent orchestration use cases."},{"composition":"autogpt","relation":"competes-with","note":"Earlier-generation autonomous-agent framework competing on the same 'autonomous AI workforce' positioning."},{"composition":"agentverse","relation":"competes-with","note":"CrewAI is the more product-oriented peer."},{"composition":"agno","relation":"competes-with","note":"Role-defined multi-agent teams with broadly similar surface area."},{"composition":"chatdev","relation":"competes-with","note":"CrewAI is the production-oriented multi-agent peer."},{"composition":"mastra","relation":"similar-shape","note":"Multi-agent + role-played agents; CrewAI is Python-only and crew-centric."},{"composition":"pydantic-ai","relation":"similar-shape","note":"Both Python + Pydantic; CrewAI is multi-agent crew-first vs Pydantic AI's single-agent-first."},{"composition":"qwen-agent","relation":"similar-shape","note":"Multi-agent role-based orchestration alternative; less RAG-centric."},{"composition":"smolagents","relation":"competes-with","note":"Role-based multi-agent crews with JSON tool shape."},{"composition":"deerflow-v1","relation":"similar-shape","note":"CrewAI composes role-based agents over tasks for similar research-automation goals; DeerFlow differs by foregrounding an explicit, human-reviewable plan and a bounded re-planning loop."}],"references":[{"type":"repo","title":"crewAIInc/crewAI","url":"https://github.com/crewAIInc/crewAI","quote":"Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.","accessed":"2026-05-20"},{"type":"doc","title":"CrewAI Introduction","url":"https://docs.crewai.com/en/introduction","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","multi-agent","role-playing","event-driven"]},{"id":"dbgpt","name":"DB-GPT","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"Eosphoros AI","language":"Python","license":"MIT","status":"active","url":"https://docs.dbgpt.cn/","repo":"https://github.com/eosphoros-ai/DB-GPT","intent":"Open-source AI-native data platform built around the AWEL agent workflow language, multi-model support, and natural-language access to databases (Text2SQL, vector + relational + graph stores) for end-to-end data agents.","description":"DB-GPT (17k+ stars, MIT) is Eosphoros AI's open-source data agent platform. It positions itself as the AI-native infrastructure for data products: a unified agent runtime over multiple LLMs (open and proprietary), the AWEL (Agentic Workflow Expression Language) for composing agent pipelines, native multi-store support (vector, relational, graph), Text2SQL with rigorous schema-aware prompting, GraphRAG with both triplet-and-structure indexing (50% token savings vs. Microsoft GraphRAG per their published numbers), and the dbgpts package ecosystem for sharable agents and data apps.","primary_use_cases":["natural-language data analytics over relational databases (Text2SQL)","agentic data workflows composed in AWEL","AI-native data products combining vector, relational, and graph storage","domain-specific Text2SQL fine-tuning via DB-GPT-Hub"],"agent_loop_shape":"AWEL defines the orchestration: each agent is a node, edges declare flow, and the runtime executes the DAG with explicit operator semantics. Each agent runs a tool-augmented loop with access to SQL execution, vector retrieval, graph queries, and Python code. Multi-model serving means routing per-task to the best-fit LLM.","key_concepts":[{"name":"AWEL (Agentic Workflow Expression Language)","summary":"DAG-based agent orchestration language.","maps_to_pattern":"orchestrator-workers"},{"name":"GraphRAG with triplet + structure","summary":"Combined triplet KG and document-structure graph; 50% token savings vs. Microsoft GraphRAG.","maps_to_pattern":"graphrag"},{"name":"DB-GPT-Hub","summary":"Text2SQL model fine-tuning toolkit."},{"name":"dbgpts","summary":"Package ecosystem for sharable data agents and apps.","maps_to_pattern":"agent-skills"}],"pattern_composition":"flowchart TD\n  USER[User question] --> AWEL[AWEL workflow runtime]\n  AWEL --> NODE1[Agent node 1]\n  AWEL --> NODE2[Agent node 2]\n  NODE1 --> ROUTE{Tool}\n  ROUTE -->|SQL| SQL[(Relational store<br/>Text2SQL)]\n  ROUTE -->|vector| VEC[(Vector store)]\n  ROUTE -->|graph| GRAPH[(GraphRAG store<br/>triplet + structure)]\n  ROUTE -->|code| PY[Python executor]\n  SQL --> NODE2\n  VEC --> NODE2\n  GRAPH --> NODE2\n  PY --> NODE2\n  NODE2 --> MODELS{Multi-model serving}\n  MODELS --> OUT[Answer]\n  DBHUB[(DB-GPT-Hub<br/>Text2SQL fine-tuning)] -.train.-> MODELS","members":[{"pattern":"agentic-rag","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"graphrag","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"naive-rag","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"hybrid-search","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"knowledge-graph-memory","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"mcp","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"orchestrator-workers","role":"first-class","note":"AWEL workflow orchestration.","evidence":[],"evidence_status":"none"},{"pattern":"supervisor","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"tool-use","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"code-as-action","role":"first-class","note":"SQL and Python.","evidence":[],"evidence_status":"none"},{"pattern":"structured-output","role":"first-class","evidence":[],"evidence_status":"none"}],"instantiates":["production-rag"],"alternatives":[{"composition":"ragflow","relation":"similar-shape","note":"Sibling Chinese open-source data-agent platform; DB-GPT specialises on relational/SQL, RAGFlow on document RAG."},{"composition":"langchain","relation":"competes-with"},{"composition":"llamaindex","relation":"competes-with"},{"composition":"kag","relation":"complements","note":"DB-GPT also ships GraphRAG; KAG's logical-form reasoning could feed into DB-GPT's AWEL workflows."}],"last_analyzed":"2026-05-22","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-05-22","status_in_practice":"mature","tags":["orchestration-framework","retrieval","memory","tool-use-environment","multi-agent","structure-data","full-code"],"references":[{"type":"doc","title":"DB-GPT — Homepage","url":"https://docs.dbgpt.cn/"},{"type":"repo","title":"DB-GPT — Source repository","url":"https://github.com/eosphoros-ai/DB-GPT"}]},{"id":"dspy","name":"DSPy","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"Stanford NLP","language":"Python","license":"MIT","status":"active","first_released":"2023-01-09","url":"https://dspy.ai","repo":"https://github.com/stanfordnlp/dspy","docs_url":"https://dspy.ai","intent":"Replace hand-tuned prompts with a declarative Python programming model in which you specify input/output behaviour as Signatures, compose Modules (Predict, ChainOfThought, ReAct, ProgramOfThought), and let Optimizers (BootstrapFewShot, MIPROv2, BootstrapFinetune) tune prompts and weights against a metric.","description":"DSPy is the MIT-licensed framework for 'programming—rather than prompting—language models'. DSPy stands for Declarative Self-improving Python. It exposes three layers: Signatures declare what an LM should do (e.g. 'question -> answer'); Modules implement reasoning shapes around those signatures (dspy.Predict, dspy.ChainOfThought, dspy.ReAct, dspy.ProgramOfThought); Optimizers consume a metric and training data to 'tune the prompts and weights of your AI modules' — synthesising few-shot demos, proposing better natural-language instructions, or fine-tuning small LMs. Most often used for classifiers, RAG pipelines, and agent loops where empirically tuned prompts beat hand-crafted strings.","primary_use_cases":["declarative programming of LM behaviour via Signatures","compiler-driven prompt + weight optimisation against a metric","ReAct-style agents whose prompts are co-tuned with the loop","RAG pipelines whose intermediate prompts get optimised"],"agent_loop_shape":"DSPy compiles to whatever loop the module declares. A dspy.ReAct agent runs the classic thought→action→observation cycle against a signature plus a tools list; dspy.ProgramOfThought emits and executes code; dspy.ChainOfThought adds a reasoning step before the signature output. At authoring time you only write the signature and pick the module. At compile time, an Optimizer (e.g. MIPROv2) iterates with a teacher LM, generates instructions and few-shot examples per step, and uses Bayesian Optimization to search the space of instructions/demonstrations.","key_concepts":[{"name":"Signature","summary":"Declarative I/O spec ('question -> answer').","url":"https://dspy.ai/learn/programming/signatures/"},{"name":"Module","summary":"Predict / ChainOfThought / ReAct / ProgramOfThought wrappers around a signature.","maps_to_pattern":"react","url":"https://dspy.ai/learn/programming/modules/"},{"name":"Optimizer","summary":"BootstrapFewShot / MIPROv2 / BootstrapFinetune tune prompts and weights against a metric.","maps_to_pattern":"evaluator-optimizer","url":"https://dspy.ai/learn/optimization/optimizers/"},{"name":"Compiler / teacher LM","summary":"Optimizer uses a teacher LM (often the program itself) to bootstrap demos."},{"name":"dspy.ReAct","summary":"Tool-using agent module over a signature and a tools list.","url":"https://dspy.ai/learn/programming/modules/"}],"pattern_composition":"flowchart TD\n  USER[Task definition] --> SIG[Signature<br/>question -> answer]\n  SIG --> MOD{Module choice}\n  MOD -->|reasoning| COT[dspy.ChainOfThought]\n  MOD -->|tools| REACT[dspy.ReAct + tools]\n  MOD -->|code| POT[dspy.ProgramOfThought]\n  MOD -->|basic| PRED[dspy.Predict]\n  COT --> LM[(LM call)]\n  REACT -->|action| TOOLS[Tool functions]\n  TOOLS -->|observation| REACT\n  REACT --> LM\n  POT --> EXEC[Code executor]\n  EXEC --> LM\n  PRED --> LM\n  LM --> OUT[Typed output]\n  TRAIN[(Trainset + metric)] --> OPT[Optimizer<br/>BootstrapFewShot / MIPROv2 / BootstrapFinetune]\n  OPT -.compiles.-> COT\n  OPT -.compiles.-> REACT\n  OPT -.compiles.-> PRED\n  OPT -->|teacher LM| TEACHER[Teacher module<br/>generates demos]","members":[{"pattern":"automatic-workflow-search","role":"first-class","note":"Optimizers use Bayesian Optimization to search instruction + demonstration space.","evidence":[{"type":"doc","title":"DSPy — Optimizers Bayesian search","url":"https://dspy.ai/learn/optimization/optimizers/","quote":"Uses Bayesian Optimization to effectively search over the space of generation instructions/demonstrations across your modules.","accessed":"2026-05-20"},{"type":"doc","title":"DSPy — MIPROv2","url":"https://dspy.ai/learn/optimization/optimizers/","quote":"MIPROv2: Generates instructions _and_ few-shot examples in each step. The instruction generation is data-aware and demonstration-aware.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"best-of-n","role":"first-class","note":"Optimizer paradigm is synthesise-many, score, keep-best; demos are bootstrapped from teacher.","evidence":[{"type":"doc","title":"DSPy — optimizer synthesis","url":"https://dspy.ai/learn/optimization/optimizers/","quote":"Different optimizers work by synthesizing good few-shot examples, proposing better natural-language instructions, or building datasets for finetuning.","accessed":"2026-05-20"},{"type":"doc","title":"DSPy — BootstrapFewShotWithRandomSearch","url":"https://dspy.ai/learn/optimization/optimizers/","quote":"Applies `BootstrapFewShot` several times with random search over generated demonstrations, and selects the best program over the optimization.","accessed":"2026-05-24"},{"type":"doc","title":"DSPy — BootstrapFewShot teacher","url":"https://dspy.ai/learn/optimization/optimizers/","quote":"Uses a `teacher` module (which defaults to your program) to generate complete demonstrations for every stage of your program, along with labeled examples in `trainset`.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"chain-of-thought","role":"first-class","note":"dspy.ChainOfThought is a named module.","evidence":[{"type":"doc","title":"DSPy — ChainOfThought","url":"https://dspy.ai/learn/programming/modules/","quote":"dspy.ChainOfThought: Teaches the LM to think step-by-step before committing to the signature's response.","accessed":"2026-05-20"},{"type":"doc","title":"DSPy — MultiChainComparison","url":"https://dspy.ai/learn/programming/modules/","quote":"dspy.MultiChainComparison: Can compare multiple outputs from ChainOfThought to produce a final prediction.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"dspy-signatures","role":"first-class","note":"Signatures are the declarative core.","evidence":[{"type":"doc","title":"DSPy — Signature definition","url":"https://dspy.ai/learn/programming/signatures/","quote":"A signature is a declarative specification of input/output behavior of a DSPy module.","accessed":"2026-05-20"},{"type":"doc","title":"DSPy — Signature what vs how","url":"https://dspy.ai/learn/programming/signatures/","quote":"Signatures allow you to tell the LM _what_ it needs to do, rather than specify _how_ we should ask the LM to do it.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"evaluator-optimizer","role":"first-class","note":"Optimizers tune prompts and weights against a metric — the canonical evaluator-optimizer loop.","evidence":[{"type":"doc","title":"DSPy — Optimizer tuning","url":"https://dspy.ai/","quote":"Optimizers tune the prompts and weights of your AI modules.","accessed":"2026-05-20"},{"type":"repo","title":"stanfordnlp/dspy README","url":"https://github.com/stanfordnlp/dspy","quote":"It allows you to iterate fast on building modular AI systems and offers algorithms for optimizing their prompts and weights","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"dspy.ReAct is a named tool-using agent module.","evidence":[{"type":"doc","title":"DSPy — ReAct module","url":"https://dspy.ai/learn/programming/modules/","quote":"dspy.ReAct: An agent that can use tools to implement the given signature.","accessed":"2026-05-20"},{"type":"doc","title":"DSPy — ReAct with tools","url":"https://dspy.ai/learn/programming/modules/","quote":"react = dspy.ReAct('question -> answer: float', tools=[evaluate_math, search_wikipedia])","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"self-refine","role":"limited","note":"Self-refine emerges through optimizer iteration; no single canonical Self-Refine module.","evidence":[{"type":"doc","title":"DSPy — instruction refinement","url":"https://dspy.ai/learn/optimization/optimizers/","quote":"Proposes and intelligently exploring better natural-language instructions for every prompt","accessed":"2026-05-20"},{"type":"doc","title":"DSPy — COPRO refines instructions","url":"https://dspy.ai/learn/optimization/optimizers/","quote":"Generates and refines new instructions for each step, and optimizes them with coordinate ascent (hill-climbing using the metric function and the `trainset`).","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"structured-output","role":"first-class","note":"Signatures enforce typed I/O; compiler builds optimised prompts/finetunes for the signature.","evidence":[{"type":"doc","title":"DSPy — signature reproducibility","url":"https://dspy.ai/learn/programming/signatures/","quote":"The DSPy compiler will figure out how to build a highly-optimized prompt for your LM (or finetune your small LM) for your signature, on your data, and within your pipeline.","accessed":"2026-05-20"},{"type":"doc","title":"DSPy — Signatures declarative","url":"https://dspy.ai/learn/programming/signatures/","quote":"A signature is a declarative specification of input/output behavior of a DSPy module.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Tool list passed to dspy.ReAct.","evidence":[{"type":"doc","title":"DSPy — tools in ReAct","url":"https://dspy.ai/learn/programming/modules/","quote":"dspy.ReAct: An agent that can use tools to implement the given signature.","accessed":"2026-05-20"},{"type":"doc","title":"DSPy — ReAct accepts tools","url":"https://dspy.ai/learn/programming/modules/","quote":"react = dspy.ReAct('question -> answer: float', tools=[evaluate_math, search_wikipedia])","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"self-consistency","role":"first-class","note":"dspy.majority (used as Ensemble reduce_fn) samples a module multiple times and returns the most frequently occurring completion, the classic self-consistency majority vote.","evidence":[{"type":"repo","title":"dspy/predict/aggregation.py — majority()","url":"https://github.com/stanfordnlp/dspy/blob/main/dspy/predict/aggregation.py","quote":"Returns the most common completion for the target field (or the last field) in the signature. When normalize returns None, that completion is ignored. In case of a tie, earlier completion are prioritized.","accessed":"2026-06-17"},{"type":"doc","title":"DSPy Cheatsheet — Ensemble","url":"https://dspy.ai/cheatsheet/","quote":"ensemble_optimizer = Ensemble(reduce_fn=dspy.majority)","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"generate-and-test-strategy","role":"first-class","note":"DSPy's BestOfN module runs a base module up to N times to generate multiple candidate predictions and then selects the first/best that passes a declared reward_fn threshold, rather than committing to the first generation.","evidence":[{"type":"doc","title":"DSPy — Refine / BestOfN","url":"https://dspy.ai/api/modules/Refine/","quote":"runs it up to N times with different rollout IDs at temperature=1.0 and returns the best prediction","accessed":"2026-06-17"},{"type":"doc","title":"DSPy — Refine / BestOfN","url":"https://dspy.ai/api/modules/Refine/","quote":"selects either the first prediction that exceeds the specified threshold or the one with the highest reward","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"zero-shot-cot","role":"first-class","note":"dspy.ChainOfThought wraps any signature so the model reasons step by step before the output, by prepending a generated reasoning/rationale field — no few-shot exemplars required.","evidence":[{"type":"doc","title":"ChainOfThought — DSPy","url":"https://dspy.ai/api/modules/ChainOfThought/","quote":"A module that reasons step by step in order to predict the output of a task.","accessed":"2026-06-17"},{"type":"doc","title":"ChainOfThought — DSPy","url":"https://dspy.ai/api/modules/ChainOfThought/","quote":"The dspy.ChainOfThought module will generally inject a reasoning before the output field(s) of your signature.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"dynamic-scaffolding","role":"first-class","note":"DSPy's KNNFewShot injects task-specific scaffolding at inference: for each input it retrieves the k most similar training examples and attaches them as demonstrations, so the prompt is scaffolded with examples relevant to that query rather than a fixed demo set — scaffolding added only when the input warrants it.","evidence":[{"type":"doc","title":"KNNFewShot | DSPy API","url":"https://dspy.ai/api/optimizers/KNNFewShot/","quote":"For each input example in a forward call, it identifies the k most similar examples from the trainset and attaches them","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"rest-em","role":"supported","note":"DSPy BootstrapFinetune ships the ReST-EM-style generate -> metric-filter -> finetune loop (framed as distillation).","evidence":[{"type":"doc","url":"https://github.com/stanfordnlp/dspy/blob/main/docs/docs/learn/optimization/optimizers.md","quote":"BootstrapFinetune: Distills a prompt-based DSPy program into weight updates. ... The bootstrapping process employs the metric to validate demonstrations, including only those that pass the metric","accessed":"2026-06-19"}],"evidence_status":"limited"},{"pattern":"star-bootstrapping","role":"supported","note":"DSPy BootstrapFinetune implements the STaR generate-rationales -> keep-correct -> finetune loop.","evidence":[{"type":"doc","url":"https://github.com/stanfordnlp/dspy/blob/main/docs/docs/learn/optimization/optimizers.md","quote":"BootstrapFinetune: Distills a prompt-based DSPy program into weight updates. ... including only those that pass the metric","accessed":"2026-06-19"}],"evidence_status":"limited"},{"pattern":"critic","role":"supported","note":"DSPy Assertions act as a programmatic critic: a failed assertion triggers a retry that dynamically adjusts the pipeline.","evidence":[{"type":"doc","url":"https://dspy.ai/learn/programming/7-assertions/","quote":"dspy.Assert initiates retry upon failure, dynamically adjusting the pipelines execution","accessed":"2026-06-19"}],"evidence_status":"limited"}],"instantiates":["eval-and-observability","reflection-and-self-correction","structured-output-stack"],"alternatives":[{"composition":"langchain","relation":"competes-with","note":"Declarative LM programming vs. imperative chain construction."},{"composition":"pydantic-ai","relation":"similar-shape","note":"Typed-I/O agent layer, but without compile-time prompt optimisation."},{"composition":"taskweaver","relation":"similar-shape","note":"Both treat the agent as code; DSPy adds prompt-program compilation, TaskWeaver adds session-stateful execution."}],"references":[{"type":"repo","title":"stanfordnlp/dspy","url":"https://github.com/stanfordnlp/dspy","accessed":"2026-05-20"},{"type":"doc","title":"DSPy docs","url":"https://dspy.ai","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-14","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","python","stanford","compiler","prompt-optimisation","signatures"]},{"id":"eko","name":"Eko","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"FellouAI","language":"TypeScript, JavaScript","license":"MIT","status":"active","url":"https://eko.fellou.ai","repo":"https://github.com/FellouAI/eko","intent":"Build production-ready agentic workflows in JavaScript/TypeScript from natural-language commands — decomposing a request into a multi-step workflow that runs multi-agent across browser and computer environments, with native MCP, human-in-the-loop, dependency-aware parallel execution, and pause/resume/interrupt with workflow recovery.","description":"Eko ('Eko Keeps Operating', FellouAI, MIT) is a TypeScript/JavaScript framework that turns a natural-language command into a multi-step agentic workflow. It runs in both browser and Node.js, ships agents for computer and browser environments, and supports multi-agent collaboration in a single task. Native MCP connects to external MCP servers. Eko 3.0 added dependency-aware parallel execution and pause/resume/interrupt with workflow recovery; 4.0 introduced chat conversations and refined agent logic. 'Stream Planning' renders the plan as it is generated; 'Loop & Listener Tasks' automate repetition. Native A2A and an Observable Chain are listed as coming soon.","primary_use_cases":["natural-language to multi-step workflow generation","multi-agent browser and computer automation in JS/TS","MCP-integrated workflows","pause / resume / interrupt long-running automations with recovery","dependency-aware parallel agent execution"],"agent_loop_shape":"Plan-then-execute workflow. From a natural-language command Eko streams a plan (Stream Planning) that decomposes the task into steps with declared dependencies; agents (e.g. BrowserAgent, FileAgent) execute the steps with awareness of dependencies for parallelism. Hooks intercept tool inputs/outputs and surface decisions for human-in-the-loop intervention. Pause / resume / interrupt controls allow workflow recovery from a snapshot. MCP servers are accessed natively; Native A2A and Observable Chain are roadmap features.","key_concepts":[{"name":"Workflow generation","summary":"One sentence to multi-step workflow with task-parallel execution.","url":"https://github.com/FellouAI/eko"},{"name":"Multi-agent","summary":"Multiple agents (BrowserAgent, FileAgent, ...) in one task."},{"name":"Native MCP","summary":"Connects seamlessly with Awesome MCP Servers.","maps_to_pattern":"mcp"},{"name":"Human-in-the-loop","summary":"Intervene when it matters most; hook system supports manual or AI intervention."},{"name":"Stream Planning","summary":"Dynamic rendering of the plan as it is generated.","maps_to_pattern":"plan-and-execute"},{"name":"Loop & Listener Tasks","summary":"Automate repetitive tasks via loop and listener constructs."},{"name":"Pause / resume / interrupt + recovery","summary":"Eko 3.0 controls for pausing, resuming, and interrupting workflows with state recovery.","maps_to_pattern":"agent-resumption"}],"pattern_composition":"flowchart TD\n  USER[Natural-language command] --> PLAN[Stream Planning<br/>decompose to workflow]\n  PLAN --> WF[Workflow<br/>dependency-aware steps]\n  WF --> BA[BrowserAgent]\n  WF --> FA[FileAgent]\n  WF --> OTHER[Other agents]\n  BA -->|tool calls| HOOK[Hook system<br/>pre/post tool]\n  HOOK --> HIL[Human intervention]\n  BA -.MCP.-> MCP[(MCP servers)]\n  WF -.pause/resume/interrupt.-> SNAP[(Workflow snapshot)]\n  SNAP --> WF\n  WF -.Loop & Listener.-> REPEAT[Repeated tasks]\n  WF -.coming soon.-> A2A[Native A2A]","members":[{"pattern":"browser-agent","role":"first-class","note":"Browser environment is one of two primary execution surfaces; e.g. stock-analysis-via-browser-automation example.","evidence":[{"type":"repo","title":"Eko README — production-ready","url":"https://github.com/FellouAI/eko","quote":"Pure JavaScript framework that enables developers to create reliable agents, from simple commands to complex workflows.","accessed":"2026-05-20"},{"type":"doc","title":"Eko site — browser automation","url":"https://fellou.ai/eko/","quote":"Browser & Computer Use: Includes stock analysis via browser automation and file management capabilities","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"computer-use","role":"first-class","note":"Computer environment (Node.js + GUI actions like clicking, screenshots, form auto-fill, command execution) is supported alongside browser.","evidence":[{"type":"doc","title":"Eko site — GUI click","url":"https://fellou.ai/eko/","quote":"Click GUI via prompt","accessed":"2026-05-24"},{"type":"repo","title":"Eko README — BrowserAgent + FileAgent","url":"https://github.com/FellouAI/eko","quote":"agents: Agent[] = [new BrowserAgent(), new FileAgent()];","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"plan-and-execute","role":"first-class","note":"Stream Planning decomposes one natural-language sentence into a multi-step workflow; Eko 3.0 documents 'dependency-aware parallel execution'.","evidence":[{"type":"repo","title":"Eko README — Stream Planning","url":"https://github.com/FellouAI/eko","quote":"Stream Planning: Dynamic rendering made easy","accessed":"2026-05-20"},{"type":"repo","title":"Eko README — one-sentence workflow","url":"https://github.com/FellouAI/eko","quote":"Workflow capabilities: One sentence to multi-step workflow with task parallel execution","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"Native MCP — connects with MCP servers as a top-level feature.","evidence":[{"type":"repo","title":"Eko README — Native MCP","url":"https://github.com/FellouAI/eko","quote":"Native MCP: Connects seamlessly with Awesome MCP Servers.🔗","accessed":"2026-05-24"},{"type":"repo","title":"Eko README — Native A2A coming soon","url":"https://github.com/FellouAI/eko","quote":"Native A2A: Coming soon","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"human-in-the-loop","role":"first-class","note":"Top-level feature: 'Intervene when it matters most'; hook system enables manual or AI intervention.","evidence":[{"type":"repo","title":"Eko README — HITL","url":"https://github.com/FellouAI/eko","quote":"Human-in-the-loop: Intervene when it matters most","accessed":"2026-05-20"},{"type":"doc","title":"Eko site — hook system","url":"https://fellou.ai/eko/","quote":"Hook System: Enables real-time monitoring of task execution, with the ability for manual or AI intervention","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"agent-resumption","role":"first-class","note":"Eko 3.0 added pause / resume / interrupt controls with workflow recovery.","evidence":[{"type":"repo","title":"Eko README — pause/resume/interrupt","url":"https://github.com/FellouAI/eko","quote":"New pause, resume, and interrupt controls with `task_snapshot` workflow recovery.","accessed":"2026-05-24"},{"type":"repo","title":"Eko README — multi-agent in one task","url":"https://github.com/FellouAI/eko","quote":"Multi-Agent: Unleash power with multiple Agents in one task.📈","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"parallelization","role":"first-class","note":"Dependency-aware parallel agent execution introduced in Eko 3.0.","evidence":[{"type":"repo","title":"Eko README — dependency-aware parallel execution","url":"https://github.com/FellouAI/eko","quote":"Eko 3.0 introduces dependency-aware parallel agent execution.","accessed":"2026-05-24"},{"type":"doc","title":"Eko site — offline planning + reusable plans","url":"https://fellou.ai/eko/","quote":"Eko allow separation of offline planning from execution and create reusable and modifiable task plans","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"8+ built-in tools (GUI click, web search, form auto-fill, command execution, screenshots); tool hooks intercept inputs/outputs.","evidence":[{"type":"doc","title":"Eko site — built-in tools","url":"https://fellou.ai/eko/","quote":"Provides various built-in tools for all platform and customize new tools easily.","accessed":"2026-05-24"},{"type":"repo","title":"Eko README — customise agents and tools","url":"https://github.com/FellouAI/eko","quote":"Agent/Tool Flexibility: Customize new Agents and Tools in just one line.🎉","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"react","role":"limited","note":"Inner agent step is tool-calling; framework framing emphasises plan-and-execute over ReAct. No 'ReAct' label found in the surveyed README or product page.","evidence":[],"evidence_status":"none"},{"pattern":"scheduled-agent","role":"limited","note":"Loop & Listener Tasks automate repetition; event/loop-driven rather than cron-style scheduled execution.","evidence":[{"type":"doc","title":"Eko — README","url":"https://github.com/FellouAI/eko","quote":"Loop & Listener Tasks: Automate any repetitive task.","accessed":"2026-05-24"},{"type":"doc","title":"Eko — README (raw)","url":"https://raw.githubusercontent.com/FellouAI/eko/main/README.md","quote":"Loop & Listener Tasks: Automate any repetitive task.","accessed":"2026-05-24"}],"evidence_status":"full"}],"references":[{"type":"repo","title":"FellouAI/eko","url":"https://github.com/FellouAI/eko","quote":"Eko (Eko Keeps Operating) - Build Production-ready Agentic Workflow with Natural Language","accessed":"2026-05-20"},{"type":"doc","title":"Eko site","url":"https://fellou.ai/eko/","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"emerging","tags":["open-source","typescript","javascript","browser-automation","computer-use","mcp","workflow"]},{"id":"hamilton","name":"Hamilton","aliases":["Apache Hamilton"],"kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"DAGWorks / Apache Software Foundation (incubating)","language":"Python","license":"Apache-2.0","status":"active","first_released":"2019","url":"https://hamilton.dagworks.io/","repo":"https://github.com/DAGWorks-Inc/hamilton","docs_url":"https://hamilton.dagworks.io/en/latest/","intent":"Express data and LLM pipelines as a directed acyclic graph of Python functions so transformations are testable, modular, and have automatic lineage and execution observability.","description":"Hamilton (Apache incubating) is a lightweight Python library that turns ordinary Python functions into nodes of a DAG: each function declares its dependencies via parameter names, and Hamilton wires them into an executable graph. It is explicitly NOT an agent framework — the maintainers (DAGWorks) ship a sister library, Burr, for loops, conditional logic, and LLM agents. Hamilton's relevance to agentic systems is structural: it organises the deterministic data/RAG pipeline that an agent calls into, and provides lineage, caching, and a UI that surface execution traces. Hamilton has been in production use since 2019 and was donated to the Apache Software Foundation.","primary_use_cases":["feature engineering and ML pipelines","LLM/RAG pipelines as DAGs of Python functions","data lineage and execution observability","structuring the deterministic backbone an agent (typically a Burr state machine) calls into"],"agent_loop_shape":"No agent loop. Hamilton executes a DAG of Python functions: each function is a node, parameter names declare upstream dependencies, and the driver resolves and runs the graph end-to-end. Loops, branching, and agentic decisions are explicitly out of scope and delegated to the sister library Burr.","key_concepts":[{"name":"Function-as-node DAG","summary":"Each Python function is a graph node; parameter names declare dependencies, which Hamilton wires into a DAG automatically.","url":"https://hamilton.dagworks.io/en/latest/"},{"name":"Lineage and execution observability","summary":"Hamilton populates a data catalog with lineage and tracing and lets users inspect execution times via the Hamilton UI.","maps_to_pattern":"lineage-tracking","url":"https://hamilton.dagworks.io/en/latest/"},{"name":"Caching (.hamilton_cache)","summary":"Per-node cache that stores metadata and results so repeated runs skip already-computed steps.","url":"https://hamilton.dagworks.io/en/latest/concepts/caching/"},{"name":"Burr (sister library)","summary":"DAGWorks' companion library for loops, conditional logic, and LLM agents — Hamilton itself is DAG-only.","url":"https://github.com/DAGWorks-Inc/burr"}],"pattern_composition":"flowchart TD\n  REQ[Request / agent step] --> DRIVER[Hamilton Driver]\n  DRIVER --> DAG{Function-as-node DAG<br/>params declare deps}\n  DAG --> N1[Node: load]\n  DAG --> N2[Node: transform]\n  DAG --> N3[Node: retrieve / LLM call]\n  N1 --> N2 --> N3\n  N3 --> CACHE[(.hamilton_cache<br/>per-node results)]\n  CACHE -.skip recompute.-> N2\n  DAG --> CATALOG[(Data catalog<br/>lineage + traces)]\n  CATALOG --> UI[Hamilton UI<br/>lineage / execution times]\n  DAG --> OUT[Final outputs]\n  AGENT[Agent loop<br/>e.g. Burr] -.calls into.-> DRIVER","members":[{"pattern":"lineage-tracking","role":"first-class","note":"Hamilton's headline observability feature: the DAG plus driver automatically populate a data catalog with lineage and execution traces visible in the Hamilton UI.","evidence":[{"type":"repo","title":"Apache Hamilton README","url":"https://github.com/DAGWorks-Inc/hamilton","quote":"It automatically populates a data catalog with lineage / tracing and provides execution observability to inspect results and debug errors.","accessed":"2026-05-20"},{"type":"doc","title":"Hamilton homepage — UI","url":"https://hamilton.dagworks.io/en/latest/","quote":"The companion Apache Hamilton UI has taken the value proposition up enormously with the ability to clearly show lineage & track execution times","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"decision-log","role":"limited","note":"Hamilton has no notion of an agent decision; its execution log captures node-level inputs/outputs and timings, which serve as a trace but not an authored decision rationale.","evidence":[{"type":"repo","title":"Apache Hamilton README — observability","url":"https://github.com/DAGWorks-Inc/hamilton","quote":"It automatically populates a data catalog with lineage / tracing and provides execution observability to inspect results and debug errors.","accessed":"2026-05-20"},{"type":"doc","title":"Hamilton UI — run history and node telemetry","url":"https://hamilton.dagworks.io/en/latest/concepts/ui","quote":"View a history of runs, telemetry on runs/comparison, and data for specific runs","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"agent-resumption","role":"limited","note":"Hamilton has no agent state machine to resume. Its per-node cache (.hamilton_cache) lets repeated runs skip already-computed steps, which approximates resumption only for deterministic pipelines.","evidence":[{"type":"doc","title":"Hamilton — caching","url":"https://hamilton.dagworks.io/en/latest/concepts/caching/","quote":"Caching enables storing execution results to be reused in later executions, effectively skipping redundant computations.","accessed":"2026-05-20"},{"type":"doc","title":"Hamilton — caching storage","url":"https://hamilton.dagworks.io/en/latest/concepts/caching/","quote":"The first execution will store metadata and results next to the current directory under ./.hamilton_cache.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"pipes-and-filters","role":"first-class","note":"Hamilton is fundamentally a pipes-and-filters DAG: parameter names declare dependencies and the driver wires them into a graph of pure functions.","evidence":[{"type":"repo","title":"Apache Hamilton README — what it is","url":"https://github.com/DAGWorks-Inc/hamilton","quote":"Apache Hamilton is a lightweight Python library for directed acyclic graphs (DAGs) of data transformations.","accessed":"2026-05-20"},{"type":"doc","title":"Hamilton homepage — dependencies via params","url":"https://hamilton.dagworks.io/en/latest/","quote":"each function defines a transformation and its parameters indicates its dependencies. Apache Hamilton automatically connects individual functions into a Directed Acyclic Graph (DAG)","accessed":"2026-05-20"}],"evidence_status":"full"}],"alternatives":[{"composition":"burr","relation":"complements","note":"Same vendor (DAGWorks). Hamilton handles DAG transformations; Burr handles agent loops and state machines."}],"references":[{"type":"repo","title":"DAGWorks-Inc/hamilton","url":"https://github.com/DAGWorks-Inc/hamilton","quote":"Apache Hamilton is a lightweight Python library for directed acyclic graphs (DAGs) of data transformations.","accessed":"2026-05-20"},{"type":"repo","title":"Hamilton README — Burr referral","url":"https://github.com/DAGWorks-Inc/hamilton","quote":"Apache Hamilton is great for DAGs, but if you need loops or conditional logic to create an LLM agent or a simulation, take a look at our sister library Burr.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","dag","data-pipeline","lineage","apache-incubating"]},{"id":"haystack","name":"Haystack","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"deepset","language":"Python","license":"Apache-2.0","status":"active","url":"https://haystack.deepset.ai/","repo":"https://github.com/deepset-ai/haystack","docs_url":"https://docs.haystack.deepset.ai/","intent":"Build production-ready LLM applications — RAG, agents, and multimodal search — as explicit pipelines of typed Components with a tool-using Agent at the top.","description":"Haystack is deepset's open-source orchestration framework for retrieval-augmented and agentic LLM applications. Its core abstractions are Components (typed building blocks) wired into Pipelines, with an Agent component that runs a tool-calling loop over chat-based LLMs. Tools include Tool, ComponentTool, PipelineTool, MCPTool, and MCPToolset, so a Haystack agent can call other pipelines, MCP servers, or arbitrary Python functions. Haystack ships first-class hybrid retrieval (BM25 + dense) backends for Weaviate, OpenSearch, Azure AI Search, and an in-memory store; cross-encoder rerankers; and Hayhooks for exposing pipelines as REST or MCP servers.","primary_use_cases":["production RAG over millions of documents","tool-using agents with explicit retrieval and routing","hybrid (BM25 + dense) search pipelines with reranking","agents exposed as MCP servers via Hayhooks"],"agent_loop_shape":"Pipeline-of-components topped by a loop-based Agent. The Agent component takes a chat-based LLM and a list of Tools (Tool, ComponentTool, PipelineTool, MCPTool, Toolset, MCPToolset, SearchableToolset) and iterates: call LLM, dispatch any tool calls, update state, repeat until an exit condition (text reply with no tool call, or a named tool being executed) or max_agent_steps is reached.","key_concepts":[{"name":"Component / Pipeline","summary":"Components are typed building blocks; Pipelines connect their outputs to compatible inputs to form a graph.","url":"https://docs.haystack.deepset.ai/docs/components"},{"name":"Agent","summary":"Tool-using loop component that drives a chat-based LLM through tool calls until an exit condition fires or max_agent_steps is hit.","maps_to_pattern":"react","url":"https://docs.haystack.deepset.ai/docs/agent"},{"name":"MCPTool / MCPToolset","summary":"Connects to external MCP servers over Streamable HTTP, SSE, or StdIO and exposes them to the Agent as tools.","maps_to_pattern":"mcp","url":"https://docs.haystack.deepset.ai/docs/mcptool"},{"name":"Hybrid retrievers","summary":"First-class BM25 + dense embedding retrievers for Weaviate, OpenSearch, Azure AI Search, and in-memory stores; results merged via RRF.","maps_to_pattern":"hybrid-search","url":"https://docs.haystack.deepset.ai/docs/weaviatehybridretriever"},{"name":"max_agent_steps + exit conditions","summary":"Agent iteration is bounded by max_agent_steps (default 100) and a list of exit conditions (e.g. 'text' or a tool name).","maps_to_pattern":"step-budget","url":"https://docs.haystack.deepset.ai/docs/agent"},{"name":"Hayhooks","summary":"Serves Haystack pipelines as REST APIs or MCP servers.","url":"https://haystack.deepset.ai/"}],"pattern_composition":"flowchart TD\n  USER[User query] --> PIPE{Pipeline}\n  PIPE --> RET[Hybrid retriever<br/>BM25 + dense, RRF]\n  RET --> RERANK[Cross-encoder reranker]\n  RERANK --> AGENT[Agent component<br/>chat LLM + tools]\n  AGENT -->|tool call| TOOLS{Tool / ComponentTool /<br/>PipelineTool / MCPTool}\n  TOOLS -->|result| AGENT\n  AGENT --> EXIT{Exit condition<br/>text or named tool}\n  EXIT -->|max_agent_steps=100| LIMIT[Stop]\n  AGENT --> RESP[Response]\n  PIPE -.serve as.-> HAY[Hayhooks<br/>REST / MCP server]","members":[{"pattern":"react","role":"first-class","note":"Haystack's Agent component is a tool-calling loop over a chat LLM; it iterates LLM call → tool dispatch → state update until an exit condition fires.","evidence":[{"type":"doc","title":"Haystack — Agent","url":"https://docs.haystack.deepset.ai/docs/agent","quote":"The Agent component is a tool-using agent that interacts with chat-based LLMs and tools to solve complex queries iteratively.","accessed":"2026-05-20"},{"type":"doc","title":"Haystack — Agent loop","url":"https://docs.haystack.deepset.ai/docs/agent","quote":"The Agent component is a loop-based system that uses a chat-based large language model (LLM) and external tools to solve complex user queries.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Agent accepts Tool, ComponentTool, PipelineTool, MCPTool, Toolset, MCPToolset, and SearchableToolset instances.","evidence":[{"type":"doc","title":"Haystack — Agent capabilities","url":"https://docs.haystack.deepset.ai/docs/agent","quote":"Dynamically select tools based on user input, maintain and validate runtime state using a schema, stream token-level outputs from the LLM.","accessed":"2026-05-20"},{"type":"doc","title":"Haystack — supported tool types","url":"https://docs.haystack.deepset.ai/docs/agent","quote":"Supported types: `Tool`, `ComponentTool`, `PipelineTool`, `MCPTool`, `Toolset`, `MCPToolset`, `SearchableToolset`.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"step-budget","role":"first-class","note":"Agent runs are bounded by max_agent_steps (default 100) and explicit exit conditions; stops when the LLM replies without a tool call or when a named tool is executed.","evidence":[{"type":"doc","title":"Haystack — max_agent_steps","url":"https://docs.haystack.deepset.ai/docs/agent","quote":"max_agent_steps: Maximum number of LLM + tool call iterations before the agent stops. Defaults to 100.","accessed":"2026-05-20"},{"type":"doc","title":"Haystack — exit conditions","url":"https://docs.haystack.deepset.ai/docs/agent","quote":"List of conditions that cause the agent to stop. Use \"text\" to stop when the LLM replies without a tool call, or a tool name to stop once that tool has been executed.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"MCPTool and MCPToolset integrate external MCP servers over Streamable HTTP, SSE, or StdIO; Hayhooks also serves Haystack pipelines as MCP servers.","evidence":[{"type":"doc","title":"Haystack — MCPTool","url":"https://docs.haystack.deepset.ai/docs/mcptool","quote":"MCPTool enables integration with external tools and services through the Model Context Protocol (MCP).","accessed":"2026-05-20"},{"type":"doc","title":"Haystack — MCPTool transports","url":"https://docs.haystack.deepset.ai/docs/mcptool","quote":"The MCPTool supports multiple transport options: Streamable HTTP for connecting to HTTP servers, SSE (Server-Sent Events) for connecting to HTTP servers (deprecated), StdIO for direct execution of local programs.","accessed":"2026-05-20"},{"type":"doc","title":"Haystack — Hayhooks MCP server","url":"https://haystack.deepset.ai/","quote":"Would you like to deploy and serve Haystack pipelines as REST APIs or MCP servers? Hayhooks provides a simple way","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"hybrid-search","role":"first-class","note":"Multiple built-in hybrid retrievers combine BM25 keyword search with dense embedding retrieval; Azure AI Search merges via RRF.","evidence":[{"type":"doc","title":"Haystack — WeaviateHybridRetriever","url":"https://docs.haystack.deepset.ai/docs/weaviatehybridretriever","quote":"combines BM25 keyword search and vector similarity to fetch documents from the Weaviate Document Store","accessed":"2026-05-20"},{"type":"doc","title":"Haystack — AzureAISearchHybridRetriever","url":"https://docs.haystack.deepset.ai/docs/azureaisearchhybridretriever","quote":"combines embedding-based retrieval and BM25 text search to find matching documents in the search index, with results merged and reordered using Reciprocal Rank Fusion (RRF)","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"cross-encoder-reranking","role":"first-class","note":"Haystack ships rerankers (SentenceTransformersDiversityRanker, LostInTheMiddleRanker, transformers cross-encoder rerankers) as Components that slot into a pipeline downstream of retrievers.","evidence":[{"type":"doc","title":"Haystack — Retrievers / hybrid pipeline","url":"https://docs.haystack.deepset.ai/docs/retrievers","quote":"A Hybrid Retriever uses both traditional keyword-based search (such as BM25) and embedding-based search to retrieve documents, combining the strengths of both approaches. The Retriever then merges and re-ranks the results from both methods.","accessed":"2026-05-20"},{"type":"doc","title":"Haystack — Rankers (cross-encoder)","url":"https://docs.haystack.deepset.ai/docs/rankers","quote":"A model-based Ranker that orders documents based on their relevance to the query. It uses a cross-encoder model to produce query and document embeddings.","accessed":"2026-05-24"},{"type":"doc","title":"Haystack — Rankers purpose","url":"https://docs.haystack.deepset.ai/docs/rankers","quote":"Rankers are a group of components that order documents by given criteria. Their goal is to improve your document retrieval results.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"agentic-rag","role":"first-class","note":"The Agent + retriever + reranker + generator pipeline is Haystack's canonical agentic-RAG shape; deepset explicitly markets RAG as a primary use case.","evidence":[{"type":"doc","title":"Haystack — intro","url":"https://docs.haystack.deepset.ai/docs/intro","quote":"Build powerful, production-ready applications with Large Language Models (LLMs) for various use cases. Whether you're creating autonomous agents, multimodal apps, or scalable RAG systems","accessed":"2026-05-20"},{"type":"doc","title":"Haystack — overview / agent workflows","url":"https://haystack.deepset.ai/overview/intro","quote":"Design explicit pipelines that control retrieval, ranking, memory, tools, and generation so every token sent to the model is intentional and traceable.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"streaming-typed-events","role":"first-class","note":"Agent streams token-level outputs from the LLM, surfaced as typed events during the run.","evidence":[{"type":"doc","title":"Haystack — Agent streaming","url":"https://docs.haystack.deepset.ai/docs/agent","quote":"stream token-level outputs from the LLM.","accessed":"2026-05-20"},{"type":"doc","title":"Haystack — streaming_callback","url":"https://docs.haystack.deepset.ai/docs/agent","quote":"Pass a callback to `streaming_callback`. Use the built-in `print_streaming_chunk` to print text tokens and tool events (tool calls and tool results).","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"structured-output","role":"limited","note":"Haystack supports state schemas and validates runtime state, but vendor docs frame structured output through state schemas rather than a first-class JSON-schema response API.","evidence":[{"type":"doc","title":"Haystack — Agent state schema","url":"https://docs.haystack.deepset.ai/docs/agent","quote":"Dynamically select tools based on user input, maintain and validate runtime state using a schema","accessed":"2026-05-20"},{"type":"doc","title":"Haystack — state_schema definition","url":"https://docs.haystack.deepset.ai/docs/agent","quote":"state_schema: Defines the agent's runtime state — a dict mapping key names to type configs (e.g. `{\"docs\": {\"type\": list[Document]}}`). Tools can read from and write to state keys via `inputs_from_state` and `outputs_to_state`.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"human-in-the-loop","role":"supported","note":"Haystack documents a Human in the Loop pattern for intercepting tool calls for human review; not the headline feature but explicitly supported.","evidence":[{"type":"doc","title":"Haystack — Agent docs (HITL link)","url":"https://docs.haystack.deepset.ai/docs/agent","quote":"Human in the Loop — intercepting tool calls for human review","accessed":"2026-05-20"},{"type":"doc","title":"Haystack — confirmation_strategies","url":"https://docs.haystack.deepset.ai/docs/agent","quote":"confirmation_strategies: A dict mapping tool names (or tuples of tool names) to a `ConfirmationStrategy`","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"hyde","role":"first-class","note":"Ships a HypotheticalDocumentEmbedder that zero-shot prompts an LLM to write a fake answer document, embeds it, and uses that embedding as the retrieval query.","evidence":[{"type":"doc","title":"Hypothetical Document Embeddings (HyDE) | Haystack Documentation","url":"https://docs.haystack.deepset.ai/docs/hypothetical-document-embeddings-hyde","quote":"Given a query, the Hypothetical Document Embeddings (HyDE) first zero-shot prompts an instruction-following language model to generate a \"fake\" hypothetical document...Then, it encodes each hypothetical document into an embedding vector and averages them.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"modular-rag","role":"core","note":"Pipelines are directed multigraphs of swappable, functionally-distinct components (retrievers, routers, rankers, generators) that can be rearranged with branching/routing rather than a fixed linear retrieve-then-generate flow.","evidence":[{"type":"doc","title":"Pipelines | Haystack Documentation","url":"https://docs.haystack.deepset.ai/docs/pipelines","quote":"The pipelines in Haystack are directed multigraphs of different Haystack components and integrations. They give you the freedom to connect these components in various ways.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"instantiates":["production-rag"],"alternatives":[{"composition":"langchain","relation":"similar-shape","note":"Peer Python agent/RAG framework."},{"composition":"llamaindex","relation":"competes-with","note":"deepset's RAG-rooted Python framework with an agent layer."}],"references":[{"type":"repo","title":"deepset-ai/haystack","url":"https://github.com/deepset-ai/haystack","quote":"open-source AI orchestration framework for building context-engineered, production-ready LLM applications","accessed":"2026-05-20"},{"type":"doc","title":"Haystack overview","url":"https://haystack.deepset.ai/overview/intro","quote":"Haystack structures agents and applications as explicit, modular pipelines composed of retrievers, routers, memory layers, tools, evaluators, and generators.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","rag","pipelines","deepset"]},{"id":"kag","name":"KAG (Knowledge Augmented Generation)","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"Ant Group (OpenSPG)","language":"Python","license":"Apache-2.0","status":"active","url":"https://openspg.yuque.com/ndx6g9/kag","repo":"https://github.com/OpenSPG/KAG","intent":"Knowledge-augmented generation framework built on the OpenSPG knowledge graph engine that translates natural-language questions into logical forms over a schema-constrained KG and combines retrieval, KG reasoning, and numerical calculation in a hybrid reasoning engine.","description":"KAG (Knowledge Augmented Generation, arXiv 2409.13731, ACM Web Conf 2025) is Ant Group's logical-form-guided RAG framework, built on the OpenSPG knowledge graph engine. KAG attacks the noise problem of OpenIE-based approaches (HippoRAG, GraphRAG) by using a schema-constrained KG with semantic alignment. It introduces mutual indexing between KG nodes and source chunks, and a hybrid reasoning engine that orchestrates four operator types — retrieval, KG reasoning, language reasoning, numerical calculation — by translating NL questions into logical forms. Published numbers significantly outperform NaiveRAG and HippoRAG on multi-hop QA (HotpotQA F1 +19.6%, 2Wiki +33.5%).","primary_use_cases":["professional-domain QA where schema-constrained knowledge is available (legal, medical, finance)","multi-hop reasoning over KG + text with quantifiable accuracy gains over RAG and HippoRAG","decision-support systems where rigorous fact-grounding is required","knowledge-graph construction with semantic alignment to reduce OpenIE noise"],"agent_loop_shape":"Offline: domain schema is defined in SPG, sources are processed into a schema-constrained KG with chunk-level mutual indexing. Online: the user question is translated to a logical form by the planner; the logical form orchestrates a sequence of operator calls (retrieval, KG reasoning, language reasoning, numerical calculation); operator outputs feed the generator. The bidirectional KG↔chunk index lets graph traversal pull supporting evidence from text.","key_concepts":[{"name":"Schema-constrained KG","summary":"Knowledge representation bound by an explicit domain schema, distinct from OpenIE-extracted graphs.","url":"https://github.com/OpenSPG/openspg","maps_to_pattern":"knowledge-graph-memory"},{"name":"Mutual KG ↔ chunk indexing","summary":"Bidirectional anchor between graph nodes and source text fragments."},{"name":"Logical-form-guided reasoning","summary":"NL questions are translated to logical forms that orchestrate planner / reasoning / retrieval operators.","maps_to_pattern":"plan-and-execute"},{"name":"Hybrid reasoning engine","summary":"Four operator types — retrieval, KG reasoning, language reasoning, numerical calculation — composed by the logical form."}],"pattern_composition":"flowchart TD\n  subgraph Offline\n    SOURCES[(Documents +<br/>structured data)] --> SCHEMA[SPG schema-constrained<br/>KG construction]\n    SCHEMA --> ALIGN[Semantic alignment<br/>reduce OpenIE noise]\n    ALIGN --> KG[(Schema-bound KG<br/>+ chunk mutual index)]\n  end\n  subgraph Online\n    Q[NL question] --> LF[Logical form planner]\n    LF --> OPS{Operator}\n    OPS -->|retrieval| R[Retrieve chunks]\n    OPS -->|KG reasoning| KGR[KG traversal]\n    OPS -->|language reasoning| LR[CoT step]\n    OPS -->|numerical| NUM[Numerical calc]\n    R --> KG\n    KGR --> KG\n    R --> GEN[Generator]\n    KGR --> GEN\n    LR --> GEN\n    NUM --> GEN\n    GEN --> ANS[Answer + citations]\n  end","members":[{"pattern":"graphrag","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"knowledge-graph-memory","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"naive-rag","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"hybrid-search","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"hippocampus-rag","role":"supported","note":"KAG is an alternative-to with explicitly published comparison numbers.","evidence":[],"evidence_status":"none"},{"pattern":"chain-of-thought","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"plan-and-execute","role":"first-class","note":"Logical form is the plan.","evidence":[],"evidence_status":"none"},{"pattern":"structured-output","role":"first-class","note":"Logical form is structured.","evidence":[],"evidence_status":"none"},{"pattern":"citation-attribution","role":"supported","evidence":[],"evidence_status":"none"}],"instantiates":["production-rag"],"alternatives":[{"composition":"ragflow","relation":"similar-shape","note":"RAGFlow is the document-RAG peer; KAG specialises on rigorous KG-based reasoning."},{"composition":"dbgpt","relation":"complements","note":"DB-GPT also ships GraphRAG; KAG's logical-form reasoning could feed into DB-GPT's AWEL workflows."}],"last_analyzed":"2026-05-22","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-05-22","status_in_practice":"emerging","tags":["orchestration-framework","retrieval","memory","reasoning","planning-control-flow","structure-data","full-code"],"references":[{"type":"doc","title":"KAG (Knowledge Augmented Generation) — Homepage","url":"https://openspg.yuque.com/ndx6g9/kag"},{"type":"repo","title":"KAG (Knowledge Augmented Generation) — Source repository","url":"https://github.com/OpenSPG/KAG"}]},{"id":"langchain","name":"LangChain","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"LangChain Inc.","language":"Python, TypeScript","license":"MIT","status":"active","first_released":"2022","url":"https://www.langchain.com/","repo":"https://github.com/langchain-ai/langchain","docs_url":"https://docs.langchain.com/oss/python/langchain/overview","intent":"Provide a standard, model-agnostic Python/TypeScript interface plus a prebuilt agent (create_agent) for building LLM applications that loop over tool calls in the ReAct shape, with first-class integrations for retrieval, structured output, MCP tools, and middleware-based human approval.","description":"LangChain is the MIT-licensed agent and LLM-application framework built by LangChain Inc., the same company that ships LangGraph and LangSmith. In its 1.x form the headline surface is create_agent, a prebuilt agent that runs a chat model in a ReAct loop over a developer-supplied list of tools, stopping when the model emits a final answer or an iteration limit is reached. The framework ships standard integrations for models, embeddings, vector stores, retrievers, MCP servers, and content-block citations; orchestration concerns beyond the loop (durable state, interrupts, multi-agent graphs) are delegated to LangGraph, which create_agent compiles down to internally.","primary_use_cases":["ReAct-shape agents calling provider-agnostic models and tools","RAG via retrievers + cross-encoder reranking + EnsembleRetriever","structured-output agents with ProviderStrategy / ToolStrategy","MCP-tool integration via langchain-mcp-adapters"],"agent_loop_shape":"ReAct tool-calling loop. The model is bound to a tool list; runtime alternates model-emit-tool-calls and tool-execute steps, and stops on final-answer-or-iteration-limit. Middleware hooks run before/after model and tool calls inside the compiled LangGraph that create_agent returns.","key_concepts":[{"name":"create_agent","summary":"Prebuilt ReAct-style agent constructor that binds a model to a tool list and loops.","maps_to_pattern":"react","url":"https://docs.langchain.com/oss/python/langchain/agents"},{"name":"Tool / @tool","summary":"Callable functions with well-defined inputs and outputs passed to a chat model.","maps_to_pattern":"tool-use","url":"https://docs.langchain.com/oss/python/langchain/tools"},{"name":"response_format (structured output)","summary":"Schema-driven structured output via ProviderStrategy (native) or ToolStrategy (tool-call emulation).","maps_to_pattern":"structured-output","url":"https://docs.langchain.com/oss/python/langchain/structured-output"},{"name":"Middleware","summary":"Hooks running inside the compiled LangGraph that create_agent returns; used for HITL, summarization, custom guards.","url":"https://docs.langchain.com/oss/python/langchain/middleware"},{"name":"Retriever","summary":"Interface returning documents given an unstructured query; integrations for many vector stores, BM25, Pinecone Hybrid Search, etc.","url":"https://docs.langchain.com/oss/python/langchain/retrieval"},{"name":"RunnableWithFallbacks (.with_fallbacks)","summary":"Runnable that falls back to other Runnables on failure; fallbacks tried in order.","maps_to_pattern":"fallback-chain","url":"https://reference.langchain.com/python/langchain_core/runnables/RunnableWithFallbacks/"}],"pattern_composition":"flowchart TD\n  USER[User input] --> AGENT[create_agent]\n  MODEL[(Chat model)] --> AGENT\n  TOOLS[Tools list<br/>@tool decorated] --> AGENT\n  MW[Middleware hooks<br/>HITL / Summarization / Custom] --> AGENT\n  AGENT -->|ReAct loop| TOOLS\n  AGENT -->|response_format| SCHEMA[(JSON schema<br/>ProviderStrategy or ToolStrategy)]\n  AGENT -->|with_fallbacks| FB[Fallback chain]\n  AGENT -->|MCP servers| MCP[langchain-mcp-adapters]\n  AGENT -->|Retriever as tool| RET[Vector store / BM25 / Hybrid]\n  RET --> CER[CrossEncoderReranker]\n  CER --> AGENT\n  AGENT --> OUT[Final answer]","members":[{"pattern":"agentic-rag","role":"first-class","note":"RAG is documented as 'a simple agent with a tool that retrieves information' — the model decides when to call the retrieval tool, mirroring the agentic-RAG shape.","evidence":[{"type":"doc","title":"LangChain — RAG","url":"https://docs.langchain.com/oss/python/langchain/rag","quote":"One formulation of a RAG application is as a simple agent with a tool that retrieves information","accessed":"2026-05-20"},{"type":"doc","title":"LangChain — RAG with retrievers","url":"https://docs.langchain.com/oss/python/langchain/rag","quote":"Given a user input, relevant splits are retrieved from storage using a Retriever.","accessed":"2026-05-24"},{"type":"doc","title":"LangChain — Agentic RAG control","url":"https://docs.langchain.com/oss/python/langchain/rag","quote":"You can add a deeper level of control and customization using the LangGraph framework directly—for example, you can add steps to grade document relevance and rewrite search queries.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"approval-queue","role":"limited","note":"Human-in-the-Loop middleware uses LangGraph's interrupt + checkpointer underneath; halts execution and saves state. Limited because the durability comes from LangGraph, not LangChain proper.","evidence":[{"type":"doc","title":"LangChain — Human-in-the-loop middleware","url":"https://docs.langchain.com/oss/python/langchain/human-in-the-loop","quote":"The Human-in-the-Loop (HITL) middleware lets you add human oversight to agent tool calls.","accessed":"2026-05-20"},{"type":"doc","title":"LangChain — HITL implementation","url":"https://docs.langchain.com/oss/python/langchain/human-in-the-loop","quote":"The middleware issues an interrupt that halts execution. The graph state is saved using LangGraph's persistence layer.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"chain-of-thought","role":"limited","note":"Not a dedicated CoT API; reasoning is exposed as 'reasoning' content blocks during streaming. The ReAct loop interleaves reasoning steps with tool calls.","evidence":[{"type":"doc","title":"LangChain — Agents (ReAct framing)","url":"https://docs.langchain.com/oss/python/langchain/agents","quote":"Agents follow the ReAct ('Reasoning + Acting') pattern, alternating between brief reasoning steps with targeted tool calls and feeding the resulting observations into subsequent decisions.","accessed":"2026-05-20"},{"type":"doc","title":"LangChain — ReAct full description","url":"https://docs.langchain.com/oss/python/langchain/agents","quote":"Agents follow the ReAct ('Reasoning + Acting') pattern, alternating between brief reasoning steps with targeted tool calls and feeding the resulting observations into subsequent decisions until they can deliver a final answer.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"citation-streaming","role":"limited","note":"Citation is a typed content-block annotation; streaming surfaces content blocks but there is no dedicated 'citation streaming' mode named in docs.","evidence":[{"type":"repo","title":"langchain_core — Citation annotation","url":"https://github.com/langchain-ai/langchain/blob/master/libs/core/langchain_core/messages/content.py","quote":"Annotation for citing data from a document.","accessed":"2026-05-20"},{"type":"doc","title":"LangChain — Messages content blocks","url":"https://docs.langchain.com/oss/python/langchain/messages","quote":"Content blocks are represented (either when creating a message or accessing the `content_blocks` property) as a list of typed dictionaries.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"context-window-packing","role":"limited","note":"Short-term-memory docs name the problem and offer SummarizationMiddleware + token trimming as the answer.","evidence":[{"type":"doc","title":"LangChain — Short-term memory","url":"https://docs.langchain.com/oss/python/langchain/short-term-memory","quote":"Long conversations pose a challenge to today's LLMs; a full history may not fit inside an LLM's context window, resulting in an context loss or errors.","accessed":"2026-05-20"},{"type":"doc","title":"LangChain — Token trimming","url":"https://docs.langchain.com/oss/python/langchain/short-term-memory","quote":"One way to decide when to truncate messages is to count the tokens in the message history and truncate whenever it approaches that limit.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"contextual-retrieval","role":"via-integration","note":"No canonical LangChain docs page for Anthropic's Contextual Retrieval technique; usable only as a recipe via custom embedding-time chunking. Downgraded from limited to via-integration. No verbatim upstream quote available for this framework-level mapping.","evidence":[],"evidence_status":"none"},{"pattern":"cross-encoder-reranking","role":"first-class","note":"CrossEncoderReranker wraps a HuggingFace cross-encoder and plugs into ContextualCompressionRetriever.","evidence":[{"type":"doc","title":"LangChain — Cross-encoder reranker","url":"https://docs.langchain.com/oss/python/integrations/document_transformers/cross_encoder_reranker/","quote":"A cross-encoder scores each `(query, document)` pair directly rather than comparing independent embeddings, which produces more accurate ordering.","accessed":"2026-05-20"},{"type":"doc","title":"LangChain — CrossEncoderReranker API","url":"https://docs.langchain.com/oss/python/integrations/document_transformers/cross_encoder_reranker/","quote":"`CrossEncoderReranker` wraps any cross-encoder and plugs into `ContextualCompressionRetriever`.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"fallback-chain","role":"first-class","note":"RunnableWithFallbacks (or .with_fallbacks()) is a documented core abstraction; fallbacks are tried in order until one succeeds.","evidence":[{"type":"api-ref","title":"langchain_core — RunnableWithFallbacks","url":"https://reference.langchain.com/python/langchain_core/runnables/RunnableWithFallbacks/","quote":"`Runnable` that can fallback to other `Runnable` objects if it fails.","accessed":"2026-05-20"},{"type":"api-ref","title":"langchain_core — fallback order","url":"https://reference.langchain.com/python/langchain_core/runnables/RunnableWithFallbacks/","quote":"Fallbacks are tried in order until one succeeds or all fail.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"handoff","role":"limited","note":"Agents transfer control via tool calls (Command(goto=...) is a LangGraph primitive re-exported by LangChain).","evidence":[{"type":"doc","title":"LangChain — Multi-agent","url":"https://docs.langchain.com/oss/python/langchain/multi-agent","quote":"Agents transfer control to each other via tool calls. Each agent can hand off to others or respond directly to the user.","accessed":"2026-05-20"},{"type":"doc","title":"LangChain — Handoff via state changes","url":"https://docs.langchain.com/oss/python/langchain/multi-agent","quote":"Behavior changes dynamically based on state. Tool calls update a state variable that triggers routing or configuration changes.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"hybrid-search","role":"supported","note":"EnsembleRetriever (langchain_classic) combines multiple retrievers via weighted Reciprocal Rank Fusion; BM25Retriever (langchain-community) provides keyword search alongside vector retrieval.","evidence":[{"type":"repo","title":"langchain — EnsembleRetriever","url":"https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain_classic/retrievers/ensemble.py","quote":"Ensemble retriever that ensemble the results of multiple retrievers by using weighted  Reciprocal Rank Fusion.","accessed":"2026-05-24"},{"type":"repo","title":"langchain-community — BM25Retriever","url":"https://github.com/langchain-ai/langchain-community/blob/main/libs/community/langchain_community/retrievers/bm25.py","quote":"`BM25` retriever without Elasticsearch.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"First-class MCP integration via langchain-mcp-adapters and MultiServerMCPClient.","evidence":[{"type":"doc","title":"LangChain — MCP","url":"https://docs.langchain.com/oss/python/langchain/mcp","quote":"An open protocol that standardizes how applications provide tools and context to LLMs.","accessed":"2026-05-20"},{"type":"doc","title":"LangChain — Multi-server MCP","url":"https://docs.langchain.com/oss/python/langchain/mcp","quote":"Enables agents to use tools defined across one or more MCP servers.","accessed":"2026-05-20"},{"type":"repo","title":"langchain-ai/langchain-mcp-adapters","url":"https://github.com/langchain-ai/langchain-mcp-adapters","quote":"This library provides a lightweight wrapper that makes Anthropic Model Context Protocol (MCP) tools compatible with LangChain","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"plan-and-execute","role":"supported","note":"Documented in LangChain Inc.'s blog rather than the docs proper; concrete implementations live in langgraph examples. Downgraded from first-class.","evidence":[{"type":"blog","title":"LangChain blog — Planning agents","url":"https://www.langchain.com/blog/planning-agents","quote":"A planner, which prompts an LLM to generate a multi-step plan to complete a large task.","accessed":"2026-05-20"},{"type":"blog","title":"LangChain blog — Planner separated from executor","url":"https://www.langchain.com/blog/planning-agents","quote":"separates an LLM-powered 'planner' from the tool execution runtime","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"query-rewriting","role":"first-class","note":"LangChain ships MultiQueryRetriever as the canonical Query Rewriting implementation: an LLM generates N variants of the user query, retrieval runs per variant, results are unioned and deduplicated. Default N=3.","evidence":[{"type":"repo","title":"langchain — MultiQueryRetriever docstring","url":"https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain_classic/retrievers/multi_query.py","quote":"Given a query, use an LLM to write a set of queries.\n\n    Retrieve docs for each query. Return the unique union of all retrieved docs.","accessed":"2026-05-24"},{"type":"repo","title":"langchain — MultiQueryRetriever default prompt (3 variants)","url":"https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain_classic/retrievers/multi_query.py","quote":"to generate 3 different versions of the given user\n    question to retrieve relevant documents from a vector  database.\n    By generating multiple perspectives on the user question","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"create_agent IS the prebuilt ReAct constructor.","evidence":[{"type":"doc","title":"LangChain — Agents (ReAct pattern)","url":"https://docs.langchain.com/oss/python/langchain/agents","quote":"Agents follow the ReAct ('Reasoning + Acting') pattern, alternating between brief reasoning steps with targeted tool calls and feeding the resulting observations into subsequent decisions.","accessed":"2026-05-20"},{"type":"doc","title":"LangChain — create_agent","url":"https://docs.langchain.com/oss/python/langchain/agents","quote":"`create_agent` provides a production-ready agent implementation.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"self-refine","role":"limited","note":"Reflection agents are blog-only; no dedicated docs page in current LangChain.","evidence":[{"type":"blog","title":"LangChain blog — Reflection agents","url":"https://www.langchain.com/blog/reflection-agents","quote":"Reflection is a prompting strategy used to improve the quality and success rate of agents and similar AI systems.","accessed":"2026-05-20"},{"type":"blog","title":"LangChain blog — Reflection critique loop","url":"https://www.langchain.com/blog/reflection-agents","quote":"The actor agent explicitly critiques each response and grounds its criticism in external data.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"session-isolation","role":"first-class","note":"Threads scope short-term memory; storing state in graph state maintains separation between threads.","evidence":[{"type":"doc","title":"LangChain — Short-term memory (threads)","url":"https://docs.langchain.com/oss/python/langchain/short-term-memory","quote":"A thread organizes multiple interactions in a session, similar to the way email groups messages in a single conversation.","accessed":"2026-05-20"},{"type":"doc","title":"LangChain — thread isolation","url":"https://docs.langchain.com/oss/python/langchain/short-term-memory","quote":"By storing these in the graph's state, the agent can access the full context for a given conversation while maintaining separation between different threads.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"skill-library","role":"limited","note":"Skills live primarily in Deep Agents (a higher-level harness built on LangChain), not core LangChain.","evidence":[{"type":"doc","title":"LangChain — Multi-agent (skills via Deep Agents)","url":"https://docs.langchain.com/oss/python/langchain/multi-agent","quote":"Specialized prompts and knowledge loaded on-demand. A single agent stays in control while loading context from skills as needed.","accessed":"2026-05-20"},{"type":"doc","title":"LangChain — Deep Agents harness","url":"https://docs.langchain.com/oss/python/langchain/multi-agent","quote":"For built-in multi-agent support, use Deep Agents: a higher-level harness built on LangChain that ships with subagents, skills, planning, a virtual filesystem, and context management.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"step-budget","role":"supported","note":"Agents stop on final-answer-or-iteration-limit but no current docs page names the iteration-limit parameter; recursion_limit is a LangGraph concept. Downgraded from first-class.","evidence":[{"type":"doc","title":"LangChain — Agents (stop condition)","url":"https://docs.langchain.com/oss/python/langchain/agents","quote":"An LLM Agent runs tools in a loop to achieve a goal. An agent runs until a stop condition is met - i.e., when the model emits a final output or an iteration limit is reached.","accessed":"2026-05-20"},{"type":"doc","title":"LangChain — iteration limit","url":"https://docs.langchain.com/oss/python/langchain/agents","quote":"An agent runs until a stop condition is met - i.e., when the model emits a final output or an iteration limit is reached.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"structured-output","role":"first-class","note":"response_format with ProviderStrategy (native) or ToolStrategy (tool-call emulation). As of LangChain 1.0, passing a schema defaults to ProviderStrategy and falls back to ToolStrategy.","evidence":[{"type":"doc","title":"LangChain — Structured output","url":"https://docs.langchain.com/oss/python/langchain/structured-output","quote":"Structured output allows agents to return data in a specific, predictable format.","accessed":"2026-05-20"},{"type":"doc","title":"LangChain — strategies","url":"https://docs.langchain.com/oss/python/langchain/structured-output","quote":"As of langchain 1.0, simply passing a schema will default to ProviderStrategy if the model supports native structured output. It will fall back to ToolStrategy otherwise.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"supervisor","role":"via-integration","note":"Closest core-LangChain concept is the Router (LLM picks among agents and invokes in parallel). The supervisor pattern is documented as a multi-agent architecture with a central supervisor coordinating worker agents.","evidence":[{"type":"doc","title":"LangChain — Router","url":"https://docs.langchain.com/oss/python/langchain/multi-agent","quote":"A routing step classifies input and directs it to one or more specialized agents. Results are synthesized into a combined response.","accessed":"2026-05-20"},{"type":"doc","title":"LangChain — Supervisor pattern definition","url":"https://docs.langchain.com/oss/python/langchain/supervisor","quote":"The **supervisor pattern** is a [multi-agent](/oss/python/langchain/multi-agent) architecture where a central supervisor agent coordinates specialized worker agents.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Tools are the primary extension point; @tool decorator + model.bind_tools(...).","evidence":[{"type":"doc","title":"LangChain — Tools","url":"https://docs.langchain.com/oss/python/langchain/tools","quote":"Tools extend what agents can do—letting them fetch real-time data, execute code, query external databases, and take actions.","accessed":"2026-05-20"},{"type":"doc","title":"LangChain — @tool decorator","url":"https://docs.langchain.com/oss/python/langchain/tools","quote":"The simplest way to create a tool is with the @tool decorator.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"semantic-response-cache","role":"supported","note":"LangChain ships RedisSemanticCache, an LLM cache that matches prompts by vector similarity rather than exact string equality and returns the stored response on a near-duplicate hit.","evidence":[{"type":"doc","title":"RedisSemanticCache | langchain_redis | LangChain Reference","url":"https://reference.langchain.com/python/langchain-redis/cache/RedisSemanticCache","quote":"This cache uses vector similarity search to find semantically similar prompts.","accessed":"2026-06-14"}],"evidence_status":"full"},{"pattern":"cross-reflection","role":"supported","note":"LLM-as-judge evaluators send one model's output to a separate judge LLM, and the judging model is configurable (PROVIDER:MODEL string or a different chat model instance), decorrelating critique from generation.","evidence":[{"type":"repo","title":"openevals README — LLM-as-Judge Evaluators","url":"https://github.com/langchain-ai/openevals","quote":"One common way to evaluate an LLM app's outputs is to use another LLM as a judge.","accessed":"2026-06-17"},{"type":"repo","title":"openevals README — configuring the judge model","url":"https://github.com/langchain-ai/openevals","quote":"pass a string formatted as `PROVIDER:MODEL` ... directly pass a LangChain chat model instance as `judge`","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"self-ask","role":"first-class","note":"create_self_ask_with_search_agent builds an agent that breaks a question into explicit follow-up sub-questions, answers each (via an Intermediate Answer search tool), then composes the final answer.","evidence":[{"type":"doc","title":"Self-ask with search — LangChain","url":"https://python.langchain.com/v0.1/docs/modules/agents/agent_types/self_ask_with_search/","quote":"This walkthrough showcases the self-ask with search agent.","accessed":"2026-06-17"},{"type":"api-ref","title":"create_self_ask_with_search_agent — LangChain","url":"https://python.langchain.com/api_reference/langchain/agents/langchain.agents.self_ask_with_search.base.create_self_ask_with_search_agent.html","quote":"For this agent, only one tool can be used and it needs to be named \"Intermediate Answer\".","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"tree-of-thoughts","role":"limited","note":"langchain_experimental ships a ToTChain that searches a tree of thoughts: a thought-generation strategy proposes children, a ToTChecker classifies each thought VALID_INTERMEDIATE/VALID_FINAL/INVALID, and a DFS-memory controller prunes and backtracks.","evidence":[{"type":"repo","title":"langchain_experimental/tot/base.py — ToTChain","url":"https://github.com/langchain-ai/langchain-experimental/blob/main/libs/experimental/langchain_experimental/tot/base.py","quote":"class ToTChain(Chain):\n    Chain implementing the Tree of Thought (ToT).","accessed":"2026-06-17"},{"type":"repo","title":"langchain_experimental/tot/thought.py — ThoughtValidity","url":"https://github.com/langchain-ai/langchain-experimental/blob/main/libs/experimental/langchain_experimental/tot/thought.py","quote":"class ThoughtValidity(Enum):\n    VALID_INTERMEDIATE = 0\n    VALID_FINAL = 1\n    INVALID = 2","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"mrkl-systems","role":"core","note":"LangChain ships MRKLChain, an implementation named after the MRKL (Modular Reasoning, Knowledge and Language) paper: an LLM dispatcher loops over tool definitions and, at each step, picks one action from [tool_names] (calculator, search, code, etc.), routing each sub-problem to the matching expert module and integrating results.","evidence":[{"type":"api-ref","title":"MRKLChain — LangChain documentation","url":"https://reference.langchain.com/python/langchain-classic/agents/","quote":"MRKLChain implements the MRKL system; the agent's format instructions tell the LLM the action to take should be one of [tool_names], with tools defined by a name, function, and description.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"alternatives":[{"composition":"langgraph","relation":"same-vendor","note":"create_agent compiles to LangGraph internally; LangChain is the high-level surface, LangGraph the low-level runtime."},{"composition":"llamaindex","relation":"similar-shape","note":"Peer Python agent/RAG framework."},{"composition":"haystack","relation":"similar-shape","note":"Peer Python agent/RAG framework."},{"composition":"pydantic-ai","relation":"similar-shape","note":"Peer Python agent framework with stronger typed-output framing."},{"composition":"vercel-ai-sdk","relation":"similar-shape","note":"Peer for the TypeScript track that LangChain.js also serves."},{"composition":"genkit","relation":"similar-shape","note":"Peer multi-provider orchestration framework."},{"composition":"agno","relation":"competes-with","note":"Wider integration set, less of a runtime/platform."},{"composition":"dbgpt","relation":"competes-with"},{"composition":"dspy","relation":"competes-with","note":"Declarative LM programming vs. imperative chain construction."},{"composition":"mastra","relation":"competes-with","note":"LangChain.js + create_agent target the same TS-agent niche; Mastra adds workflow engine + native suspend/resume."},{"composition":"openagents","relation":"competes-with","note":"LangChain is the broader framework; OpenAgents ships three pre-configured agents as a product."},{"composition":"qwen-agent","relation":"similar-shape","note":"Western OSS counterpart — comparable function-calling + RAG + ReAct surface, model-agnostic rather than Qwen-coupled."},{"composition":"ragflow","relation":"competes-with","note":"LangChain is broader; RAGFlow is RAG-specialised with document-understanding as the differentiator."},{"composition":"semantic-kernel","relation":"competes-with","note":"Both are general orchestration SDKs spanning multiple languages."},{"composition":"smolagents","relation":"competes-with","note":"Same role layer, JSON-first tool-calling, much larger surface area."}],"references":[{"type":"repo","title":"langchain-ai/langchain","url":"https://github.com/langchain-ai/langchain","quote":"The agent engineering platform.","accessed":"2026-05-20"},{"type":"doc","title":"LangChain overview (Python)","url":"https://docs.langchain.com/oss/python/langchain/overview","accessed":"2026-05-20"},{"type":"repo","title":"langchain-ai/langchainjs","url":"https://github.com/langchain-ai/langchainjs","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-06-19","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","model-agnostic","python","typescript","react-loop"],"anti_patterns_avoided":[{"pattern":"tool-explosion","note":"Dynamic tool calling limits which tools are exposed early, reducing wrong turns from an over-large tool surface.","evidence":[{"type":"doc","title":"LangChain — dynamic tool calling","url":"https://changelog.langchain.com/announcements/dynamic-tool-calling-in-langgraph-agents","quote":"Reduce wrong turns by limiting unnecessary tools early.","accessed":"2026-06-19"}]}]},{"id":"langgraph","name":"LangGraph","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"LangChain Inc.","language":"Python, TypeScript","license":"MIT","status":"active","first_released":"2024","url":"https://www.langchain.com/langgraph","repo":"https://github.com/langchain-ai/langgraph","docs_url":"https://docs.langchain.com/oss/python/langgraph/overview","intent":"Provide low-level orchestration infrastructure for long-running, stateful agents with durable execution, persistent memory, and built-in human-in-the-loop interrupts.","description":"LangGraph is the MIT-licensed orchestration framework from LangChain Inc. that models an agent as a graph of nodes operating over a shared, checkpointed state object. Production users (Klarna, Replit, Elastic) build durable agents that resume from failures, pause for human approval via interrupt(), and share short-term thread-scoped memory plus long-term cross-session memory in a Store. LangGraph draws structural inspiration from Pregel and Apache Beam; it can be used standalone or paired with LangChain, LangSmith for observability, and LangGraph Platform for hosted runs. Multi-agent topologies (supervisor, swarm, hierarchical) ship as separate prebuilt libraries.","primary_use_cases":["long-running stateful agents resilient to failures","multi-agent supervisor / swarm / hierarchical topologies","agents with mid-run human-approval steps","agents needing both working memory and cross-session persistent memory"],"agent_loop_shape":"Graph of nodes with explicit state edges; the runtime checkpoints state after every super-step so any node can be paused, resumed, or restarted, and Command-typed return values can re-route control between nodes or subgraphs.","key_concepts":[{"name":"StateGraph","summary":"The core abstraction: a graph whose nodes read from and write to a typed state object the runtime checkpoints.","url":"https://docs.langchain.com/oss/python/langgraph/overview"},{"name":"Checkpointer","summary":"Persistence layer that saves state at each super-step so executions are durable and resumable.","maps_to_pattern":"agent-resumption","url":"https://docs.langchain.com/oss/python/langgraph/durable-execution"},{"name":"interrupt()","summary":"Pauses graph execution at a node and waits indefinitely for external (human) input before continuing.","maps_to_pattern":"human-in-the-loop","url":"https://docs.langchain.com/oss/python/langgraph/interrupts"},{"name":"Command","summary":"Return value from a node combining a state update with a goto routing decision; Command.PARENT crosses subgraph boundaries.","maps_to_pattern":"handoff","url":"https://docs.langchain.com/oss/python/langchain/multi-agent/handoffs"},{"name":"create_react_agent","summary":"Prebuilt tool-calling ReAct-style agent that loops over tool calls until the model emits a final answer.","maps_to_pattern":"react","url":"https://langchain-ai.github.io/langgraph/reference/prebuilt/"},{"name":"Subgraphs","summary":"Nested graphs that let a node delegate to a complete inner graph (the basis for hierarchical agents).","maps_to_pattern":"hierarchical-agents"},{"name":"Store (long-term memory)","summary":"Cross-thread persistence of long-term memories under custom namespaces and keys.","maps_to_pattern":"cross-session-memory","url":"https://docs.langchain.com/oss/python/langgraph/memory"},{"name":"recursion_limit / GraphRecursionError","summary":"Cap on total super-steps (default 25); exceeding it raises GraphRecursionError.","maps_to_pattern":"step-budget","url":"https://docs.langchain.com/oss/python/langgraph/errors/GRAPH_RECURSION_LIMIT"}],"pattern_composition":"flowchart TD\n  USER[User input] --> SG[StateGraph]\n  SG --> NODE1[Node A<br/>read/write state]\n  SG --> NODE2[create_react_agent<br/>ReAct tool loop]\n  SG --> NODE3[Tool node]\n  NODE2 -->|Command(goto=...)| HANDOFF[Routed to other node]\n  NODE3 -.->|interrupt()| HITL[Paused — wait for human]\n  HITL -.->|resume| NODE3\n  SG -->|every super-step| CKPT[(Checkpointer<br/>persists state)]\n  CKPT -.->|resume after failure| SG\n  CKPT -.->|checkpoint_id| REPLAY[Replay / time-travel]\n  SG --> MEM_S[Short-term<br/>thread-scoped]\n  SG --> STORE[(Store<br/>long-term cross-session)]\n  SG -.->|Subgraph| INNER[Nested StateGraph]\n  SG --> SUP[langgraph-supervisor<br/>create_supervisor]\n  SG --> SWARM[langgraph-swarm<br/>create_swarm]\n  SG -.recursion_limit.-> LIMIT[GraphRecursionError]\n  SG --> OUT[Final state]","members":[{"pattern":"agent-resumption","role":"core","note":"Checkpointer persists graph state at each super-step; on restart the agent resumes from where it left off, even after failure.","evidence":[{"type":"repo","title":"LangGraph README — Why use LangGraph","url":"https://github.com/langchain-ai/langgraph","quote":"Build agents that persist through failures and can run for extended periods, automatically resuming from exactly where they left off.","accessed":"2026-05-20"},{"type":"doc","title":"Durable execution","url":"https://docs.langchain.com/oss/python/langgraph/durable-execution","quote":"LangGraph's built-in persistence layer provides durable execution for workflows, ensuring that the state of each execution step is saved to a durable store.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"human-in-the-loop","role":"core","note":"interrupt() pauses the graph and waits indefinitely until an external caller resumes it; state can be inspected and modified before resume.","evidence":[{"type":"doc","title":"Interrupts","url":"https://docs.langchain.com/oss/python/langgraph/interrupts","quote":"Interrupts allow you to pause graph execution at specific points and wait for external input before continuing.","accessed":"2026-05-20"},{"type":"doc","title":"Interrupts — indefinite wait","url":"https://docs.langchain.com/oss/python/langgraph/interrupts","quote":"When an interrupt is triggered, LangGraph saves the graph state using its persistence layer and waits indefinitely until you resume execution.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"handoff","role":"first-class","note":"Multi-agent handoff is expressed by a node returning Command(goto=<target>, update=...) with optional state updates; Command.PARENT scopes the goto across subgraph boundaries.","evidence":[{"type":"doc","title":"Handoffs — Command","url":"https://docs.langchain.com/oss/python/langchain/multi-agent/handoffs","quote":"return Command(goto='sales_agent', update={'active_agent': 'sales_agent', 'messages': [last_ai_message, transfer_message]}, graph=Command.PARENT)","accessed":"2026-05-20"},{"type":"repo","title":"langgraph-swarm README","url":"https://github.com/langchain-ai/langgraph-swarm-py","quote":"A swarm is a type of multi-agent architecture where agents dynamically hand off control to one another based on their specializations.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"session-isolation","role":"first-class","note":"Short-term memory is thread-scoped: each thread holds its own checkpointed state and message history; thread_id is required to scope persistence per session.","evidence":[{"type":"doc","title":"Memory — short-term thread-scoped memory","url":"https://docs.langchain.com/oss/python/langgraph/memory","quote":"Short-term memory, or thread-scoped memory, tracks the ongoing conversation by maintaining message history within a session.","accessed":"2026-05-20"},{"type":"doc","title":"LangGraph — thread identifier","url":"https://docs.langchain.com/oss/python/langgraph/persistence","quote":"A thread is a unique ID or thread identifier assigned to each checkpoint saved by a checkpointer.","accessed":"2026-05-24"},{"type":"doc","title":"LangGraph — thread_id required","url":"https://docs.langchain.com/oss/python/langgraph/persistence","quote":"When invoking a graph with a checkpointer, you **must** specify a `thread_id` as part of the `configurable` portion of the config","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"cross-session-memory","role":"first-class","note":"LangGraph provides a Store for long-term memory keyed by namespace+key, shared across threads/sessions.","evidence":[{"type":"doc","title":"Memory — long-term cross-session","url":"https://docs.langchain.com/oss/python/langgraph/memory","quote":"Long-term memory stores user-specific or application-level data across sessions and is shared across conversational threads.","accessed":"2026-05-20"},{"type":"doc","title":"Memory — Store and namespaces","url":"https://docs.langchain.com/oss/python/langgraph/memory","quote":"Each memory is organized under a custom namespace (similar to a folder) and a distinct key (like a file name).","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"supervisor","role":"first-class","note":"langgraph-supervisor prebuilt library exposes create_supervisor; a central supervisor agent picks which worker agent runs next.","evidence":[{"type":"repo","title":"langgraph-supervisor README","url":"https://github.com/langchain-ai/langgraph-supervisor-py","quote":"Hierarchical systems are a type of multi-agent architecture where specialized agents are coordinated by a central supervisor agent.","accessed":"2026-05-20"},{"type":"repo","title":"langgraph-supervisor — control flow","url":"https://github.com/langchain-ai/langgraph-supervisor-py","quote":"The supervisor controls all communication flow and task delegation, making decisions about which agent to invoke based on the current context.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"hierarchical-agents","role":"first-class","note":"Subgraphs let a node delegate to a complete inner graph; langgraph-supervisor markets itself as 'hierarchical multi-agent systems'.","evidence":[{"type":"repo","title":"langgraph-supervisor — hierarchical","url":"https://github.com/langchain-ai/langgraph-supervisor-py","quote":"A Python library for creating hierarchical multi-agent systems using LangGraph.","accessed":"2026-05-20"},{"type":"doc","title":"LangGraph — subgraph as node","url":"https://docs.langchain.com/oss/python/langgraph/use-subgraphs","quote":"A subgraph is a [graph](/oss/python/langgraph/graph-api#graphs) that is used as a [node](/oss/python/langgraph/graph-api#nodes) in another graph.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"swarm","role":"first-class","note":"langgraph-swarm provides create_swarm: agents dynamically hand off control to one another and a shared 'messages' state.","evidence":[{"type":"repo","title":"langgraph-swarm — architecture","url":"https://github.com/langchain-ai/langgraph-swarm-py","quote":"A swarm is a type of multi-agent architecture where agents dynamically hand off control to one another based on their specializations.","accessed":"2026-05-20"},{"type":"repo","title":"langgraph-swarm — last-active","url":"https://github.com/langchain-ai/langgraph-swarm-py","quote":"The system remembers which agent was last active, ensuring that on subsequent interactions, the conversation resumes with that agent.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"create_react_agent is the prebuilt tool-calling ReAct-style agent constructor in langgraph.prebuilt; its docstring describes a graph that loops over tool calls until a stopping condition is met.","evidence":[{"type":"doc","title":"LangGraph prebuilt — ReAct-style agent","url":"https://langchain-ai.github.io/langgraph/reference/prebuilt/","quote":"tool-calling ReAct-style agent","accessed":"2026-05-20"},{"type":"repo","title":"langgraph — create_react_agent docstring","url":"https://github.com/langchain-ai/langgraph/blob/main/libs/prebuilt/langgraph/prebuilt/chat_agent_executor.py","quote":"Creates an agent graph that calls tools in a loop until a stopping condition is met.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"create_react_agent and the underlying StateGraph wire tool nodes that the model can invoke through LangChain tool bindings.","evidence":[{"type":"doc","title":"LangGraph prebuilt — ReAct-style agent","url":"https://langchain-ai.github.io/langgraph/reference/prebuilt/","quote":"tool-calling ReAct-style agent","accessed":"2026-05-20"},{"type":"repo","title":"langgraph — create_react_agent tool loop","url":"https://github.com/langchain-ai/langgraph/blob/main/libs/prebuilt/langgraph/prebuilt/chat_agent_executor.py","quote":"Creates an agent graph that calls tools in a loop until a stopping condition is met.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"step-budget","role":"first-class","note":"recursion_limit caps total super-steps (default 25); exceeding it raises GraphRecursionError.","evidence":[{"type":"doc","title":"GRAPH_RECURSION_LIMIT error","url":"https://docs.langchain.com/oss/python/langgraph/errors/GRAPH_RECURSION_LIMIT","quote":"Your LangGraph StateGraph reached the maximum number of steps before hitting a stop condition.","accessed":"2026-05-20"},{"type":"doc","title":"GRAPH_RECURSION_LIMIT — override","url":"https://docs.langchain.com/oss/python/langgraph/errors/GRAPH_RECURSION_LIMIT","quote":"If you have a complex graph, you can pass in a higher recursion_limit value into your config object when invoking your graph","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"replay-time-travel","role":"first-class","note":"Per-super-step checkpoints expose a state history; graph.get_state_history(config) returns the full execution history and the graph can be invoked with a prior checkpoint_id to replay.","evidence":[{"type":"doc","title":"Persistence — replay","url":"https://docs.langchain.com/oss/python/langgraph/persistence","quote":"Replay re-executes steps from a prior checkpoint. Invoke the graph with a prior checkpoint_id to re-run nodes after that checkpoint.","accessed":"2026-05-20"},{"type":"doc","title":"Persistence — history","url":"https://docs.langchain.com/oss/python/langgraph/persistence","quote":"You can get the full history of the graph execution for a given thread by calling graph.get_state_history(config).","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"approval-queue","role":"first-class","note":"interrupt() at a tool-call boundary lets the caller review and accept/reject/modify the proposed action; this is LangGraph's documented HITL approval shape.","evidence":[{"type":"repo","title":"LangGraph README — inspect & modify","url":"https://github.com/langchain-ai/langgraph","quote":"Seamlessly incorporate human oversight by inspecting and modifying agent state at any point during execution.","accessed":"2026-05-20"},{"type":"doc","title":"LangGraph — pause before tool execution","url":"https://docs.langchain.com/oss/python/langgraph/interrupts","quote":"pause before executing tool calls to review and edit the tool call before execution","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"orchestrator-workers","role":"first-class","note":"Supervisor-with-workers is the canonical orchestrator-workers topology in LangGraph, shipped via langgraph-supervisor.","evidence":[{"type":"repo","title":"langgraph-supervisor — delegation","url":"https://github.com/langchain-ai/langgraph-supervisor-py","quote":"The supervisor controls all communication flow and task delegation, making decisions about which agent to invoke based on the current context.","accessed":"2026-05-20"},{"type":"repo","title":"langgraph-supervisor — create_supervisor","url":"https://github.com/langchain-ai/langgraph-supervisor-py","quote":"Create a supervisor agent to orchestrate multiple specialized agents","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"plan-and-execute","role":"supported","note":"Plan-and-execute is documented as an idiomatic LangGraph shape (planner node → executor nodes → replan), built on the StateGraph; the canonical LangChain Inc. blog post on planning agents describes it as a planner + executor split.","evidence":[{"type":"repo","title":"LangGraph README — stateful agent foundation","url":"https://github.com/langchain-ai/langgraph","quote":"Low-level orchestration framework for building stateful agents.","accessed":"2026-05-20"},{"type":"blog","title":"LangChain blog — Planning agents (planner + executor)","url":"https://www.langchain.com/blog/planning-agents","quote":"A planner, which prompts an LLM to generate a multi-step plan to complete a large task.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"subagent-isolation","role":"limited","note":"Subgraphs scope state per inner graph; isolation is achieved by giving the subgraph a different state schema (no shared keys) or by transforming state between parent and child.","evidence":[{"type":"doc","title":"Handoffs — Command.PARENT","url":"https://docs.langchain.com/oss/python/langchain/multi-agent/handoffs","quote":"return Command(goto='sales_agent', update={'active_agent': 'sales_agent', 'messages': [last_ai_message, transfer_message]}, graph=Command.PARENT)","accessed":"2026-05-20"},{"type":"doc","title":"LangGraph — subgraph isolation via different schemas","url":"https://docs.langchain.com/oss/python/langgraph/use-subgraphs","quote":"Parent and subgraph have **different state schemas** (no shared keys), or you need to transform state between them","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"mcp","role":"supported","note":"LangGraph integrates MCP through LangChain's langchain-mcp-adapters (MultiServerMCPClient), which converts MCP tools into LangChain tools usable by create_react_agent.","evidence":[{"type":"repo","title":"langchain-mcp-adapters — convert MCP tools","url":"https://github.com/langchain-ai/langchain-mcp-adapters","quote":"Convert MCP tools into LangChain tools that can be used with LangGraph agents","accessed":"2026-05-24"},{"type":"repo","title":"langchain-mcp-adapters — multi-server","url":"https://github.com/langchain-ai/langchain-mcp-adapters","quote":"The library also allows you to connect to multiple MCP servers and load tools from them","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"composable-termination-conditions","role":"supported","note":"interrupt() pauses graph execution and returns to the caller; interrupt_before / interrupt_after pin pause points on specific nodes. Composes naturally with the checkpointer to preserve state across the pause.","evidence_status":"full","evidence":[{"type":"doc","title":"LangGraph — Interrupts","url":"https://docs.langchain.com/oss/python/langgraph/interrupts","quote":"The interrupt() function pauses graph execution and returns a value to the caller.","accessed":"2026-05-23"},{"type":"doc","title":"LangGraph — interrupt_before","url":"https://docs.langchain.com/oss/python/langgraph/interrupts","quote":"interrupt_before specifies the nodes where execution should pause before the node is executed.","accessed":"2026-05-23"}]},{"pattern":"interruptible-agent-execution","role":"first-class","note":"interrupt() + checkpointer is the canonical pattern: graph execution pauses, state is durable in the checkpointer, and the caller resumes by re-invoking with the same thread id.","evidence_status":"full","evidence":[{"type":"doc","title":"LangGraph — Interrupts","url":"https://docs.langchain.com/oss/python/langgraph/interrupts","quote":"The interrupt() function pauses graph execution and returns a value to the caller.","accessed":"2026-05-23"},{"type":"doc","title":"LangGraph — Checkpointers","url":"https://reference.langchain.com/python/langgraph/checkpoints","quote":"Checkpoints allow LangGraph agents to persist their state within and across multiple interactions; a checkpoint is a snapshot of the graph state at a given point in time.","accessed":"2026-05-23"}]},{"pattern":"journaled-llm-call","role":"supported","note":"LangGraph durable execution journals non-deterministic step results so a resumed workflow reuses recorded values rather than re-running the model.","evidence_status":"limited","evidence":[{"type":"repo","title":"langchain-ai/langgraph - README","url":"https://github.com/langchain-ai/langgraph","quote":"Build agents that persist through failures and can run for extended periods, automatically resuming from exactly where they left off.","accessed":"2026-06-17"}]},{"pattern":"reversibility-aware-action-filter","role":"limited","note":"Its human-in-the-loop interrupt middleware sits between the policy and the environment and pauses side-effecting tool calls (file writes, SQL) for review, gating the irreversible step rather than re-sampling, but it does not itself classify actions by a reversibility estimate.","evidence":[{"type":"doc","title":"Human-in-the-loop - Docs by LangChain","url":"https://docs.langchain.com/oss/python/langchain/human-in-the-loop","quote":"When a model proposes an action that might require review—for example, writing to a file or executing SQL—the middleware can pause execution and wait for a decision.","accessed":"2026-06-14"}],"evidence_status":"limited"},{"pattern":"short-term-memory","role":"core","note":"LangGraph checkpointers give an agent thread-scoped short-term memory, persisting the conversation state per thread_id and replaying the relevant slice into context across turns within a session.","evidence":[{"type":"doc","title":"Persistence - Docs by LangChain","url":"https://docs.langchain.com/oss/python/langgraph/persistence","quote":"Checkpointers persist a thread's graph state as checkpoints. Use them for short-term, thread-scoped memory, including conversation continuity, human-in-the-loop workflows, time travel, and fault tolerance.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"map-reduce","role":"first-class","note":"The Send API fans out an arbitrary number of parallel subgraph runs over a runtime collection (map), and a state reducer combines the workers' results (reduce).","evidence":[{"type":"doc","title":"Map-Reduce with the Send() API (LangGraph blog/tutorial summary)","url":"https://www.langchain.com/blog/planning-agents","quote":"The map phase uses the Send API to create an arbitrary number of parallel executions based on runtime data, where the Send object specifies the target node and the state to pass to it.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"rewoo","role":"supported","note":"A planner emits a complete plan up front with placeholder variables (#E1, #E2) standing in for tool outputs; a worker later executes the tools and substitutes the observations into the plan.","evidence":[{"type":"blog","title":"Plan-and-Execute Agents (ReWOO section)","url":"https://www.langchain.com/blog/planning-agents","quote":"The planner generates a plan list consisting of interleaving 'Plan' (reasoning) and 'E#' lines ... the planner can reference previous outputs using syntax like `#E2` ... [the worker] replaces variables with their results when calling subsequent calls.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"llm-compiler","role":"supported","note":"A planner streams a DAG of tasks with dependencies; a Task Fetching Unit substitutes placeholder dependencies and dispatches ready tasks to an executor for parallel function calling.","evidence":[{"type":"blog","title":"Plan-and-Execute Agents (LLMCompiler section)","url":"https://www.langchain.com/blog/planning-agents","quote":"The Planner streams a DAG of tasks. Each task contains a tool, arguments, and list of dependencies. [The] Task Fetching Unit schedules and executes the tasks [by] scheduling tasks once their dependencies are met.","accessed":"2026-06-17"},{"type":"repo","title":"SqueezeAILab/LLMCompiler","url":"https://github.com/SqueezeAILab/LLMCompiler","quote":"LLMCompiler is a framework that enables an efficient and effective orchestration of parallel function calling with LLMs ... LLMCompiler automatically computes an optimized orchestration for the function calls.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"lats","role":"supported","note":"Lifts the agent loop into a Monte-Carlo tree search where the LLM acts as action generator, value function, and reflection mechanism, with selection/expansion/backpropagation and backtracking.","evidence":[{"type":"blog","title":"Reflection Agents (LATS)","url":"https://www.langchain.com/blog/reflection-agents","quote":"Language Agent Tree Search (LATS) ... combines reflection/evaluation and search (specifically Monte-Carlo trees search) to achieve better overall task performance ... replacing the RL agents, value functions, and optimizer all with calls to an LLM.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"single-path-plan-generator","role":"supported","note":"A planner prompts the LLM to generate one linear multi-step plan up front, then an executor goes through that plan one item at a time.","evidence":[{"type":"blog","title":"Plan-and-Execute Agents","url":"https://www.langchain.com/blog/planning-agents","quote":"A planner, which prompts an LLM to generate a multi-step plan to complete a large task ... The core idea is to first come up with a multi-step plan, and then go through that plan one item at a time.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"human-reflection","role":"first-class","note":"interrupt() pauses the graph so a human can review and edit a proposed tool call/plan or reject-with-feedback, and that human message/edit is resumed back into the agent's state — feedback that revises the plan, not a binary approval gate.","evidence":[{"type":"doc","title":"Human-in-the-loop - LangChain Docs","url":"https://docs.langchain.com/oss/python/langchain/human-in-the-loop","quote":"Use `edit` to modify the tool call before execution. ... reject ... explain why the action was rejected and whether the agent should retry. ... Respond: Tool execution is skipped; the human's message becomes the tool result.","accessed":"2026-06-17"},{"type":"blog","title":"Making it easier to build human-in-the-loop agents with interrupt","url":"https://www.langchain.com/blog/making-it-easier-to-build-human-in-the-loop-agents-with-interrupt","quote":"Review Tool Calls: A human can review and edit the output from the LLM before proceeding.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"planner-executor-observer","role":"supported","note":"LangGraph Plan-and-Execute: a planner emits a multi-step plan, an executor runs each step, and the agent is re-invoked to re-plan or finish.","evidence":[{"type":"doc","url":"https://www.langchain.com/blog/planning-agents","quote":"Once execution is completed, the agent is called again with a re-planning prompt, letting it decide whether to finish with a response or whether to generate a follow-up plan","accessed":"2026-06-19"}],"evidence_status":"full"},{"pattern":"critic","role":"supported","note":"LangGraph Reflexion: an actor critiques each response, grounds the critique in external data, and enumerates missing/superfluous aspects to drive revision.","evidence":[{"type":"doc","url":"https://www.langchain.com/blog/reflection-agents","quote":"the actor agent explicitly critiques each response and grounds its criticism in external data. It is forced to generate citations and explicitly enumerate superfluous and missing aspects of the generated response.","accessed":"2026-06-19"}],"evidence_status":"full"},{"pattern":"control-flow-integrity","role":"supported","note":"Stateful graph fixes edges at compile time; node outputs cannot rewire the graph at runtime. Cited as a CFI-style defence in Del Rosario et al. (2025).","evidence":[{"type":"doc","url":"https://langchain-ai.github.io/langgraph/","quote":"Stateful graph fixes edges at compile time; node outputs cannot rewire the graph at runtime. Cited as a CFI-style defence in Del Rosario et al. (2025).","accessed":"2026-06-19"}],"evidence_status":"full"}],"instantiates":["long-running-autonomous-agent","multi-agent-coordination"],"alternatives":[{"composition":"autogen","relation":"competes-with","note":"Graph-of-nodes-over-checkpointed-state vs event-driven actors + group chat."},{"composition":"crewai","relation":"competes-with","note":"Lower-level graph runtime vs role-driven multi-agent crew."},{"composition":"burr","relation":"similar-shape","note":"Both model agents as graphs over a typed state object with checkpointing."},{"composition":"microsoft-agent-framework","relation":"competes-with","note":"Both offer graph-based agent/workflow runtimes."},{"composition":"langchain","relation":"same-vendor","note":"create_agent compiles to LangGraph internally; LangChain is the high-level surface, LangGraph the low-level runtime."},{"composition":"llamaindex","relation":"similar-shape","note":"Both express orchestration as a graph over typed state with checkpointable execution."},{"composition":"mastra","relation":"similar-shape","note":"Both ship a graph engine with checkpointed/suspendable state; LangGraph in Py+TS, Mastra in TS-first."},{"composition":"pydantic-ai","relation":"competes-with","note":"Both target production-grade agents; LangGraph delivers durability via built-in Checkpointer, Pydantic AI delegates to Temporal/DBOS/Prefect/Restate."},{"composition":"smolagents","relation":"competes-with","note":"Same vendor space (LangChain Inc.); graph-of-nodes shape with checkpointed durable state — smolagents is much smaller and has no checkpointer."},{"composition":"deerflow-v1","relation":"complements","note":"DeerFlow's graph is built on LangGraph, which supplies the stateful nodes, conditional edges, and the interrupt() primitive behind the human-feedback plan review."}],"references":[{"type":"repo","title":"langchain-ai/langgraph","url":"https://github.com/langchain-ai/langgraph","quote":"Low-level orchestration framework for building stateful agents.","accessed":"2026-05-20"},{"type":"doc","title":"Durable execution","url":"https://docs.langchain.com/oss/python/langgraph/durable-execution","quote":"LangGraph's built-in persistence layer provides durable execution for workflows, ensuring that the state of each execution step is saved to a durable store.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-06-14","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","graph-based","stateful","checkpointed"],"anti_patterns_avoided":[{"pattern":"blocking-sync-calls-in-agent-loop","note":"Async nodes plus asyncio.to_thread wrap blocking I/O so a synchronous call does not stall the agent event loop.","evidence":[{"type":"doc","title":"LangGraph — fault tolerance","url":"https://docs.langchain.com/oss/python/langgraph/fault-tolerance","quote":"To wrap blocking I/O, use asyncio.to_thread inside an async node.","accessed":"2026-06-19"}]},{"pattern":"replay-divergence","note":"Wrapping side-effecting operations as durable tasks means a resumed run retrieves their persisted results instead of re-executing them, avoiding divergent replay.","evidence":[{"type":"doc","title":"LangGraph — durable execution","url":"https://docs.langchain.com/oss/python/langgraph/durable-execution","quote":"wrap each operation in a separate task. This ensures that when the workflow is resumed, the operations are not repeated, and their results are retrieved from the persistence layer.","accessed":"2026-06-19"}]}]},{"id":"letta","name":"Letta","aliases":["MemGPT"],"kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"Letta","language":"Python","license":"Apache-2.0","status":"active","url":"https://docs.letta.com/","repo":"https://github.com/letta-ai/letta","docs_url":"https://docs.letta.com/","intent":"Build stateful LLM agents that remember, learn, and improve over time by self-managing a tiered memory (in-context blocks plus archival/recall stores) via tool calls.","description":"Letta (formerly MemGPT) is the open-source platform created by the authors of the MemGPT paper for building stateful agents with advanced memory. Letta agents process messages through a tool-calling loop in which the model can read and edit its own memory: in-context memory blocks live inside the prompt, archival memory is a vector store queried on demand, and recall memory logs full conversational history. All agent state — memory blocks, messages, reasoning, and tool calls — is persisted in a database so nothing is lost even when content is evicted from the context window. Letta exposes its agents as a stateful REST API.","primary_use_cases":["long-running personal/companion agents that retain user context across sessions","research agents that accumulate facts in archival memory","stateful customer-support agents exposed as a REST service","MemGPT-style virtual-context experiments and continual learning"],"agent_loop_shape":"Tool-calling loop on top of MemGPT-style virtual context. Memory blocks are always-visible XML-like sections prepended to the prompt; the agent can call memory tools to edit them, archive content, or search recall/archival memory. All state — memory, messages, reasoning, tool calls — is written to a database after each step, so the agent persists across server restarts and is addressable through the Letta REST API.","key_concepts":[{"name":"Memory blocks (core memory)","summary":"Structured sections of the prompt that persist across all interactions and are always visible to the agent; the agent edits them via memory tools.","maps_to_pattern":"cross-session-memory","url":"https://docs.letta.com/guides/agents/memory-blocks"},{"name":"Archival memory","summary":"Vector-DB store of long-term facts and knowledge; queried on demand via tools because contents cannot be pinned to the context window.","maps_to_pattern":"vector-memory","url":"https://docs.letta.com/guides/agents/archival-memory/"},{"name":"Recall memory","summary":"Database table that logs the agent's full conversational history; surfaces older messages when needed.","url":"https://docs.letta.com/advanced/memory-management/"},{"name":"MemGPT virtual context","summary":"OS-inspired paging mechanism from the MemGPT paper: the agent self-manages what lives in the limited context window via tool calls.","maps_to_pattern":"memgpt-paging","url":"https://www.letta.com/research"},{"name":"Letta REST API / stateful messages.create","summary":"Agents are addressed as stateful resources; client SDKs hit a REST endpoint that processes one user message and returns assistant, reasoning, tool-call, and tool-return messages.","url":"https://docs.letta.com/api-reference/agents/messages/create"}],"pattern_composition":"flowchart TD\n  USER[User message] --> API[POST agents/messages/create]\n  API --> AGENT[Letta agent loop]\n  AGENT --> CTX{Prompt = memory blocks + recent msgs}\n  AGENT -->|tool call| TOOLS[Tools<br/>memory edit / search / user-defined]\n  TOOLS --> CORE[Memory blocks<br/>always visible]\n  TOOLS --> ARCH[(Archival memory<br/>vector DB)]\n  TOOLS --> RECALL[(Recall memory<br/>conversation log)]\n  AGENT --> RESP[Assistant / Reasoning /<br/>ToolCall / ToolReturn messages]\n  RESP --> DB[(Database<br/>all state persisted)]\n  DB -.resume.-> AGENT\n  ARCH -.semantic search.-> AGENT\n  RECALL -.paged in.-> AGENT","members":[{"pattern":"memgpt-paging","role":"core","note":"Letta is the production reimplementation of the MemGPT paper: the agent self-manages a finite context window via tool calls that page content between core, recall, and archival memory.","evidence":[{"type":"blog","title":"Letta research — MemGPT origin","url":"https://www.letta.com/research","quote":"MemGPT: Towards LLMs as Operating Systems","accessed":"2026-05-20"},{"type":"repo","title":"letta-ai/letta README","url":"https://github.com/letta-ai/letta","quote":"Letta (formerly MemGPT)","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"cross-session-memory","role":"core","note":"Memory blocks plus archival and recall memory persist across all interactions; the agent retains user/world facts between sessions.","evidence":[{"type":"doc","title":"Letta — memory blocks","url":"https://docs.letta.com/guides/agents/memory-blocks","quote":"Memory blocks are structured sections of the agent's context window that persist across all interactions.","accessed":"2026-05-20"},{"type":"doc","title":"Letta — agent memory","url":"https://docs.letta.com/guides/agents/memory/","quote":"All state, includes memories, user messages, reasoning, tool calls, are all persisted in a database, so they are never lost, even once evicted from the context window","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"vector-memory","role":"first-class","note":"Archival memory is a semantically searchable vector DB that the agent queries on demand via tools when content cannot fit in core memory.","evidence":[{"type":"doc","title":"Letta — archival memory","url":"https://docs.letta.com/guides/agents/archival-memory/","quote":"Archival memory is a semantically searchable database where agents can store facts, knowledge, and information for long-term retrieval.","accessed":"2026-05-20"},{"type":"doc","title":"Letta — archival on demand","url":"https://docs.letta.com/guides/agents/archival-memory/","quote":"Unlike memory blocks, archival memory fragments cannot be pinned to the context window, and must be queried on-demand via tools.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"agent-resumption","role":"first-class","note":"All agent state (memory, messages, reasoning, tool calls) is persisted in a database after every interaction, so the agent resumes from where it was on the next API call or after a restart.","evidence":[{"type":"doc","title":"Letta — persistence","url":"https://docs.letta.com/guides/agents/memory/","quote":"All state, includes memories, user messages, reasoning, tool calls, are all persisted in a database, so they are never lost, even once evicted from the context window","accessed":"2026-05-20"},{"type":"doc","title":"Letta — messages.create endpoint","url":"https://docs.letta.com/api-reference/agents/messages/create","quote":"Process a user message and return the agent's response. This endpoint accepts a message from a user and processes it through the agent.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"core","note":"Letta agents act exclusively through tool calls — memory edits, searches, and user-defined tools; the messages.create response surfaces ToolCallMessage and ToolReturnMessage.","evidence":[{"type":"doc","title":"Letta — ToolCall / ToolReturn messages","url":"https://docs.letta.com/api-reference/agents/messages/create","quote":"A message representing a request to call a tool (generated by the LLM to trigger tool execution)","accessed":"2026-05-20"},{"type":"doc","title":"Letta — client-side tools","url":"https://docs.letta.com/api-reference/agents/messages/create","quote":"Client-side tools that the agent can call. When the agent calls a client-side tool, execution pauses and returns control to the client to execute the tool and provide the result via a ToolReturn.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"five-tier-memory-cascade","role":"limited","note":"Letta exposes a tiered memory (in-context blocks, recall, archival) inherited from MemGPT; not exactly five tiers but the same cascade idea. Mapping is approximate, not vendor-named.","evidence":[{"type":"doc","title":"Letta — memory blocks vs archival","url":"https://docs.letta.com/guides/agents/memory-blocks","quote":"Memory blocks are Letta's core abstraction.","accessed":"2026-05-20"},{"type":"doc","title":"Letta — archival","url":"https://docs.letta.com/guides/agents/archival-memory/","quote":"Archival memory is a semantically searchable database where agents can store facts, knowledge, and information for long-term retrieval.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"structured-output","role":"first-class","note":"Letta supports JSON mode and JSON-schema strict outputs via response_format in model_settings; the setting persists as part of agent state. The messages.create response is also a typed envelope of Assistant/Reasoning/ToolCall/ToolReturn messages.","evidence":[{"type":"doc","title":"Letta — structured outputs / response_format","url":"https://docs.letta.com/guides/agents/json-mode","quote":"Letta supports structured JSON output from agents using the `response_format` parameter in `model_settings`.","accessed":"2026-05-24"},{"type":"doc","title":"Letta — response_format persistence","url":"https://docs.letta.com/guides/agents/json-mode","quote":"Once set, `response_format` becomes a persistent part of the agent's state - all future responses will follow the format until explicitly changed","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"sleep-time-compute","role":"first-class","note":"Letta lets you create sleep-time agents that share a primary agent's memory and run in the background during downtime to reorganize and pre-compute over context, shifting load off the latency-sensitive user-query path.","evidence":[{"type":"blog","title":"Sleep-time Compute | Letta","url":"https://www.letta.com/blog/sleep-time-compute","quote":"agents should be running even while they 'sleep', using their downtime to reorganize information and reason through the information they have available in advance.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"hippocampal-rehearsal","role":"supported","note":"Letta/MemGPT stores fragments in archival memory that cannot be pinned and must be reactivated on demand via archival_memory_search.","evidence":[{"type":"doc","url":"https://docs.letta.com/guides/core-concepts/memory/archival-memory","quote":"Unlike memory blocks, archival memory fragments cannot be pinned to the context window, and must be queried on-demand via tools.","accessed":"2026-06-19"}],"evidence_status":"full"}],"references":[{"type":"repo","title":"letta-ai/letta","url":"https://github.com/letta-ai/letta","quote":"Letta is the platform for building stateful agents: AI with advanced memory that can learn and self-improve over time.","accessed":"2026-05-20"},{"type":"blog","title":"Letta research — founding team","url":"https://www.letta.com/research","quote":"Founded by the creators of MemGPT from UC Berkeley's Sky Computing Lab","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-14","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","memory","memgpt","stateful"]},{"id":"llamaindex","name":"LlamaIndex","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"LlamaIndex Inc.","language":"Python, TypeScript","license":"MIT","status":"active","first_released":"2022-11-02","url":"https://www.llamaindex.ai/","repo":"https://github.com/run-llama/llama_index","docs_url":"https://developers.llamaindex.ai/python/framework/","intent":"Provide an open-source Python/TypeScript framework for context-augmented LLM and agent applications combining RAG primitives (data connectors, indexes, query engines, retrievers, rerankers) with an event-driven Workflow runtime and a multi-agent AgentWorkflow orchestrator over FunctionAgent / ReActAgent / CodeActAgent prebuilts.","description":"LlamaIndex began as the leading data framework for retrieval-augmented LLM applications and has, since 2024, grown an explicit agent layer on top of that retrieval substrate. Today LlamaIndex Inc. ships LlamaIndex OSS (MIT, Python and TypeScript) as 'an open-source framework to build agentic applications' with Workflows as the underlying event-driven runtime and AgentWorkflow as a multi-agent orchestrator. The framework positions RAG as one tool an agent can call rather than the whole application: 'LlamaIndex provides a framework for building agents including the ability to use RAG pipelines as one of many tools to complete a task'.","primary_use_cases":["agentic RAG with multiple retrievers and rerankers","event-driven Workflow orchestration","multi-agent topologies via AgentWorkflow with handoff","graph-RAG via PropertyGraphIndex"],"agent_loop_shape":"Event-driven Workflow: steps decorated with @step are triggered by typed Events and emit further Events. On top, prebuilt agents FunctionAgent (native tool calling) and ReActAgent (prompted ReAct) plus the multi-agent AgentWorkflow orchestrator routing via can_handoff_to and a root_agent.","key_concepts":[{"name":"Workflow","summary":"Event-driven, step-based execution runtime.","maps_to_pattern":"event-driven-agent","url":"https://developers.llamaindex.ai/python/framework/understanding/workflows/"},{"name":"@step + Event","summary":"Pydantic events trigger @step-decorated methods; events flow between steps to drive control.","url":"https://developers.llamaindex.ai/python/framework/understanding/workflows/"},{"name":"FunctionAgent","summary":"Native function-calling agent that uses provider tool/function APIs.","maps_to_pattern":"tool-use","url":"https://developers.llamaindex.ai/python/framework/module_guides/deploying/agents/"},{"name":"ReActAgent","summary":"Reason+Act prompted agent for models without native tool calling.","maps_to_pattern":"react","url":"https://developers.llamaindex.ai/python/examples/agent/react_agent"},{"name":"AgentWorkflow","summary":"Orchestrator for one or more agents with can_handoff_to routing and a designated root_agent.","maps_to_pattern":"orchestrator-workers","url":"https://developers.llamaindex.ai/python/examples/agent/agent_workflow_basic"},{"name":"Index / QueryEngine","summary":"RAG building blocks; VectorStoreIndex, PropertyGraphIndex, etc.","maps_to_pattern":"agentic-rag","url":"https://developers.llamaindex.ai/python/framework/understanding/rag/"}],"pattern_composition":"flowchart TD\n  USER[User input] --> AW[AgentWorkflow]\n  ROOT[root_agent] --> AW\n  FA[FunctionAgent] --> AW\n  RA[ReActAgent] --> AW\n  CA[CodeActAgent] --> AW\n  AW -->|can_handoff_to| AW\n  AW -->|@step + Event| WF[(Workflow runtime<br/>event-driven)]\n  WF -->|WorkflowCheckpointer| CKPT[(Checkpoints<br/>run_from)]\n  WF -->|Context| STATE[(Per-run state<br/>serializable)]\n  TOOLS[FunctionTool / QueryEngineTool] --> AW\n  MCP[get_tools_from_mcp_url] --> TOOLS\n  QE[QueryEngine] --> TOOLS\n  IDX[(VectorStoreIndex<br/>PropertyGraphIndex)] --> QE\n  RET[QueryFusionRetriever<br/>vector + BM25] --> QE\n  RERANK[SentenceTransformerRerank<br/>CohereRerank] --> QE\n  AW -->|Pydantic structured output| OUT[Final answer]\n  AW -->|workflow_as_mcp| MCPOUT[Expose as MCP server]","members":[{"pattern":"agent-resumption","role":"limited","note":"WorkflowCheckpointer.run() wraps Workflow.run() and injects checkpoint callbacks; resume via run_from(). No automatic per-step checkpointing.","evidence":[{"type":"doc","title":"LlamaIndex — Checkpointing workflows","url":"https://developers.llamaindex.ai/python/examples/workflow/checkpointing_workflows/","quote":"The `WorkflowCheckpointer.run()` method is a wrapper over the `Workflow.run()` method, which injects a checkpointer callback in order to create and store checkpoints.","accessed":"2026-05-20"},{"type":"doc","title":"LlamaIndex — run_from","url":"https://developers.llamaindex.ai/python/examples/workflow/checkpointing_workflows/","quote":"To run from a chosen `Checkpoint` we can use the `WorkflowCheckpointer.run_from()` method.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"agentic-rag","role":"first-class","note":"RAG pipelines are exposed as tools the agent decides to call.","evidence":[{"type":"doc","title":"LlamaIndex framework — RAG as agent tool","url":"https://developers.llamaindex.ai/python/framework/","quote":"LlamaIndex provides a framework for building agents including the ability to use RAG pipelines as one of many tools to complete a task.","accessed":"2026-05-20"},{"type":"doc","title":"LlamaIndex — agents as LLM-powered knowledge assistants","url":"https://developers.llamaindex.ai/python/framework/","quote":"Agents are LLM-powered knowledge assistants that use tools to perform tasks like research, data extraction, and more.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"contextual-retrieval","role":"supported","note":"Cookbook recipe composing Claude + chunk enrichment with LlamaIndex primitives; not a core API.","evidence":[{"type":"doc","title":"LlamaIndex — Contextual Retrieval cookbook","url":"https://developers.llamaindex.ai/python/examples/cookbooks/contextual_retrieval","quote":"In this notebook we will demonstrate how you can implement Anthropic's Contextual Retrieval using LlamaIndex abstractions.","accessed":"2026-05-24"},{"type":"doc","title":"LlamaIndex — Contextual Retrieval observations","url":"https://developers.llamaindex.ai/python/examples/cookbooks/contextual_retrieval","quote":"We observed improved metrics with contextual retrieval; however, our experiments showed that much depends on the queries, chunk size, chunk overlap, and other variables.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"crag","role":"limited","note":"CRAG ships as a workflow example/pack, not a top-level API; the pack uses an LLM to grade retrieved-node relevance before generation.","evidence":[{"type":"doc","title":"LlamaIndex — Corrective RAG pack","url":"https://developers.llamaindex.ai/python/examples/workflow/corrective_rag_pack","quote":"Corrective Retrieval Augmented Generation (CRAG) is a method designed to enhance the robustness of language model generation by evaluating and augmenting the relevance of retrieved documents","accessed":"2026-05-20"},{"type":"doc","title":"LlamaIndex — Corrective RAG relevance evaluation","url":"https://developers.llamaindex.ai/python/examples/workflow/corrective_rag_pack","quote":"Uses an LLM to determine whether the retrieved nodes are relevant to the query given the content of the nodes.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"cross-encoder-reranking","role":"first-class","note":"SentenceTransformerRerank + CohereRerank as node post-processors.","evidence":[{"type":"doc","title":"LlamaIndex — SentenceTransformerRerank","url":"https://developers.llamaindex.ai/python/framework/module_guides/querying/node_postprocessors/node_postprocessors","quote":"Uses the cross-encoders from the `sentence-transformer` package to re-order nodes, and returns the top N nodes.","accessed":"2026-05-20"},{"type":"doc","title":"LlamaIndex — CohereRerank","url":"https://developers.llamaindex.ai/python/framework/module_guides/querying/node_postprocessors/node_postprocessors","quote":"Uses the \"Cohere ReRank\" functionality to re-order nodes, and returns the top N nodes.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"event-driven-agent","role":"first-class","note":"Workflow IS the event-driven runtime; @step methods are triggered by Events and emit further Events.","evidence":[{"type":"doc","title":"LlamaIndex — Workflows overview","url":"https://developers.llamaindex.ai/python/framework/understanding/workflows/","quote":"A workflow is an event-driven, step-based way to control the execution flow of an application.","accessed":"2026-05-20"},{"type":"doc","title":"LlamaIndex — Step/Event chaining","url":"https://developers.llamaindex.ai/python/framework/understanding/workflows/","quote":"Your application is divided into sections called Steps which are triggered by Events, and themselves emit Events which trigger further steps.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"graphrag","role":"first-class","note":"PropertyGraphIndex with built-in graph retrievers (TextToCypher, CypherTemplate, LLMSynonym, VectorContext).","evidence":[{"type":"blog","title":"LlamaIndex blog — Property Graph Index","url":"https://www.llamaindex.ai/blog/introducing-the-property-graph-index-a-powerful-new-way-to-build-knowledge-graphs-with-llms","quote":"Introducing the Property Graph Index! A Powerful New Way to Build Knowledge Graphs with LLMs","accessed":"2026-05-20"},{"type":"doc","title":"LlamaIndex — PropertyGraphIndex guide","url":"https://developers.llamaindex.ai/python/framework/module_guides/indexing/lpg_index_guide","quote":"A property graph is a knowledge collection of labeled nodes (i.e. entity categories, text labels, etc.) with properties (i.e. metadata), linked together by relationships into structured paths.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"hybrid-search","role":"first-class","note":"QueryFusionRetriever combines vector retrieval with BM25; Relative Score Fusion applies a MinMax scaler per result set and produces a weighted sum.","evidence":[{"type":"doc","title":"LlamaIndex — Relative-score fusion retriever","url":"https://developers.llamaindex.ai/python/examples/retrievers/relative_score_dist_fusion","quote":"In this step, we fuse our index with a BM25 based retriever. This will enable us to capture both semantic relations and keywords in our input queries.","accessed":"2026-05-20"},{"type":"doc","title":"LlamaIndex — Relative Score Fusion algorithm","url":"https://developers.llamaindex.ai/python/examples/retrievers/relative_score_dist_fusion","quote":"The following example uses the Relative Score Fusion algorithm from Weaviate, which applies a MinMax scaler to each result set, then makes a weighted sum.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"get_tools_from_mcp_url returns FunctionTools from an MCP server; BasicMCPClient supports HTTP/SSE/stdio; workflow_as_mcp exposes Workflows as MCP servers.","evidence":[{"type":"doc","title":"LlamaIndex — MCP tools","url":"https://developers.llamaindex.ai/python/examples/tools/mcp","quote":"Using the `get_tools_from_mcp_url` or `aget_tools_from_mcp_url` function, you can get a list of `FunctionTool`s from an MCP server.","accessed":"2026-05-20"},{"type":"doc","title":"LlamaIndex — MCP transports","url":"https://developers.llamaindex.ai/python/examples/tools/mcp","quote":"Support for HTTP, SSE, and stdio transports.","accessed":"2026-05-20"},{"type":"doc","title":"LlamaIndex — workflow_as_mcp","url":"https://developers.llamaindex.ai/python/examples/tools/mcp","quote":"The `workflow_as_mcp` function enables converting custom LlamaIndex Workflows into MCP applications, automatically generate[ing] a `FastMCP` server","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"naive-rag","role":"first-class","note":"Canonical 5-line example: SimpleDirectoryReader + VectorStoreIndex.from_documents + as_query_engine.","evidence":[{"type":"doc","title":"LlamaIndex — RAG basics","url":"https://developers.llamaindex.ai/python/framework/understanding/rag/","quote":"Retrieval-Augmented Generation (RAG) solves this problem by adding your data to the data LLMs already have access to.","accessed":"2026-05-24"},{"type":"doc","title":"LlamaIndex — Loading and indexing","url":"https://developers.llamaindex.ai/python/framework/understanding/rag/","quote":"Indexing means creating a data structure that allows for querying the data...this nearly always means creating vector embeddings.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"orchestrator-workers","role":"first-class","note":"Orchestrator pattern: expose each agent's run as a tool and give those tools to a new top-level FunctionAgent.","evidence":[{"type":"doc","title":"LlamaIndex — Orchestrator pattern","url":"https://developers.llamaindex.ai/python/framework/understanding/agent/multi_agent/","quote":"You expose each agent's `run` method as a tool and give those tools to a new top-level agent – the _Orchestrator_.","accessed":"2026-05-20"},{"type":"doc","title":"LlamaIndex — orchestrator chooses sub-agents","url":"https://developers.llamaindex.ai/python/framework/understanding/agent/multi_agent/","quote":"an 'orchestrator' agent chooses which sub-agent to call next; those sub-agents are exposed to it as tools","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"ReActAgent works with any LLM regardless of whether it supports function calling; reasoning is step-by-step over tools.","evidence":[{"type":"doc","title":"LlamaIndex — ReActAgent","url":"https://developers.llamaindex.ai/python/examples/agent/react_agent","quote":"The main advantage of the ReAct agent over a Function Calling agent is that it can work with any LLM regardless of whether it supports function calling.","accessed":"2026-05-20"},{"type":"doc","title":"LlamaIndex — ReAct step-by-step","url":"https://developers.llamaindex.ai/python/examples/agent/react_agent","quote":"We show how it can reason step-by-step over different tools to achieve the end goal.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"self-rag","role":"via-integration","note":"Self-RAG is shipped as a LlamaPack — SelfRAGQueryEngine is the official module name. Not a core framework primitive.","evidence":[{"type":"doc","title":"LlamaIndex — Self-RAG pack API reference","url":"https://developers.llamaindex.ai/python/framework-api-reference/packs/self_rag/","quote":"Simple short form self RAG query engine","accessed":"2026-05-24"},{"type":"doc","title":"LlamaIndex — Self-RAG pack API reference","url":"https://developers.llamaindex.ai/python/framework-api-reference/packs/self_rag/","quote":"Simple short form Self-RAG pack","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"session-isolation","role":"supported","note":"Context is per-run and serializable across sessions; AgentWorkflow is stateless between runs by default. Downgraded from first-class — closer to per-run isolation than thread-scoped session memory.","evidence":[{"type":"doc","title":"LlamaIndex — Agent state","url":"https://developers.llamaindex.ai/python/framework/understanding/agent/state/","quote":"By default, the `AgentWorkflow` is stateless between runs.","accessed":"2026-05-20"},{"type":"doc","title":"LlamaIndex — Context serializable","url":"https://developers.llamaindex.ai/python/framework/understanding/agent/state/","quote":"serializable, allowing you to persist state across sessions.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"step-budget","role":"first-class","note":"BaseWorkflowAgent sets DEFAULT_MAX_ITERATIONS = 20; agent run accepts max_iterations and triggers an early-stopping branch once num_iterations >= max_iterations.","evidence":[{"type":"repo","title":"llama_index — DEFAULT_MAX_ITERATIONS","url":"https://github.com/run-llama/llama_index/blob/main/llama-index-core/llama_index/core/agent/workflow/base_agent.py","quote":"DEFAULT_MAX_ITERATIONS = 20","accessed":"2026-05-24"},{"type":"repo","title":"llama_index — max_iterations enforcement","url":"https://github.com/run-llama/llama_index/blob/main/llama-index-core/llama_index/core/agent/workflow/base_agent.py","quote":"max_iterations = await ctx.store.get(\n            \"max_iterations\", default=DEFAULT_MAX_ITERATIONS\n        )","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"structured-output","role":"first-class","note":"Pydantic models drive both JSON-schema generation and coercion of LLM output.","evidence":[{"type":"doc","title":"LlamaIndex — Extraction overview","url":"https://developers.llamaindex.ai/python/framework/understanding/extraction/","quote":"The core of the way structured data extraction works in LlamaIndex is Pydantic classes: you define a data structure in Pydantic and LlamaIndex works with Pydantic to coerce the output of the LLM into that structure.","accessed":"2026-05-20"},{"type":"doc","title":"LlamaIndex — JSON-schema for LLMs","url":"https://developers.llamaindex.ai/python/framework/understanding/extraction/","quote":"these JSON-formatted schemas are often passed to LLMs and the LLMs in turn use them as instructions on how to return data","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"supervisor","role":"first-class","note":"The Orchestrator pattern + root_agent designation is the supervisor shape.","evidence":[{"type":"doc","title":"LlamaIndex — Orchestrator pattern","url":"https://developers.llamaindex.ai/python/framework/understanding/agent/multi_agent/","quote":"You expose each agent's `run` method as a tool and give those tools to a new top-level agent – the _Orchestrator_.","accessed":"2026-05-20"},{"type":"doc","title":"LlamaIndex — orchestrator chooses sub-agents","url":"https://developers.llamaindex.ai/python/framework/understanding/agent/multi_agent/","quote":"an 'orchestrator' agent chooses which sub-agent to call next; those sub-agents are exposed to it as tools","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Tool has __call + metadata; FunctionTool wraps any callable; QueryEngineTool wraps a query engine.","evidence":[{"type":"doc","title":"LlamaIndex — Tools","url":"https://developers.llamaindex.ai/python/framework/module_guides/deploying/agents/tools/","quote":"A Tool implements a very generic interface - simply define `__call` and also return some basic metadata (name, description, function schema).","accessed":"2026-05-20"},{"type":"doc","title":"LlamaIndex — QueryEngineTool","url":"https://developers.llamaindex.ai/python/framework/module_guides/deploying/agents/tools/","quote":"Any query engine can be turned into a tool, using `QueryEngineTool`","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"semantic-response-cache","role":"supported","note":"LlamaIndex consumes the semantic response cache through its first-class GPTCache integration, so near-duplicate queries return a stored LLM answer without re-running the pipeline.","evidence":[{"type":"repo","title":"GPTCache README — integrations","url":"https://github.com/zilliztech/GPTCache","quote":"Fully integrated with LangChain and llama_index.","accessed":"2026-06-14"}],"evidence_status":"limited"},{"pattern":"tenant-scoped-tool-binding","role":"supported","note":"Per-user query engines are built with MetadataFilters/ExactMatchFilter keyed on a user field, so each retriever only fetches documents tagged with that tenant's identifier at query time.","evidence":[{"type":"doc","title":"Multi-Tenancy RAG with LlamaIndex","url":"https://developers.llamaindex.ai/python/examples/multi_tenancy/multi_tenancy_rag/","quote":"filters=MetadataFilters(filters=[ExactMatchFilter(key=\"user\", value=\"Jerry\")])","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"map-reduce","role":"first-class","note":"A @step(num_workers=N) runs the same step concurrently over many emitted events (map), and ctx.collect_events gathers all results before a combine step merges them (reduce).","evidence":[{"type":"doc","title":"Parallel Execution of Same Event Example","url":"https://developers.llamaindex.ai/python/examples/workflow/parallel_execution/","quote":"By setting the `num_workers` parameter in `@step` decorator, we can control the number of steps executed simultaneously, enabling efficient parallel processing.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"query-decomposition-agent","role":"first-class","note":"Breaks a complex query into sub-questions each routed to a relevant data source, gathers the intermediate answers, and synthesizes them into a single final response.","evidence":[{"type":"doc","title":"Sub Question Query Engine","url":"https://developers.llamaindex.ai/python/examples/query_engine/sub_question_query_engine/","quote":"It first breaks down the complex query into sub questions for each relevant data source, then gather all the intermediate reponses and synthesizes a final response.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"hierarchical-retrieval","role":"first-class","note":"AutoMergingRetriever over a HierarchicalNodeParser tree retrieves fine leaf chunks first, then escalates to coarser parent nodes when enough children match, pushing each decision to the cheapest tier.","evidence":[{"type":"doc","title":"Auto Merging Retriever | LlamaIndex OSS Documentation","url":"https://developers.llamaindex.ai/python/examples/retrievers/auto_merging_retriever/","quote":"looks at a set of leaf nodes and recursively \"merges\" subsets of leaf nodes that reference a parent node beyond a given threshold. This allows consolidation of potentially disparate, smaller contexts into a larger context that might help synthesis.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"mcp-bidirectional-bridge","role":"supported","note":"LlamaIndex exposes MCP client tooling and a workflow_as_mcp helper for serving Workflows over MCP.","evidence":[{"type":"doc","url":"https://developers.llamaindex.ai/python/shared/mcp/","quote":"LlamaIndex exposes MCP client tooling and a workflow_as_mcp helper for serving Workflows over MCP.","accessed":"2026-06-19"}],"evidence_status":"full"}],"instantiates":["production-rag","multi-agent-coordination"],"alternatives":[{"composition":"langchain","relation":"competes-with","note":"Same model-agnostic LLM-app framework niche; LangChain delegates orchestration to LangGraph, LlamaIndex uses its own Workflow runtime."},{"composition":"langgraph","relation":"similar-shape","note":"Both express orchestration as a graph over typed state with checkpointable execution."},{"composition":"haystack","relation":"competes-with","note":"deepset's RAG-rooted Python framework with an agent layer."},{"composition":"crewai","relation":"competes-with","note":"Multi-agent role-based orchestration alternative."},{"composition":"dbgpt","relation":"competes-with"},{"composition":"mastra","relation":"competes-with","note":"LlamaIndex.ts has stronger RAG; Mastra has stronger workflow/MCP-server story."},{"composition":"openagents","relation":"competes-with"},{"composition":"qwen-agent","relation":"similar-shape","note":"Stronger RAG focus; weaker on Qwen-native function-calling templates."},{"composition":"ragflow","relation":"competes-with","note":"Both are full RAG frameworks; RAGFlow emphasises DeepDoc parsing and no-code workflow editor."}],"references":[{"type":"repo","title":"run-llama/llama_index","url":"https://github.com/run-llama/llama_index","quote":"LlamaIndex is the leading document agent and OCR platform","accessed":"2026-05-20"},{"type":"doc","title":"LlamaIndex framework overview","url":"https://developers.llamaindex.ai/python/framework/","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-06-14","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","python","typescript","rag-first","graph-rag","event-driven"],"anti_patterns_avoided":[{"pattern":"lost-in-the-middle","note":"LongContextReorder re-orders retrieved nodes so the most relevant ones sit at the start/end of the context, where models attend best, instead of the lost middle.","evidence":[{"type":"doc","title":"LlamaIndex — LongContextReorder","url":"https://docs.llamaindex.ai/en/stable/examples/node_postprocessor/LongContextReorder/","quote":"This module will re-order the retrieved nodes, which can be helpful in cases where a large top-k is needed.","accessed":"2026-06-19"}]}]},{"id":"marvin","name":"Marvin","aliases":["ControlFlow (predecessor)"],"kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"Prefect Technologies","language":"Python","license":"Apache-2.0","status":"active","url":"https://github.com/PrefectHQ/marvin","repo":"https://github.com/PrefectHQ/marvin","intent":"Express agentic AI work as a set of structured Tasks executed by portable Agents inside Threads, producing validated Pydantic outputs.","description":"Marvin is PrefectHQ's Python framework for structured AI outputs and task-based agentic workflows. Marvin 3.x merged the developer ergonomics of Marvin 2.x with the agentic engine of ControlFlow (and supersedes ControlFlow). The core abstractions are Task (a structured unit of work with a result type), Agent (a portable LLM configuration assigned to tasks), and Thread (a conversation context that maintains history and shared state across tasks). Internally Marvin uses Pydantic AI for LLM interactions and supports the LLM providers Pydantic AI supports. Threads persist in SQLite; structured outputs are validated against Pydantic types.","primary_use_cases":["extracting and validating structured data from unstructured text","composing AI workflows from typed Tasks","sequencing tool-using agents within a Thread","casting/classifying/summarising with Pydantic-validated returns"],"agent_loop_shape":"Task-centric agentic loop. Each Task names a result type, an agent, and optional tools; the agent runs Pydantic AI under the hood, calling tools until the result conforms to the declared type. Threads compose Tasks into customizable chains that share history; thread state is persisted to SQLite so a thread can be resumed by id.","key_concepts":[{"name":"Task","summary":"Structured unit of work with a declared result type, assigned tools, and an assigned Agent; the result is validated before return.","maps_to_pattern":"structured-output","url":"https://github.com/PrefectHQ/marvin"},{"name":"Agent","summary":"Portable LLM configuration (model, instructions, tools) that can be assigned to one or more tasks.","url":"https://github.com/PrefectHQ/marvin"},{"name":"Thread","summary":"Conversation context that spans tasks; persisted to SQLite so it can be resumed by id.","maps_to_pattern":"session-isolation","url":"https://github.com/PrefectHQ/marvin"},{"name":"Pydantic AI backbone","summary":"Marvin 3 uses Pydantic AI for LLM interactions and inherits its provider matrix.","url":"https://github.com/PrefectHQ/marvin"}],"pattern_composition":"flowchart TD\n  USER[Caller code] --> TASK{Task<br/>result_type + tools}\n  TASK --> AGENT[Agent<br/>portable LLM config]\n  AGENT --> PAI[Pydantic AI<br/>LLM + tool calling]\n  PAI -->|tool call| TOOLS[Functions / tools]\n  TOOLS --> PAI\n  PAI --> VALIDATE[Validate against<br/>Pydantic result_type]\n  VALIDATE --> OUT[Validated result]\n  TASK -.inside.-> THREAD[Thread<br/>shared history]\n  THREAD --> SQLITE[(SQLite<br/>thread persistence)]\n  SQLITE -.resume by id.-> THREAD","members":[{"pattern":"structured-output","role":"core","note":"Tasks declare a result_type; Pydantic AI validates the LLM output against it. The framework's primary purpose is structured outputs.","evidence":[{"type":"repo","title":"Marvin README — purpose","url":"https://github.com/PrefectHQ/marvin","quote":"Cast, classify, extract, and generate structured data from any inputs.","accessed":"2026-05-20"},{"type":"repo","title":"Marvin docs — task result_type","url":"https://github.com/PrefectHQ/marvin/blob/main/docs/concepts/tasks.mdx","quote":"A task's result type indicates the type of value that the task will return. This is used to validate the task's result.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Tasks and Agents both accept a list of tool Python functions; agents call tools during task execution and use them to mark task status.","evidence":[{"type":"repo","title":"Marvin docs — task tools","url":"https://github.com/PrefectHQ/marvin/blob/main/docs/concepts/tasks.mdx","quote":"The tools of a task are a list of tools that the task requires. These tools will be made available to the agent during execution.","accessed":"2026-05-20"},{"type":"repo","title":"Marvin docs — agent tools","url":"https://github.com/PrefectHQ/marvin/blob/main/docs/concepts/agents.mdx","quote":"Tools are Python functions that the agent can call to perform specific actions or computations.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"react","role":"supported","note":"Marvin 3 delegates LLM interactions to Pydantic AI, which runs a tool-calling loop until the typed result is produced; agents use tools to mark tasks SUCCESSFUL or FAILED.","evidence":[{"type":"repo","title":"Marvin README — Pydantic AI","url":"https://github.com/PrefectHQ/marvin","quote":"Marvin 3.0 uses Pydantic AI for LLM interactions, and supports the full range of LLM providers that Pydantic AI supports.","accessed":"2026-05-20"},{"type":"repo","title":"Marvin docs — agents use tools to mark task status","url":"https://github.com/PrefectHQ/marvin/blob/main/docs/concepts/tasks.mdx","quote":"Agents use tools to mark tasks as `SUCCESSFUL` or `FAILED`. Successful tasks will also have a `result` property, which contains the task's final output.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"session-isolation","role":"first-class","note":"Threads are per-conversation contexts that scope shared state; each thread maintains its own conversation history.","evidence":[{"type":"repo","title":"Marvin docs — thread context","url":"https://github.com/PrefectHQ/marvin/blob/main/docs/concepts/threads.mdx","quote":"A thread in Marvin represents a conversation context that can span multiple interactions.","accessed":"2026-05-20"},{"type":"repo","title":"Marvin docs — each thread has its own history","url":"https://github.com/PrefectHQ/marvin/blob/main/docs/concepts/threads.mdx","quote":"Each thread:\n- Maintains its own conversation history\n- Can be persisted to a database\n- Can be referenced across different parts of your application","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"agent-resumption","role":"limited","note":"Threads can be persisted to SQLite and created with an explicit id so a later run resumes the same conversation; not per-step checkpointing.","evidence":[{"type":"repo","title":"Marvin docs — thread persistence","url":"https://github.com/PrefectHQ/marvin/blob/main/docs/concepts/threads.mdx","quote":"Threads are automatically persisted to a database when configured","accessed":"2026-05-20"},{"type":"repo","title":"Marvin docs — resume by id","url":"https://github.com/PrefectHQ/marvin/blob/main/docs/concepts/threads.mdx","quote":"You can create a thread with a specific ID, which is useful for resuming conversations","accessed":"2026-05-20"}],"evidence_status":"full"}],"anti_patterns_avoided":[],"alternatives":[{"composition":"pydantic-ai","relation":"wraps","note":"Marvin 3 is built on Pydantic AI for the underlying LLM interactions."}],"references":[{"type":"repo","title":"PrefectHQ/marvin","url":"https://github.com/PrefectHQ/marvin","quote":"Marvin is a Python framework for producing structured outputs and building agentic AI workflows.","accessed":"2026-05-20"},{"type":"repo","title":"Marvin README — supersedes ControlFlow","url":"https://github.com/PrefectHQ/marvin","quote":"Marvin 3.0 combines the DX of Marvin 2.0 with the powerful agentic engine of ControlFlow (thereby superseding ControlFlow).","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"emerging","tags":["open-source","structured-output","task-based","pydantic"]},{"id":"mastra","name":"Mastra","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"Kepler Software, Inc.","language":"TypeScript","license":"Apache-2.0 (ee/ directories under Mastra Enterprise License)","status":"active","first_released":"2024-08-06","url":"https://mastra.ai","repo":"https://github.com/mastra-ai/mastra","docs_url":"https://mastra.ai/docs","intent":"Provide a TypeScript-native framework for AI agents and multi-step workflows, where Agents run an LLM tool-calling loop bounded by maxSteps, Workflows give graph-based control flow with suspend/resume snapshots, and integrations cover MCP (client+server), RAG, memory, voice, and evals.","description":"Mastra is an Apache-2.0 TypeScript framework from Kepler Software, Inc. ('from the team behind Gatsby', per repo). It ships two execution shapes: Agents which 'reason about goals, decide which tools to use, retain conversation memory, and iterate internally until the model emits a final answer or an optional stop condition is met' (docs/agents/overview), and Workflows which give 'full control over how tasks are broken down' via .then() / .branch() / .parallel() / .dountil() / .dowhile() / .foreach() over typed steps. Both can suspend and resume — snapshots persist to the configured storage provider across deployments and restarts. Integrates with 40+ model providers, exposes Mastra tools/agents/workflows over MCP via MCPServer (and consumes external MCP via MCPClient), and ships built-in evals (Scorers), memory, RAG, and voice (TTS/STT/STS).","primary_use_cases":["TypeScript-native agents with workflow orchestration","long-running agents that pause and resume across deployments","supervisor agents with subagents","multi-channel voice + chat assistants"],"agent_loop_shape":"Imperative tool-calling agent loop with explicit maxSteps cap (default 5). Starting v0.14.0 Mastra owns the loop directly rather than delegating to AI SDK. Workflows compose typed steps via graph operators (.then/.branch/.parallel/.dountil/.dowhile/.foreach) with suspend()/resume() snapshots. Workflows and agents compose — workflow steps can call agents, agents can be configured with sub-agents (supervisor pattern), agents can be exposed over MCP.","key_concepts":[{"name":"Agent","summary":"Reasons about goals, uses tools, retains memory, iterates internally.","maps_to_pattern":"react","url":"https://mastra.ai/docs/agents/overview"},{"name":"Workflow","summary":"Graph-based orchestration with .then/.branch/.parallel/.dountil/.dowhile/.foreach.","url":"https://mastra.ai/docs/workflows/overview"},{"name":"createTool","summary":"Custom tools with Zod-typed input/output schemas.","maps_to_pattern":"tool-use","url":"https://mastra.ai/reference/tools/create-tool"},{"name":"Memory","summary":"Working memory + semantic recall + conversation history across interactions.","url":"https://mastra.ai/docs/memory/overview"},{"name":"Suspend & Resume","summary":"suspend() persists workflow state to storage; resume continues across deployments and restarts.","maps_to_pattern":"agent-resumption","url":"https://mastra.ai/docs/workflows/suspend-and-resume"},{"name":"Supervisor Agents","summary":"Supervisor coordinates subagents (configured on the agents property); migration path from deprecated AgentNetwork.","maps_to_pattern":"supervisor","url":"https://mastra.ai/docs/agents/supervisor-agents"},{"name":"MCPServer / MCPClient","summary":"Bidirectional MCP — consume external MCP and expose Mastra tools/agents/workflows over MCP.","maps_to_pattern":"mcp","url":"https://mastra.ai/docs/mcp/overview"}],"pattern_composition":"flowchart TD\n  USER[User input] --> AGENT[Mastra Agent]\n  MODELS[(provider/model<br/>40+ providers)] --> AGENT\n  TOOLS[createTool<br/>Zod schemas] --> AGENT\n  AGENT -->|maxSteps loop| TOOLS\n  AGENT -->|response_format| ZOD[(Zod / JSON Schema)]\n  AGENT --> MEM[Memory<br/>working + semantic recall + history]\n  AGENT -->|MCPClient| MCPEXT[External MCP servers]\n  AGENT -->|MCPServer exposes| MCPOUT[Mastra as MCP server]\n  AGENT -->|agents property| SUB[Subagents<br/>supervisor pattern]\n  AGENT -->|RAG tool| RAG[Vector store + embeddings]\n  AGENT -->|Scorers| EVAL[Live evals]\n  WF[Workflow<br/>.then .branch .parallel .dountil .dowhile .foreach] -.calls.-> AGENT\n  WF -->|suspend snapshot| STORE[(Storage provider)]\n  STORE -->|resume| WF\n  VOICE[Voice TTS/STT/STS] -.-> AGENT","members":[{"pattern":"agent-resumption","role":"first-class","note":"Workflow suspend()/resume() saves snapshot to storage provider; persists across deployments and restarts. README explicitly highlights this as a differentiator.","evidence":[{"type":"doc","title":"Mastra — Suspend & Resume","url":"https://mastra.ai/docs/workflows/suspend-and-resume","quote":"Use `suspend()` to pause workflow execution at a specific step.","accessed":"2026-05-20"},{"type":"doc","title":"Mastra — snapshot persistence","url":"https://mastra.ai/docs/workflows/suspend-and-resume","quote":"Snapshots are stored in your configured storage provider and persist across deployments and application restarts.","accessed":"2026-05-20"},{"type":"repo","title":"Mastra README","url":"https://github.com/mastra-ai/mastra","quote":"Suspend an agent or workflow and await user input or approval before resuming. Mastra uses storage to remember execution state, so you can pause indefinitely and resume where you left off.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"agentic-rag","role":"first-class","note":"Vector Query Tool gives agents direct access to vector DB queries; the agent decides what to retrieve and how to filter. Upgraded from via-integration.","evidence":[{"type":"doc","title":"Mastra — RAG overview","url":"https://mastra.ai/docs/rag/overview","quote":"RAG in Mastra helps you enhance LLM outputs by incorporating relevant context from your own data sources, improving accuracy and grounding responses in real information.","accessed":"2026-05-20"},{"type":"doc","title":"Mastra — Retrieval (agent-driven query)","url":"https://mastra.ai/docs/rag/retrieval","quote":"The Vector Query Tool allows your agent to be in charge of retrieval decisions, combining semantic search with optional filtering.","accessed":"2026-05-24"},{"type":"doc","title":"Mastra — Retrieval (agent queries vector DB)","url":"https://mastra.ai/docs/rag/retrieval","quote":"Sometimes you want to give your agent the ability to query a vector database directly.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"evaluator-optimizer","role":"limited","note":"Scorers measure but do not close the loop to regenerate; live evals are async observation, not driven optimization. Could be re-classified as agent-as-judge.","evidence":[{"type":"doc","title":"Mastra — Evals (Scorers)","url":"https://mastra.ai/docs/evals/overview","quote":"Scorers are automated tests that evaluate Agents outputs using model-graded, rule-based, and statistical methods.","accessed":"2026-05-20"},{"type":"doc","title":"Mastra — Live evaluations","url":"https://mastra.ai/docs/evals/overview","quote":"Live evaluations allow you to automatically score AI outputs in real-time as your agents and workflows operate.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"Both directions: MCPClient consumes external MCP servers; MCPServer exposes Mastra tools/agents/workflows to MCP clients.","evidence":[{"type":"doc","title":"Mastra — MCPClient","url":"https://mastra.ai/docs/mcp/overview","quote":"Connects to one or many MCP servers to access their tools, resources, prompts, and handle elicitation requests.","accessed":"2026-05-20"},{"type":"doc","title":"Mastra — MCPServer","url":"https://mastra.ai/docs/mcp/overview","quote":"Exposes Mastra tools, agents, workflows, prompts, and resources to MCP-compatible clients.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"Agent loop reasons → tool-calls → iterates. v0.14.0 brought the loop in-framework. Mastra docs don't use the literal 'ReAct' label.","evidence":[{"type":"doc","title":"Mastra — Agents overview","url":"https://mastra.ai/docs/agents/overview","quote":"Agents use LLMs and tools to solve open-ended tasks. They reason about goals, decide which tools to use, retain conversation memory, and iterate internally until the model emits a final answer or an optional stop condition is met.","accessed":"2026-05-20"},{"type":"blog","title":"Mastra v0.14.0 announcement","url":"https://mastra.ai/blog/introducing-mastra-improved-agent-orchestration-ai-sdk-v5-support","quote":"Starting v0.14.0, Mastra now controls the agent loop, as well as tool calling.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"structured-output","role":"first-class","note":"Schema-based structured output via Zod / Valibot / ArkType / raw JSON Schema; Zod recommended for TS type inference.","evidence":[{"type":"doc","title":"Mastra — Structured output","url":"https://mastra.ai/docs/agents/structured-output","quote":"Structured output lets an agent return an object that matches the shape defined by a schema instead of returning text.","accessed":"2026-05-20"},{"type":"doc","title":"Mastra — schema libraries","url":"https://mastra.ai/docs/agents/structured-output","quote":"Agents can return structured data by defining the expected output with either Standard JSON Schema (Zod, Valibot, ArkType, etc.) or JSON Schema.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"supervisor","role":"first-class","note":"Upgraded from limited per upstream docs — Mastra has a dedicated Supervisor Agents docs page and a migration guide deprecating AgentNetwork in favour of this exact pattern.","evidence":[{"type":"doc","title":"Mastra — Supervisor agents","url":"https://mastra.ai/docs/agents/supervisor-agents","quote":"A supervisor agent coordinates multiple subagents using `Agent.stream()` or `Agent.generate()`.","accessed":"2026-05-20"},{"type":"doc","title":"Mastra — Subagents on agents property","url":"https://mastra.ai/docs/agents/supervisor-agents","quote":"You configure subagents on the supervisor's `agents` property, and the supervisor uses its instructions and each subagent's `description` to decide when and how to delegate tasks.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"createTool() with description / inputSchema / outputSchema / execute. Tools can also come from MCP.","evidence":[{"type":"doc","title":"Mastra — createTool","url":"https://mastra.ai/reference/tools/create-tool","quote":"The `createTool()` function is used to define custom tools that your Mastra agents can execute.","accessed":"2026-05-20"},{"type":"doc","title":"Mastra — Tools framing","url":"https://mastra.ai/docs/agents/using-tools-and-mcp","quote":"Tools give agents capabilities beyond language generation by providing structured access to data and performing clearly defined operations.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp-bidirectional-bridge","role":"supported","note":"Mastra ships an MCPClient for consuming external servers and an MCPServer for exposing Mastra tools, agents, workflows, prompts, and resources.","evidence":[{"type":"doc","url":"https://mastra.ai/docs/mcp/overview","quote":"Mastra ships an MCPClient for consuming external servers and an MCPServer for exposing Mastra tools, agents, workflows, prompts, and resources.","accessed":"2026-06-19"}],"evidence_status":"full"},{"pattern":"unified-voice-interface","role":"supported","note":"Mastra's Voice system documents a unified interface spanning TTS, STT, and real-time STS.","evidence":[{"type":"doc","url":"https://mastra.ai/docs/voice/overview","quote":"Mastra's Voice system documents a unified interface spanning TTS, STT, and real-time STS.","accessed":"2026-06-19"}],"evidence_status":"full"}],"instantiates":["long-running-autonomous-agent","multi-agent-coordination","structured-output-stack"],"alternatives":[{"composition":"langgraph","relation":"similar-shape","note":"Both ship a graph engine with checkpointed/suspendable state; LangGraph in Py+TS, Mastra in TS-first."},{"composition":"langchain","relation":"competes-with","note":"LangChain.js + create_agent target the same TS-agent niche; Mastra adds workflow engine + native suspend/resume."},{"composition":"llamaindex","relation":"competes-with","note":"LlamaIndex.ts has stronger RAG; Mastra has stronger workflow/MCP-server story."},{"composition":"vercel-ai-sdk","relation":"complements","note":"Mastra historically used AI SDK for the agent loop; v0.14.0 took over directly. AI SDK remains a peer for model bindings."},{"composition":"crewai","relation":"similar-shape","note":"Multi-agent + role-played agents; CrewAI is Python-only and crew-centric."}],"references":[{"type":"repo","title":"mastra-ai/mastra","url":"https://github.com/mastra-ai/mastra","quote":"From the team behind Gatsby, Mastra is a framework for building AI-powered applications and agents with a modern TypeScript stack.","accessed":"2026-05-20"},{"type":"doc","title":"Mastra docs","url":"https://mastra.ai/docs","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-14","date_added":"2026-04-30","status_in_practice":"emerging","tags":["open-source","typescript","workflow-engine","suspend-resume","supervisor"]},{"id":"metagpt","name":"MetaGPT","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"DeepWisdom / FoundationAgents","language":"Python","license":"MIT","status":"active","url":"https://github.com/FoundationAgents/MetaGPT","repo":"https://github.com/FoundationAgents/MetaGPT","docs_url":"https://docs.deepwisdom.ai/","intent":"Materialise software-engineering Standard Operating Procedures as multi-agent teams of Roles whose communication is mediated by an Environment.","description":"MetaGPT is DeepWisdom's open-source multi-agent framework whose stated philosophy is 'Code = SOP(Team)' — it encodes the SOPs of a software company as a Team of Role agents (Product Manager, Architect, Project Manager, Engineer, Data Analyst). Roles observe Messages from a shared Environment and emit Actions; the Environment broadcasts messages to subscribed roles until everyone is idle. MetaGPT has also published the AFlow paper on automatic agentic workflow generation (ICLR 2025 oral) and the Data Interpreter agent that writes and runs code to solve analytics tasks.","primary_use_cases":["code generation from a one-line requirement via a simulated software company","multi-role SOP-encoded multi-agent workflows","data analysis via the Data Interpreter code-as-action agent","automatic workflow search via AFlow"],"agent_loop_shape":"Role-based multi-agent loop. A Team contains Role agents, each with a list of Actions and a watch list of Message types to subscribe to. The shared Environment broadcasts messages to subscribed roles; each role observes, decides which Action to run, executes it, and publishes a Message back. The Team runs for n_round steps or until env.is_idle (no role has new messages).","key_concepts":[{"name":"Code = SOP(Team)","summary":"Core philosophy: a Standard Operating Procedure for a software team is materialised as code executed by an LLM team.","url":"https://github.com/geekan/MetaGPT"},{"name":"Role","summary":"Agent abstraction with a system prompt, a list of Actions, and a watch list of Message types it subscribes to.","maps_to_pattern":"role-assignment","url":"https://docs.deepwisdom.ai/main/en/guide/tutorials/multi_agent_101.html"},{"name":"Environment","summary":"Shared message bus that broadcasts published messages to roles based on their subscriptions; tracks env.is_idle for termination.","maps_to_pattern":"inter-agent-communication","url":"https://docs.deepwisdom.ai/main/en/guide/in_depth_guides/agent_communication.html"},{"name":"Team / company.hire / company.run","summary":"Container that wires Roles together, invests budget, and runs the SOP for n_round rounds.","maps_to_pattern":"sop-encoded-multi-agent","url":"https://docs.deepwisdom.ai/main/en/guide/get_started/quickstart.html"},{"name":"Data Interpreter","summary":"Code-as-action agent that writes and executes Python code to solve data analysis tasks.","maps_to_pattern":"code-as-action","url":"https://github.com/geekan/MetaGPT"},{"name":"AFlow","summary":"Automated agentic workflow generation (ICLR 2025 oral) shipped in MetaGPT.","maps_to_pattern":"automatic-workflow-search","url":"https://github.com/geekan/MetaGPT"}],"pattern_composition":"flowchart TD\n  REQ[One-line requirement] --> TEAM[Team / company]\n  TEAM --> ROLES{Roles<br/>PM / Architect / PrjM / Engineer}\n  ROLES --> PM[Product Manager Role]\n  ROLES --> ARCH[Architect Role]\n  ROLES --> ENG[Engineer Role]\n  ROLES --> DA[Data Interpreter<br/>code-as-action]\n  PM -->|publish Message| ENV[(Environment<br/>broadcast bus)]\n  ARCH --> ENV\n  ENG --> ENV\n  ENV -->|subscribed types| ARCH\n  ENV -->|subscribed types| ENG\n  ROLES -->|_act / Action| OUT[Artifacts:<br/>specs / code / docs]\n  ENV -.env.is_idle.-> STOP[Stop]\n  TEAM -.n_round limit.-> STOP\n  TEAM -.AFlow.-> SEARCH[Automatic<br/>workflow search]","members":[{"pattern":"sop-encoded-multi-agent","role":"core","note":"MetaGPT's stated philosophy is 'Code = SOP(Team)'; the framework materialises a software-company SOP as code executed by an LLM team.","evidence":[{"type":"repo","title":"MetaGPT README — philosophy","url":"https://github.com/geekan/MetaGPT","quote":"Code = SOP(Team) is the core philosophy. We materialize SOP and apply it to teams composed of LLMs.","accessed":"2026-05-20"},{"type":"doc","title":"MetaGPT docs — SOPs","url":"https://docs.deepwisdom.ai/main/en/guide/get_started/introduction.html","quote":"It provides the entire process of a software company along with carefully orchestrated SOPs.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"role-assignment","role":"first-class","note":"Off-the-shelf roles (Product Manager, Architect, Project Manager, Engineer) are imported and hired into a Team via company.hire().","evidence":[{"type":"repo","title":"MetaGPT README — roles","url":"https://github.com/geekan/MetaGPT","quote":"MetaGPT includes product managers / architects / project managers / engineers.","accessed":"2026-05-20"},{"type":"doc","title":"MetaGPT — quickstart roles","url":"https://docs.deepwisdom.ai/main/en/guide/get_started/quickstart.html","quote":"Architect, Engineer, ProductManager, ProjectManager","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"inter-agent-communication","role":"core","note":"Communication is mediated by an Environment object that broadcasts messages to roles based on their subscription; env.is_idle terminates the run.","evidence":[{"type":"doc","title":"MetaGPT — agent communication","url":"https://docs.deepwisdom.ai/main/en/guide/in_depth_guides/agent_communication.html","quote":"The exchange of messages between agents is facilitated through attributes provided in the Message class and the publish_message capability provided by the Environment.","accessed":"2026-05-20"},{"type":"doc","title":"MetaGPT — environment broadcast","url":"https://docs.deepwisdom.ai/main/en/guide/in_depth_guides/agent_communication.html","quote":"The Environment object is responsible for broadcasting messages to the various agents according to their subscription requirements.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"supervisor","role":"first-class","note":"The SOP defines an implicit supervision order; Team.run advances through rounds where each role acts on subscribed messages — upstream roles publish work, downstream roles observe and act.","evidence":[{"type":"doc","title":"MetaGPT — multi-agent 101 (team)","url":"https://docs.deepwisdom.ai/main/en/guide/tutorials/multi_agent_101.html","quote":"Initialize all roles, create a team with an environment to put them in, and enable them to interact with each other","accessed":"2026-05-20"},{"type":"doc","title":"MetaGPT — upstream/downstream role flow","url":"https://docs.deepwisdom.ai/main/en/guide/tutorials/multi_agent_101.html","quote":"Making each role observe the corresponding output from upstream, and publish its own for the downstream","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"code-as-action","role":"first-class","note":"Data Interpreter is MetaGPT's code-as-action agent that writes and executes Python code to solve analytics tasks; the framework also outputs full code projects through the software-company simulation.","evidence":[{"type":"repo","title":"MetaGPT README — outputs","url":"https://github.com/geekan/MetaGPT","quote":"MetaGPT takes a one line requirement as input and outputs user stories / competitive analysis / requirements / data structures / APIs / documents, etc.","accessed":"2026-05-20"},{"type":"repo","title":"MetaGPT README — Data Interpreter","url":"https://github.com/geekan/MetaGPT","quote":"Data Interpreter to write code","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"automatic-workflow-search","role":"first-class","note":"AFlow (ICLR 2025 oral, top 1.8%) is shipped in MetaGPT and is the framework's automatic agentic workflow generation engine; the paper reformulates workflow optimisation as MCTS over code-represented workflows.","evidence":[{"type":"repo","title":"MetaGPT README — AFlow","url":"https://github.com/geekan/MetaGPT","quote":"AFlow: Automating Agentic Workflow Generation","accessed":"2026-05-20"},{"type":"paper","title":"AFlow paper — MCTS over code-represented workflows","url":"https://arxiv.org/abs/2410.10762","quote":"efficiently explores this space using Monte Carlo Tree Search, iteratively refining workflows through code modification, tree-structured experience, and execution feedback","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"orchestrator-workers","role":"first-class","note":"Team.run advances rounds where roles act on subscribed messages; the Team object orchestrates Roles with SOPs as workers in a software-company simulation.","evidence":[{"type":"doc","title":"MetaGPT — environment broadcast","url":"https://docs.deepwisdom.ai/main/en/guide/in_depth_guides/agent_communication.html","quote":"The Environment object is responsible for broadcasting messages to the various agents according to their subscription requirements.","accessed":"2026-05-20"},{"type":"repo","title":"MetaGPT team.py — Team class docstring","url":"https://github.com/FoundationAgents/MetaGPT/blob/main/metagpt/team.py","quote":"Team: Possesses one or more roles (agents), SOP (Standard Operating Procedures), and a env for instant messaging, dedicated to env any multi-agent activity, such as collaboratively writing executable code.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"supported","note":"Actions can invoke tools (e.g. Data Interpreter runs code, the SD skill is invoked from UIDesign); MetaGPT does not market tool-use as its headline pattern, the SOP/roles do.","evidence":[{"type":"repo","title":"MetaGPT README — Data Interpreter","url":"https://github.com/geekan/MetaGPT","quote":"Data Interpreter to write code","accessed":"2026-05-20"},{"type":"doc","title":"MetaGPT — Concepts (tool-using actions)","url":"https://docs.deepwisdom.ai/main/en/guide/tutorials/concepts.html","quote":"Agents can also execute tool-using actions, including searching the web for weather, using a calculator to do maths, and more.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"spec-driven-loop","role":"first-class","note":"The team produces user stories, requirements, and data structures before code — a literal spec-driven flow where each artefact becomes the next role's input, orchestrated by SOPs.","evidence":[{"type":"repo","title":"MetaGPT README — outputs sequence","url":"https://github.com/geekan/MetaGPT","quote":"MetaGPT takes a one line requirement as input and outputs user stories / competitive analysis / requirements / data structures / APIs / documents, etc.","accessed":"2026-05-20"},{"type":"doc","title":"MetaGPT introduction — software-company SOPs","url":"https://docs.deepwisdom.ai/main/en/guide/get_started/introduction.html","quote":"It provides the entire process of a software company along with carefully orchestrated SOPs.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"agent-as-judge","role":"supported","note":"MetaGPT introduced the SPO (Self-supervised Prompt Optimisation) paper which uses an LLM evaluator over pairwise output comparisons; treated as research rather than a headline feature of the core framework.","evidence":[{"type":"repo","title":"MetaGPT README — SPO paper","url":"https://github.com/geekan/MetaGPT","quote":"We introduced two papers: SPO and AOT","accessed":"2026-05-20"},{"type":"paper","title":"SPO paper — LLM evaluator over pairwise outputs","url":"https://arxiv.org/abs/2502.06855","quote":"SPO selects superior prompts through pairwise output comparisons evaluated by an LLM evaluator","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"stigmergic-coordination","role":"core","note":"Agents coordinate indirectly via a shared message pool: they publish structured messages and subscribe by role-specific interest instead of querying each other directly.","evidence":[{"type":"paper","title":"MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework","url":"https://ar5iv.labs.arxiv.org/html/2308.00352","quote":"agents not only publish their structured messages in the pool but also access messages from other entities transparently. [...] Any agent can directly retrieve required information from the shared pool, eliminating the need to inquire about other agents and await their responses.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"instantiates":["multi-agent-coordination"],"alternatives":[{"composition":"agentverse","relation":"competes-with"},{"composition":"chatdev","relation":"competes-with","note":"Both are Chinese multi-agent SDLC frameworks with role-specialised agents; MetaGPT emphasises SOP encoding."},{"composition":"crewai","relation":"similar-shape","note":"MetaGPT's role-played software-company crew (PM, Architect, Engineer) is the closest sibling to CrewAI's Crew+Agent+Process model."},{"composition":"openmanus","relation":"same-vendor","note":"Same FoundationAgents team that maintains MetaGPT."},{"composition":"smolagents","relation":"similar-shape","note":"Also code-as-action first-class."}],"references":[{"type":"repo","title":"geekan/MetaGPT","url":"https://github.com/geekan/MetaGPT","quote":"Assign different roles to GPTs to form a collaborative entity for complex tasks.","accessed":"2026-05-20"},{"type":"doc","title":"MetaGPT — introduction","url":"https://docs.deepwisdom.ai/main/en/guide/get_started/introduction.html","quote":"Code = SOP(Team) is the core philosophy. We materialize SOP and apply it to teams composed of LLMs.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","multi-agent","sop","software-company"]},{"id":"modelscope-agent","name":"ModelScope-Agent","aliases":["MS-Agent"],"kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"Alibaba ModelScope","language":"Python","license":"Apache-2.0","status":"active","first_released":"2023-08-07","url":"https://github.com/modelscope/modelscope-agent","repo":"https://github.com/modelscope/modelscope-agent","intent":"Provide a lightweight, extensible Chinese-ecosystem agent framework with RolePlay agents, tool calling, hybrid RAG, and MCP-mediated multi-agent workflows.","description":"ModelScope-Agent (now also branded MS-Agent) is Alibaba ModelScope's open-source agent framework. It started in August 2023 as a RolePlay-style agent with tool calling and a published preprint (arXiv:2309.00986), and has since added an AgentFabric UI for creating agents tailored to applications, hybrid search combining FAISS dense retrieval with BM25, MCP-based tool calling, and code-generation artifacts. The framework integrates Qwen and other ModelScope-hosted models and ships RAG over LlamaIndex.","primary_use_cases":["Chinese-ecosystem tool-using agents over Qwen and ModelScope APIs","agent chat with MCP-mediated external tools","hybrid (BM25 + dense) RAG over local code and docs","user-built agents via the AgentFabric UI"],"agent_loop_shape":"RolePlay-style tool-calling agent. Agents are configured with a role, tools, and an LLM (typically Qwen or OpenAI-SDK-compatible). The loop is the standard reason-tool-observe pattern with MCP servers and local tools as the action space; a multi-agent path orchestrates specialised agents for task decomposition.","key_concepts":[{"name":"RolePlay agent","summary":"Primary agent class — a role-prompted, tool-using agent supporting image inputs (e.g. GPT-4o) and OpenAI-SDK tool calling.","url":"https://github.com/modelscope/modelscope-agent"},{"name":"AgentFabric","summary":"Interactive UI for creating agents tailored to applications, released Nov 17 2023.","url":"https://github.com/modelscope/modelscope-agent"},{"name":"MCP-based tool calling","summary":"Agent chat with tool-calling capabilities based on the Model Context Protocol.","maps_to_pattern":"mcp","url":"https://github.com/modelscope/modelscope-agent"},{"name":"Hybrid search","summary":"FAISS dense retrieval combined with BM25 sparse retrieval for RAG.","maps_to_pattern":"hybrid-search","url":"https://github.com/modelscope/modelscope-agent"},{"name":"Multi-Agent (general purpose)","summary":"Orchestrates multiple specialised agents for task decomposition, data collection, quantitative analysis.","maps_to_pattern":"orchestrator-workers","url":"https://github.com/modelscope/modelscope-agent"}],"pattern_composition":"flowchart TD\n  USER[User input] --> RP[RolePlay agent<br/>Qwen / OpenAI SDK]\n  RP -->|tool call| MCP[(MCP servers)]\n  RP -->|tool call| LOCAL[Local tools<br/>code gen / artifacts]\n  RP --> RAG[Hybrid search<br/>FAISS + BM25]\n  RAG --> LLAMA[LlamaIndex RAG flow]\n  RP --> AF[AgentFabric UI]\n  RP --> MULTI{Multi-Agent orchestration}\n  MULTI --> SUB1[Specialised agent: task decomp]\n  MULTI --> SUB2[Specialised agent: data collection]\n  MULTI --> SUB3[Specialised agent: quant analysis]\n  RP --> OUT[Response / artifacts]","members":[{"pattern":"tool-use","role":"core","note":"RolePlay agent supports OpenAI-SDK-style tool calling on Qwen and other backends; MCP servers extend the tool surface.","evidence":[{"type":"repo","title":"MS-Agent README — tool calling","url":"https://github.com/modelscope/modelscope-agent","quote":"Multi-Agent for general purpose: Chat with agent with tool-calling capabilities based on MCP.","accessed":"2026-05-20"},{"type":"repo","title":"MS-Agent README — Qwen tool calling","url":"https://github.com/modelscope/modelscope-agent","quote":"Qwen2 could be used by OpenAI SDK with tool calling ability","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"LLMAgent.run_loop is documented as a llm-calling and tool-calling loop with per-step think→tool-call→tool-result iteration; the framework's autonomous exploration is built on this iterative tool-using loop.","evidence":[{"type":"repo","title":"MS-Agent README — autonomous exploration","url":"https://github.com/modelscope/modelscope-agent","quote":"MS-Agent is a lightweight framework designed to empower agents with autonomous exploration capabilities.","accessed":"2026-05-20"},{"type":"repo","title":"ms-agent LLMAgent.run_loop docstring","url":"https://github.com/modelscope/ms-agent/blob/main/ms_agent/agent/llm_agent.py","quote":"Run the agent, mainly contains a llm calling and tool calling loop.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"MCP-based tool calling is highlighted on the README; the agent connects to MCP servers for external tools.","evidence":[{"type":"repo","title":"MS-Agent README — MCP tool calling","url":"https://github.com/modelscope/modelscope-agent","quote":"Multi-Agent for general purpose: Chat with agent with tool-calling capabilities based on MCP.","accessed":"2026-05-20"},{"type":"repo","title":"MS-Agent README — MCP","url":"https://github.com/modelscope/modelscope-agent","quote":"Support for Agent chat with MCP (Model Context Protocol)","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"hybrid-search","role":"first-class","note":"Hybrid Search is named as a feature: FAISS dense retrieval combined with BM25 sparse retrieval; the Skill Module also uses it for skill retrieval.","evidence":[{"type":"repo","title":"MS-Agent README — hybrid search","url":"https://github.com/modelscope/modelscope-agent","quote":"Hybrid Search: Combines FAISS dense retrieval with BM25 sparse retrieval","accessed":"2026-05-20"},{"type":"repo","title":"ms-agent Skill Module README — hybrid search","url":"https://github.com/modelscope/ms-agent/blob/main/ms_agent/skill/README.md","quote":"Skill Discovery: Using hybrid retrieval (dense + sparse) to find relevant skills from a skill library","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"agentic-rag","role":"first-class","note":"RAG flow is built on LlamaIndex with hybrid retrieval; more recent versions add Sirchmunk-based knowledge search over local codebases and documentation invoked during agent conversations.","evidence":[{"type":"repo","title":"MS-Agent README — RAG flow","url":"https://github.com/modelscope/modelscope-agent","quote":"RAG flow based on LLama-index, allow user to hybrid search knowledge by different strategies and modalities.","accessed":"2026-05-20"},{"type":"repo","title":"MS-Agent README — Knowledge Search","url":"https://github.com/modelscope/ms-agent","quote":"Knowledge Search: Integrated Sirchmunk for intelligent retrieval over local codebases and documentation during agent conversations.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"code-as-action","role":"first-class","note":"Agents generate and emit code as artifacts; Code Genesis is the multi-agent code-generation project that produces production-ready software projects from natural language requirements.","evidence":[{"type":"repo","title":"MS-Agent README — code generation","url":"https://github.com/modelscope/modelscope-agent","quote":"Support code generation tasks with artifacts.","accessed":"2026-05-20"},{"type":"repo","title":"Code Genesis README — production-ready code from NL","url":"https://github.com/modelscope/ms-agent/blob/main/projects/code_genesis/README.md","quote":"An open-source multi-agent framework that generates production-ready software projects from natural language requirements.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"orchestrator-workers","role":"first-class","note":"The Multi-Agent path orchestrates specialised agents; FinResearch is the concrete five-agent DAG: Orchestrator decomposes tasks, Searcher/Collector gather data, Analyst computes, Aggregator finalises.","evidence":[{"type":"repo","title":"MS-Agent README — multi-agent orchestration","url":"https://github.com/modelscope/modelscope-agent","quote":"Orchestrates multiple specialized agents to handle task decomposition, data collection, quantitative analysis, sentiment research","accessed":"2026-05-20"},{"type":"repo","title":"FinResearch README — Orchestrator agent","url":"https://github.com/modelscope/ms-agent/blob/main/projects/fin_research/README.md","quote":"Orchestrator Agent - Decomposes user queries into three components: task description and scope, financial data tasks, and public sentiment tasks.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"skill-library","role":"first-class","note":"The MS-Agent Skill Module is an explicit implementation of Anthropic's Agent Skills protocol with hybrid skill retrieval, DAG-based execution, and progressive skill analysis. AgentFabric is a separate UI for assembling agents.","evidence":[{"type":"repo","title":"ms-agent Skill Module README — purpose","url":"https://github.com/modelscope/ms-agent/blob/main/ms_agent/skill/README.md","quote":"A powerful, extensible skill execution framework that enables LLM agents to automatically discover, analyze, and execute domain-specific skills for complex task completion.","accessed":"2026-05-24"},{"type":"repo","title":"ms-agent Skill Module README — Anthropic protocol","url":"https://github.com/modelscope/ms-agent/blob/main/ms_agent/skill/README.md","quote":"The **MS-Agent Skill Module** is **Implementation** of [Anthropic-Agent-Skills](https://platform.claude.com/docs/en/agents-and-tools/agent-skills) Protocol.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"structured-output","role":"limited","note":"Tool-call schemas via OpenAI SDK provide a structured channel; Agentic Insight v2 also persists structured artifacts to disk as a file-system-as-context substrate. No first-class structured-response_format API beyond that.","evidence":[{"type":"repo","title":"MS-Agent README — OpenAI SDK","url":"https://github.com/modelscope/modelscope-agent","quote":"Qwen2 could be used by OpenAI SDK with tool calling ability","accessed":"2026-05-20"},{"type":"repo","title":"MS-Agent README — file system as structured context","url":"https://github.com/modelscope/ms-agent","quote":"File system as context: Structured artifacts persisted to disk for low-loss context handoff and stable long-horizon runs (resume-friendly).","accessed":"2026-05-24"}],"evidence_status":"full"}],"alternatives":[{"composition":"qwen-agent","relation":"same-vendor","note":"Sister project from Alibaba's ModelScope team — also Python, also ReAct + tool use, slightly different abstractions."}],"references":[{"type":"repo","title":"modelscope/modelscope-agent","url":"https://github.com/modelscope/modelscope-agent","quote":"MS-Agent is a lightweight framework designed to empower agents with autonomous exploration capabilities.","accessed":"2026-05-20"},{"type":"paper","title":"ModelScope-Agent preprint","url":"https://arxiv.org/abs/2309.00986","quote":"Sep 2, 2023: The preprint paper associated with this project was published.","accessed":"2026-05-20"},{"type":"repo","title":"MS-Agent README — initial release","url":"https://github.com/modelscope/modelscope-agent","quote":"Aug 7, 2023: The initial version of the modelscope-agent repository was released.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"emerging","tags":["open-source","alibaba","qwen","modelscope","chinese-ecosystem"]},{"id":"openagents","name":"OpenAgents","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"XLang Lab / HKU NLP","language":"Python, TypeScript","license":"Apache-2.0","status":"active","url":"https://github.com/xlang-ai/OpenAgents","repo":"https://github.com/xlang-ai/OpenAgents","intent":"Open platform for language agents in the wild, bundling three specialised agents — Data Agent (Python/SQL data analysis), Plugins Agent (200+ APIs), and Web Agent (autonomous browsing) — under a unified web UI and local-deployment story.","description":"OpenAgents (COLM 2024) is HKU XLang Lab's open platform for production-style language agents. It ships three pre-configured agents: a Data Agent that uses Python and SQL for data analysis with tabular tools; a Plugins Agent that exposes 200+ daily-use APIs (weather, calendar, search, etc.) as tools; and a Web Agent for autonomous web browsing. The platform optimises for end-user UX (swift response, common-failure mitigation) and developer deployability (local setup with documented extension points). XLang has since extended to OSWorld (computer-use evaluation env) and OpenCUA (open foundations for computer-use agents).","primary_use_cases":["real-world data analysis with Python + SQL inside a chat UI","plugin-augmented assistant with 200+ APIs (weather, calendar, mail, etc.)","autonomous web browsing with screenshot + DOM observation","research baseline for language agents in the wild"],"agent_loop_shape":"Per-agent ReAct loop with the agent's specialised tool palette. Data Agent runs Python and SQL queries against uploaded data; Plugins Agent calls API tools and aggregates responses; Web Agent observes browser state and emits browser actions. A unified Next.js UI fronts all three with chat history, file upload, and tool-trace display.","key_concepts":[{"name":"Data Agent / Plugins Agent / Web Agent","summary":"Three specialised agents with their own tool palettes.","maps_to_pattern":"tool-loadout"},{"name":"200+ plugin APIs","summary":"Curated tool catalog for the Plugins Agent.","maps_to_pattern":"tool-discovery"},{"name":"Real-world UX optimisation","summary":"Common-failure handling and swift-response design for end users."}],"pattern_composition":"flowchart TD\n  USER[User] --> UI[Web UI<br/>Next.js chat]\n  UI --> ROUTE{Agent}\n  ROUTE -->|Data Agent| DATA[Data Agent<br/>Python + SQL]\n  ROUTE -->|Plugins Agent| PLUGINS[Plugins Agent<br/>200+ APIs]\n  ROUTE -->|Web Agent| WEB[Web Agent<br/>browser observation]\n  DATA --> DTOOLS[Pandas / SQL / file tools]\n  PLUGINS --> PTOOLS[API catalog<br/>weather / mail / search / ...]\n  WEB --> WTOOLS[Browser actions]\n  DATA --> UI\n  PLUGINS --> UI\n  WEB --> UI","members":[{"pattern":"react","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"tool-use","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"code-as-action","role":"first-class","note":"Data Agent runs Python.","evidence":[],"evidence_status":"none"},{"pattern":"code-execution","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"tool-loadout","role":"first-class","note":"Different tool palettes per agent.","evidence":[],"evidence_status":"none"},{"pattern":"browser-agent","role":"first-class","note":"Web Agent.","evidence":[],"evidence_status":"none"},{"pattern":"agent-skills","role":"supported","note":"Plugins Agent treats each API as a skill.","evidence":[],"evidence_status":"none"},{"pattern":"supervisor","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"structured-output","role":"supported","evidence":[],"evidence_status":"none"}],"alternatives":[{"composition":"langchain","relation":"competes-with","note":"LangChain is the broader framework; OpenAgents ships three pre-configured agents as a product."},{"composition":"llamaindex","relation":"competes-with"},{"composition":"autogen","relation":"similar-shape","note":"Both are research platforms shipping agents in the wild."}],"last_analyzed":"2026-05-22","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-05-22","status_in_practice":"emerging","tags":["orchestration-framework","planning-control-flow","tool-use-environment","multi-agent","structure-data","full-code"],"references":[{"type":"doc","title":"OpenAgents — Homepage","url":"https://github.com/xlang-ai/OpenAgents"}]},{"id":"openmanus","name":"OpenManus","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"FoundationAgents (MetaGPT team)","language":"Python","license":"MIT","status":"active","url":"https://github.com/FoundationAgents/OpenManus","repo":"https://github.com/FoundationAgents/OpenManus","intent":"Provide an open, no-invite-code clone of the Manus general AI agent that combines a ReAct tool-calling loop with browser-use, code execution, and MCP tools.","description":"OpenManus is FoundationAgents' (the MetaGPT team's) open-source clone of the Manus general AI agent. The architecture is a class hierarchy of agents: a base ReActAgent defines the think/act loop, ToolCallAgent layers tool dispatch with a default max_steps budget of 30, and the headline Manus agent is a versatile general-purpose agent with PythonExecute, BrowserUseTool, StrReplaceEditor, AskHuman, and Terminate tools plus support for remote MCP tools over SSE and stdio. The project explicitly credits anthropic-computer-use, browser-use, and crawl4ai for foundational support.","primary_use_cases":["general-purpose tool-using agent without an invite code","browser automation via browser-use","code execution and data analysis (DataAnalysis Agent)","MCP-connected agent with remote tool servers"],"agent_loop_shape":"ReAct think/act loop. The base ReActAgent defines an async step() that calls think() (decide whether to act) then act() (execute the decided action). ToolCallAgent fills in the loop with a ToolCollection and a max_steps cap (default 30); Manus narrows max_steps to 20 and wires in PythonExecute, BrowserUseTool, StrReplaceEditor, AskHuman, Terminate, plus dynamically discovered MCP tools.","key_concepts":[{"name":"ReActAgent","summary":"Abstract base class defining the async think → act loop.","maps_to_pattern":"react","url":"https://github.com/FoundationAgents/OpenManus/blob/main/app/agent/react.py"},{"name":"ToolCallAgent","summary":"Concrete ReAct agent layering tool dispatch with a ToolCollection and a default max_steps of 30.","maps_to_pattern":"tool-use","url":"https://github.com/FoundationAgents/OpenManus/blob/main/app/agent/toolcall.py"},{"name":"Manus agent","summary":"Versatile general-purpose agent wiring PythonExecute, BrowserUseTool, StrReplaceEditor, AskHuman, Terminate and MCP tools; max_steps=20.","url":"https://github.com/FoundationAgents/OpenManus/blob/main/app/agent/manus.py"},{"name":"MCP clients (SSE + stdio)","summary":"Manus connects to remote MCP servers over both SSE and stdio and dynamically adds discovered tools.","maps_to_pattern":"mcp","url":"https://github.com/FoundationAgents/OpenManus/blob/main/app/agent/manus.py"},{"name":"max_steps budget","summary":"Each agent enforces a hard step budget; default 30 in ToolCallAgent, 20 in Manus.","maps_to_pattern":"step-budget","url":"https://github.com/FoundationAgents/OpenManus/blob/main/app/agent/toolcall.py"},{"name":"DataAnalysis Agent","summary":"Multi-agent flow alternative tailored for data analysis and visualisation tasks.","url":"https://github.com/FoundationAgents/OpenManus"}],"pattern_composition":"flowchart TD\n  USER[User goal] --> MANUS[Manus agent<br/>max_steps=20]\n  MANUS --> RA[ReActAgent.step]\n  RA --> THINK[think]\n  THINK -->|act?| ACT[act]\n  ACT --> TOOLS{ToolCollection}\n  TOOLS --> PY[PythonExecute]\n  TOOLS --> BROWSER[BrowserUseTool<br/>browser-use]\n  TOOLS --> EDITOR[StrReplaceEditor]\n  TOOLS --> ASK[AskHuman]\n  TOOLS --> TERM[Terminate]\n  TOOLS --> MCP[(MCP servers<br/>SSE + stdio)]\n  MCP -.dynamic tools.-> TOOLS\n  ACT --> OBS[Observation]\n  OBS --> RA\n  RA -.max_steps reached.-> STOP[Stop]\n  TERM -.special tool.-> FIN[Finished state]\n  MANUS -.alt flow.-> DA[DataAnalysis Agent]","members":[{"pattern":"react","role":"core","note":"ReActAgent base class defines the async step() = think() then act() pattern that all OpenManus agents inherit.","evidence":[{"type":"repo","title":"OpenManus — ReActAgent.step","url":"https://github.com/FoundationAgents/OpenManus/blob/main/app/agent/react.py","quote":"async def step(self) -> str: \"\"\"Execute a single step: think and act.\"\"\" should_act = await self.think() if not should_act: return \"Thinking complete - no action needed\" return await self.act()","accessed":"2026-05-20"},{"type":"repo","title":"OpenManus — ReActAgent.think","url":"https://github.com/FoundationAgents/OpenManus/blob/main/app/agent/react.py","quote":"async def think(self) -> bool: \"\"\"Process current state and decide next action\"\"\"","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"core","note":"ToolCallAgent layers a ToolCollection over the ReAct loop; Manus wires in PythonExecute, BrowserUseTool, StrReplaceEditor, AskHuman, Terminate plus MCP tools.","evidence":[{"type":"repo","title":"OpenManus — ToolCallAgent tools","url":"https://github.com/FoundationAgents/OpenManus/blob/main/app/agent/toolcall.py","quote":"available_tools: ToolCollection = ToolCollection(CreateChatCompletion(), Terminate())","accessed":"2026-05-20"},{"type":"repo","title":"OpenManus — Manus tools","url":"https://github.com/FoundationAgents/OpenManus/blob/main/app/agent/manus.py","quote":"A versatile general-purpose agent with support for both local and MCP tools.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"step-budget","role":"first-class","note":"ToolCallAgent.max_steps defaults to 30; Manus tightens it to 20; Terminate is a special tool that ends the loop.","evidence":[{"type":"repo","title":"OpenManus — ToolCallAgent max_steps","url":"https://github.com/FoundationAgents/OpenManus/blob/main/app/agent/toolcall.py","quote":"max_steps: int = 30","accessed":"2026-05-20"},{"type":"repo","title":"OpenManus — Manus max_steps","url":"https://github.com/FoundationAgents/OpenManus/blob/main/app/agent/manus.py","quote":"max_steps: int = 20","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"Manus connects to MCP servers over both SSE and stdio and dynamically adds the discovered tools to its ToolCollection.","evidence":[{"type":"repo","title":"OpenManus — Manus MCP support","url":"https://github.com/FoundationAgents/OpenManus/blob/main/app/agent/manus.py","quote":"A versatile agent that can solve various tasks using multiple tools including MCP-based tools","accessed":"2026-05-20"},{"type":"repo","title":"OpenManus README — MCP runner","url":"https://github.com/FoundationAgents/OpenManus","quote":"For MCP tool version, you can run: python run_mcp.py","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"browser-agent","role":"first-class","note":"BrowserUseTool integrates the browser-use library (acknowledged in README) so Manus can drive a browser as one of its tools.","evidence":[{"type":"repo","title":"OpenManus — Manus BrowserUseTool","url":"https://github.com/FoundationAgents/OpenManus/blob/main/app/agent/manus.py","quote":"PythonExecute(), BrowserUseTool(), StrReplaceEditor(), AskHuman(), Terminate()","accessed":"2026-05-20"},{"type":"repo","title":"OpenManus README — acknowledgements","url":"https://github.com/FoundationAgents/OpenManus","quote":"Thanks to anthropic-computer-use, browser-use and crawl4ai for providing basic support for this project!","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"code-execution","role":"first-class","note":"PythonExecute is one of Manus's headline tools; the DataAnalysis Agent is dedicated to executable code analysis.","evidence":[{"type":"repo","title":"OpenManus — Manus PythonExecute","url":"https://github.com/FoundationAgents/OpenManus/blob/main/app/agent/manus.py","quote":"PythonExecute(), BrowserUseTool(), StrReplaceEditor(), AskHuman(), Terminate()","accessed":"2026-05-20"},{"type":"repo","title":"OpenManus README — DataAnalysis Agent","url":"https://github.com/FoundationAgents/OpenManus","quote":"Currently, besides the general OpenManus Agent, we have also integrated the DataAnalysis Agent, which is suitable for data analysis and data visualization tasks.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"computer-use","role":"supported","note":"OpenManus credits Anthropic's computer-use demo as foundational support; full computer-use is delivered via the BrowserUseTool path rather than a separate desktop-control surface. The Manus agent description explicitly names MCP-based tools.","evidence":[{"type":"repo","title":"OpenManus README — anthropic-computer-use","url":"https://github.com/FoundationAgents/OpenManus","quote":"Thanks to anthropic-computer-use, browser-use and crawl4ai for providing basic support for this project!","accessed":"2026-05-20"},{"type":"repo","title":"OpenManus manus.py — Manus description","url":"https://github.com/FoundationAgents/OpenManus/blob/main/app/agent/manus.py","quote":"A versatile agent that can solve various tasks using multiple tools including MCP-based tools","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"human-in-the-loop","role":"first-class","note":"AskHuman is one of Manus's built-in tools, providing an explicit ask-the-user step inside the loop; its self-described purpose is to ask a human for help.","evidence":[{"type":"repo","title":"OpenManus — AskHuman tool","url":"https://github.com/FoundationAgents/OpenManus/blob/main/app/agent/manus.py","quote":"PythonExecute(), BrowserUseTool(), StrReplaceEditor(), AskHuman(), Terminate()","accessed":"2026-05-20"},{"type":"repo","title":"OpenManus — AskHuman tool description","url":"https://github.com/FoundationAgents/OpenManus/blob/main/app/tool/ask_human.py","quote":"description: str = \"Use this tool to ask human for help.\"","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"supervisor","role":"limited","note":"PlanningFlow is a BaseFlow that supervises a plan: it creates an actionable plan with the PlanningTool, then assigns each step to an executor agent; FlowFactory currently only ships the PLANNING flow type. No first-class supervisor abstraction comparable to LangGraph or AutoGen.","evidence":[{"type":"repo","title":"OpenManus README — multi-agent flow","url":"https://github.com/FoundationAgents/OpenManus","quote":"Currently, besides the general OpenManus Agent, we have also integrated the DataAnalysis Agent, which is suitable for data analysis and data visualization tasks.","accessed":"2026-05-20"},{"type":"repo","title":"OpenManus planning.py — PlanningFlow docstring","url":"https://github.com/FoundationAgents/OpenManus/blob/main/app/flow/planning.py","quote":"A flow that manages planning and execution of tasks using agents.","accessed":"2026-05-24"}],"evidence_status":"full"}],"alternatives":[{"composition":"metagpt","relation":"same-vendor","note":"Same FoundationAgents team that maintains MetaGPT."},{"composition":"manus","relation":"similar-shape","note":"Open-source clone reproducing Manus's browser + sandbox + plan-file loop."},{"composition":"autogpt","relation":"similar-shape","note":"General-computer-using descendant."},{"composition":"chatdev","relation":"similar-shape","note":"OpenMANUS is from the same FoundationAgents lineage as MetaGPT."},{"composition":"xagent","relation":"similar-shape","note":"Both target general-purpose autonomy with sandboxed tools."}],"references":[{"type":"repo","title":"FoundationAgents/OpenManus","url":"https://github.com/FoundationAgents/OpenManus","quote":"Manus is incredible, but OpenManus can achieve any idea without an Invite Code","accessed":"2026-05-20"},{"type":"repo","title":"OpenManus README — acknowledgements","url":"https://github.com/FoundationAgents/OpenManus","quote":"Thanks to anthropic-computer-use, browser-use and crawl4ai for providing basic support for this project!","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"experimental","tags":["open-source","manus-clone","react","browser-use","mcp"]},{"id":"picoagents","name":"picoagents","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"Victor Dibia","language":"Python","license":"MIT","status":"active","url":"https://github.com/victordibia/designing-multiagent-systems","intent":"Teach the building blocks of production multi-agent systems through small, testable primitives.","description":"Minimal multi-agent framework published as companion code for Victor Dibia's book Designing Multi-Agent Systems (O'Reilly 2025). Pedagogic in scope but published on PyPI and actively maintained; demonstrates composable termination conditions, a pre/post/error middleware chain, and first-class interruptibility (cancellation + external signals) as separately-tested primitives.","primary_use_cases":["Book companion code for working through multi-agent design exercises","Reference implementation of composable termination and middleware contracts"],"members":[{"pattern":"composable-termination-conditions","role":"first-class","note":"Full termination package — MaxMessageTermination, TokenUsageTermination, TextMentionTermination, TimeoutTermination, HandoffTermination, ExternalTermination, CancellationTermination, FunctionCallTermination, plus a CompositeTermination combinator with any/all semantics.","evidence_status":"full","evidence":[{"type":"repo","title":"picoagents/termination/__init__.py","url":"https://github.com/victordibia/designing-multiagent-systems/blob/main/picoagents/src/picoagents/termination/__init__.py","quote":"This package provides various termination conditions that determine when orchestration should stop, following the PRD specification.","accessed":"2026-05-23"},{"type":"repo","title":"picoagents/termination/_composite.py","url":"https://github.com/victordibia/designing-multiagent-systems/blob/main/picoagents/src/picoagents/termination/_composite.py","quote":"Composite termination condition for combining multiple conditions; supports any (OR) and all (AND) modes.","accessed":"2026-05-23"}]},{"pattern":"agent-middleware-chain","role":"first-class","note":"BaseMiddleware exposes process_request / process_response / process_error; MiddlewareChain runs the chain forward through process_request, then reverse through process_response.","evidence_status":"full","evidence":[{"type":"repo","title":"picoagents/_middleware.py","url":"https://github.com/victordibia/designing-multiagent-systems/blob/main/picoagents/src/picoagents/_middleware.py","quote":"BaseMiddleware requires three async generator methods: process_request (pre-execution), process_response (post-execution), process_error (exception handling).","accessed":"2026-05-23"},{"type":"doc","title":"picoagents on PyPI","url":"https://pypi.org/project/picoagents/","quote":"A minimal multi-agent framework for educational purposes designed to teach multi-agent systems from first principles.","accessed":"2026-05-23"}]},{"pattern":"interruptible-agent-execution","role":"first-class","note":"CancellationToken + CancellationTermination + ExternalTermination are first-class primitives. The cancellation token follows AutoGen's design for cancelling pending async operations.","evidence_status":"full","evidence":[{"type":"blog","title":"4 UX Design Principles for Multi-Agent Systems — Victor Dibia","url":"https://newsletter.victordibia.com/p/4-ux-design-principles-for-multi","quote":"Interruptibility: allow users to pause, resume or cancel agent actions. Provide pause/stop controls enabling users to halt execution mid-task while preserving system state.","accessed":"2026-05-23"},{"type":"repo","title":"picoagents/_cancellation_token.py","url":"https://github.com/victordibia/designing-multiagent-systems/blob/main/picoagents/src/picoagents/_cancellation_token.py","quote":"This module provides a thread-safe cancellation mechanism following AutoGen's proven design for cancelling pending async operations in a consistent manner.","accessed":"2026-05-24"},{"type":"repo","title":"picoagents/termination/_cancellation.py","url":"https://github.com/victordibia/designing-multiagent-systems/blob/main/picoagents/src/picoagents/termination/_cancellation.py","quote":"class CancellationTermination(BaseTermination): \"\"\"Terminates when cancellation token is triggered.\"\"\"","accessed":"2026-05-24"}]},{"pattern":"actor-model-agents","role":"supported","note":"Picoagents has shared-conversation orchestration (RoundRobin, AIOrchestrator, plan-based) over a shared message list, not per-agent mailboxes — closest analogue but no actor-model abstraction.","evidence_status":"full","evidence":[{"type":"repo","title":"Picoagents — RoundRobin orchestrator","url":"https://github.com/victordibia/designing-multiagent-systems/blob/main/picoagents/src/picoagents/orchestration/_round_robin.py","quote":"Cycles through agents in fixed order, giving each agent access to the complete shared conversation history.","accessed":"2026-05-24"},{"type":"repo","title":"Picoagents — AI orchestrator","url":"https://github.com/victordibia/designing-multiagent-systems/blob/main/picoagents/src/picoagents/orchestration/_ai.py","quote":"Uses LLM reasoning with structured output to select the most appropriate next agent based on conversation context and agent capabilities.","accessed":"2026-05-24"}]},{"pattern":"autogen-conversational","role":"first-class","note":"RoundRobinOrchestrator cycles agents over a shared conversation history; AIOrchestrator picks the next speaker via LLM-driven structured-output selection — both classic conversational multi-agent patterns.","evidence_status":"full","evidence":[{"type":"repo","title":"picoagents/orchestration/_round_robin.py","url":"https://github.com/victordibia/designing-multiagent-systems/blob/main/picoagents/src/picoagents/orchestration/_round_robin.py","quote":"Cycles through agents in fixed order, giving each agent access to the complete shared conversation history.","accessed":"2026-05-24"},{"type":"repo","title":"picoagents/orchestration/_ai.py","url":"https://github.com/victordibia/designing-multiagent-systems/blob/main/picoagents/src/picoagents/orchestration/_ai.py","quote":"AI-driven conversation orchestration pattern. This module provides the AISelectorOrchestrator that uses LLM reasoning with structured output to select the most appropriate next agent.","accessed":"2026-05-24"}]}],"instantiates":["agent-runtime-cross-cutting"],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","status_in_practice":"emerging","pattern_composition":"flowchart TD\n  fw[\"picoagents\"]\n  fw --> p1[\"Composable Termination Conditions<br/>(first-class)\"]\n  fw --> p2[\"Agent Middleware Chain<br/>(first-class)\"]\n  fw --> p3[\"Interruptible Agent Execution<br/>(first-class)\"]\n  fw --> p4[\"Conversational Multi-Agent<br/>(first-class)\"]\n  fw --> p5[\"Actor-Model Agents<br/>(supported)\"]","tags":["orchestration-framework","safety-control","governance-observability","multi-agent","full-code"],"references":[{"type":"doc","title":"picoagents — Homepage","url":"https://github.com/victordibia/designing-multiagent-systems"}],"key_concepts":[{"name":"Composable Termination Conditions","summary":"Full termination package — MaxMessageTermination, TokenUsageTermination, TextMentionTermination, TimeoutTermination, HandoffTermination, ExternalTermination, CancellationTermination, FunctionCallTermination, plus a CompositeTermination combinator with any/all semantics.","maps_to_pattern":"composable-termination-conditions"},{"name":"Agent Middleware Chain","summary":"BaseMiddleware exposes process_request / process_response / process_error; MiddlewareChain runs the chain forward through process_request, then reverse through process_response.","maps_to_pattern":"agent-middleware-chain"},{"name":"Interruptible Agent Execution","summary":"CancellationToken + CancellationTermination + ExternalTermination are first-class primitives. The cancellation token follows AutoGen's design for cancelling pending async operations.","maps_to_pattern":"interruptible-agent-execution"},{"name":"Actor-Model Agents","summary":"Picoagents has shared-conversation orchestration (RoundRobin, AIOrchestrator, plan-based) over a shared message list, not per-agent mailboxes — closest analogue but no actor-model abstraction.","maps_to_pattern":"actor-model-agents"},{"name":"Conversational Multi-Agent","summary":"RoundRobinOrchestrator cycles agents over a shared conversation history; AIOrchestrator picks the next speaker via LLM-driven structured-output selection — both classic conversational multi-agent patterns.","maps_to_pattern":"autogen-conversational"}],"agent_loop_shape":"An imperative async agent loop assembled from small, separately-tested primitives: each step passes through a pre/post/error middleware chain, the loop keeps running until a composable termination condition fires, and execution stays interruptible throughout via cancellation tokens and external signals. Multi-agent work is coordinated by actor-style message passing between independently-addressable agents."},{"id":"pocketflow","name":"PocketFlow","aliases":["Pocket Flow"],"kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"The Pocket","language":"Python","license":"MIT","status":"active","url":"https://github.com/The-Pocket/PocketFlow","repo":"https://github.com/The-Pocket/PocketFlow","docs_url":"https://the-pocket.github.io/PocketFlow/","intent":"Capture the core graph abstraction of LLM frameworks in 100 lines of zero-dependency Python so Agent, Multi-Agent, Workflow, and RAG patterns can be assembled on top.","description":"PocketFlow is a 100-line, zero-dependency minimalist LLM framework. Its only abstractions are Node (a small LLM/task unit) and Flow (which connects Nodes via labelled Actions); batch and async variants extend the same primitives. The Agent design pattern is realised by branching from a decision Node to one of several action Nodes and looping; the Multi-Agent pattern uses message queues in shared storage for peer-to-peer coordination. Workflow and RAG are documented as compositions of the same primitives.","primary_use_cases":["learning the core graph abstraction behind LLM frameworks","tiny tool-using agents with explicit action spaces","peer-to-peer multi-agent message-queue prototypes","lightweight workflow decompositions and RAG pipelines"],"agent_loop_shape":"Graph of Nodes connected by Action labels. A Node computes (often via an LLM call), returns an Action label, and the Flow uses that label to pick the next Node — enabling branching, looping, and agent-style 'decide-then-act'. Multi-agent coordination is implemented with asyncio.Queue message queues in shared storage; the framework explicitly does not ship a supervisor or scheduler abstraction.","key_concepts":[{"name":"Node","summary":"Smallest unit of computation; handles a single (LLM) task.","url":"https://the-pocket.github.io/PocketFlow/"},{"name":"Flow","summary":"Connects Nodes through Actions (labelled edges); the basis of branching and looping.","url":"https://the-pocket.github.io/PocketFlow/"},{"name":"Agent design pattern","summary":"Branch from a decision Node to each action Node; let the agent direct flow between Nodes and loop back for multi-step.","maps_to_pattern":"react","url":"https://the-pocket.github.io/PocketFlow/design_pattern/agent.html"},{"name":"Multi-Agent message queues","summary":"Multi-agent coordination through asyncio.Queue message queues in shared storage; peer-to-peer rather than supervised.","url":"https://the-pocket.github.io/PocketFlow/design_pattern/multi_agent.html"},{"name":"Batch / Async variants","summary":"Batch Nodes/Flows handle data-intensive tasks; Async Nodes/Flows wait for asynchronous tasks.","url":"https://the-pocket.github.io/PocketFlow/"}],"pattern_composition":"flowchart TD\n  USER[Input] --> N0[Decision Node<br/>LLM picks action]\n  N0 -->|action: search| N1[Search Node]\n  N0 -->|action: code| N2[Code Node]\n  N0 -->|action: answer| N3[Answer Node]\n  N1 --> N0\n  N2 --> N0\n  N3 --> OUT[Output]\n  subgraph MULTI[Multi-Agent]\n    A1[Agent A] -->|asyncio.Queue| SHARED[(Shared storage<br/>message queues)]\n    SHARED -->|messages| A2[Agent B]\n  end","members":[{"pattern":"react","role":"first-class","note":"The Agent design pattern wires a decision Node to action Nodes and loops back — the canonical decide-then-act ReAct shape, expressed in the framework's own minimal vocabulary.","evidence":[{"type":"doc","title":"PocketFlow — Agent pattern","url":"https://the-pocket.github.io/PocketFlow/design_pattern/agent.html","quote":"Agent is a powerful design pattern in which nodes can take dynamic actions based on the context.","accessed":"2026-05-20"},{"type":"doc","title":"PocketFlow — Agent loop","url":"https://the-pocket.github.io/PocketFlow/design_pattern/agent.html","quote":"Use branching to connect each action node to an agent node. Use action to allow the agent to direct the flow between nodes—and potentially loop back for multi-step.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Tools are modelled as action Nodes the agent chooses among; the framework names 'Action Space' design (parameterised actions, undo, chunked feeds) as a first-class concern.","evidence":[{"type":"doc","title":"PocketFlow — Action Space","url":"https://the-pocket.github.io/PocketFlow/design_pattern/agent.html","quote":"Provide a well-structured and unambiguous set of actions","accessed":"2026-05-20"},{"type":"doc","title":"PocketFlow — parameterised actions","url":"https://the-pocket.github.io/PocketFlow/design_pattern/agent.html","quote":"enable parameterized (columns to select) or programmable (SQL queries) actions","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"inter-agent-communication","role":"first-class","note":"Multi-agent coordination is documented as message queues in shared storage; this is the explicit communication mechanism between agents.","evidence":[{"type":"doc","title":"PocketFlow — multi-agent communication","url":"https://the-pocket.github.io/PocketFlow/design_pattern/multi_agent.html","quote":"Communication between agents is typically implemented using message queues in shared storage.","accessed":"2026-05-20"},{"type":"doc","title":"PocketFlow — multi-agent subtasks","url":"https://the-pocket.github.io/PocketFlow/design_pattern/multi_agent.html","quote":"Multiple Agents can work together by handling subtasks and communicating the progress.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"pipes-and-filters","role":"first-class","note":"Flow connects Nodes through labelled Actions; the core abstraction is a graph of small functional units.","evidence":[{"type":"doc","title":"PocketFlow — Flow","url":"https://the-pocket.github.io/PocketFlow/","quote":"Flow connects nodes through Actions (labeled edges).","accessed":"2026-05-20"},{"type":"doc","title":"PocketFlow — Graph core","url":"https://github.com/The-Pocket/PocketFlow","quote":"The 100 lines capture the core abstraction of LLM frameworks: Graph!","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"supervisor","role":"limited","note":"Multi-agent coordination is peer-to-peer via message queues in shared storage; the Supervisor tutorial example is the closest first-class supervisor analogue.","evidence":[{"type":"doc","title":"PocketFlow — Multi-Agents pattern","url":"https://the-pocket.github.io/PocketFlow/design_pattern/multi_agent.html","quote":"Communication between agents is typically implemented using message queues in shared storage.","accessed":"2026-05-24"},{"type":"doc","title":"PocketFlow — Supervisor cookbook example","url":"https://github.com/The-Pocket/PocketFlow","quote":"Research agent is getting unreliable... Let's build a supervision process","accessed":"2026-05-24"}],"evidence_status":"full"}],"anti_patterns_avoided":[{"pattern":"vendor-lock-in","note":"PocketFlow markets zero dependencies and zero vendor lock-in as a deliberate design choice.","evidence":[{"type":"repo","title":"PocketFlow README — zero lock-in","url":"https://github.com/The-Pocket/PocketFlow","quote":"Just 100 lines. Zero bloat, zero dependencies, zero vendor lock-in.","accessed":"2026-05-20"}]}],"references":[{"type":"repo","title":"The-Pocket/PocketFlow","url":"https://github.com/The-Pocket/PocketFlow","quote":"Pocket Flow is a 100-line minimalist LLM framework","accessed":"2026-05-20"},{"type":"doc","title":"PocketFlow design patterns","url":"https://the-pocket.github.io/PocketFlow/","quote":"Everything you love—(Multi-)Agents, Workflow, RAG, and more.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"emerging","tags":["open-source","minimalist","graph","100-line"]},{"id":"pydantic-ai","name":"Pydantic AI","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"Pydantic Services Inc.","language":"Python","license":"MIT","status":"active","first_released":"2024-12-02","url":"https://ai.pydantic.dev/","repo":"https://github.com/pydantic/pydantic-ai","docs_url":"https://pydantic.dev/docs/ai/overview/","intent":"Provide a Python-first, model-agnostic agent framework that brings the 'FastAPI feeling' to GenAI by making Pydantic-validated structured output, type-safe dependency injection, and a graph-based async agent loop the primary surface for production LLM applications.","description":"Pydantic AI is the MIT-licensed Python agent framework from the Pydantic team. The framework is built around an Agent class generic in both a dependency type and an output type, so each agent run is a strongly typed function from inputs to a Pydantic-validated result. Its distinguishing thesis: structured output via Pydantic models is the primary action format. Model-agnostic across major providers (OpenAI, Anthropic, Gemini, DeepSeek, Cohere, Mistral, Bedrock, Ollama, Groq, OpenRouter, …); MCP-first toolset shape; durable execution via TemporalAgent, DBOSAgent, PrefectAgent, RestateAgent wrappers.","primary_use_cases":["type-safe agents with Pydantic-validated final answers","tool-using agents with dependency injection (deps_type / RunContext.deps)","MCP-tool integration in Python","durable agents via Temporal/DBOS/Prefect/Restate wrappers"],"agent_loop_shape":"Agent class generic in deps_type and output_type. Agent.run() returns AgentRunResult; Agent.iter() exposes the run as a pydantic-graph of nodes — UserPromptNode → ModelRequestNode → CallToolsNode → End. UsageLimits caps requests/tokens/tool-calls before each request and raises UsageLimitExceeded on breach.","key_concepts":[{"name":"Agent","summary":"Primary interface; generic in DepsT and OutputT (Agent[Foobar, list[str]]).","url":"https://pydantic.dev/docs/ai/core-concepts/agent/"},{"name":"deps_type + RunContext","summary":"Type-safe dependency injection — HTTP clients, DB sessions, API keys reach tools/system-prompts via RunContext.deps.","url":"https://pydantic.dev/docs/ai/overview/"},{"name":"output_type","summary":"Three methods to coerce LLM output to a schema: Tool Output, Native Output, Prompted Output.","maps_to_pattern":"structured-output","url":"https://pydantic.dev/docs/ai/core-concepts/output/"},{"name":"@agent.tool / @agent.tool_plain","summary":"Tool registration; tool_plain skips RunContext access.","maps_to_pattern":"tool-use","url":"https://pydantic.dev/docs/ai/tools-toolsets/tools/"},{"name":"Toolset (incl. MCP)","summary":"A collection of tools registered to an agent in one go; MCP servers fit this shape.","maps_to_pattern":"mcp","url":"https://pydantic.dev/docs/ai/tools-toolsets/toolsets/"},{"name":"UsageLimits","summary":"Caps requests/tokens/tool_calls; raises UsageLimitExceeded if breached.","maps_to_pattern":"step-budget","url":"https://pydantic.dev/docs/ai/core-concepts/agent/"},{"name":"ModelRetry","summary":"Exception tools/validators raise to request a model retry with feedback.","url":"https://pydantic.dev/docs/ai/api/pydantic-ai/exceptions/"}],"pattern_composition":"flowchart TD\n  USER[User prompt] --> AGENT[\"Agent[DepsT, OutputT]\"]\n  DEPS[(deps: DepsT)] -.injected at run-time.-> AGENT\n  INSTR[instructions / system_prompt] --> AGENT\n  AGENT --> GRAPH[pydantic-graph run]\n  GRAPH --> UPN[UserPromptNode]\n  UPN --> MRN[ModelRequestNode]\n  MRN --> CTN[CallToolsNode]\n  CTN -->|loop| MRN\n  CTN --> TOOLS[Tools<br/>@agent.tool / @agent.tool_plain]\n  CTN --> TOOLSETS[Toolsets incl. MCP]\n  TOOLSETS --> MCPSERVERS[MCPServerStdio / SSE / StreamableHTTP]\n  TOOLS -.RunContext.deps.-> DEPS\n  TOOLS -.raises ModelRetry on validation failure.-> MRN\n  MRN -.UsageLimits checked.-> LIMITS[UsageLimits]\n  GRAPH --> END[End node]\n  END --> RESULT[AgentRunResult<br/>OutputT validated via Pydantic]\n  AGENT -.optional wrap.-> DURABLE[TemporalAgent / DBOSAgent / PrefectAgent / RestateAgent]\n  DURABLE -.-> CHECKPOINT[(Durable store)]","members":[{"pattern":"agent-resumption","role":"via-integration","note":"Durability delegated to Temporal/DBOS/Prefect/Restate via the durable_exec module — TemporalAgent / DBOSAgent / PrefectAgent / RestateAgent wrappers. No built-in checkpointer.","evidence":[{"type":"doc","title":"Pydantic AI — Durable execution overview","url":"https://pydantic.dev/docs/ai/integrations/durable_execution/overview/","quote":"build durable agents that can preserve their progress across transient API failures and application errors or restarts","accessed":"2026-05-20"},{"type":"doc","title":"Pydantic AI — four official solutions","url":"https://pydantic.dev/docs/ai/integrations/durable_execution/overview/","quote":"Pydantic AI officially supports four durable execution solutions: Temporal, DBOS, Prefect, Restate","accessed":"2026-05-20"},{"type":"api-ref","title":"Pydantic AI — TemporalAgent","url":"https://pydantic.dev/docs/ai/api/pydantic-ai/durable_exec/","quote":"Wrap an agent to enable it to be used inside a Temporal workflow, by automatically offloading model requests, tool calls, and MCP server communication to Temporal activities.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"evaluator-optimizer","role":"limited","note":"pydantic_evals is an evaluation harness (Datasets, Cases, Evaluators, ReportEvaluators) that measures but does NOT close the loop to regenerate. Per-tool retries are handled by ModelRetry.","evidence":[{"type":"doc","title":"Pydantic Evals — framework framing","url":"https://pydantic.dev/docs/ai/evals/evals/","quote":"Pydantic Evals is a powerful evaluation framework for systematically testing and evaluating AI systems, from simple LLM calls to complex multi-agent applications.","accessed":"2026-05-20"},{"type":"api-ref","title":"Pydantic AI — ModelRetry","url":"https://pydantic.dev/docs/ai/api/pydantic-ai/exceptions/","quote":"Exception to raise to request a model retry.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"Three MCP transports: MCPServerStdio (subprocess), MCPServerStreamableHTTP, plus exposing Agents as MCP servers themselves.","evidence":[{"type":"doc","title":"Pydantic AI — MCP client","url":"https://pydantic.dev/docs/ai/mcp/overview/","quote":"Pydantic AI can act as an MCP client and connect directly to local and remote MCP servers.","accessed":"2026-05-20"},{"type":"doc","title":"Pydantic AI — MCP server hosting","url":"https://pydantic.dev/docs/ai/mcp/overview/","quote":"Agents can be used within MCP servers.","accessed":"2026-05-20"},{"type":"api-ref","title":"Pydantic AI — MCPServerStdio","url":"https://pydantic.dev/docs/ai/api/pydantic-ai/mcp/","quote":"Runs an MCP server in a subprocess and communicates with it over stdin/stdout.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"Node graph (UserPromptNode → ModelRequestNode → CallToolsNode → End) implements the canonical reason→tool→observation→reason cycle.","evidence":[{"type":"doc","title":"Pydantic AI — Agent loop shape","url":"https://pydantic.dev/docs/ai/core-concepts/agent/","quote":"the agent reasons, calls tools, observes results, and reasons again","accessed":"2026-05-20"},{"type":"doc","title":"Pydantic AI — pydantic-graph backed","url":"https://pydantic.dev/docs/ai/core-concepts/agent/","quote":"Under the hood, each Agent in Pydantic AI uses pydantic-graph to manage its execution flow.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"step-budget","role":"first-class","note":"UsageLimits caps requests, tokens, and tool calls before each request; UsageLimitExceeded raised on breach.","evidence":[{"type":"doc","title":"Pydantic AI — UsageLimits","url":"https://pydantic.dev/docs/ai/core-concepts/agent/","quote":"Pydantic AI offers a UsageLimits structure to help you limit your usage (tokens, requests, and tool calls) on model runs.","accessed":"2026-05-20"},{"type":"api-ref","title":"Pydantic AI — request limit","url":"https://pydantic.dev/docs/ai/api/pydantic-ai/usage/","quote":"The request count is tracked by pydantic_ai, and the request limit is checked before each request.","accessed":"2026-05-20"},{"type":"api-ref","title":"Pydantic AI — tool_calls_limit","url":"https://pydantic.dev/docs/ai/api/pydantic-ai/usage/","quote":"The maximum number of successful tool calls allowed to be executed.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"structured-output","role":"core","note":"The framework's defining shape. Three strategies — Tool Output, Native Output, Prompted Output. Validation failure forces a retry.","evidence":[{"type":"doc","title":"Pydantic AI overview — JSON schema from Pydantic","url":"https://pydantic.dev/docs/ai/overview/","quote":"This Pydantic model is used to constrain the structured data returned by the agent. From this simple definition, Pydantic builds the JSON Schema that tells the LLM how to return the data.","accessed":"2026-05-20"},{"type":"doc","title":"Pydantic AI — three output methods","url":"https://pydantic.dev/docs/ai/core-concepts/output/","quote":"Pydantic AI implements three different methods to get a model to output structured data: Tool Output, Native Output, and Prompted Output.","accessed":"2026-05-20"},{"type":"repo","title":"pydantic/pydantic-ai README — validation-retry","url":"https://github.com/pydantic/pydantic-ai","quote":"The response from the agent will be guaranteed to be a SupportOutput, if validation fails the agent is prompted to try again.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"@agent.tool decorator + RunContext access; @agent.tool_plain for context-free tools; schemas auto-extracted from function signature + docstring.","evidence":[{"type":"doc","title":"Pydantic AI — @agent.tool","url":"https://pydantic.dev/docs/ai/tools-toolsets/tools/","quote":"There are a number of ways to register tools with an agent: via the @agent.tool decorator — for tools that need access to the agent context","accessed":"2026-05-20"},{"type":"doc","title":"Pydantic AI — schema from signature","url":"https://pydantic.dev/docs/ai/tools-toolsets/tools/","quote":"Function parameters are extracted from the function signature, and all parameters except RunContext are used to build the schema","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp-server-side-sampling","role":"supported","note":"","evidence":[{"type":"doc","title":"MCP Client - Pydantic AI (Sampling)","url":"https://ai.pydantic.dev/mcp/client/","quote":"In MCP sampling is a system by which an MCP server can make LLM calls via the MCP client - effectively proxying requests to an LLM via the client over whatever transport is being used.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"stochastic-deterministic-boundary","role":"first-class","note":"Output validators verify each proposed model output and raise ModelRetry to reject and re-ask, so an output must pass validation (consuming the run's retry budget) before it is committed.","evidence":[{"type":"doc","title":"Output | Pydantic AI Docs","url":"https://ai.pydantic.dev/output/","quote":"you can raise ModelRetry to ask the model to try again with modified arguments (or with a different output type).","accessed":"2026-06-17"},{"type":"doc","title":"Output | Pydantic AI Docs","url":"https://ai.pydantic.dev/output/","quote":"Each ModelRetry raised here consumes one unit of the run's output retry budget.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"polymorphic-record","role":"supported","note":"Pydantic discriminated (tagged) unions model one record type with type-tagged variant payloads validated by a discriminator field.","evidence":[{"type":"doc","url":"https://docs.pydantic.dev/latest/concepts/unions/","quote":"Discriminated unions are sometimes referred to as Tagged unions.","accessed":"2026-06-19"}],"evidence_status":"full"},{"pattern":"mcp-bidirectional-bridge","role":"supported","note":"Pydantic-AI documents both directions — agents can connect to MCP servers and agents can be used within MCP servers.","evidence":[{"type":"doc","url":"https://pydantic.dev/docs/ai/mcp/overview/","quote":"Pydantic-AI documents both directions — agents can connect to MCP servers and agents can be used within MCP servers.","accessed":"2026-06-19"}],"evidence_status":"full"}],"anti_patterns_avoided":[{"pattern":"schema-free-output","note":"Pydantic AI rejects free-text outputs by design — validation failure triggers a retry, not a silent pass.","evidence":[{"type":"repo","title":"pydantic/pydantic-ai README","url":"https://github.com/pydantic/pydantic-ai","quote":"The response from the agent will be guaranteed to be a SupportOutput, if validation fails the agent is prompted to try again.","accessed":"2026-05-20"}]},{"pattern":"unbounded-loop","note":"UsageLimits enforces request, token, and tool-call ceilings before each request.","evidence":[{"type":"api-ref","title":"Pydantic AI — request limit checked before each request","url":"https://pydantic.dev/docs/ai/api/pydantic-ai/usage/","quote":"The request count is tracked by pydantic_ai, and the request limit is checked before each request.","accessed":"2026-05-20"}]}],"instantiates":["structured-output-stack","long-running-autonomous-agent"],"alternatives":[{"composition":"langchain","relation":"competes-with","note":"Broader, JSON-tool-call-first; Pydantic AI's output_type is comparable to LangChain's response_format but is the headline shape, not an option."},{"composition":"langgraph","relation":"competes-with","note":"Both target production-grade agents; LangGraph delivers durability via built-in Checkpointer, Pydantic AI delegates to Temporal/DBOS/Prefect/Restate."},{"composition":"smolagents","relation":"competes-with","note":"Closest stylistic peer in 'small library' framing."},{"composition":"crewai","relation":"similar-shape","note":"Both Python + Pydantic; CrewAI is multi-agent crew-first vs Pydantic AI's single-agent-first."},{"composition":"marvin","relation":"similar-shape","note":"Pydantic-typed-output-forward Python agent library."},{"composition":"strands-agents","relation":"similar-shape","note":"Peer Python-first typed agent SDK."},{"composition":"dspy","relation":"similar-shape","note":"Typed-I/O agent layer, but without compile-time prompt optimisation."},{"composition":"semantic-kernel","relation":"competes-with","note":"Type-first agent framework with similar plugin model."}],"references":[{"type":"repo","title":"pydantic/pydantic-ai","url":"https://github.com/pydantic/pydantic-ai","quote":"AI Agent Framework, the Pydantic way","accessed":"2026-05-20"},{"type":"doc","title":"Pydantic AI overview","url":"https://pydantic.dev/docs/ai/overview/","quote":"A Python agent framework designed to help you quickly, confidently, and painlessly build production grade applications and workflows with Generative AI.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-20","verification_status":"verified","last_updated":"2026-06-14","date_added":"2026-04-30","status_in_practice":"emerging","tags":["open-source","python","pydantic-typed","durable-execution","mcp"]},{"id":"qwen-agent","name":"Qwen-Agent","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"Alibaba Qwen team (QwenLM)","language":"Python","license":"Apache-2.0","status":"active","first_released":"2023-09-22","url":"https://github.com/QwenLM/Qwen-Agent","repo":"https://github.com/QwenLM/Qwen-Agent","docs_url":"https://github.com/QwenLM/Qwen-Agent","intent":"Provide a Python framework for building LLM applications that exercise the instruction-following, tool-use, planning, and memory capabilities of Alibaba's Qwen models, with built-in support for function calling, MCP, a Docker-sandboxed code interpreter, and RAG over very long documents.","description":"Qwen-Agent is the Apache-2.0 framework maintained by Alibaba's Qwen team for building agentic applications on top of Qwen models (Qwen >= 3.0). It ships in two languages — the English README and a parallel 中文 README_CN — and centres on three first-class capabilities advertised in the GitHub repo description: Function Calling, MCP, Code Interpreter, RAG. Distinctive characteristics: tight coupling to Qwen's native tool-call template (parallel function calls supported by default), Mandarin-first documentation with full Chinese-language abstractions and prompts, and a bundled fast-RAG pipeline aimed at super-long-document QA rather than generic vector-store glue.","primary_use_cases":["Qwen-model-native function calling and tool use","RAG over super-long documents","ReAct-style agents with sandboxed code execution","agentic apps with Mandarin-first prompts and docs"],"agent_loop_shape":"Class-hierarchy-based imperative loop. The Agent base class defines an iterator interface; subclasses compose function calling, ReAct-style reasoning, and RAG: FnCallAgent is the function-calling base, ReActChat wraps the loop in a ReAct Thought/Action/Observation prompt, Assistant layers RAG over function-calling. Multi-agent shapes (group_chat, group_chat_auto_router, router) live in sibling modules.","key_concepts":[{"name":"class Agent","summary":"Base class: 'An agent can receive messages and provide response by LLM or Tools. Different agents have distinct workflows for processing messages and generating responses in the _run method.'","url":"https://github.com/QwenLM/Qwen-Agent/blob/main/qwen_agent/agent.py"},{"name":"class FnCallAgent","summary":"Function-calling base agent integrating LLM and tool use.","url":"https://github.com/QwenLM/Qwen-Agent/blob/main/qwen_agent/agents/fncall_agent.py"},{"name":"class Assistant","summary":"Default agent class integrating RAG and function calling.","maps_to_pattern":"agentic-rag","url":"https://github.com/QwenLM/Qwen-Agent/blob/main/qwen_agent/agents/assistant.py"},{"name":"class ReActChat","summary":"ReAct-format agent carrying the embedded Thought/Action/Action Input/Observation prompt template.","maps_to_pattern":"react","url":"https://github.com/QwenLM/Qwen-Agent/blob/main/qwen_agent/agents/react_chat.py"},{"name":"Code Interpreter tool","summary":"Docker-sandboxed code execution as a built-in tool the agent invokes autonomously.","maps_to_pattern":"code-as-action","url":"https://github.com/QwenLM/Qwen-Agent"}],"pattern_composition":"flowchart TD\n  USER[User query] --> AGENT_BASE[class Agent<br/>base — _run iterator]\n  AGENT_BASE -.->|inherits| FN[class FnCallAgent<br/>function calling]\n  FN -.->|inherits| REACT[class ReActChat<br/>Thought/Action/Observation prompt]\n  FN -.->|inherits| ASSIST[class Assistant<br/>RAG + function calling]\n  REACT -->|tool call| TOOLS[(Tools<br/>built-in or @tool)]\n  ASSIST -->|tool call| TOOLS\n  TOOLS --> CI[Code Interpreter<br/>Docker-sandboxed]\n  TOOLS --> MCP[MCP servers]\n  ASSIST --> RAG[(Long-document RAG<br/>doc_qa subpackage)]\n  RAG --> KB[(Knowledge base files)]\n  AGENT_BASE -.->|backed by| QWEN[Qwen >= 3.0 LLM]","members":[{"pattern":"agentic-rag","role":"first-class","note":"Bundled fast-RAG pipeline aimed at super-long-document QA; the default Assistant class integrates RAG and function calling. A dedicated qwen_agent/agents/doc_qa/ subpackage exists.","evidence":[{"type":"repo","title":"Qwen-Agent README — long-document RAG","url":"https://github.com/QwenLM/Qwen-Agent","quote":"We have released a fast RAG solution, as well as an expensive but competitive agent, for doing question-answering over super-long documents.","accessed":"2026-05-20"},{"type":"repo","title":"Qwen-Agent README_CN — long-document RAG (Mandarin)","url":"https://github.com/QwenLM/Qwen-Agent/blob/main/README_CN.md","quote":"我们已发布了一个快速的RAG解决方案，以及一个虽运行成本较高但准确度较高的智能体，用于在超长文档中进行问答。","accessed":"2026-05-20"},{"type":"repo","title":"assistant.py class docstring","url":"https://github.com/QwenLM/Qwen-Agent/blob/main/qwen_agent/agents/assistant.py","quote":"This is a widely applicable agent integrated with RAG capabilities and function call ability.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"code-as-action","role":"first-class","note":"The agent autonomously writes code and executes it in a Docker sandbox via the built-in Code Interpreter tool; observation = execution result.","evidence":[{"type":"repo","title":"Qwen-Agent README — Code Interpreter","url":"https://github.com/QwenLM/Qwen-Agent","quote":"We implement a code interpreter tool based on local Docker containers.","accessed":"2026-05-20"},{"type":"repo","title":"Qwen-Agent README — code interpreter usage","url":"https://github.com/QwenLM/Qwen-Agent","quote":"You can enable the built-in code interpreter tool for your agent, allowing it to autonomously write code according to specific scenarios, execute it securely within an isolated sandbox environment, and return the execution results.","accessed":"2026-05-20"},{"type":"repo","title":"Qwen-Agent README_CN — Code Interpreter (Mandarin)","url":"https://github.com/QwenLM/Qwen-Agent/blob/main/README_CN.md","quote":"我们提供了一种基于本地Docker容器的代码解释器实现。您可以为智能体启用内置的code interpreter工具。","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"MCP is one of the four headline features in the repo's own one-line description; users configure MCP servers from the open-source MCP catalog.","evidence":[{"type":"repo","title":"Qwen-Agent README — How to Use MCP","url":"https://github.com/QwenLM/Qwen-Agent","quote":"You can select the required tools on the open-source MCP server website and configure the relevant environment.","accessed":"2026-05-20"},{"type":"repo","title":"Qwen-Agent README_CN — How to Use MCP (Mandarin)","url":"https://github.com/QwenLM/Qwen-Agent/blob/main/README_CN.md","quote":"可以在开源的MCP Server网站上选择需要的工具，并配置相关环境。","accessed":"2026-05-20"},{"type":"api-ref","title":"Qwen-Agent repo description","url":"https://api.github.com/repos/QwenLM/Qwen-Agent","quote":"Agent framework and applications built upon Qwen>=3.0, featuring Function Calling, MCP, Code Interpreter, RAG, Chrome extension, etc.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"ReActChat is a named, exported agent class carrying the canonical ReAct prompt template (Thought / Action / Action Input / Observation).","evidence":[{"type":"repo","title":"react_chat.py class docstring","url":"https://github.com/QwenLM/Qwen-Agent/blob/main/qwen_agent/agents/react_chat.py","quote":"This agent use ReAct format to call tools","accessed":"2026-05-20"},{"type":"repo","title":"react_chat.py embedded ReAct prompt","url":"https://github.com/QwenLM/Qwen-Agent/blob/main/qwen_agent/agents/react_chat.py","quote":"Thought: you should always think about what to do Action: the action to take, should be one of [{tool_names}] Action Input: the input to the action Observation: the result of the action","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Function calling is built into the LLM base class; FnCallAgent, ReActChat, and Assistant all inherit from it.","evidence":[{"type":"repo","title":"Qwen-Agent README — function calling","url":"https://github.com/QwenLM/Qwen-Agent","quote":"Yes. The LLM classes provide function calling. Additionally, some Agent classes also are built upon the function calling capability, e.g., FnCallAgent and ReActChat.","accessed":"2026-05-20"},{"type":"repo","title":"Qwen-Agent README_CN — function calling (Mandarin)","url":"https://github.com/QwenLM/Qwen-Agent/blob/main/README_CN.md","quote":"支持，LLM类提供了函数调用的支持。此外，一些Agent类如FnCallAgent和ReActChat也是基于函数调用功能构建的。","accessed":"2026-05-20"},{"type":"repo","title":"fncall_agent.py class docstring","url":"https://github.com/QwenLM/Qwen-Agent/blob/main/qwen_agent/agents/fncall_agent.py","quote":"This is a widely applicable function call agent integrated with llm and tool use ability.","accessed":"2026-05-20"}],"evidence_status":"full"}],"instantiates":["production-rag"],"alternatives":[{"composition":"modelscope-agent","relation":"same-vendor","note":"Sister project from Alibaba's ModelScope team — also Python, also ReAct + tool use, slightly different abstractions."},{"composition":"agentscope","relation":"same-vendor","note":"Alibaba DAMO Academy framework with a more multi-agent/conversational shape; not Qwen-coupled."},{"composition":"langchain","relation":"similar-shape","note":"Western OSS counterpart — comparable function-calling + RAG + ReAct surface, model-agnostic rather than Qwen-coupled."},{"composition":"crewai","relation":"similar-shape","note":"Multi-agent role-based orchestration alternative; less RAG-centric."},{"composition":"llamaindex","relation":"similar-shape","note":"Stronger RAG focus; weaker on Qwen-native function-calling templates."}],"references":[{"type":"repo","title":"QwenLM/Qwen-Agent","url":"https://github.com/QwenLM/Qwen-Agent","quote":"Agent framework and applications built upon Qwen>=3.0, featuring Function Calling, MCP, Code Interpreter, RAG, Chrome extension, etc.","accessed":"2026-05-20"},{"type":"repo","title":"QwenLM/Qwen-Agent 中文 README","url":"https://github.com/QwenLM/Qwen-Agent/blob/main/README_CN.md","accessed":"2026-05-20"}],"last_analyzed":"2026-05-20","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"emerging","tags":["open-source","chinese-market","qwen-native","rag","mandarin-docs"]},{"id":"ragflow","name":"RAGFlow","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"InfiniFlow","language":"Python, TypeScript","license":"Apache-2.0","status":"active","url":"https://ragflow.io/","repo":"https://github.com/infiniflow/ragflow","docs_url":"https://ragflow.io/docs/","intent":"Open-source RAG engine that pairs deep document-understanding (DeepDoc) layout-aware parsing with an agentic, graph-orchestrated workflow runtime, MCP support, and an extensive citation/traceability surface.","description":"RAGFlow (78k+ GitHub stars, Apache-2.0) is InfiniFlow's open-source RAG engine. Its defining feature is DeepDoc — a layout-aware, template-driven document parser that handles PDF, Word, Excel, slides, scanned copies, structured data, and web pages with format-specific chunking strategies rather than naive splitting. From v0.8 onward RAGFlow became agentic: a graph-based workflow editor lets users compose retrieval + reasoning + tool stages, with MCP support, GraphRAG, and (from late 2025) memory for AI agents. Citations are first-class and traceable to source chunks with visualisations.","primary_use_cases":["enterprise document QA over complex format mixes (PDF, Office, scans, structured data)","agentic RAG with no-code workflow composition over retrieval + reasoning + tools","high-fidelity citation rendering with chunk-level provenance to source","MCP-server integration so external agents can call RAGFlow's retrieval as tools"],"agent_loop_shape":"Document ingestion runs DeepDoc parsing → template-based chunking → embedding → indexing (with optional GraphRAG knowledge-graph construction). At query time, a no-code graph workflow defines retrieval + rerank + reasoning + tool stages. Each query enters the graph at the start node and traverses with branch/loop semantics. MCP exposes retrieval to external agents.","key_concepts":[{"name":"DeepDoc","summary":"Layout-aware, template-driven parser for complex document formats.","url":"https://github.com/infiniflow/ragflow/blob/main/deepdoc/README.md"},{"name":"Graph workflow editor","summary":"No-code DAG for composing RAG + agent stages.","maps_to_pattern":"orchestrator-workers"},{"name":"Template-based chunking","summary":"Per-format chunking templates instead of naive token splits."},{"name":"Citation rendering","summary":"Traceable chunk-level citations attached to answers.","maps_to_pattern":"citation-attribution"}],"pattern_composition":"flowchart TD\n  subgraph Ingest\n    DOC[(Documents<br/>PDF/Word/Excel/scans)] --> DEEPDOC[DeepDoc parser<br/>layout-aware, template-based]\n    DEEPDOC --> CHUNK[Per-format chunking]\n    CHUNK --> IDX[(Index:<br/>vector + sparse + optional KG)]\n  end\n  subgraph Query\n    Q[User query] --> WORKFLOW[Graph workflow editor<br/>no-code DAG]\n    WORKFLOW --> RET[Retrieval node]\n    RET --> RERANK[Rerank node]\n    RERANK --> REASON[Reasoning / generator]\n    REASON --> CITE[Citation node]\n    CITE --> ANS[Answer + traceable citations]\n  end\n  IDX --> RET\n  MCP[MCP server] -.expose retrieval.-> EXT[External agents]","members":[{"pattern":"naive-rag","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"agentic-rag","role":"first-class","note":"v0.8+ explicitly agentic.","evidence":[],"evidence_status":"none"},{"pattern":"hybrid-search","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"cross-encoder-reranking","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"graphrag","role":"first-class","note":"GraphRAG supported as a knowledge graph option.","evidence":[],"evidence_status":"none"},{"pattern":"citation-attribution","role":"first-class","note":"Chunk-level citations with visualisation.","evidence":[],"evidence_status":"none"},{"pattern":"citation-streaming","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"contextual-retrieval","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"mcp","role":"first-class","note":"MCP server exposes retrieval as tools.","evidence":[],"evidence_status":"none"},{"pattern":"query-rewriting","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"event-driven-agent","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"orchestrator-workers","role":"supported","evidence":[],"evidence_status":"none"}],"instantiates":["production-rag"],"alternatives":[{"composition":"llamaindex","relation":"competes-with","note":"Both are full RAG frameworks; RAGFlow emphasises DeepDoc parsing and no-code workflow editor."},{"composition":"langchain","relation":"competes-with","note":"LangChain is broader; RAGFlow is RAG-specialised with document-understanding as the differentiator."},{"composition":"dbgpt","relation":"similar-shape","note":"Both are Chinese open-source agentic data platforms; DB-GPT emphasises structured/SQL, RAGFlow emphasises unstructured documents."},{"composition":"kag","relation":"similar-shape","note":"RAGFlow is the document-RAG peer; KAG specialises on rigorous KG-based reasoning."}],"last_analyzed":"2026-05-22","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-05-22","status_in_practice":"mature","tags":["orchestration-framework","retrieval","streaming-ux","tool-use-environment","planning-control-flow","multi-agent","full-code"],"references":[{"type":"doc","title":"RAGFlow — Documentation","url":"https://ragflow.io/docs/"},{"type":"doc","title":"RAGFlow — Homepage","url":"https://ragflow.io/"},{"type":"repo","title":"RAGFlow — Source repository","url":"https://github.com/infiniflow/ragflow"}]},{"id":"semantic-kernel","name":"Semantic Kernel","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"Microsoft","language":".NET, Python, Java","license":"MIT","status":"maintenance","successor":"microsoft-agent-framework","first_released":"2023-02-27","url":"https://aka.ms/semantic-kernel","repo":"https://github.com/microsoft/semantic-kernel","docs_url":"https://learn.microsoft.com/en-us/semantic-kernel/","intent":"Lightweight model-agnostic SDK (C#/Python/Java) that turns existing code into Plugins of KernelFunctions so LLMs can call them via auto function calling, with first-class observability, MCP/OpenAPI extensibility, and an Agent Framework for single/multi-agent orchestration.","description":"Semantic Kernel is Microsoft's open-source middleware that wraps any chat-completion model behind a Kernel and exposes user code as KernelFunctions grouped into Plugins. The framework drives an auto function-calling loop (FunctionChoiceBehavior.Auto()) that serialises plugin schemas as JSON, lets the model emit function calls, marshals arguments, invokes the function, and feeds results back — repeating until the model returns a chat reply. Plugins import natively or from OpenAPI specs or MCP Servers (MCPStdioPlugin, MCPSsePlugin, MCPStreamableHttpPlugin). The Agent Framework adds ChatCompletionAgent, OpenAIAssistantAgent, AzureAIAgent, and OpenAIResponsesAgent plus an orchestration package. OpenTelemetry-compatible logs/metrics/spans emit as Microsoft.SemanticKernel activities. Now succeeded by Microsoft Agent Framework 1.0; SK itself remains stable with v1.x releases continuing through late 2025.","primary_use_cases":[".NET / Python / Java enterprise apps adding LLM tool calling","Plugin-based extension of LLMs with native code, OpenAPI, or MCP","Multi-agent orchestration via Agent Framework","OpenTelemetry-instrumented LLM apps in regulated environments"],"agent_loop_shape":"Kernel sits between chat-completion service and user code. With FunctionChoiceBehavior.Auto() set, SK serialises all registered plugin functions to JSON schema, sends them with chat history, parses any tool_calls the model returns, marshals JSON into the function's typed parameters, invokes the function (optionally with DI), appends a FunctionCallContent/FunctionResultContent pair to the chat history, and re-prompts the model — looping until the model returns a chat message or a max-iteration cap is reached. Agent Framework instantiates ChatCompletionAgent / OpenAIAssistantAgent / AzureAIAgent / OpenAIResponsesAgent over this same kernel + plugin substrate.","key_concepts":[{"name":"Kernel","summary":"Middleware container coordinating chat services + plugins.","url":"https://learn.microsoft.com/en-us/semantic-kernel/overview/"},{"name":"Plugin","summary":"Group of KernelFunctions exposed to the LLM as callable tools.","maps_to_pattern":"skill-library","url":"https://learn.microsoft.com/en-us/semantic-kernel/concepts/plugins/"},{"name":"Auto function calling","summary":"FunctionChoiceBehavior.Auto() drives the JSON-schema tool-call loop.","maps_to_pattern":"tool-use","url":"https://learn.microsoft.com/en-us/semantic-kernel/concepts/ai-services/chat-completion/function-calling/"},{"name":"Agent Framework","summary":"ChatCompletionAgent / OpenAIAssistantAgent / AzureAIAgent over the kernel.","url":"https://learn.microsoft.com/en-us/semantic-kernel/frameworks/agent/"},{"name":"MCP plugins","summary":"MCPStdioPlugin, MCPSsePlugin, MCPStreamableHttpPlugin import MCP-server tools as plugins.","maps_to_pattern":"mcp","url":"https://learn.microsoft.com/en-us/semantic-kernel/concepts/plugins/adding-mcp-plugins"},{"name":"OpenTelemetry observability","summary":"Spans + metrics + logs emitted under Microsoft.SemanticKernel namespace.","url":"https://learn.microsoft.com/en-us/semantic-kernel/concepts/enterprise-readiness/observability/"}],"pattern_composition":"flowchart TD\n  USER[User input] --> KERNEL[Kernel<br/>orchestrator]\n  KERNEL --> CHAT[IChatCompletionService<br/>FunctionChoiceBehavior.Auto]\n  CHAT -->|tool_call| INVOKE{Auto function<br/>invocation loop}\n  INVOKE -->|serialize JSON schema| PLUGINS[(Plugins<br/>KernelFunction)]\n  PLUGINS -->|native code| NATIVE[Native plugin]\n  PLUGINS -->|OpenAPI spec| OPENAPI[OpenAPI plugin]\n  PLUGINS -->|MCP server| MCP[MCPStdioPlugin / MCPSsePlugin]\n  INVOKE -->|FunctionResultContent| CHAT\n  CHAT -->|final ChatResponse| OUT[Reply]\n  KERNEL -.-> AGENTS[Agent Framework<br/>ChatCompletionAgent<br/>AzureAIAgent<br/>OpenAIAssistantAgent]\n  AGENTS -.orchestration package.-> MULTI[Multi-agent<br/>collaboration]\n  KERNEL -.OpenTelemetry.-> OTEL[(spans + metrics + histograms)]","members":[{"pattern":"agent-resumption","role":"limited","note":"Per-turn ChatHistory can be serialised by the caller; no built-in checkpoint/resume API like LangGraph's Checkpointer.","evidence":[{"type":"doc","title":"Semantic Kernel — AgentThread abstraction","url":"https://learn.microsoft.com/en-us/semantic-kernel/frameworks/agent/agent-architecture","quote":"The abstract `AgentThread` class serves as the core abstraction for threads or conversation state. It abstracts away the different ways in which conversation state may be managed for different agents.","accessed":"2026-05-24"},{"type":"doc","title":"Semantic Kernel — conversation state managed locally or in service","url":"https://learn.microsoft.com/en-us/semantic-kernel/frameworks/agent/agent-architecture","quote":"Stateful agent services often store conversation state in the service, and you can interact with it via an id. Other agents may require the entire chat history to be passed to the agent on each invocation, in which case the conversation state is managed locally in the application.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"Multiple MCP plugin types (stdio, SSE, Streamable HTTP); MCP is a first-class plugin importer.","evidence":[{"type":"doc","title":"Semantic Kernel — MCP plugins","url":"https://learn.microsoft.com/en-us/semantic-kernel/concepts/plugins/adding-mcp-plugins","quote":"Semantic Kernel allows you to add plugins from a MCP Server to your agents.","accessed":"2026-05-20"},{"type":"doc","title":"Semantic Kernel — MCP types","url":"https://learn.microsoft.com/en-us/semantic-kernel/concepts/plugins/adding-mcp-plugins","quote":"Semantic Kernel supports multiple MCP plugin types, including MCPStdioPlugin and MCPStreamableHttpPlugin.","accessed":"2026-05-20"},{"type":"repo","title":"microsoft/semantic-kernel README","url":"https://github.com/microsoft/semantic-kernel","quote":"Extend with native code functions, prompt templates, OpenAPI specs, or Model Context Protocol (MCP)","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"limited","note":"SK does not name ReAct; tool loop is auto function calling. Thought/action/observation pattern is emergent rather than a named module.","evidence":[{"type":"doc","title":"Semantic Kernel — function calling loop (steps 2-6 repeat)","url":"https://learn.microsoft.com/en-us/semantic-kernel/concepts/ai-services/chat-completion/function-calling/","quote":"The result of the function is then sent back to the model as part of the chat history. Steps 2-6 are then repeated until the model returns a chat message or the max iteration number has been reached.","accessed":"2026-05-24"},{"type":"doc","title":"Semantic Kernel — model decides when to call functions","url":"https://learn.microsoft.com/en-us/semantic-kernel/concepts/ai-services/chat-completion/function-calling/","quote":"With function calling, the model can decide *when* to call a function and *what* information to provide.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"skill-library","role":"first-class","note":"Plugins are the canonical skill library — three import paths (native, OpenAPI, MCP).","evidence":[{"type":"doc","title":"Semantic Kernel — Plugins","url":"https://learn.microsoft.com/en-us/semantic-kernel/concepts/plugins/","quote":"Plugins are a key component of Semantic Kernel...With plugins, you can encapsulate your existing APIs into a collection that can be used by an AI.","accessed":"2026-05-20"},{"type":"doc","title":"Semantic Kernel — Plugin definition","url":"https://learn.microsoft.com/en-us/semantic-kernel/concepts/plugins/","quote":"a plugin is a group of functions that can be exposed to AI apps and services.","accessed":"2026-05-20"},{"type":"doc","title":"Semantic Kernel — Plugin import paths","url":"https://learn.microsoft.com/en-us/semantic-kernel/concepts/plugins/","quote":"There are three primary ways of importing plugins into Semantic Kernel: using native code, using an OpenAPI specification or from a MCP Server","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"structured-output","role":"limited","note":"Function parameters are JSON-schema-typed and return types can be Pydantic/typed classes, but no top-level response_model API at the Kernel level.","evidence":[{"type":"doc","title":"Semantic Kernel — JSON schema serialisation","url":"https://learn.microsoft.com/en-us/semantic-kernel/concepts/ai-services/chat-completion/function-calling/","quote":"All of the available functions (and its input parameters) in the kernel are serialized using JSON schema.","accessed":"2026-05-20"},{"type":"doc","title":"Semantic Kernel — parameter type / enum schema","url":"https://learn.microsoft.com/en-us/semantic-kernel/concepts/ai-services/chat-completion/function-calling/","quote":"With the schema, you can specify the type of each parameter. This is important for the model to understand the expected input.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"supervisor","role":"limited","note":"Agents.Orchestration package provides multi-agent collaboration; no first-class supervisor abstraction.","evidence":[{"type":"doc","title":"Semantic Kernel — Agent collaboration","url":"https://learn.microsoft.com/en-us/semantic-kernel/frameworks/agent/","quote":"Multiple agents may 'collaborate' on tasks. For example, one agent might handle data collection while another analyzes it and yet another uses the results to make decisions","accessed":"2026-05-20"},{"type":"doc","title":"Semantic Kernel — Orchestration package","url":"https://learn.microsoft.com/en-us/semantic-kernel/frameworks/agent/","quote":"Provides the orchestration framework for the Agent Framework.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Auto function calling is the framework's defining loop.","evidence":[{"type":"doc","title":"Semantic Kernel — function calling headline","url":"https://learn.microsoft.com/en-us/semantic-kernel/concepts/ai-services/chat-completion/function-calling/","quote":"The most powerful feature of chat completion is the ability to call functions from the model.","accessed":"2026-05-20"},{"type":"doc","title":"Semantic Kernel — auto-described functions","url":"https://learn.microsoft.com/en-us/semantic-kernel/concepts/ai-services/chat-completion/function-calling/","quote":"we simplify the process of using function calling by automatically describing your functions and their parameters to the model and then handling the back-and-forth communication","accessed":"2026-05-20"},{"type":"doc","title":"Semantic Kernel — function calling rationale","url":"https://learn.microsoft.com/en-us/semantic-kernel/concepts/plugins/","quote":"Semantic Kernel leverages function calling, a native feature of most of the latest LLMs to allow LLMs, to perform planning and to invoke your APIs.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp-server-side-sampling","role":"supported","note":"","evidence":[{"type":"blog","title":"Semantic Kernel adds Model Context Protocol (MCP) support for Python - Microsoft DevBlogs","url":"https://devblogs.microsoft.com/semantic-kernel/semantic-kernel-adds-model-context-protocol-mcp-support-for-python/","quote":"By default, whenever a MCP Plugin is added to a Kernel, either directly, or through an agent, it will be able to use all of the Chat Completion services registered in that kernel to perform sampling with","accessed":"2026-06-17"}],"evidence_status":"limited"}],"instantiates":["structured-output-stack","eval-and-observability","multi-agent-coordination"],"alternatives":[{"composition":"langchain","relation":"competes-with","note":"Both are general orchestration SDKs spanning multiple languages."},{"composition":"autogen","relation":"same-vendor","note":"Microsoft's other agent framework; partly converging into Microsoft Agent Framework."},{"composition":"pydantic-ai","relation":"competes-with","note":"Type-first agent framework with similar plugin model."},{"composition":"microsoft-agent-framework","relation":"successor","note":"Semantic Kernel's plugin and connector model was merged into this framework."}],"references":[{"type":"repo","title":"microsoft/semantic-kernel","url":"https://github.com/microsoft/semantic-kernel","accessed":"2026-05-20"},{"type":"doc","title":"Semantic Kernel overview","url":"https://learn.microsoft.com/en-us/semantic-kernel/overview/","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","microsoft","dotnet","python","java","succeeded-by-maf"]},{"id":"smolagents","name":"smolagents","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"Hugging Face (Paris, France)","language":"Python","license":"Apache-2.0","status":"active","first_released":"2024-12-05","url":"https://huggingface.co/docs/smolagents","repo":"https://github.com/huggingface/smolagents","docs_url":"https://huggingface.co/docs/smolagents","intent":"Provide a barebones (~1k LoC) Python library for multi-step ReAct agents whose default action format is executable Python code rather than JSON tool calls, with first-class sandboxed execution and Hugging Face Hub sharing.","description":"smolagents is Hugging Face's Apache-2.0 Python library for 'agents that think in code'. Its distinguishing design choice is CodeAgent: the LLM writes each action as a Python code snippet that is parsed and executed by a custom AST-walking interpreter (LocalPythonExecutor) or a remote sandbox (e2b, blaxel, modal, docker, wasm), with the print output returned as the observation for the next ReAct step. Alongside CodeAgent it also ships ToolCallingAgent for the conventional JSON tool-calling shape. The framework deliberately keeps abstractions minimal — the docs claim 'The logic for agents fits in ~thousand lines of code' — and integrates with the Hugging Face Hub for sharing agents and tools.","primary_use_cases":["code-as-action agents that emit Python instead of JSON tool calls","ReAct agents with sandboxed execution (local or e2b/modal/docker)","Hub-shareable agents and tools","minimal-footprint replacements for larger frameworks"],"agent_loop_shape":"MultiStepAgent runs a textbook ReAct loop: SystemPromptStep + TaskStep go to memory, then while budget allows the agent serialises memory to chat messages, calls the Model, parses the completion into an action (Python code for CodeAgent, JSON tool call for ToolCallingAgent), executes the action (LocalPythonExecutor or remote sandbox), and records the result as an ActionStep observation. Iteration stops on final_answer or max_steps (default 20). An optional planning_interval inserts PlanningStep revisions without performing a tool call.","key_concepts":[{"name":"CodeAgent","summary":"Writes actions as Python code snippets that are parsed and executed; enables natural composability (function nesting, loops, conditionals).","maps_to_pattern":"code-as-action","url":"https://huggingface.co/docs/smolagents/index"},{"name":"ToolCallingAgent","summary":"Conventional JSON/text-based tool-calling agent for users who prefer that paradigm.","url":"https://huggingface.co/docs/smolagents/index"},{"name":"Tool / @tool","summary":"A callable function with metadata the LLM uses; @tool decorator is the recommended way to define simple tools.","maps_to_pattern":"tool-use","url":"https://huggingface.co/docs/smolagents/tutorials/tools"},{"name":"executor_type","summary":"Sandbox selector — local | blaxel | e2b | modal | docker | wasm. Default 'local' uses LocalPythonExecutor.","url":"https://huggingface.co/docs/smolagents/tutorials/secure_code_execution"},{"name":"LocalPythonExecutor","summary":"AST-walking interpreter: imports allow-listed, op count capped; safer than exec() but not fully secure.","url":"https://huggingface.co/docs/smolagents/tutorials/secure_code_execution"},{"name":"managed_agents","summary":"Constructor parameter that gives an agent a list of sub-agents to call like tools.","maps_to_pattern":"agent-as-tool-embedding","url":"https://huggingface.co/docs/smolagents/examples/multiagents"}],"pattern_composition":"flowchart TD\n  TASK[Task] --> MEM[(Agent memory<br/>SystemPromptStep + TaskStep)]\n  MEM --> MS[MultiStepAgent loop]\n  MS --> MODEL[Model call]\n  MODEL --> PARSE{Parse completion}\n  PARSE -->|CodeAgent| CODE[Python code snippet]\n  PARSE -->|ToolCallingAgent| JSON[JSON tool call]\n  CODE --> EXEC[LocalPythonExecutor<br/>or e2b/modal/docker sandbox]\n  JSON --> TOOL[Tool function]\n  EXEC --> OBS[ActionStep observation]\n  TOOL --> OBS\n  OBS --> MS\n  MS -->|final_answer| OUT[Final answer]\n  MS -.->|managed_agents call| SUB[Sub-agent<br/>also a MultiStepAgent]\n  SUB -.->|returns final| MS","members":[{"pattern":"agent-as-tool-embedding","role":"first-class","note":"Managed agents are passed via managed_agents=[...] on a parent agent; the parent calls them through the same prompt scaffolding as tools.","evidence":[{"type":"doc","title":"smolagents — Building good agents","url":"https://huggingface.co/docs/smolagents/tutorials/building_good_agents","quote":"You can also give tasks to team members. Calling a team member works similarly to calling a tool: provide the task description as the 'task' argument.","accessed":"2026-05-20"},{"type":"doc","title":"smolagents — Multi-agents example","url":"https://huggingface.co/docs/smolagents/examples/multiagents","quote":"Then we create a manager agent, and upon initialization we pass our managed agent to it in its `managed_agents` argument.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"code-as-action","role":"core","note":"The framework's defining thesis. CodeAgent writes actions as code rather than JSON tool calls. Bumped from first-class to core because removing this pattern removes the framework.","evidence":[{"type":"repo","title":"huggingface/smolagents README","url":"https://github.com/huggingface/smolagents","quote":"Our `CodeAgent` writes its actions in code (as opposed to 'agents being used to write code').","accessed":"2026-05-20"},{"type":"repo","title":"smolagents README — research support","url":"https://github.com/huggingface/smolagents","quote":"Writing actions as code snippets is demonstrated to work better than the current industry practice of letting the LLM output a dictionary of the tools it wants to call","accessed":"2026-05-20"},{"type":"doc","title":"smolagents — Intro to agents","url":"https://huggingface.co/docs/smolagents/conceptual_guides/intro_agents","quote":"we crafted our code languages specifically to express the actions performed by a computer... do you think their programming will be easier in blocks of Python or JSON?","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"react","role":"core","note":"MultiStepAgent IS a ReAct abstraction per upstream docs. Bumped from first-class to core because the whole agent class is described as that.","evidence":[{"type":"doc","title":"smolagents — ReAct guide","url":"https://huggingface.co/docs/smolagents/conceptual_guides/react","quote":"The ReAct framework (Yao et al., 2022) is currently the main approach to building agents.","accessed":"2026-05-20"},{"type":"doc","title":"smolagents — MultiStepAgent is a ReAct abstraction","url":"https://huggingface.co/docs/smolagents/conceptual_guides/react","quote":"All agents in `smolagents` are based on singular `MultiStepAgent` class, which is an abstraction of ReAct framework.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"step-budget","role":"first-class","note":"max_steps is a top-level constructor parameter with default 20.","evidence":[{"type":"api-ref","title":"smolagents — Agent API reference","url":"https://huggingface.co/docs/smolagents/reference/agents","quote":"max_steps (`int`, default `20`) : Maximum number of steps the agent can take to solve the task.","accessed":"2026-05-20"},{"type":"doc","title":"smolagents — MultiStepAgent ReAct cycle","url":"https://huggingface.co/docs/smolagents/reference/agents","quote":"Agent class that solves the given task step by step, using the ReAct framework: While the objective is not reached, the agent will perform a cycle of action (given by the LLM) and observation (obtained from the environment).","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"structured-output","role":"limited","note":"Structured outputs are opt-in: structured_output=True on MCPClient/ToolCollection, use_structured_outputs_internally=False default on CodeAgent.","evidence":[{"type":"doc","title":"smolagents — Tools / structured_output","url":"https://huggingface.co/docs/smolagents/tutorials/tools","quote":"To enable structured output support, pass `structured_output=True` when initializing the `MCPClient`.","accessed":"2026-05-20"},{"type":"doc","title":"smolagents — structured_output default","url":"https://huggingface.co/docs/smolagents/tutorials/tools","quote":"The `structured_output` parameter currently defaults to `False` to maintain backwards compatibility.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"subagent-isolation","role":"limited","note":"Managed agents are supported but Approach-1 sandboxing (per-snippet executor_type=e2b/blaxel/modal/docker) doesn't cover multi-agents; full multi-agent isolation requires running the whole agent in E2B.","evidence":[{"type":"doc","title":"smolagents — Secure code execution / multi-agent caveat","url":"https://huggingface.co/docs/smolagents/tutorials/secure_code_execution","quote":"Running individual code snippets in a sandbox... it's simpler to set up using `executor_type=\"blaxel\"`, `executor_type=\"e2b\"`, `executor_type=\"modal\"`, or `executor_type=\"docker\"`, but it doesn't support multi-agents","accessed":"2026-05-20"},{"type":"doc","title":"smolagents — multi-agents in E2B","url":"https://huggingface.co/docs/smolagents/tutorials/secure_code_execution","quote":"To use multi-agents in an E2B sandbox, you need to run your agents completely from within E2B.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Tools are the primary extension point; @tool decorator + first-class loaders for MCP, LangChain, and Hub Spaces.","evidence":[{"type":"doc","title":"smolagents — Tools","url":"https://huggingface.co/docs/smolagents/tutorials/tools","quote":"A tool is mostly a function that an LLM can use in an agentic system.","accessed":"2026-05-20"},{"type":"doc","title":"smolagents — index, tool ecosystem","url":"https://huggingface.co/docs/smolagents/index","quote":"You can use tools from any MCP server, from LangChain, you can even use a Hub Space as a tool.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-output-arithmetic-trust","role":"core","note":"Its first-class CodeAgent writes actions as executable code to perform computations rather than generating aggregates in free-form text, which is exactly the offload this anti-pattern omits.","evidence":[{"type":"doc","title":"smolagents documentation — index","url":"https://huggingface.co/docs/smolagents/en/index","quote":"CodeAgent writes its actions in code (as opposed to \"agents being used to write code\") to invoke tools or perform computations, enabling natural composability (function nesting, loops, conditionals).","accessed":"2026-06-14"}],"evidence_status":"full"},{"pattern":"incremental-model-querying","role":"core","note":"MultiStepAgent/CodeAgent generates the next action at each step from prior observations (Thought-Code-Observation cycle) rather than producing the whole plan upfront.","evidence":[{"type":"repo","title":"huggingface/smolagents","url":"https://github.com/huggingface/smolagents","quote":"To solve the task, the agent must proceed in a series of steps, in a cycle of 'Thought:', 'Code:', and 'Observation:' sequences ... each step consisting of one thought, then one tool call and execution.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"anti_patterns_avoided":[{"pattern":"json-only-action-schema","note":"smolagents explicitly argues that JSON tool-call dicts are inferior to executable code as the action format; supported by cited research and the @tool surface.","evidence":[{"type":"doc","title":"smolagents — Secure code execution / JSON critique","url":"https://huggingface.co/docs/smolagents/tutorials/secure_code_execution","quote":"Multiple research papers have shown that having the LLM write its actions (the tool calls) in code is much better than the current standard format for tool calling, which is across the industry different shades of 'writing actions as a JSON of tools names and arguments to use'.","accessed":"2026-05-20"}]}],"alternatives":[{"composition":"langchain","relation":"competes-with","note":"Same role layer, JSON-first tool-calling, much larger surface area."},{"composition":"langgraph","relation":"competes-with","note":"Same vendor space (LangChain Inc.); graph-of-nodes shape with checkpointed durable state — smolagents is much smaller and has no checkpointer."},{"composition":"crewai","relation":"competes-with","note":"Role-based multi-agent crews with JSON tool shape."},{"composition":"pydantic-ai","relation":"competes-with","note":"Closest stylistic peer in 'small library' framing; structured-output-first."},{"composition":"autogen","relation":"similar-shape","note":"Already marked code-as-action / code-execution first-class in this catalog."},{"composition":"metagpt","relation":"similar-shape","note":"Also code-as-action first-class."}],"references":[{"type":"repo","title":"huggingface/smolagents","url":"https://github.com/huggingface/smolagents","quote":"🤗 smolagents: a barebones library for agents that think in code.","accessed":"2026-05-20"},{"type":"blog","title":"Introducing smolagents","url":"https://huggingface.co/blog/smolagents","quote":"Today we are launching `smolagents`, a very simple library that unlocks agentic capabilities for language models.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"emerging","tags":["open-source","code-as-action","french-vendor","minimal","hub-shareable"]},{"id":"taskweaver","name":"TaskWeaver","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"Microsoft Research Asia (Beijing)","language":"Python","license":"MIT","status":"active","url":"https://microsoft.github.io/TaskWeaver/","repo":"https://github.com/microsoft/TaskWeaver","intent":"Code-first agent framework that converts user requests into executable Python code, preserves in-memory state (variables, DataFrames) across turns, and orchestrates user-defined plugins as callable functions.","description":"TaskWeaver (arXiv 2311.17541) is Microsoft Research Asia's code-first agent framework for data analytics. Its distinguishing design choice is preserving the Python interpreter's in-memory state across conversational turns — DataFrames, intermediate variables, and rich data structures persist so subsequent turns can refer to them by name rather than re-loading or re-serialising. A planner decomposes the user request into tasks, a code generator emits Python that calls user-defined plugins, and an executor runs the code with the persisted state. Code verification before execution and reflective adjustment after errors are first-class.","primary_use_cases":["multi-turn data analytics where in-memory state (DataFrames) carries across turns","domain adaptation via plugins encapsulating proprietary algorithms","verifiable code-execution flows (pre-execution check + post-execution reflection)","research baseline for code-first vs. text-first agent architectures"],"agent_loop_shape":"Planner emits a typed plan over the user request. CodeInterpreter generates Python that invokes the plan's user-defined plugins (each plugin is a typed Python callable). The Executor runs the code in a persistent Python interpreter session, so all state from prior turns is alive. Code is checked before execution and the agent reflects on errors to adjust subsequent code generation.","key_concepts":[{"name":"Code-first interface","summary":"User requests become executable Python rather than text-only plans.","maps_to_pattern":"code-as-action"},{"name":"Stateful interpreter session","summary":"Variables, DataFrames, and in-memory state persist across turns."},{"name":"Plugin functions","summary":"User-defined typed callables the code generator can invoke.","maps_to_pattern":"tool-use"},{"name":"Pre-execution code verification","summary":"Generated code is checked for safety and correctness before running.","maps_to_pattern":"sandbox-isolation"}],"pattern_composition":"flowchart TD\n  USER[Request] --> PLANNER[Planner<br/>typed plan]\n  PLANNER --> CI[Code Interpreter<br/>generate Python]\n  CI --> VERIFY[Pre-execution<br/>code verification]\n  VERIFY --> EXEC[Executor<br/>persistent Python session]\n  EXEC --> STATE[(In-memory state<br/>vars / DataFrames<br/>persists across turns)]\n  STATE --> EXEC\n  EXEC --> PLUGINS[User-defined<br/>plugin functions]\n  EXEC -->|error| REFLECT[Reflect on error]\n  REFLECT --> CI\n  EXEC --> RESULT[Return result]\n  RESULT --> PLANNER","members":[{"pattern":"code-as-action","role":"first-class","note":"Core architecture is code-first.","evidence":[],"evidence_status":"none"},{"pattern":"code-execution","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"plan-and-execute","role":"first-class","note":"Planner → CodeInterpreter → Executor.","evidence":[],"evidence_status":"none"},{"pattern":"hierarchical-agents","role":"first-class","note":"Planner over CodeInterpreter over Executor.","evidence":[],"evidence_status":"none"},{"pattern":"tool-use","role":"first-class","note":"Plugins as typed callables.","evidence":[],"evidence_status":"none"},{"pattern":"self-refine","role":"first-class","note":"Reflection on execution errors adjusts next code.","evidence":[],"evidence_status":"none"},{"pattern":"structured-output","role":"first-class","note":"Typed plan and typed plugins.","evidence":[],"evidence_status":"none"},{"pattern":"sandbox-isolation","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"step-budget","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"tool-output-arithmetic-trust","role":"core","note":"A code-first agent that interprets user requests as code snippets and executes them through plugins to perform data-analytics computation, routing aggregation to a deterministic run instead of in-head arithmetic.","evidence":[{"type":"doc","title":"TaskWeaver — Overview","url":"https://microsoft.github.io/TaskWeaver/docs/overview/","quote":"This innovative framework interprets user requests through coded snippets and efficiently coordinates a variety of plugins in the form of functions to execute data analytics or workflow automation tasks.","accessed":"2026-06-14"}],"evidence_status":"full"}],"alternatives":[{"composition":"openai-swarm","relation":"competes-with","note":"Both are code-first agent frameworks; Swarm uses agent-as-tool handoffs, TaskWeaver uses stateful Python sessions."},{"composition":"dspy","relation":"similar-shape","note":"Both treat the agent as code; DSPy adds prompt-program compilation, TaskWeaver adds session-stateful execution."},{"composition":"autogen","relation":"competes-with","note":"Same MSR family; AutoGen is conversation-first, TaskWeaver is code-first."},{"composition":"claude-agent-sdk","relation":"similar-shape"}],"last_analyzed":"2026-05-22","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-05-22","status_in_practice":"mature","tags":["orchestration-framework","tool-use-environment","planning-control-flow","multi-agent","verification-reflection","structure-data","safety-control","anti-patterns"],"references":[{"type":"doc","title":"TaskWeaver — Homepage","url":"https://microsoft.github.io/TaskWeaver/"},{"type":"repo","title":"TaskWeaver — Source repository","url":"https://github.com/microsoft/TaskWeaver"}]},{"id":"xagent","name":"XAgent","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"OpenBMB / Tsinghua","language":"Python","license":"Apache-2.0","status":"maintenance","url":"https://x-agent.net/","repo":"https://github.com/OpenBMB/XAgent","docs_url":"https://github.com/OpenBMB/XAgent-doc","intent":"Autonomous LLM agent for complex task solving with an outer planner / inner actor architecture, a dispatcher that spawns specialised sub-agents, a sandboxed ToolServer for actions, and human-in-the-loop assistance.","description":"XAgent is OpenBMB's general-purpose autonomous agent. Its distinctive architecture splits responsibilities across three components: a Planner that decomposes the user task into subtasks and milestones, an Actor (executor) that runs tools to achieve each subtask, and a Dispatcher that dynamically spawns specialised sub-agents and adds them to the agent pool. All actions run inside a Docker-isolated ToolServer that exposes File Editor, Python Notebook, Web Browser, Shell, and RapidAPI tools. Human-in-the-loop is first-class — the agent asks for guidance when blocked.","primary_use_cases":["autonomous research, data collection, and report generation tasks","research on outer-inner planner-executor architectures","Docker-isolated tool execution for safer agent experiments","human-in-the-loop assisted task completion"],"agent_loop_shape":"Three-tier loop. The outer Planner decomposes the user goal into a milestone-annotated subtask list. For each subtask, the Dispatcher selects (or spawns) a specialised Actor and hands it the subtask. The Actor runs an inner ReAct loop calling ToolServer tools (file edit, Python notebook, web browser, shell, RapidAPI) until the subtask succeeds, fails, or escalates. Failures bubble back to the Planner which can re-plan or ask the human for help.","key_concepts":[{"name":"Planner / Actor / Dispatcher","summary":"Three-role architectural split: planning, execution, agent provisioning.","maps_to_pattern":"hierarchical-agents"},{"name":"ToolServer","summary":"Docker-isolated tool execution environment shared by all Actors.","maps_to_pattern":"sandbox-isolation"},{"name":"Inner-outer loop","summary":"Outer planning loop wraps inner per-subtask ReAct execution loops.","maps_to_pattern":"hierarchical-agents"}],"pattern_composition":"flowchart TD\n  USER[User task] --> PLANNER[Outer Planner<br/>decompose into subtasks + milestones]\n  PLANNER --> DISPATCH[Dispatcher<br/>spawn or pick Actor]\n  DISPATCH --> ACTOR[Inner Actor<br/>ReAct loop per subtask]\n  ACTOR --> TS[ToolServer<br/>Docker-isolated]\n  TS --> T1[File Editor]\n  TS --> T2[Python Notebook]\n  TS --> T3[Web Browser]\n  TS --> T4[Shell]\n  TS --> T5[RapidAPI]\n  ACTOR -->|success / fail| DISPATCH\n  DISPATCH -->|escalate| HUMAN[Human-in-the-loop]\n  HUMAN --> PLANNER\n  DISPATCH -.all subtasks done.-> DONE[Return result]","members":[{"pattern":"hierarchical-agents","role":"first-class","note":"Outer planner over inner actor loops.","evidence":[],"evidence_status":"none"},{"pattern":"plan-and-execute","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"react","role":"first-class","note":"Inner Actor loop is ReAct.","evidence":[],"evidence_status":"none"},{"pattern":"orchestrator-workers","role":"first-class","note":"Dispatcher spawns Actor workers per subtask.","evidence":[],"evidence_status":"none"},{"pattern":"tool-use","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"sandbox-isolation","role":"first-class","note":"Docker-isolated ToolServer.","evidence":[],"evidence_status":"none"},{"pattern":"human-in-the-loop","role":"first-class","evidence":[],"evidence_status":"none"},{"pattern":"agent-resumption","role":"supported","evidence":[],"evidence_status":"none"},{"pattern":"code-execution","role":"first-class","note":"Python notebook tool.","evidence":[],"evidence_status":"none"},{"pattern":"browser-agent","role":"supported","note":"Web Browser tool.","evidence":[],"evidence_status":"none"}],"alternatives":[{"composition":"autogpt","relation":"competes-with","note":"Both are autonomous LLM agents; XAgent adds Docker-isolated ToolServer and explicit Planner/Actor/Dispatcher split."},{"composition":"babyagi","relation":"competes-with","note":"BabyAGI is the simpler ancestor of the autonomous-agent shape."},{"composition":"openmanus","relation":"similar-shape","note":"Both target general-purpose autonomy with sandboxed tools."},{"composition":"autogen","relation":"similar-shape","note":"AutoGen offers similar multi-agent shape without the outer-inner planner split."}],"last_analyzed":"2026-05-22","verification_status":"needs-verification","last_updated":"2026-06-17","date_added":"2026-05-22","status_in_practice":"emerging","tags":["orchestration-framework","multi-agent","planning-control-flow","tool-use-environment","safety-control","governance-observability","full-code"],"references":[{"type":"doc","title":"XAgent — Documentation","url":"https://github.com/OpenBMB/XAgent-doc"},{"type":"doc","title":"XAgent — Homepage","url":"https://x-agent.net/"},{"type":"repo","title":"XAgent — Source repository","url":"https://github.com/OpenBMB/XAgent"}]},{"id":"hipporag","name":"HippoRAG","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"OSU NLP Group","language":"Python","license":"Apache-2.0","status":"active","url":"https://github.com/OSU-NLP-Group/HippoRAG","repo":"https://github.com/OSU-NLP-Group/HippoRAG","intent":"Hippocampus-inspired RAG framework that builds a knowledge graph from documents and uses Personalized PageRank for multi-hop retrieval, replacing naive top-k vector search.","description":"HippoRAG is a research RAG framework from OSU NLP Group that draws on hippocampal indexing theory: documents are decomposed into entity-relation triples that form a knowledge graph, and retrieval runs Personalized PageRank from question entities across that graph. The shape targets multi-hop (associative) questions and sense-making over large corpora where naive dense retrieval fails. HippoRAG 2 (the current implementation) builds on the same PPR core and adds deeper passage integration and more effective online use of an LLM. It is distributed as a Python library and reference implementation accompanying the NeurIPS 2024 HippoRAG paper and the ICML 2025 HippoRAG 2 paper.","primary_use_cases":["multi-hop (associative) question answering over a document corpus","knowledge-graph-augmented retrieval pipelines","sense-making over long / interconnected contexts","research baselines comparing graph retrieval to dense top-k and to other graph-RAG systems (GraphRAG, RAPTOR, LightRAG)"],"agent_loop_shape":"Two-phase pipeline. Offline indexing: documents are passed through an LLM to extract OpenIE-style entity-relation triples that are merged into a persistent knowledge graph with vector embeddings on nodes. Online retrieval: at query time the system extracts entities from the question, seeds Personalized PageRank from those entity nodes over the KG, and returns the top-scoring passages associated with the highest-ranked nodes for an LLM reader (the rag_qa step in the Python API).","key_concepts":[{"name":"Hippocampal indexing theory","summary":"Cognitive-neuroscience theory that the hippocampus stores sparse pointers (indices) into neocortical patterns, used as the design metaphor for HippoRAG's KG + retrieval split.","maps_to_pattern":"hippocampus-rag","url":"https://arxiv.org/abs/2405.14831"},{"name":"OpenIE knowledge graph","summary":"Documents are decomposed by an LLM into entity-relation triples that are merged into a persistent knowledge graph with vector embeddings on nodes (the offline indexing phase).","maps_to_pattern":"graphrag","url":"https://github.com/OSU-NLP-Group/HippoRAG"},{"name":"Personalized PageRank retrieval","summary":"At query time, entities extracted from the question seed Personalized PageRank over the KG; top-scoring nodes' associated passages are returned to the reader LLM.","maps_to_pattern":"hippocampus-rag","url":"https://arxiv.org/abs/2405.14831"},{"name":"Neocortex / hippocampus analogy","summary":"The framework's design metaphor: the LLM plays the neocortex (pattern completion / language) and the KG + PPR plays the hippocampus (associative pointer index).","maps_to_pattern":"hippocampus-rag","url":"https://arxiv.org/abs/2405.14831"},{"name":"Associative / factual / sense-making memory","summary":"HippoRAG 2 explicitly targets three memory regimes: associative (multi-hop), factual, and sense-making (integrating large complex contexts), reporting gains over baseline RAG on each.","url":"https://arxiv.org/abs/2502.14802"},{"name":"HippoRAG Python API","summary":"Library surface: HippoRAG(save_dir, llm_model_name, embedding_model_name).index(docs), .retrieve(queries), .rag_qa(queries) — combined or separate retrieval-and-QA calls.","url":"https://github.com/OSU-NLP-Group/HippoRAG"}],"members":[{"pattern":"hippocampus-rag","role":"first-class","note":"HippoRAG is the eponymous implementation of the hippocampus-inspired retrieval shape: a knowledge graph plus Personalized PageRank as the retrieval primitive, mirroring the hippocampal indexing theory of human long-term memory.","evidence":[{"type":"repo","title":"HippoRAG GitHub README","url":"https://github.com/OSU-NLP-Group/HippoRAG","quote":"HippoRAG 2 is a powerful memory framework for LLMs that enhances their ability to recognize and utilize connections in new knowledge—mirroring a key function of human long-term memory.","accessed":"2026-05-24"},{"type":"paper","title":"HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models (NeurIPS 2024)","url":"https://arxiv.org/abs/2405.14831","quote":"we introduce HippoRAG, a novel retrieval framework inspired by the hippocampal indexing theory of human long-term memory... HippoRAG synergistically orchestrates LLMs, knowledge graphs, and the Personalized PageRank algorithm to mimic the different roles of neocortex and hippocampus in human memory.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"graphrag","role":"supported","note":"HippoRAG is in the GraphRAG family (KG-augmented retrieval) but differs in retrieval primitive: Personalized PageRank over entity nodes rather than community summaries. The README explicitly positions it against GraphRAG, RAPTOR, and LightRAG on indexing cost.","evidence":[{"type":"repo","title":"HippoRAG GitHub README","url":"https://github.com/OSU-NLP-Group/HippoRAG","quote":"Like its predecessor, HippoRAG 2 remains cost and latency efficient in online processes, while using significantly fewer resources for offline indexing compared to other graph-based solutions such as GraphRAG, RAPTOR, and LightRAG.","accessed":"2026-05-24"},{"type":"paper","title":"From RAG to Memory: Non-Parametric Continual Learning for Large Language Models (ICML 2025, HippoRAG 2)","url":"https://arxiv.org/abs/2502.14802","quote":"Recent RAG approaches augment vector embeddings with various structures like knowledge graphs to address some of these gaps, namely sense-making and associativity.","accessed":"2026-05-24"}],"evidence_status":"full"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-05-24","status_in_practice":"experimental","pattern_composition":"flowchart TD\n  fw[\"HippoRAG\"]\n  fw --> p1[\"HippoRAG<br/>(first-class)\"]\n  fw --> p2[\"GraphRAG<br/>(supported)\"]","tags":["orchestration-framework","retrieval","full-code"],"references":[{"type":"doc","title":"HippoRAG — Homepage","url":"https://github.com/OSU-NLP-Group/HippoRAG"}]},{"id":"routellm","name":"RouteLLM","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"LM-Sys","language":"Python","license":"Apache-2.0","status":"active","url":"https://github.com/lm-sys/RouteLLM","repo":"https://github.com/lm-sys/RouteLLM","intent":"Research framework for training and serving LLM routers that dynamically dispatch each query between a stronger, more expensive model and a cheaper but weaker model based on a learned difficulty score and a per-request cost threshold.","description":"RouteLLM is LM-Sys's open-source framework for cost-effective LLM serving. It trains learned routers — matrix factorisation (mf), similarity-weighted Elo ranking (sw_ranking), and a BERT classifier — on preference data so that incoming queries are dispatched to a strong (expensive) or weak (cheap) model based on a predicted win-rate score against a caller-supplied cost threshold. The project ships an OpenAI-compatible server that drops in as a routing layer in front of two model endpoints, plus benchmarks documenting >2x cost reduction at fixed quality on MT-Bench, MMLU, and GSM8K. Companion paper: arXiv 2406.18665.","primary_use_cases":["research on learned LLM routing","cost-aware serving across strong/weak model pairs","drop-in OpenAI-compatible routing proxy","benchmarking routing strategies on MT-Bench / MMLU / GSM8K"],"agent_loop_shape":"Routing proxy in front of two model endpoints. A trained router (mf / sw_ranking / bert) scores each incoming request for predicted strong-vs-weak win rate; if the score exceeds the request's cost threshold, the query is dispatched to the strong model, otherwise to the weak model. The response is returned through an OpenAI-compatible API. Routing is stateless per request; the router is trained offline on preference data and loaded as a model artefact.","key_concepts":[{"name":"Matrix factorisation router (mf)","summary":"Recommended router type; a matrix-factorisation model trained on preference data to score strong-vs-weak win rate for a query.","url":"https://github.com/lm-sys/RouteLLM"},{"name":"BERT classifier router (bert)","summary":"BERT classifier trained on preference data to predict whether the strong model would win against the weak model on this query.","url":"https://github.com/lm-sys/RouteLLM"},{"name":"Similarity-weighted Elo router (sw_ranking)","summary":"Weighted Elo calculation where each preference vote is weighted by similarity to the user's prompt.","url":"https://github.com/lm-sys/RouteLLM"},{"name":"Strong vs weak two-model routing","summary":"RouteLLM focuses on routing between exactly two models: one stronger and more expensive, one cheaper but weaker.","url":"https://arxiv.org/abs/2406.18665"},{"name":"Cost threshold","summary":"Per-request scalar that determines the cost-quality tradeoff; if the router's predicted win rate exceeds it the strong model is used.","maps_to_pattern":"complexity-based-routing","url":"https://github.com/lm-sys/RouteLLM"},{"name":"Preference-data training","summary":"Routers are trained offline on human preference data (e.g. Chatbot Arena votes plus augmentations) rather than hand-coded rules.","url":"https://arxiv.org/abs/2406.18665"}],"members":[{"pattern":"multi-model-routing","role":"first-class","note":"Core function: route every query to one of two configured models (strong / weak) via a learned router.","evidence":[{"type":"repo","title":"RouteLLM README — overview","url":"https://github.com/lm-sys/RouteLLM","quote":"route simpler queries to cheaper models, saving costs while maintaining quality","accessed":"2026-05-24"},{"type":"paper","title":"RouteLLM: Learning to Route LLMs with Preference Data (arXiv:2406.18665) — abstract","url":"https://arxiv.org/abs/2406.18665","quote":"we propose several efficient router models that dynamically select between a stronger and a weaker LLM during inference, aiming to optimize the balance between cost and response quality","accessed":"2026-05-24"},{"type":"repo","title":"RouteLLM README — router types","url":"https://github.com/lm-sys/RouteLLM","quote":"1. `mf`: Uses a matrix factorization model trained on the preference data (recommended). 2. `sw_ranking`: Uses a weighted Elo calculation for routing, where each vote is weighted according to how similar it is to the user's prompt. 3. `bert`: Uses a BERT classifier trained on the preference data.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"complexity-based-routing","role":"first-class","note":"The trained router scores predicted strong-vs-weak win rate (a proxy for query difficulty) and compares it against a caller-supplied cost threshold — the canonical complexity-based routing shape.","evidence":[{"type":"repo","title":"RouteLLM README — cost threshold","url":"https://github.com/lm-sys/RouteLLM","quote":"Each request is also associated with a _cost threshold_ that determines the cost-quality tradeoff of that request - a higher cost threshold leads to lower cost but may lead to lower-quality responses.","accessed":"2026-05-24"},{"type":"repo","title":"RouteLLM README — routing decision rule","url":"https://github.com/lm-sys/RouteLLM","quote":"if this win rate is great than user-specified cost threshold, then the request is routed to the strong model","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"open-weight-cascade","role":"core","note":"RouteLLM is the canonical strong/weak cascade: a trained router sends easy queries to a cheaper weak model (their reference weak tier is an open-weight model such as Mixtral-8x7B that can run inside the operator's boundary) and only escalates hard queries to the expensive frontier strong model. Cost-threshold per request tunes the cascade.","evidence":[{"type":"blog","title":"RouteLLM: An Open-Source Framework for Cost-Effective LLM Routing (LMSYS)","url":"https://www.lmsys.org/blog/2024-07-01-routellm/","quote":"all queries that can be handled by weaker models should be routed to these models, with all other queries routed to stronger models, minimizing cost while maintaining response quality.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-05-24","status_in_practice":"experimental","pattern_composition":"flowchart TD\n  fw[\"RouteLLM\"]\n  fw --> p1[\"Open-Weight Cascade<br/>(core)\"]\n  fw --> p2[\"Multi-Model Routing<br/>(first-class)\"]\n  fw --> p3[\"Complexity-Based Routing<br/>(first-class)\"]","tags":["orchestration-framework","routing-composition","full-code"],"references":[{"type":"doc","title":"RouteLLM — Homepage","url":"https://github.com/lm-sys/RouteLLM"}]},{"id":"openrouter","name":"OpenRouter","kind":"framework","category":"orchestration-framework","build_surface":"closed-application","vendor":"OpenRouter","language":"API (REST, OpenAI-compatible)","license":"proprietary","status":"active","url":"https://openrouter.ai/","docs_url":"https://openrouter.ai/docs","intent":"Hosted LLM aggregator that exposes a single OpenAI-compatible endpoint over hundreds of models from many providers, with built-in provider routing, automatic fallback, price-weighted load balancing, and an Auto Router that picks the best model per prompt.","description":"OpenRouter is a commercial LLM aggregation service. Applications point at a single OpenAI-compatible endpoint and select a model by string identifier; OpenRouter handles authentication to the underlying provider, load-balances across providers prioritising price, falls back to the next provider on error, and offers an Auto Router (openrouter/auto, powered by NotDiamond) that selects the best model per prompt. Callers can customise routing via a `provider` object (e.g. explicit `order` of provider slugs). It is not a developer SDK in the build-your-agent sense — it is a hosted service end-user applications consume.","primary_use_cases":["unified access to many model providers from one endpoint","provider fallback and price-weighted load distribution","cost-aware automatic model selection via Auto Router","OpenAI-compatible drop-in for multi-provider apps"],"agent_loop_shape":"Hosted routing gateway. The application issues an OpenAI-style chat completion request to OpenRouter's endpoint with a model identifier (a specific model or a routing variant such as openrouter/auto). OpenRouter dispatches to one of the backing providers — by default load-balancing across the top providers weighted by inverse-square of price — applies fallback on errors, accounts cost, and streams the response back. Stateless per request; routing policy lives in the model slug, the optional `provider` object, and account settings.","key_concepts":[{"name":"Unified OpenAI-compatible endpoint","summary":"A single API URL fronting hundreds of models from many providers; clients keep using the OpenAI SDK and just change the base URL and model slug.","url":"https://openrouter.ai/docs/quickstart"},{"name":"Price-weighted load balancing","summary":"Default routing across stable providers selects one weighted by inverse square of price, favouring cheaper backends without sacrificing uptime.","url":"https://openrouter.ai/docs/guides/routing/provider-selection"},{"name":"Automatic provider fallback","summary":"If a provider errors or times out, OpenRouter transparently retries the next provider in the routing list.","url":"https://openrouter.ai/docs/guides/routing/provider-selection"},{"name":"Provider routing preferences","summary":"Callers can pass a `provider` object (e.g. `order: [\"anthropic\", \"openai\"]`) to constrain or reorder the provider list per request.","url":"https://openrouter.ai/docs/guides/routing/provider-selection"},{"name":"Auto Router (openrouter/auto)","summary":"Meta-model slug powered by NotDiamond that picks the best underlying model per prompt, routing simpler prompts to efficient models and complex ones to capable models.","maps_to_pattern":"cost-aware-action-delegation","url":"https://openrouter.ai/docs/guides/routing/routers/auto-router"},{"name":"Latest-alias model slugs","summary":"Slugs like `openai/gpt-latest` resolve to the current flagship of a provider so client code keeps pointing at the freshest model without redeploying.","url":"https://openrouter.ai/docs/quickstart"}],"members":[{"pattern":"multi-model-routing","role":"first-class","note":"Whole product is built around routing each request across a catalogue of models and providers behind one OpenAI-compatible endpoint.","evidence":[{"type":"doc","title":"OpenRouter Quickstart","url":"https://openrouter.ai/docs/quickstart","quote":"OpenRouter provides a unified API that gives you access to hundreds of AI models through a single endpoint","accessed":"2026-05-24"},{"type":"doc","title":"OpenRouter Provider Routing guide","url":"https://openrouter.ai/docs/guides/routing/provider-selection","quote":"OpenRouter routes requests to the best available providers for your model. By default, requests are load balanced across the top providers to maximize uptime","accessed":"2026-05-24"},{"type":"doc","title":"OpenRouter Provider Routing — fallback","url":"https://openrouter.ai/docs/guides/routing/provider-selection","quote":"If Provider A fails, then Provider C will be tried next. If Provider C also fails, Provider B will be tried last","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"cost-aware-action-delegation","role":"first-class","note":"Default load balancing prioritises price; the Auto Router delegates model selection to a meta-model that routes simpler tasks to cheaper models — both let the caller delegate cost-aware model choice without changing client code.","evidence":[{"type":"doc","title":"OpenRouter Quickstart — automatic routing","url":"https://openrouter.ai/docs/quickstart","quote":"while automatically handling fallbacks and selecting the most cost-effective options","accessed":"2026-05-24"},{"type":"doc","title":"OpenRouter Provider Routing — price-weighted load balancing","url":"https://openrouter.ai/docs/guides/routing/provider-selection","quote":"Prioritize providers that have not seen significant outages in the last 30 seconds. For the stable providers, look at the lowest-cost candidates and select one weighted by inverse square of the price","accessed":"2026-05-24"},{"type":"doc","title":"OpenRouter Auto Router","url":"https://openrouter.ai/docs/guides/routing/routers/auto-router","quote":"The Auto Router automatically selects the best model for your prompt, powered by NotDiamond.","accessed":"2026-05-24"},{"type":"doc","title":"OpenRouter Auto Router — use cases","url":"https://openrouter.ai/docs/guides/routing/routers/auto-router","quote":"Let the router choose efficient models for simpler tasks","accessed":"2026-05-24"}],"evidence_status":"full"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-05-24","status_in_practice":"mature","pattern_composition":"flowchart TD\n  fw[\"OpenRouter\"]\n  fw --> p1[\"Multi-Model Routing<br/>(first-class)\"]\n  fw --> p2[\"Cost-Aware Action Delegation<br/>(first-class)\"]","tags":["orchestration-framework","routing-composition","safety-control","closed-application"],"references":[{"type":"doc","title":"OpenRouter — Documentation","url":"https://openrouter.ai/docs"},{"type":"doc","title":"OpenRouter — Homepage","url":"https://openrouter.ai/"}]},{"id":"not-diamond","name":"Not Diamond","kind":"framework","category":"orchestration-framework","build_surface":"closed-application","vendor":"Not Diamond","language":"API","license":"proprietary","status":"active","url":"https://www.notdiamond.ai/","intent":"Commercial intelligent model-routing service that predicts the best-performing model per query and dispatches automatically across configured endpoints.","description":"Not Diamond is a hosted model router for LLM applications. Customers register the set of candidate models they want to consider and call Not Diamond's API with a query; a meta-model that learns when to call each LLM predicts the best target and the request is dispatched there. The product positions itself around quality maximisation and cost reduction — simple queries are routed to cheap models, complex queries to frontier models — without the customer hand-coding a routing policy. A pre-trained router gets users started quickly; custom routers can be trained on the customer's own preference data.","primary_use_cases":["automatic best-model selection per query","cost reduction over single-model deployments by routing simple queries to cheap models","quality-maximising routing across heterogeneous models","managed routing without authoring rules","custom router training on customer preference data"],"agent_loop_shape":"Hosted decision API. The application sends each prompt to Not Diamond with the configured candidate model list; Not Diamond analyses the input and predicts which model will provide the highest quality response at the lowest cost for that specific query, then (optionally) executes the call. Routing is stateless per request; configuration lives in the Not Diamond dashboard / API call parameters, and learned preference data drives the meta-model behind the routing decision.","key_concepts":[{"name":"Meta-model router","summary":"A learned model that decides which underlying LLM to call for each query, combining the candidate models into a single routed endpoint.","maps_to_pattern":"multi-model-routing","url":"https://github.com/Not-Diamond/notdiamond-python"},{"name":"Per-query analysis","summary":"Router analyses each input and predicts the highest-quality, lowest-cost model for that specific query.","maps_to_pattern":"complexity-based-routing","url":"https://docs.notdiamond.ai/docs/what-is-model-routing"},{"name":"Pre-trained and custom routers","summary":"Customers can start with Not Diamond's pre-trained router or train a custom router on their own preference data.","url":"https://www.notdiamond.ai/"},{"name":"Cost-vs-quality tradeoff routing","summary":"Routing policy explicitly trades cost against quality: simple queries to cheap models, complex queries to frontier models.","maps_to_pattern":"complexity-based-routing","url":"https://docs.notdiamond.ai/docs/what-is-model-routing"}],"members":[{"pattern":"multi-model-routing","role":"first-class","note":"The product is a router across a configured set of models — that is its single function.","evidence":[{"type":"doc","title":"Not Diamond Docs — What is Model Routing?","url":"https://docs.notdiamond.ai/docs/what-is-model-routing","quote":"Model routing intelligently selects the best LLM from your candidate models to respond to each query in your application to improve accuracy while radically reducing cost.","accessed":"2026-05-24"},{"type":"repo","title":"notdiamond-python README","url":"https://github.com/Not-Diamond/notdiamond-python","quote":"Not Diamond is an AI model router that automatically determines which LLM is best-suited to respond to any query","accessed":"2026-05-24"},{"type":"repo","title":"notdiamond-python README — meta-model framing","url":"https://github.com/Not-Diamond/notdiamond-python","quote":"combining multiple LLMs into a meta-model that learns when to call each LLM","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"complexity-based-routing","role":"first-class","note":"Routing decision is driven by a learned model that predicts which model will perform best on a given query; the docs explicitly frame the policy as simple-vs-complex query routing.","evidence":[{"type":"doc","title":"Not Diamond Docs — What is Model Routing? (complexity framing)","url":"https://docs.notdiamond.ai/docs/what-is-model-routing","quote":"Save costs by routing simple queries to cost-effective models while reserving frontier models for complex tasks.","accessed":"2026-05-24"},{"type":"doc","title":"Not Diamond Docs — per-query analysis","url":"https://docs.notdiamond.ai/docs/what-is-model-routing","quote":"Not Diamond analyzes each input and predicts which model will provide the highest quality response at the lowest cost for that specific query.","accessed":"2026-05-24"}],"evidence_status":"full"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-06-19","date_added":"2026-05-24","status_in_practice":"mature","pattern_composition":"flowchart TD\n  fw[\"Not Diamond\"]\n  fw --> p1[\"Multi-Model Routing<br/>(first-class)\"]\n  fw --> p2[\"Complexity-Based Routing<br/>(first-class)\"]","tags":["orchestration-framework","routing-composition","closed-application"],"references":[{"type":"doc","title":"Not Diamond — Homepage","url":"https://www.notdiamond.ai/"}],"anti_patterns_avoided":[{"pattern":"top-tier-model-for-everything","note":"Model routing sends simple queries to cost-effective models and reserves frontier models for complex tasks, instead of using the top model for everything.","evidence":[{"type":"doc","title":"Not Diamond — model routing","url":"https://docs.notdiamond.ai/docs/what-is-model-routing","quote":"Save costs by routing simple queries to cost-effective models while reserving frontier models for complex tasks","accessed":"2026-06-19"}]}]},{"id":"anp","name":"Agent Network Protocol (ANP)","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"Agent Network Protocol","language":"Python, TypeScript","license":"MIT","status":"active","url":"https://agent-network-protocol.com/","repo":"https://github.com/agent-network-protocol/AgentNetworkProtocol","intent":"Open specification (and reference implementation) for decentralised agent-to-agent communication: cross-platform DID identity, dynamic protocol negotiation, and an application-layer description / discovery protocol so agents from different vendors can talk without a central broker.","description":"Agent Network Protocol (ANP) is an open communication protocol for intelligent agents, positioned by its authors as 'the HTTP of the Agentic Web era'. It defines a three-layer architecture: an Identity and Secure Communication Layer based on W3C DID (with the did:wba method) for cross-platform identity and end-to-end encrypted channels; a Meta-Protocol Layer that lets agents negotiate the actual wire protocol in natural language and then run AI-generated code over it; and an Application Protocol Layer with the Agent Description Protocol (ADP) and an agent discovery protocol. The reference Python SDK lives in the AgentConnect repo.","primary_use_cases":["cross-organisation agent-to-agent communication without a central broker","decentralised agent identity (W3C DID / did:wba) and end-to-end encrypted channels","agent capability description (ADP) and peer discovery","interop between heterogeneous agent runtimes / vendors","research on self-organising, self-negotiating agent networks"],"agent_loop_shape":"Network-protocol layer rather than a single agent loop. Each participating agent embeds an ANP client. Step 1: identity / handshake — the client signs the first HTTP request with its DID key; the peer fetches the DID document and verifies the signature, returning a token for subsequent calls. Step 2: meta-protocol negotiation in natural language to agree the wire protocol (or reuse a cached one). Step 3: discovery via ADP description documents and the discovery protocol. Step 4: application-layer message exchange over the negotiated protocol, tagged with a 2-bit Protocol Type (meta / application / natural-language / verification).","key_concepts":[{"name":"Three-layer protocol architecture","summary":"Identity & Secure Communication Layer + Meta-Protocol Layer + Application Protocol Layer — the spec's top-level structure separating trust, negotiation, and capability description.","url":"https://github.com/agent-network-protocol/AgentNetworkProtocol/blob/main/01-agentnetworkprotocol-technical-white-paper.md"},{"name":"W3C DID (did:wba method)","summary":"Decentralised identifiers and DID documents give each agent a cross-platform identity; ANP's did:wba method extends did:web for agent communication so peers can verify each other without a central authority.","maps_to_pattern":"decentralized-agent-network","url":"https://github.com/agent-network-protocol/AgentNetworkProtocol/blob/main/03-did-wba-method-design-specification.md"},{"name":"Agent Description Protocol (ADP)","summary":"Application-layer protocol by which an agent publishes a structured description document declaring its capabilities and supported application protocols.","maps_to_pattern":"inter-agent-communication","url":"https://github.com/agent-network-protocol/AgentNetworkProtocol/blob/main/07-anp-agent-description-protocol-specification.md"},{"name":"Agent Discovery Protocol","summary":"Mechanism for one agent to locate peers and fetch their ADP description documents — the discovery surface for an open agent network.","maps_to_pattern":"decentralized-agent-network","url":"https://github.com/agent-network-protocol/AgentNetworkProtocol/blob/main/08-ANP-Agent-Discovery-Protocol-Specification.md"},{"name":"Meta-protocol negotiation","summary":"Two agents negotiate the actual wire protocol in natural language, then generate code to handle it; cached negotiation results can be reused on subsequent calls.","url":"https://github.com/agent-network-protocol/AgentNetworkProtocol/blob/main/06-anp-agent-communication-meta-protocol-specification.md"},{"name":"Protocol Type (PT) header","summary":"2-bit field at the head of every decrypted ANP message labelling the speech-act category: 00 meta protocol, 01 application protocol, 10 natural language protocol, 11 verification protocol.","maps_to_pattern":"performative-message","url":"https://github.com/agent-network-protocol/AgentNetworkProtocol/blob/main/06-anp-agent-communication-meta-protocol-specification.md"},{"name":"End-to-end encrypted channel (ECDHE)","summary":"Each agent's DID key pair is used both for authentication and for an ECDHE-based end-to-end encryption scheme, so intermediate forwarders cannot decrypt traffic.","url":"https://github.com/agent-network-protocol/AgentNetworkProtocol/blob/main/01-agentnetworkprotocol-technical-white-paper.md"}],"members":[{"pattern":"decentralized-agent-network","role":"first-class","note":"ANP's whole purpose is a decentralised agent network: W3C-DID-based mutual identity verification without a central authority, and the explicit goal of an 'open, secure, and efficient collaboration network for billions of agents'.","evidence":[{"type":"repo","title":"AgentNetworkProtocol README (Vision and Positioning)","url":"https://github.com/agent-network-protocol/AgentNetworkProtocol","quote":"Agent Network Protocol (ANP) is an open-source communication protocol for intelligent agents. Agent Network Protocol (ANP) aims to become the HTTP of the Agentic Web era. Our vision is to define how agents connect with each other, building an open, secure, and efficient collaboration network for billions of agents.","accessed":"2026-05-24"},{"type":"spec","title":"ANP Technical White Paper, section 3.2 Identity and Secure Communication Layer","url":"https://github.com/agent-network-protocol/AgentNetworkProtocol/blob/main/01-agentnetworkprotocol-technical-white-paper.md","quote":"ANP, based on the W3C DID standard, has designed a lightweight, extensible decentralized identity authentication mechanism (such as the did:wba method), ensuring that any two agents can securely verify each other's identity and establish private, reliable encrypted communication channels without central authority intervention.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"inter-agent-communication","role":"first-class","note":"ANP exists to make heterogeneous agents talk: the README's 'Why We Need ANP' section frames interconnection, native interfaces, and efficient collaboration as the three problems the protocol solves, and the spec defines the on-the-wire message layers.","evidence":[{"type":"repo","title":"AgentNetworkProtocol README (Why We Need ANP)","url":"https://github.com/agent-network-protocol/AgentNetworkProtocol","quote":"Interconnection: Enabling all agents to communicate with each other, breaking down data silos, and allowing AI to access complete contextual information... Efficient Collaboration: Using AI, agents can self-organize and self-negotiate to build a more cost-effective and efficient collaboration network than the existing internet.","accessed":"2026-05-24"},{"type":"spec","title":"ANP Technical White Paper, section 3.1 Protocol Layer Architecture","url":"https://github.com/agent-network-protocol/AgentNetworkProtocol/blob/main/01-agentnetworkprotocol-technical-white-paper.md","quote":"Through a three-layer protocol system—identity and encrypted communication layer, meta-protocol negotiation layer, and application protocol layer—ANP systematically solves the problems of agent identity authentication, dynamic negotiation, and capability discovery interoperability.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"performative-message","role":"supported","note":"Every ANP message carries a 2-bit Protocol Type (PT) in its binary header that labels the speech-act category of the payload — meta-protocol, application-protocol, natural-language, or verification — a performative-message style envelope rather than implicit content typing.","evidence":[{"type":"spec","title":"ANP Agent Communication Meta-Protocol Specification, Message Format Definition","url":"https://github.com/agent-network-protocol/AgentNetworkProtocol/blob/main/06-anp-agent-communication-meta-protocol-specification.md","quote":"PT: Protocol Type, 2 bits, indicating the protocol type — 00: meta protocol; 01: application protocol; 10: natural language protocol; 11: verification protocol. Reserved: 6 bits, reserved field, not used yet. Protocol Data: variable length, indicating the specific content of the protocol.","accessed":"2026-05-24"},{"type":"spec","title":"ANP Agent Communication Meta-Protocol Specification, Background","url":"https://github.com/agent-network-protocol/AgentNetworkProtocol/blob/main/06-anp-agent-communication-meta-protocol-specification.md","quote":"We define the meta-protocol message as a semi-structured format. The core protocol negotiation part uses natural language to maintain the flexibility of negotiation, while the process control uses structured JSON to keep the protocol negotiation process controllable.","accessed":"2026-05-24"}],"evidence_status":"full"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-05-24","status_in_practice":"experimental","pattern_composition":"flowchart TD\n  fw[\"Agent Network Protocol (ANP)\"]\n  fw --> p1[\"Decentralized Agent Network<br/>(first-class)\"]\n  fw --> p2[\"Inter-Agent Communication<br/>(first-class)\"]\n  fw --> p3[\"Performative Message<br/>(supported)\"]","tags":["orchestration-framework","multi-agent","full-code"],"references":[{"type":"doc","title":"Agent Network Protocol (ANP) — Homepage","url":"https://agent-network-protocol.com/"},{"type":"repo","title":"Agent Network Protocol (ANP) — Source repository","url":"https://github.com/agent-network-protocol/AgentNetworkProtocol"}]},{"id":"microsoft-agent-framework","name":"Microsoft Agent Framework","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"Microsoft","language":"C#/.NET, Python","license":"MIT","status":"active","first_released":"2025","url":"https://learn.microsoft.com/en-us/agent-framework/overview/agent-framework-overview","repo":"https://github.com/microsoft/agent-framework","docs_url":"https://learn.microsoft.com/en-us/agent-framework/overview/agent-framework-overview","intent":"Unify Microsoft's two prior agent stacks — AutoGen's multi-agent orchestration and Semantic Kernel's enterprise plugin and connector model — into one SDK for building single- and multi-agent systems over shared abstractions for agents, tools, threads, and workflows.","description":"Microsoft Agent Framework is an open-source SDK announced in October 2025 that merges AutoGen and Semantic Kernel into a single agent stack for .NET and Python. It provides a unified agent abstraction with tool calling, multiple multi-agent orchestration shapes (sequential, concurrent, group chat, handoff), and a graph-based workflow runtime, alongside enterprise concerns — observability, approvals, and connectors — inherited from Semantic Kernel. Microsoft positions it as the successor to both predecessors, with general availability targeted for early 2026.","primary_use_cases":["enterprise multi-agent applications",".NET agent development","tool-using assistants","workflow orchestration"],"agent_loop_shape":"A unified agent abstraction over a tool-calling loop, with pluggable multi-agent orchestrators (sequential, concurrent, group chat, handoff) and a separate graph-based workflow runtime for deterministic control flow.","members":[{"pattern":"tool-use","role":"core","note":"Agents invoke typed functions and tools as a first-class capability, carried over from Semantic Kernel's plugin/function model.","evidence":[{"type":"doc","title":"Microsoft Agent Framework overview","url":"https://learn.microsoft.com/en-us/agent-framework/overview/agent-framework-overview","accessed":"2026-06-02"}],"evidence_status":"none"},{"pattern":"orchestrator-workers","role":"first-class","note":"Group-chat and concurrent orchestration shapes are ported from AutoGen and exposed as built-in orchestrators.","evidence":[{"type":"repo","title":"microsoft/agent-framework","url":"https://github.com/microsoft/agent-framework","accessed":"2026-06-02"}],"evidence_status":"none"}],"instantiates":["multi-agent-coordination","planning-loops"],"alternatives":[{"composition":"autogen","relation":"predecessor","note":"AutoGen's multi-agent orchestration was merged into this framework."},{"composition":"semantic-kernel","relation":"predecessor","note":"Semantic Kernel's plugin and connector model was merged into this framework."},{"composition":"langgraph","relation":"competes-with","note":"Both offer graph-based agent/workflow runtimes."}],"references":[{"type":"doc","title":"Microsoft Agent Framework overview","url":"https://learn.microsoft.com/en-us/agent-framework/overview/agent-framework-overview","accessed":"2026-06-02"},{"type":"repo","title":"microsoft/agent-framework","url":"https://github.com/microsoft/agent-framework","accessed":"2026-06-02"},{"type":"doc","title":"Microsoft Agent Framework docs","url":"https://learn.microsoft.com/en-us/agent-framework/","accessed":"2026-06-04"}],"last_analyzed":"2026-06-02","verification_status":"partial","last_updated":"2026-06-17","date_added":"2026-06-02","status_in_practice":"emerging","pattern_composition":"flowchart TD\n  fw[\"Microsoft Agent Framework\"]\n  fw --> p1[\"Tool Use<br/>(core)\"]\n  fw --> p2[\"Orchestrator-Workers<br/>(first-class)\"]","tags":["orchestration-framework","tool-use-environment","multi-agent","full-code"],"key_concepts":[{"name":"Tool Use","summary":"Agents invoke typed functions and tools as a first-class capability, carried over from Semantic Kernel's plugin/function model.","maps_to_pattern":"tool-use"},{"name":"Orchestrator-Workers","summary":"Group-chat and concurrent orchestration shapes are ported from AutoGen and exposed as built-in orchestrators.","maps_to_pattern":"orchestrator-workers"}]},{"id":"genkit","name":"Genkit","kind":"framework","category":"orchestration-framework","build_surface":"full-code","vendor":"Google (Firebase)","language":"TypeScript, Go, Python","license":"Apache-2.0","status":"active","url":"https://genkit.dev","repo":"https://github.com/firebase/genkit","docs_url":"https://genkit.dev/docs/","intent":"Provider-neutral framework for building AI-powered apps with unified APIs, tool calling and agentic workflows.","description":"Google/Firebase full-stack AI framework with unified APIs across providers (Google, OpenAI, Anthropic, Ollama), structured output, RAG, multimodal support and agentic tool-calling flows. Production-ready in JavaScript/TypeScript and Go, with Python in beta; built and used in production by Google.","primary_use_cases":["building production AI features with one API across multiple model providers","tool-calling flows and lightweight agentic workflows in JS/Go/Python","RAG and structured-output pipelines"],"agent_loop_shape":"Developers compose named flows that call models with tools and structured-output schemas; flows can chain steps and run agentic tool-calling loops over a unified provider abstraction.","key_concepts":[{"name":"Flows","summary":"Composable, observable functions chaining model and tool steps.","url":"https://genkit.dev/docs/flows/"},{"name":"Dotprompt","summary":"Versioned, model-agnostic prompt files."},{"name":"Tools","summary":"Typed tool calling for agentic steps.","maps_to_pattern":"tool-use"}],"pattern_composition":"flowchart TD\n  APP[App request] --> FLOW[Genkit flow]\n  FLOW --> MODEL[Unified model API<br/>Google / OpenAI / Anthropic / Ollama]\n  MODEL --> TOOLS[Tool calling]\n  TOOLS --> MODEL\n  MODEL --> SCHEMA[Structured output]\n  FLOW --> RAG[(RAG / retrievers)]\n  RAG --> MODEL\n  SCHEMA --> OUT[Typed result]","members":[{"pattern":"tool-use","role":"first-class","note":"Use streamlined APIs to build AI features with structured output, agentic tool calling, context-aware generation, multi-modal input/output, and more.","evidence":[{"type":"repo","title":"firebase/genkit README","url":"https://github.com/firebase/genkit","quote":"Use streamlined APIs to build AI features with structured output, agentic tool calling, context-aware generation, multi-modal input/output, and more.","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"structured-output","role":"first-class","note":"Type-safe, structured data generation","evidence":[{"type":"repo","title":"firebase/genkit README","url":"https://github.com/firebase/genkit","quote":"Type-safe, structured data generation","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"multi-model-routing","role":"first-class","note":"Use a unified interface to integrate with hundreds of models from providers like Google, OpenAI, Anthropic, Ollama, and more.","evidence":[{"type":"repo","title":"firebase/genkit README","url":"https://github.com/firebase/genkit","quote":"Use a unified interface to integrate with hundreds of models from providers like Google, OpenAI, Anthropic, Ollama, and more.","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"agentic-rag","role":"supported","note":"AI-powered data retrieval (RAG)","evidence":[{"type":"repo","title":"firebase/genkit README","url":"https://github.com/firebase/genkit","quote":"AI-powered data retrieval (RAG)","accessed":"2026-06-04"}],"evidence_status":"limited"}],"instantiates":["structured-output-stack"],"alternatives":[{"composition":"vercel-ai-sdk","relation":"similar-shape","note":"Peer provider-neutral TS-first AI app framework."},{"composition":"langchain","relation":"similar-shape","note":"Peer multi-provider orchestration framework."}],"references":[{"type":"repo","title":"firebase/genkit","url":"https://github.com/firebase/genkit","accessed":"2026-06-04"},{"type":"doc","title":"Genkit documentation","url":"https://genkit.dev/docs/","accessed":"2026-06-04"}],"last_analyzed":"2026-06-04","verification_status":"partial","last_updated":"2026-06-17","date_added":"2026-06-04","status_in_practice":"emerging","tags":["google","firebase","typescript","go","open-source","multi-provider"]},{"id":"a2a-protocol","name":"Agent2Agent (A2A) Protocol","kind":"framework","category":"orchestration-framework","vendor":"Google / Linux Foundation","language":"N/A","license":"Apache-2.0","status":"active","first_released":"2025-04-09","url":"https://a2a-protocol.org/","repo":"https://github.com/a2aproject/A2A","docs_url":"https://a2a-protocol.org/latest/specification/","intent":"A2A is an open protocol that lets independent AI agents built by different vendors or frameworks discover each other and exchange tasks, messages, and results over a standard wire format.","description":"Agent2Agent (A2A) is an open protocol specification originally developed by Google Cloud and donated to the Linux Foundation in 2025. Each A2A server publishes an Agent Card describing its identity, skills, and endpoint, and clients send messages to it that the server either answers directly or turns into an asynchronous Task with a defined lifecycle. The protocol defines transport bindings over JSON-RPC, gRPC, and HTTP/REST so agents can communicate across process and vendor boundaries without sharing an internal framework.","primary_use_cases":["cross-vendor agent interoperability","runtime agent discovery via Agent Cards","delegating tasks between enterprise agents","multi-agent coordination across frameworks"],"agent_loop_shape":"A2A is not an agent loop itself but a request/response and task-lifecycle protocol between agents: a client agent discovers a remote agent through its Agent Card, sends a message, and the remote agent returns a direct response or creates a long-running Task whose state the client polls or streams until completion.","build_surface":"full-code","members":[{"pattern":"agent-capability-manifest","role":"core","note":"Each A2A server publishes an Agent Card JSON (identity, skills, service endpoint, auth) at the well-known URI /.well-known/agent-card.json so clients discover and bind to it by capability at runtime.","evidence":[{"type":"spec","title":"Agent2Agent (A2A) Protocol Specification","url":"https://a2a-protocol.org/latest/specification/","quote":"Agent Card: A JSON metadata document published by an A2A Server, describing its identity, capabilities, skills, service endpoint, and authentication requirements.","accessed":"2026-06-17"},{"type":"doc","title":"Agent Discovery - A2A Protocol","url":"https://a2a-protocol.org/latest/topics/agent-discovery/","quote":"A2A Servers make their Agent Card discoverable by hosting it at a standardized, `well-known` URI on their domain. The standard path is `https://{agent-server-domain}/.well-known/agent-card.json`","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"cross-domain-agent-network","role":"core","note":"A2A lets vendor- and framework-independent agents, each grounded in its own enterprise system, securely exchange information and coordinate actions across enterprise platforms via a standard inter-agent protocol.","evidence":[{"type":"blog","title":"Announcing the Agent2Agent Protocol (A2A)","url":"https://developers.googleblog.com/en/a2a-a-new-era-of-agent-interoperability/","quote":"securely exchange information, and coordinate actions on top of various enterprise platforms or applications","accessed":"2026-06-17"},{"type":"blog","title":"Announcing the Agent2Agent Protocol (A2A)","url":"https://developers.googleblog.com/en/a2a-a-new-era-of-agent-interoperability/","quote":"A2A empowers developers to build agents capable of connecting with any other agent built using the protocol","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"inter-agent-communication","role":"core","note":"A2A is itself the inter-agent communication protocol: it defines a common message and task exchange model so agents built on different frameworks, languages, or vendors can interoperate.","evidence":[{"type":"spec","title":"Agent2Agent (A2A) Protocol Specification","url":"https://a2a-protocol.org/latest/specification/","quote":"The Agent2Agent (A2A) Protocol is an open standard designed to facilitate communication and interoperability between independent, potentially opaque AI agent systems.","accessed":"2026-06-17"},{"type":"spec","title":"Agent2Agent (A2A) Protocol Specification","url":"https://a2a-protocol.org/latest/specification/","quote":"In an ecosystem where agents might be built using different frameworks, languages, or by different vendors, A2A provides a common language and interaction model.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"async-tool-handle","role":"supported","note":"A message that cannot be answered immediately becomes a stateful Task with a unique id; the client retrieves its current state (status, artifacts, history) later via a separate Get Task call rather than blocking on the original request.","evidence":[{"type":"spec","title":"Agent2Agent (A2A) Protocol Specification — Core Concepts","url":"https://a2a-protocol.org/latest/specification/","quote":"Task: The fundamental unit of work managed by A2A, identified by a unique ID. Tasks are stateful and progress through a defined lifecycle.","accessed":"2026-06-17"},{"type":"spec","title":"Agent2Agent (A2A) Protocol Specification — Get Task","url":"https://a2a-protocol.org/latest/specification/","quote":"Retrieves the current state (including status, artifacts, and optionally history) of a previously initiated task.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"streaming-typed-events","role":"supported","note":"Over a streaming connection built on Server-Sent Events, the remote agent pushes typed TaskStatusUpdateEvent and TaskArtifactUpdateEvent objects as work progresses, and the stream closes when the task reaches a terminal state.","evidence":[{"type":"spec","title":"Agent2Agent (A2A) Protocol Specification — Send Streaming Message","url":"https://a2a-protocol.org/latest/specification/","quote":"The stream MUST close when the task reaches a terminal state (`TASK_STATE_COMPLETED`, `TASK_STATE_FAILED`, `TASK_STATE_CANCELED`, `TASK_STATE_REJECTED`).","accessed":"2026-06-17"},{"type":"spec","title":"Agent2Agent (A2A) Protocol Specification — Guiding Principles","url":"https://a2a-protocol.org/latest/specification/","quote":"Simple: Reuse existing, well-understood standards (HTTP, JSON-RPC 2.0, Server-Sent Events).","accessed":"2026-06-17"}],"evidence_status":"full"}],"key_concepts":[{"name":"Agent Card","summary":"A JSON metadata document published by an A2A server at a well-known URI describing its identity, capabilities, skills, service endpoint, and authentication requirements so clients can discover and bind to it.","maps_to_pattern":"agent-capability-manifest","url":"https://a2a-protocol.org/latest/specification/"},{"name":"Task","summary":"The fundamental unit of work, identified by a unique id and stateful through a defined lifecycle (submitted, working, input-required, completed, failed, canceled, rejected), with results stored as artifacts.","maps_to_pattern":"async-tool-handle","url":"https://a2a-protocol.org/latest/specification/"},{"name":"Message and Part","summary":"A communication turn between client and agent, composed of typed Parts (text, file, structured data), exchanged via message/send or the streaming message/stream method.","maps_to_pattern":"inter-agent-communication","url":"https://a2a-protocol.org/latest/specification/"},{"name":"Transport bindings","summary":"A2A is async-first and reuses HTTP, JSON-RPC 2.0, gRPC, and Server-Sent Events so streaming updates and long-running tasks ride on standard infrastructure.","maps_to_pattern":"streaming-typed-events","url":"https://a2a-protocol.org/latest/specification/"}],"references":[{"type":"spec","title":"Agent2Agent (A2A) Protocol Specification","url":"https://a2a-protocol.org/latest/specification/","quote":"The Agent2Agent (A2A) Protocol is an open standard designed to facilitate communication and interoperability between independent, potentially opaque AI agent systems.","accessed":"2026-06-17"},{"type":"blog","title":"Announcing the Agent2Agent Protocol (A2A) — Google Developers Blog","url":"https://developers.googleblog.com/en/a2a-a-new-era-of-agent-interoperability/","quote":"A2A is an open protocol that complements Anthropic's Model Context Protocol (MCP), which provides helpful tools and context to agents.","accessed":"2026-06-17"},{"type":"repo","title":"a2aproject/A2A — GitHub","url":"https://github.com/a2aproject/A2A","accessed":"2026-06-17"}],"alternatives":[{"composition":"anp","relation":"competes-with","note":"Agent Network Protocol pursues the same cross-vendor inter-agent interoperability goal, but emphasises decentralized DID-based identity and discovery rather than A2A's Agent Card at a well-known URI."},{"composition":"google-ap2","relation":"same-vendor","note":"AP2 is a companion Google-originated open protocol layering agent-initiated payments on top of agent-to-agent interaction; both come from the same A2A/AP2 protocol family."}],"verification_status":"needs-verification","status_in_practice":"emerging","last_analyzed":"2026-06-17","last_updated":"2026-06-19","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Agent2Agent (A2A) Protocol\"]\n  fw --> p1[\"Agent Capability Manifest<br/>(core)\"]\n  fw --> p2[\"Cross-Domain Enterprise Agent Network<br/>(core)\"]\n  fw --> p3[\"Inter-Agent Communication<br/>(core)\"]\n  fw --> p4[\"Async Tool Handle<br/>(supported)\"]\n  fw --> p5[\"Streaming Typed Events<br/>(supported)\"]","tags":["orchestration-framework","multi-agent","tool-use-environment","streaming-ux","full-code"],"anti_patterns_avoided":[{"pattern":"insecure-inter-agent-channel","note":"The A2A server must authenticate every request, and Agent Cards are JWS-signed, so inter-agent messages are authenticated and integrity-checked rather than implicitly trusted.","evidence":[{"type":"doc","title":"A2A Protocol — specification","url":"https://a2a-protocol.org/v0.3.0/specification/","quote":"The A2A Server MUST authenticate every incoming request based on the provided HTTP credentials and its declared authentication requirements from its Agent Card. ... Agent Card signatures use JSON Web Signatures (JWS) to verify the integrity of the AgentCard.","accessed":"2026-06-19"}]}]},{"id":"nemo-guardrails","name":"NVIDIA NeMo Guardrails","kind":"framework","category":"orchestration-framework","vendor":"NVIDIA","language":"Python","license":"Apache-2.0","status":"active","first_released":"2023","url":"https://github.com/NVIDIA-NeMo/Guardrails","repo":"https://github.com/NVIDIA-NeMo/Guardrails","docs_url":"https://docs.nvidia.com/nemo/guardrails","intent":"NeMo Guardrails adds programmable rails around an LLM-based conversational application so that inputs and outputs can be checked, rewritten, or rejected outside the model's own discretion.","description":"NeMo Guardrails is an open-source Python toolkit from NVIDIA for adding rule-based controls to LLM applications. Developers express rails in Colang, a flow language, and the runtime applies them at fixed stages: input rails on the incoming user message, dialog rails that steer the conversation, and output rails on the generated response. Each rail can reject, alter, or rephrase the text it inspects, so safety and topic checks run as deterministic code around the model rather than as model behavior.","primary_use_cases":["topic and scope control for domain-specific assistants","input sanitization and PII masking on user messages","output filtering and response rewriting","refusal of disallowed subjects","enforcing predefined conversational flows"],"agent_loop_shape":"A wrap-around guard pipeline: user input passes through input rails, then dialog rails select a flow and the LLM generates a response, then output rails inspect and may reject or rewrite that response before it returns to the user. Rails are deterministic Colang subflows that bracket each model call rather than a free agent loop.","build_surface":"full-code","members":[{"pattern":"enforced-advisory-disclaimer","role":"supported","note":"Output rails run after generation and can modify the bot response programmatically; a Colang output-rail subflow rewrites $bot_message outside the model's discretion — the mechanism for attaching a non-suppressible disclaimer.","evidence":[{"type":"repo","title":"Configuration Reference - Rail Types (Output rails)","url":"https://raw.githubusercontent.com/NVIDIA-NeMo/Guardrails/develop/docs/configure-rails/configuration-reference.mdx","quote":"Validate, filter, or modify bot responses","accessed":"2026-06-17"},{"type":"repo","title":"Sample config.co - output rail subflow appending to bot message","url":"https://raw.githubusercontent.com/NVIDIA-NeMo/Guardrails/develop/examples/configs/sample/config.co","quote":"define subflow dummy output rail\n  $bot_message = $bot_message + \"!!\"","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"refusal","role":"first-class","note":"Programmable rails control LLM output to refuse disallowed subjects and respond in prescribed ways to specific user requests, keeping the assistant on its permitted topics.","evidence":[{"type":"repo","title":"NVIDIA-NeMo/Guardrails — README","url":"https://github.com/NVIDIA-NeMo/Guardrails","quote":"Guardrails (or \"rails\" for short) are specific ways of controlling the output of a large language model, such as not talking about politics, responding in a particular way to specific user requests, following a predefined dialog path, using a particular language style, and more.","accessed":"2026-06-17"},{"type":"repo","title":"NVIDIA-NeMo/Guardrails — README (domain-specific assistants)","url":"https://github.com/NVIDIA-NeMo/Guardrails","quote":"Domain-specific Assistants (a.k.a. chatbots): Ensure the assistant stays on topic and follows the designed conversational flows.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"scope-of-practice-gate","role":"first-class","note":"Topical/dialog rails restrict the bot to its purpose and enforce code-defined Colang flows (user ask about X -> bot refuse to respond about X) that block off-topic professional advice such as investing advice.","evidence":[{"type":"repo","title":"Topical Rails tutorial (NeMo Guardrails docs)","url":"https://raw.githubusercontent.com/NVIDIA-NeMo/Guardrails/develop/docs/configure-rails/colang/colang-1/tutorials/6-topical-rails/README.mdx","quote":"Topical rails keep the bot talking only about the topics related to its purpose. In the case of the ABC Bot, for example, it should not talk about cooking or giving investing advice.","accessed":"2026-06-17"},{"type":"repo","title":"Topical Rails tutorial - disallowed-topics refusal flow","url":"https://raw.githubusercontent.com/NVIDIA-NeMo/Guardrails/develop/docs/configure-rails/colang/colang-1/tutorials/6-topical-rails/README.mdx","quote":"define flow\n  user ask about cooking\n  bot refuse to respond about cooking","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"supervisor-plus-gate","role":"core","note":"Output rails run programmable deterministic checks on the LLM's generated output and can reject it (blocking delivery) or alter it before it reaches the user or any downstream side-effect.","evidence":[{"type":"doc","title":"Guardrails Process — NeMo Guardrails docs source","url":"https://github.com/NVIDIA-NeMo/Guardrails/blob/main/docs/user-guides/guardrails-process.md","quote":"Output rails: applied to the output generated by the LLM; an output rail can reject the output, preventing it from being returned to the user or alter it (e.g., removing sensitive data).","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"input-output-guardrails","role":"core","note":"Input rails inspect the incoming user message before it reaches the LLM and can reject it or alter it (masking sensitive data, rephrasing); output rails do the symmetric check on the generated response, so the model call is bracketed by guard stages on both sides.","evidence":[{"type":"doc","title":"Guardrails Process — Input rails","url":"https://github.com/NVIDIA-NeMo/Guardrails/blob/main/docs/user-guides/guardrails-process.md","quote":"Input rails: applied to the input from the user; an input rail can reject the input (stopping any additional processing) or alter the input (e.g., to mask potentially sensitive data, to rephrase).","accessed":"2026-06-17"},{"type":"doc","title":"Guardrails Process — Output rails","url":"https://github.com/NVIDIA-NeMo/Guardrails/blob/main/docs/user-guides/guardrails-process.md","quote":"Output rails: applied to the output generated by the LLM; an output rail can reject the output, preventing it from being returned to the user or alter it (e.g., removing sensitive data).","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"prompt-injection-defense","role":"first-class","note":"An input rail screens the user message for exploitation attempts (prompt/code injection and jailbreak) before it reaches the LLM, and on a match returns a refusal rather than passing the crafted input through — a guard stage explicitly aimed at agentic defense-in-depth.","evidence":[{"type":"doc","title":"Configuring Injection Detection — NVIDIA NeMo","url":"https://docs.nvidia.com/nemo/microservices/latest/guardrails/tutorials/injection-detection.html","quote":"Detect potential exploitation attempts (such as code injection, cross-site scripting, SQL injection, and template injection)","accessed":"2026-06-17"},{"type":"doc","title":"Configuring Injection Detection — NVIDIA NeMo","url":"https://docs.nvidia.com/nemo/microservices/latest/guardrails/tutorials/injection-detection.html","quote":"Injection detection is primarily intended for agentic systems as part of a defense-in-depth strategy.","accessed":"2026-06-17"}],"evidence_status":"full"}],"key_concepts":[{"name":"Colang","summary":"A modeling language for conversational flows in which developers write rails and dialog flows as subflows that the runtime executes deterministically around the LLM.","maps_to_pattern":"deterministic-control-flow-not-prompt","url":"https://github.com/NVIDIA-NeMo/Guardrails"},{"name":"Input / dialog / output rails","summary":"The three rail stages: input rails check the user message before the model, dialog rails steer which flow runs, and output rails check the generated response before it returns to the user.","maps_to_pattern":"input-output-guardrails","url":"https://github.com/NVIDIA-NeMo/Guardrails/blob/main/docs/user-guides/guardrails-process.md"},{"name":"Self-check rails","summary":"Built-in rails that issue a separate LLM query to moderate input or output, including jailbreak/prompt-injection detection, before content is accepted.","maps_to_pattern":"prompt-injection-defense","url":"https://docs.nvidia.com/nemo/microservices/latest/guardrails/tutorials/injection-detection.html"}],"references":[{"type":"repo","title":"NVIDIA-NeMo/Guardrails — GitHub README","url":"https://github.com/NVIDIA-NeMo/Guardrails","quote":"Guardrails (or \"rails\" for short) are specific ways of controlling the output of a large language model, such as not talking about politics, responding in a particular way to specific user requests, following a predefined dialog path, using a particular language style, and more.","accessed":"2026-06-17"},{"type":"doc","title":"Guardrails Process — NVIDIA NeMo Guardrails docs","url":"https://github.com/NVIDIA-NeMo/Guardrails/blob/main/docs/user-guides/guardrails-process.md","quote":"Input rails: applied to the input from the user; an input rail can reject the input (stopping any additional processing) or alter the input (e.g., to mask potentially sensitive data, to rephrase).","accessed":"2026-06-17"}],"alternatives":[{"composition":"guardrails-ai","relation":"competes-with","note":"Guardrails AI provides the same wrap-around input/output validation role for LLM applications, but specifies validators in Python/RAIL rather than NeMo's Colang flow language."},{"composition":"bedrock-guardrails","relation":"competes-with","note":"Bedrock Guardrails is the managed-cloud equivalent on AWS — configurable content filters, denied topics, and PII redaction around model calls — versus NeMo's self-hosted open-source toolkit."}],"verification_status":"needs-verification","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-19","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"NVIDIA NeMo Guardrails\"]\n  fw --> p1[\"Supervisor-Plus-Gate<br/>(core)\"]\n  fw --> p2[\"Input/Output Guardrails<br/>(core)\"]\n  fw --> p3[\"Refusal<br/>(first-class)\"]\n  fw --> p4[\"Scope-of-Practice Boundary Gate<br/>(first-class)\"]\n  fw --> p5[\"Prompt Injection Defense<br/>(first-class)\"]\n  fw --> p6[\"Enforced Advisory Disclaimer<br/>(supported)\"]","tags":["orchestration-framework","safety-control","full-code"],"anti_patterns_avoided":[{"pattern":"over-helpfulness","note":"Topical/dialog rails keep the assistant on topic and on its designed flows, so it declines out-of-scope requests rather than over-helpfully complying.","evidence":[{"type":"doc","title":"NVIDIA NeMo Guardrails","url":"https://github.com/NVIDIA-NeMo/Guardrails","quote":"ensure the assistant stays on topic and follows the designed conversational flows","accessed":"2026-06-19"}]}]},{"id":"gptswarm","name":"GPTSwarm","kind":"framework","category":"orchestration-framework","vendor":"metauto-ai","language":"Python","license":"MIT","status":"active","first_released":"2024","url":"https://github.com/metauto-ai/GPTSwarm","repo":"https://github.com/metauto-ai/GPTSwarm","docs_url":"https://gptswarm.org/","intent":"Represent LLM-based agents and their connections as optimizable graphs whose edges and prompts are tuned automatically toward better swarm performance.","description":"GPTSwarm is a graph-based framework for LLM-based agents. It builds agents from computational graphs and connects them into swarms whose inter-agent edges can be pruned or created at run time. The framework includes optimization algorithms that tune prompts and the graph itself toward better task performance. It is written in Python and released under the MIT license; the underlying work was published as 'Language Agents as Optimizable Graphs' and accepted at ICML 2024.","primary_use_cases":["building LLM agents from computational graphs","optimizing inter-agent connections at run time","automatically tuning agent prompts and swarm topology"],"agent_loop_shape":"Agents are constructed as graphs of operations and connected into a swarm; an optimizer adjusts the inter-agent edges and prompts against an evaluation signal, pruning or creating connections so the collaboration structure is learned rather than fixed.","build_surface":"full-code","members":[{"pattern":"dynamic-topology-routing","role":"core","note":"Inter-agent connections in the agent graph are optimized by pruning or creating edges at run time, reshaping the collaboration topology instead of using a fixed structure.","evidence":[{"type":"repo","title":"metauto-ai/GPTSwarm README","url":"https://github.com/metauto-ai/GPTSwarm","quote":"Notice that within an agent, the edges are fixed, whereas the inter-agent connections are getting optimized towards either edge pruning (value 0, blue) or creation (value 1, red).","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"automatic-workflow-search","role":"core","note":"GPTSwarm treats the agent swarm as an optimizable graph and ships optimization algorithms that tune prompts and the graph structure to improve agent and swarm performance.","evidence":[{"type":"repo","title":"metauto-ai/GPTSwarm README","url":"https://github.com/metauto-ai/GPTSwarm","quote":"It enables the customized and automatic self-organization of agent swarms with self-improvement capabilities.","accessed":"2026-06-17"},{"type":"paper","title":"Language Agents as Optimizable Graphs","url":"https://github.com/metauto-ai/GPTSwarm","quote":"Language Agents as Optimizable Graphs","accessed":"2026-06-17"}],"evidence_status":"full"}],"key_concepts":[{"name":"Agent as graph","summary":"Each agent is built as a computational graph of operation nodes; GPTSwarm lets you construct LLM-based agents from these graphs rather than as monolithic prompts.","maps_to_pattern":"automatic-workflow-search","url":"https://github.com/metauto-ai/GPTSwarm"},{"name":"Swarm","summary":"A composite graph that connects multiple agent graphs together, whose inter-agent edges are the optimizable surface tuned at run time.","maps_to_pattern":"dynamic-topology-routing","url":"https://github.com/metauto-ai/GPTSwarm"},{"name":"swarm.optimizer","summary":"The module of optimization algorithms that tunes prompts and graph edges via reinforcement learning and prompt optimization to raise agent and swarm performance.","maps_to_pattern":"automatic-workflow-search","url":"https://github.com/metauto-ai/GPTSwarm"}],"references":[{"type":"repo","title":"metauto-ai/GPTSwarm — GitHub README","url":"https://github.com/metauto-ai/GPTSwarm","quote":"It lets you build LLM-based agents from graphs.","accessed":"2026-06-17"},{"type":"paper","title":"Language Agents as Optimizable Graphs (ICML 2024)","url":"https://arxiv.org/abs/2402.16823","accessed":"2026-06-17"}],"alternatives":[{"composition":"agentverse","relation":"similar-shape","note":"AgentVerse also studies multi-agent self-organization and emergent collaboration, but configures agent groups declaratively rather than learning the inter-agent graph through an optimizer."},{"composition":"autogen","relation":"similar-shape","note":"AutoGen composes multi-agent systems as configurable conversation graphs; GPTSwarm differs by making the graph and prompts an optimization target tuned automatically toward a reward."}],"verification_status":"partial","status_in_practice":"experimental","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"GPTSwarm\"]\n  fw --> p1[\"Dynamic Topology Routing<br/>(core)\"]\n  fw --> p2[\"Automatic Workflow Search<br/>(core)\"]","tags":["orchestration-framework","multi-agent","routing-composition","full-code"]},{"id":"guardrails-ai","name":"Guardrails AI","kind":"framework","category":"orchestration-framework","vendor":"Guardrails AI","language":"Python","license":"Apache-2.0","status":"active","first_released":"2023","url":"https://www.guardrailsai.com/","repo":"https://github.com/guardrails-ai/guardrails","docs_url":"https://guardrailsai.com/docs","intent":"Wrap LLM calls with composable input and output guards built from validators that detect, quantify, and mitigate specific risks.","description":"Guardrails AI is a Python framework that runs validators around LLM calls. Validators compose into Input and Output Guards that intercept the inputs and outputs of LLMs. The guards detect, quantify, and mitigate risks such as policy violations, hallucinations, and data leakage before output reaches users. Validators are distributed through the Guardrails Hub, and the framework is released under the Apache 2.0 license.","primary_use_cases":["validating LLM inputs and outputs against risk checks","composing validators into reusable guards","generating structured data from LLMs"],"agent_loop_shape":"Inputs pass through an Input Guard composed of validators before reaching the model, and the model's output passes through an Output Guard before reaching the user; each guard detects, quantifies, and mitigates the configured risks, blocking or correcting violations.","build_surface":"full-code","members":[{"pattern":"context-minimization","role":"first-class","note":"Validators such as ValidChoices (allow-listed enums) and ValidLength (max lengths) compose into Input Guards that intercept and constrain untrusted LLM inputs to a strictly formatted, typed interface.","evidence":[{"type":"doc","title":"Validators — Guardrails AI","url":"https://guardrailsai.com/docs/concepts/validators","quote":"Validators can be combined together into Input and Output Guards that intercept the inputs and outputs of LLMs.","accessed":"2026-06-17"},{"type":"repo","title":"guardrails-ai/valid_choice","url":"https://github.com/guardrails-ai/valid_choice","quote":"Guardrails AI: Valid choices - validates that a value is within the acceptable choices","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"input-output-guardrails","role":"core","note":"Guardrails runs Input and Output Guards that detect, quantify, and mitigate risks such as hallucinations and data leakage, blocking bad outputs before they reach users.","evidence":[{"type":"repo","title":"guardrails-ai/guardrails (README)","url":"https://github.com/guardrails-ai/guardrails","quote":"Guardrails runs Input/Output Guards in your application that detect, quantify and mitigate the presence of specific types of risks.","accessed":"2026-06-17"},{"type":"doc","title":"Guardrails AI","url":"https://www.guardrailsai.com/","quote":"runtime guardrails that detect policy violations, hallucinations, and data leakage. Block bad outputs before they reach users.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"structured-output","role":"first-class","note":"A Guard can be created from a Pydantic model so that the LLM is called in a manner that formats the output to that class, yielding validated structured data rather than free-form text.","evidence":[{"type":"repo","title":"guardrails-ai/guardrails (README)","url":"https://github.com/guardrails-ai/guardrails","quote":"Guardrails help you generate structured data from LLMs.","accessed":"2026-06-17"},{"type":"doc","title":"Generating Structured Data — Guardrails AI","url":"https://www.guardrailsai.com/docs/how_to_guides/generate_structured_data","quote":"The Guard can be used to call the LLM in a manner so that the output is formatted to the `Pet` class.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"evaluator-optimizer","role":"supported","note":"On the reask on-fail action a validator re-prompts the LLM with the validation error message, looping generation against the validator's correctness criteria until the output passes.","evidence":[{"type":"doc","title":"On-Fail Actions — Guardrails AI","url":"https://guardrailsai.com/docs/concepts/validator_on_fail_actions","quote":"Reask the LLM to generate an output that meets the correctness criteria specified in the validator.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"pii-redaction","role":"supported","note":"The DetectPII validator uses Microsoft Presidio to detect personally identifiable information and, on failure, applies a programmatic fix that anonymizes the offending text before it is returned.","evidence":[{"type":"repo","title":"guardrails-ai/detect_pii (README)","url":"https://github.com/guardrails-ai/detect_pii","quote":"This validator ensures that any given text does not contain PII. This validator uses Microsoft's Presidio to detect PII in the text. If PII is detected, the validator will fail with a programmatic fix that anonymizes the text.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"Validator","summary":"A unit that checks one specific property of an input or output (a choice, a length, the absence of PII) and returns a pass or a FailResult that triggers a configured on-fail action.","maps_to_pattern":"input-output-guardrails","url":"https://guardrailsai.com/docs/concepts/validators"},{"name":"Guard","summary":"A composition of validators that wraps an LLM call as an Input Guard (on the prompt) or an Output Guard (on the response), running its validators in sequence.","maps_to_pattern":"input-output-guardrails","url":"https://guardrailsai.com/docs/concepts/guard"},{"name":"On-fail action","summary":"The per-validator policy chosen when validation fails: reask (re-prompt the model), fix (substitute a corrected value), filter, refrain, exception, or noop.","maps_to_pattern":"evaluator-optimizer","url":"https://guardrailsai.com/docs/concepts/validator_on_fail_actions"},{"name":"Guardrails Hub","summary":"A registry from which pre-built validators are installed and composed into guards, so risk checks are shared rather than re-implemented per project.","url":"https://guardrailsai.com/hub"}],"references":[{"type":"repo","title":"guardrails-ai/guardrails (GitHub)","url":"https://github.com/guardrails-ai/guardrails","quote":"Guardrails runs Input/Output Guards in your application that detect, quantify and mitigate the presence of specific types of risks.","accessed":"2026-06-17"},{"type":"doc","title":"Validators — Guardrails AI documentation","url":"https://guardrailsai.com/docs/concepts/validators","accessed":"2026-06-17"},{"type":"doc","title":"Guardrails Hub","url":"https://guardrailsai.com/hub","accessed":"2026-06-17"}],"alternatives":[{"composition":"nemo-guardrails","relation":"competes-with","note":"Both wrap LLM calls with configurable input and output guardrails; NeMo Guardrails uses a Colang rail definition where Guardrails AI composes Python validators."},{"composition":"llamafirewall","relation":"competes-with","note":"Both are open-source guardrail layers that screen LLM inputs and outputs; LlamaFirewall focuses on alignment and prompt-injection checks rather than a general validator library."},{"composition":"invariant-labs","relation":"competes-with","note":"Invariant Guardrails also intercepts agent traffic with declarative checks, targeting MCP/tool flows where Guardrails AI targets the LLM input/output boundary."}],"verification_status":"partial","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Guardrails AI\"]\n  fw --> p1[\"Input/Output Guardrails<br/>(core)\"]\n  fw --> p2[\"Context Minimization<br/>(first-class)\"]\n  fw --> p3[\"Structured Output<br/>(first-class)\"]\n  fw --> p4[\"Evaluator-Optimizer<br/>(supported)\"]\n  fw --> p5[\"PII Redaction<br/>(supported)\"]","tags":["orchestration-framework","safety-control","structure-data","verification-reflection","full-code"]},{"id":"jade","name":"JADE (Java Agent DEvelopment Framework)","kind":"framework","category":"orchestration-framework","vendor":"Telecom Italia Lab (TILAB)","language":"Java","license":"LGPL-2.0-only","status":"active","first_released":"2000","url":"https://jade.tilab.com/","docs_url":"https://jade-project.gitlab.io/","intent":"Provide a FIPA-compliant Java platform where agents communicate through typed ACL messages and standard interaction protocols such as Contract Net.","description":"JADE is a Java software framework for developing multi-agent applications in compliance with the FIPA specifications. Agents exchange messages encoded in the FIPA Agent Communication Language, where each message carries a performative such as REQUEST or INFORM. The framework ships behaviour classes implementing standard FIPA interaction protocols, including Contract Net. It was first released as open source in 2000 by Telecom Italia under the LGPL license.","primary_use_cases":["building FIPA-compliant multi-agent systems in Java","exchanging typed ACL messages between agents","running standard interaction protocols such as Contract Net"],"agent_loop_shape":"Each agent runs its behaviours and communicates with peers by sending FIPA ACL messages typed by performative; the runtime handles message transport, encoding, and the agent life-cycle, and interaction-protocol behaviour classes drive multi-step exchanges such as a Contract Net negotiation.","build_surface":"full-code","members":[{"pattern":"contract-net-protocol","role":"core","note":"JADE ships first-class ContractNetInitiator and ContractNetResponder behaviour classes implementing the FIPA Contract Net protocol: a manager solicits proposals via a CFP, responders propose, and the initiator awards via accept-proposal.","evidence":[{"type":"api-ref","title":"ContractNetResponder (JADE API)","url":"https://staff.fmi.uvt.ro/~horia.popa/IS/Jade/jade/doc/api/jade/proto/ContractNetResponder.html","quote":"This behaviour implements the `fipa-contract-net` interaction protocol from the point of view of a responder to a call for proposal (`cfp`) message.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"performative-message","role":"core","note":"JADE implements the FIPA Agent Communication Language, where each inter-agent message is an ACLMessage carrying a performative (such as REQUEST, INFORM, or PROPOSE) rather than free-form prose.","evidence":[{"type":"doc","title":"About JADE","url":"https://jade-project.gitlab.io/page/about/","quote":"a software framework to develop agent-based applications in compliance with the FIPA specifications for interoperable intelligent multi-agent systems","accessed":"2026-06-17"},{"type":"tutorial","title":"Communication - Start-Jade","url":"https://startjade.gitlab.io/agents-implementation/communication/","quote":"Create the message ACLMessage msg=new ACLMessage(ACLMessage.INFORM);//FIPA performative to choose","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"tool-agent-registry","role":"core","note":"The Directory Facilitator (DF) is JADE's yellow-pages registry: provider agents register the service types they perform and client agents search the DF for providers of a given service, through the DFService utility class.","evidence":[{"type":"doc","title":"JADE Discovery (Directory Facilitator)","url":"https://www.cs.sjsu.edu/faculty/pearce/modules/lectures/abs/jadeweb/discovery.htm","quote":"The Directory Facilitator agent (DF) is a yellow pages server. Provider agents may register (and deregister) the types of services they perform with the DF. ... Client agents may search the DF for providers of a given service.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"tool-discovery","role":"supported","note":"Agents resolve which peers provide which services at runtime by querying the DF rather than hardcoding agent addresses, via DFService.search().","evidence":[{"type":"doc","title":"JADE Discovery (Directory Facilitator)","url":"https://www.cs.sjsu.edu/faculty/pearce/modules/lectures/abs/jadeweb/discovery.htm","quote":"Client agents may search the DF for providers of a given service. The interface to the DF agent is through the DFService utility class.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"inter-agent-communication","role":"core","note":"JADE defines how agents exchange messages across boundaries: same-container delivery uses Java events, agents in different containers on one platform use Java RMI, and cross-platform delivery uses IIOP, all carrying FIPA-ACL messages.","evidence":[{"type":"doc","title":"JADE for Autonomous Agent Development — SmythOS","url":"https://smythos.com/developers/agent-development/jade-java-agent-development-framework/","quote":"For agents running in the same container, JADE uses efficient Java events. When agents are in different containers but on the same platform, it employs Java RMI. For inter-platform communication, JADE leverages IIOP (Internet Inter-ORB Protocol) to ensure reliable message delivery.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"actor-model-agents","role":"supported","note":"Each JADE agent runs on its own thread of execution and maintains a private message queue into which the runtime posts incoming ACL messages, processing them asynchronously rather than sharing mutable state with peers.","evidence":[{"type":"doc","title":"JADE for Autonomous Agent Development — SmythOS","url":"https://smythos.com/developers/agent-development/jade-java-agent-development-framework/","quote":"JADE's messaging system utilizes an asynchronous message passing approach, where each agent maintains a private message queue for receiving and processing communications.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"ACLMessage","summary":"The unit of inter-agent communication: a FIPA-ACL message with a performative (communicative act), sender, receivers, content, content language, and ontology.","maps_to_pattern":"performative-message","url":"https://startjade.gitlab.io/agents-implementation/communication/"},{"name":"Behaviour","summary":"A task an agent carries out; an agent schedules several behaviours cooperatively (non-preemptively) on its single thread, and interaction-protocol behaviours encode multi-step exchanges.","url":"https://jade-project.gitlab.io/page/about/"},{"name":"Directory Facilitator (DF)","summary":"The platform's yellow-pages agent where providers register service types and clients search for providers; DFs can federate across platforms.","maps_to_pattern":"tool-agent-registry","url":"https://www.cs.sjsu.edu/faculty/pearce/modules/lectures/abs/jadeweb/discovery.htm"},{"name":"Agent Platform / container","summary":"The distributed runtime: agents live in containers across one or more hosts, and JADE handles message transport, encoding, and the agent life-cycle between them.","maps_to_pattern":"inter-agent-communication","url":"https://jade-project.gitlab.io/page/about/"}],"references":[{"type":"doc","title":"About JADE","url":"https://jade-project.gitlab.io/page/about/","quote":"JADE (Java Agent DEvelopment Framework) is a software framework to develop agent-based applications in compliance with the FIPA specifications for interoperable intelligent multi-agent systems.","accessed":"2026-06-17"},{"type":"paper","title":"JADE — A FIPA-compliant agent framework (Bellifemine, Poggi, Rimassa, 1999)","url":"https://jmvidal.cse.sc.edu/library/jade.pdf","accessed":"2026-06-17"},{"type":"tutorial","title":"Communication — Start-Jade","url":"https://startjade.gitlab.io/agents-implementation/communication/","accessed":"2026-06-17"}],"alternatives":[{"composition":"jason-bdi","relation":"similar-shape","note":"Both are classical pre-LLM multi-agent platforms; Jason runs AgentSpeak BDI agents and commonly interoperates with JADE for FIPA messaging and infrastructure."},{"composition":"camel-ai","relation":"competes-with","note":"Both build communicating multi-agent systems, but CAMEL-AI orchestrates role-playing LLM agents in natural language rather than FIPA-typed ACL messages."}],"verification_status":"partial","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"JADE (Java Agent DEvelopment Framework)\"]\n  fw --> p1[\"Contract Net Protocol<br/>(core)\"]\n  fw --> p2[\"Performative Message<br/>(core)\"]\n  fw --> p3[\"Tool/Agent Registry<br/>(core)\"]\n  fw --> p4[\"Inter-Agent Communication<br/>(core)\"]\n  fw --> p5[\"Tool Discovery<br/>(supported)\"]\n  fw --> p6[\"Actor-Model Agents<br/>(supported)\"]","tags":["orchestration-framework","multi-agent","tool-use-environment","full-code"]},{"id":"together-moa","name":"Together Mixture-of-Agents (MoA)","kind":"framework","category":"orchestration-framework","vendor":"Together AI","language":"Python","license":"Apache-2.0","status":"active","first_released":"2024-06-01","url":"https://docs.together.ai/docs/mixture-of-agents","repo":"https://github.com/togethercomputer/MoA","docs_url":"https://docs.together.ai/docs/mixture-of-agents","intent":"Together Mixture-of-Agents sends a prompt to several open-source LLMs acting as proposers and has a final aggregator LLM synthesize their responses into one answer.","description":"Mixture-of-Agents is a method and reference implementation from Together AI that runs several LLMs in parallel on the same prompt, then passes their outputs to a final aggregator LLM that synthesizes a single response. The proposer models run independently in one layer; the aggregator combines their results in a second layer. The architecture can be stacked across multiple layers, where each layer comprises several LLM agents. The reference code is published under Apache 2.0.","primary_use_cases":["combining multiple open-source LLMs on one prompt","synthesizing several model responses into one answer","quality improvement through proposer-aggregator layering"],"agent_loop_shape":"A prompt is sent in parallel to several proposer LLMs, each producing an independent response. The collected responses are passed to a final aggregator LLM whose instruction is to synthesize them into a single high-quality answer. The layer can be repeated, feeding aggregated output back into another round of proposers and aggregation for further refinement.","build_surface":"full-code","members":[{"pattern":"heterogeneous-model-council-with-judge","role":"core","note":"Sends a prompt to several different LLMs acting as proposers, then a final aggregator LLM synthesizes their outputs into one answer; realises the role-specialized-personas-on-different-architectures plus synthesis core, though the aggregator is not isolated to structured-JSON-only.","evidence":[{"type":"doc","title":"Together Mixture of Agents (MoA)","url":"https://docs.together.ai/docs/mixture-of-agents","quote":"given a prompt, like `tell me the best things to do in SF`, it sends it to 4 different OSS LLMs. It then combines results from all 4, sends it to a final LLM, and asks it to combine all 4 responses into an ideal response","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"parallel-fan-out-gather","role":"core","note":"The proposer LLMs run in parallel on the same prompt and a dedicated aggregator LLM gathers and reconciles their independent results into a single output, with the layered architecture stackable for further refinement.","evidence":[{"type":"doc","title":"Together Mixture of Agents (MoA)","url":"https://docs.together.ai/docs/mixture-of-agents","quote":"it sends it to 4 different OSS LLMs. It then combines results from all 4, sends it to a final LLM","accessed":"2026-06-17"},{"type":"doc","title":"Together Mixture of Agents (MoA)","url":"https://docs.together.ai/docs/mixture-of-agents","quote":"By employing a layered architecture where each layer comprises several LLM agents, MoA significantly outperforms","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"self-refine","role":"supported","note":"Stacking layers makes MoA iterative: each additional layer re-processes the prompt together with the previous layer's aggregated outputs as auxiliary information, refining the answer across rounds (rounds == layers - 1).","evidence":[{"type":"repo","title":"togethercomputer/MoA (README)","url":"https://github.com/togethercomputer/MoA","quote":"--rounds: Number of rounds to process the input for refinement. (num rounds == num of MoA layers - 1)","accessed":"2026-06-17"},{"type":"paper","title":"Mixture-of-Agents Enhances Large Language Model Capabilities (arXiv:2406.04692)","url":"https://arxiv.org/abs/2406.04692","quote":"Each agent takes all the outputs from agents in the previous layer as auxiliary information in generating its response.","accessed":"2026-06-17"}],"evidence_status":"full"}],"key_concepts":[{"name":"Proposer","summary":"An LLM in a layer that generates a candidate response to the prompt; several proposers run in parallel and the next layer treats their outputs as auxiliary context.","maps_to_pattern":"parallel-fan-out-gather","url":"https://docs.together.ai/docs/mixture-of-agents"},{"name":"Aggregator","summary":"The final LLM that synthesizes the proposers' responses into a single high-quality answer rather than picking one verbatim.","maps_to_pattern":"heterogeneous-model-council-with-judge","url":"https://docs.together.ai/docs/mixture-of-agents"},{"name":"Layered architecture","summary":"The stacked structure in which each layer comprises several LLM agents and each agent consumes the previous layer's outputs, enabling additional refinement rounds.","maps_to_pattern":"self-refine","url":"https://arxiv.org/abs/2406.04692"}],"references":[{"type":"paper","title":"Mixture-of-Agents Enhances Large Language Model Capabilities (Wang et al., arXiv:2406.04692)","url":"https://arxiv.org/abs/2406.04692","quote":"In our approach, we construct a layered MoA architecture wherein each layer comprises multiple LLM agents.","accessed":"2026-06-17"},{"type":"doc","title":"Together Mixture of Agents (MoA)","url":"https://docs.together.ai/docs/mixture-of-agents","quote":"synthesize these responses into a single, high-quality response","accessed":"2026-06-17"},{"type":"repo","title":"togethercomputer/MoA (GitHub)","url":"https://github.com/togethercomputer/MoA","accessed":"2026-06-17"}],"alternatives":[{"composition":"gptswarm","relation":"similar-shape","note":"Both compose multiple LLM agents into a graph that produces a combined answer; GPTSwarm optimizes the agent graph topology where MoA fixes a proposer-aggregator layering."},{"composition":"routellm","relation":"complements","note":"RouteLLM picks one model per query to cut cost, the inverse trade-off to MoA, which spends several model calls per query to raise quality; the two can be combined."}],"verification_status":"partial","status_in_practice":"emerging","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Together Mixture-of-Agents (MoA)\"]\n  fw --> p1[\"Heterogeneous-Model Council with Synthesis Judge<br/>(core)\"]\n  fw --> p2[\"Parallel Fan-Out / Gather<br/>(core)\"]\n  fw --> p3[\"Self-Refine<br/>(supported)\"]","tags":["orchestration-framework","multi-agent","verification-reflection","full-code"]},{"id":"dendron","name":"Dendron","kind":"framework","category":"orchestration-framework","vendor":"Richard Kelley","language":"Python","license":"MIT","status":"active","first_released":"2024-04-01","url":"https://github.com/RichardKelley/dendron","repo":"https://github.com/RichardKelley/dendron","docs_url":"https://richardkelley.io/dendron/","intent":"Dendron is a Python library for programming LLM agents as behavior trees whose leaves are model and tool actions composed by control nodes.","description":"Dendron is a library for building software agents using behavior trees and language models. Leaf action nodes wrap LLM calls (CausalLMAction, ImageLMAction, LogLikelihoodAction) and tools, and they are composed by control nodes such as Sequence and Fallback. Every behavior tree has an associated Blackboard, a key-value store accessible by all nodes, which is the mechanism by which nodes share state since they otherwise communicate only by ticking children and returning a NodeStatus.","primary_use_cases":["composing LLM agents as behavior trees","reactive control flow over model and tool actions","reusing simple behaviors as larger agent programs","structured programming of language model agents"],"agent_loop_shape":"An agent is expressed as a behavior tree. The root is ticked, control nodes such as Sequence and Fallback tick their children in order, and leaf action nodes run an LLM call or tool and return a NodeStatus of SUCCESS, FAILURE, or RUNNING. Nodes that need to share information beyond status read and write a shared Blackboard key-value store attached to the tree.","build_surface":"full-code","members":[{"pattern":"agentic-behavior-tree","role":"core","note":"A Python library for programming LLM agents as behavior trees: LLM action leaves (CausalLMAction etc.) return success/failure and are composed by control nodes (sequence, fallback/selector, decorators).","evidence":[{"type":"repo","title":"RichardKelley/dendron","url":"https://github.com/RichardKelley/dendron","quote":"Dendron is a library for building software agents using behavior trees and language models.","accessed":"2026-06-17"},{"type":"doc","title":"Dendron documentation (node types)","url":"https://richardkelley.io/dendron/","quote":"behavior trees make it easy to compose simple behaviors (pieces of code) into collections of complex behaviors. [Control nodes include Sequence and Fallback (Selector); action types include CausalLMAction, ImageLMAction, LogLikelihoodAction.]","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"blackboard","role":"core","note":"Every Dendron behavior tree has an associated Blackboard, a key-value mapping accessible by all nodes in the tree, which is how nodes share state with one another since direct node-to-node communication is limited to ticking children and returning a NodeStatus.","evidence":[{"type":"doc","title":"Dendron Blackboard — API reference","url":"https://richardkelley.io/dendron/api/blackboard/","quote":"A blackboard for a Behavior Tree. Implements a key-value mapping that is accessible by all of the nodes in a behavior tree.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"fallback-chain","role":"supported","note":"The Fallback (Selector) control node ticks its children in order and returns on the first SUCCESS, only failing if all children fail, so a primary action falls through to alternative recovery behaviors.","evidence":[{"type":"doc","title":"Dendron Fallback node — API reference","url":"https://richardkelley.io/dendron/api/controls/fallback/","quote":"A Fallback node is a control node that ticks its children in sequence, until a child returns `SUCCESS`, at which point it returns success. If all children return `FAILURE`, then the Fallback node returns `FAILURE`.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"key_concepts":[{"name":"Action node","summary":"A leaf node that performs work and returns a NodeStatus; Dendron's LLM action leaves include CausalLMAction, ImageLMAction, and LogLikelihoodAction.","maps_to_pattern":"agentic-behavior-tree","url":"https://richardkelley.io/dendron/"},{"name":"Control node","summary":"An internal node (Sequence, Fallback/Selector, decorators) that orchestrates control flow by ticking its children and combining their NodeStatus results.","maps_to_pattern":"agentic-behavior-tree","url":"https://richardkelley.io/dendron/"},{"name":"Blackboard","summary":"A key-value store attached to a tree and readable/writable by every node, the only channel for sharing state beyond the tick/NodeStatus contract.","maps_to_pattern":"blackboard","url":"https://richardkelley.io/dendron/api/blackboard/"},{"name":"NodeStatus","summary":"The return value a ticked node yields — SUCCESS, FAILURE, or RUNNING — which control nodes use to decide how the tree proceeds.","url":"https://richardkelley.io/dendron/"}],"references":[{"type":"paper","title":"Behavior Trees Enable Structured Programming of Language Model Agents (Kelley, arXiv:2404.07439)","url":"https://arxiv.org/abs/2404.07439","quote":"We introduce Dendron, a Python library for programming language model agents using behavior trees.","accessed":"2026-06-17"},{"type":"repo","title":"RichardKelley/dendron (GitHub)","url":"https://github.com/RichardKelley/dendron","quote":"Behavior trees are a technique for building complex reactive agents by composing simpler behaviors in a principled way.","accessed":"2026-06-17"},{"type":"doc","title":"Dendron documentation","url":"https://richardkelley.io/dendron/","accessed":"2026-06-17"}],"alternatives":[{"composition":"pocketflow","relation":"similar-shape","note":"Both express an agent as a small graph of composable nodes with explicit control flow; PocketFlow uses a directed node graph where Dendron uses the behavior-tree formalism."},{"composition":"burr","relation":"similar-shape","note":"Both give the agent loop explicit, inspectable control structure instead of free-form prompting; Burr models it as a state machine rather than a behavior tree."}],"verification_status":"partial","status_in_practice":"experimental","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Dendron\"]\n  fw --> p1[\"Agentic Behavior Tree<br/>(core)\"]\n  fw --> p2[\"Blackboard<br/>(core)\"]\n  fw --> p3[\"Fallback Chain<br/>(supported)\"]","tags":["orchestration-framework","planning-control-flow","multi-agent","routing-composition","full-code"]},{"id":"inspect-ai","name":"Inspect AI","kind":"framework","category":"orchestration-framework","vendor":"UK AI Safety Institute (AISI)","language":"Python","license":"MIT","status":"active","first_released":"2024-05-01","url":"https://inspect.aisi.org.uk/","repo":"https://github.com/UKGovernmentBEIS/inspect_ai","docs_url":"https://inspect.aisi.org.uk/","intent":"Inspect AI is an evaluation framework that runs labelled datasets through solvers and scorers to measure language-model and agent performance.","description":"Inspect is an open-source framework for large language model evaluations from the UK AI Security Institute. An Inspect evaluation is a Task that combines a Dataset of labelled samples, a Solver that produces an answer for each sample, and a Scorer that evaluates the output. Solvers range from a single generate() call to a full tool-using agent, and scorers range from text comparison to model grading. Its model_graded_qa() scorer runs a separate grader model that sees only the question, answer, criterion, and instructions.","primary_use_cases":["running held-out evaluation datasets against models and agents","model-graded scoring of open-ended answers","agentic and reasoning task evaluation","safety and capability benchmarking"],"agent_loop_shape":"An evaluation is defined as a Task binding a Dataset, a Solver, and a Scorer. For each sample, the solver produces an answer, which may be a single generate() call or a full agent that uses tools over many turns. The scorer then evaluates the output by text comparison or by model grading, where model_graded_qa() invokes a separate grader model given only question, answer, criterion, and instructions, optionally with several graders voting.","build_surface":"full-code","members":[{"pattern":"blind-grader-with-isolated-context","role":"core","note":"Its model_graded_qa() scorer runs a separate grader model whose prompt template exposes only question, answer, criterion and instructions, so the grader judges the answer against the rubric without ever seeing the producing model's reasoning trace.","evidence":[{"type":"api-ref","title":"inspect_ai.scorer — model_graded_qa reference","url":"https://inspect.aisi.org.uk/reference/inspect_ai.scorer.html","quote":"This template has four variables: question, criterion, answer, and instructions (which is fed from the instructions parameter). ... Model or models to use for grading. If a list is provided, each model grades independently and the final grade is computed by majority vote.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"eval-harness","role":"core","note":"An Inspect evaluation is a Task binding a Dataset of labelled samples, a Solver that produces an answer per sample, and a Scorer that evaluates the output, so a held-out dataset is run against models and agents to measure performance and detect regressions.","evidence":[{"type":"doc","title":"Inspect — home","url":"https://inspect.aisi.org.uk/","quote":"An Inspect evaluation is a Task that brings together three things: Dataset that provides labelled samples, Solver that produces an answer for each sample, and Scorer that evaluates the output.","accessed":"2026-06-17"},{"type":"doc","title":"Inspect — home","url":"https://inspect.aisi.org.uk/","quote":"Solver that produces an answer for each sample. This can be as simple as a single generate() call to the model, or as sophisticated as a full agent that uses tools over many turns.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"react","role":"supported","note":"The agents module ships a general-purpose react agent solver that runs a tool loop, interleaving reasoning and tool calls until the model invokes the special submit() tool to signal completion.","evidence":[{"type":"doc","title":"Inspect — ReAct Agent","url":"https://inspect.aisi.org.uk/react-agent.html","quote":"It runs a tool loop until the model calls a special `submit()` tool indicating it is done.","accessed":"2026-06-17"},{"type":"doc","title":"Inspect — Agents","url":"https://inspect.aisi.org.uk/agents.html","quote":"The agents module includes a flexible, general-purpose react agent","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"sandbox-isolation","role":"supported","note":"When a solver's tools execute arbitrary model-generated code (shell or Python), Inspect provisions sandboxes — Docker containers by default — so untrusted code runs in an isolated environment rather than the evaluation process.","evidence":[{"type":"doc","title":"Inspect — Sandboxing","url":"https://inspect.aisi.org.uk/sandboxing.html","quote":"Inspect provides support for _sandboxing_, which typically involves provisioning containers for tools to execute code within.","accessed":"2026-06-17"},{"type":"doc","title":"Inspect — Sandboxing","url":"https://inspect.aisi.org.uk/sandboxing.html","quote":"Support for Docker sandboxes is built in, and the Extension API enables the creation of additional sandbox types.","accessed":"2026-06-17"}],"evidence_status":"full"}],"key_concepts":[{"name":"Task","summary":"The unit of an Inspect evaluation, binding a Dataset, a Solver, and a Scorer into a runnable, repeatable evaluation.","maps_to_pattern":"eval-harness","url":"https://inspect.aisi.org.uk/"},{"name":"Solver","summary":"The component that produces an answer for each sample, ranging from a single generate() call to a full tool-using agent run over many turns.","maps_to_pattern":"react","url":"https://inspect.aisi.org.uk/"},{"name":"Scorer","summary":"The component that evaluates a solver's output, from text comparison (match, pattern) to model grading (model_graded_qa) against a target rubric.","maps_to_pattern":"blind-grader-with-isolated-context","url":"https://inspect.aisi.org.uk/scorers.html"},{"name":"Sandbox environment","summary":"A provisioned isolated runtime (Docker by default) in which tools execute model-generated code, keeping untrusted execution out of the main evaluation process.","maps_to_pattern":"sandbox-isolation","url":"https://inspect.aisi.org.uk/sandboxing.html"}],"references":[{"type":"repo","title":"UKGovernmentBEIS/inspect_ai (GitHub)","url":"https://github.com/UKGovernmentBEIS/inspect_ai","quote":"Welcome to Inspect, a framework for large language model evaluations created by the UK AI Security Institute.","accessed":"2026-06-17"},{"type":"doc","title":"Inspect — home","url":"https://inspect.aisi.org.uk/","quote":"An Inspect evaluation is a Task that brings together three things: Dataset that provides labelled samples, Solver that produces an answer for each sample, and Scorer that evaluates the output.","accessed":"2026-06-17"},{"type":"doc","title":"Inspect — Scorers","url":"https://inspect.aisi.org.uk/scorers.html","accessed":"2026-06-17"}],"alternatives":[{"composition":"deepeval","relation":"competes-with","note":"Both are open-source LLM evaluation frameworks with model-graded metrics; DeepEval targets pytest-style application testing where Inspect targets dataset-driven safety and capability evals."},{"composition":"promptfoo","relation":"competes-with","note":"Both run held-out test cases against models with assertions and graders; Promptfoo is config-and-CLI driven while Inspect is a Python Task/Solver/Scorer framework."},{"composition":"ragas","relation":"complements","note":"Ragas specializes in RAG-pipeline metrics and synthetic test-set generation; Inspect is a general agent/model eval harness, and the two cover different evaluation scopes."}],"verification_status":"partial","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Inspect AI\"]\n  fw --> p1[\"Blind Grader with Isolated Context<br/>(core)\"]\n  fw --> p2[\"Eval Harness<br/>(core)\"]\n  fw --> p3[\"ReAct<br/>(supported)\"]\n  fw --> p4[\"Sandbox Isolation<br/>(supported)\"]","tags":["orchestration-framework","verification-reflection","governance-observability","planning-control-flow","tool-use-environment","full-code"]},{"id":"ragas","name":"Ragas (synthetic testset generation)","kind":"framework","category":"orchestration-framework","vendor":"Exploding Gradients","language":"Python","license":"Apache-2.0","status":"active","first_released":"2023-07-01","url":"https://docs.ragas.io/","repo":"https://github.com/vibrantlabsai/ragas","docs_url":"https://docs.ragas.io/","intent":"Ragas generates synthetic test sets across named question dimensions and scores RAG and agent outputs with LLM-based evaluation metrics.","description":"Ragas is a Python library for evaluating retrieval-augmented and agentic LLM applications. Its synthetic test data generator enumerates named question types (such as reasoning, conditioning, and multi-context) with a configurable distribution and named personas to produce a diverse evaluation set instead of relying on free-form LLM prompting that mode-collapses. Its metrics include LLM-based metrics that use a configured LLM to score outputs against criteria. Ragas is released under Apache 2.0.","primary_use_cases":["synthetic test set generation for RAG","LLM-based evaluation of RAG and agent outputs","measuring retrieval and answer quality","regression testing of LLM pipelines"],"agent_loop_shape":"From a set of source documents, Ragas synthesises test questions by enumerating question evolution types across a configurable distribution and seeding generation with named personas, producing a diverse held-out set. At evaluation time, each sample is scored by metrics, where LLM-based metrics issue one or more LLM calls to grade the output against criteria, yielding scores aligned with human judgement.","build_surface":"full-code","members":[{"pattern":"dimensional-synthetic-eval-set","role":"core","note":"Ragas rejects naive LLM prompting (which mode-collapses) and instead enumerates named question evolution types (simple, reasoning, multi_context, conditional) with a configurable distribution plus named personas to seed test generation across defined dimensions.","evidence":[{"type":"doc","title":"Synthetic Test Data generation | Ragas","url":"https://docs.ragas.io/en/v0.1.21/concepts/testset_generation.html","quote":"LLMs by default are not good at creating diverse samples as it tends to follow common paths. ... questions with different characteristics such as reasoning, conditioning, multi-context, and more are systematically crafted from the provided set of documents.","accessed":"2026-06-17"},{"type":"doc","title":"Persona Generation | Ragas","url":"https://docs.ragas.io/en/stable/howtos/customizations/testgenerator/_persona_generator/","quote":"You can add different persona's to the testset generation process by defining the Persona class with the name and role description ... you can use these personas in the testset generation process by passing them to the TestsetGenerator class.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"llm-as-judge","role":"core","note":"Ragas scores outputs with LLM-based metrics that use an LLM underneath to perform the evaluation, issuing one or more LLM calls per sample to grade the output against criteria, an approach it states is closer to human evaluation than non-LLM metrics.","evidence":[{"type":"doc","title":"Metrics Overview | Ragas","url":"https://docs.ragas.io/en/stable/concepts/metrics/overview/","quote":"These metrics use LLM underneath to do the evaluation. There might be one or more LLM calls that are performed to arrive at the score or result.","accessed":"2026-06-17"},{"type":"doc","title":"Metrics Overview | Ragas","url":"https://docs.ragas.io/en/stable/concepts/metrics/overview/","quote":"All LLM based metrics in ragas are inherited from `MetricWithLLM` class. These metrics expects a LLM object to be set before scoring.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"eval-harness","role":"supported","note":"The evaluate() function runs a held-out EvaluationDataset of samples through a chosen list of metrics and returns a per-metric score for the pipeline, the offline batch run used to measure and compare RAG quality.","evidence":[{"type":"doc","title":"Evaluate a simple RAG system | Ragas","url":"https://docs.ragas.io/en/stable/getstarted/rag_eval/","quote":"Now, we can evaluate our RAG system on the collected dataset using a set of commonly used RAG evaluation metrics.","accessed":"2026-06-17"},{"type":"api-ref","title":"evaluate() | Ragas","url":"https://docs.ragas.io/en/stable/references/evaluate/","quote":"result = evaluate(dataset=evaluation_dataset, metrics=[LLMContextRecall(), Faithfulness(), FactualCorrectness()], llm=evaluator_llm)","accessed":"2026-06-17"}],"evidence_status":"full"}],"key_concepts":[{"name":"Question evolution types","summary":"Named transformations (simple, reasoning, multi_context, conditional) applied with a configurable distribution to craft diverse questions from source documents.","maps_to_pattern":"dimensional-synthetic-eval-set","url":"https://docs.ragas.io/en/v0.1.21/concepts/testset_generation.html"},{"name":"Persona","summary":"A named role with a description used to seed test-set generation so questions reflect different user viewpoints rather than one generic asker.","maps_to_pattern":"dimensional-synthetic-eval-set","url":"https://docs.ragas.io/en/stable/howtos/customizations/testgenerator/_persona_generator/"},{"name":"Faithfulness metric","summary":"An LLM-based metric that breaks the response into claims and scores the fraction supported by the retrieved context, measuring groundedness against hallucination.","maps_to_pattern":"llm-as-judge","url":"https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/faithfulness/"},{"name":"EvaluationDataset","summary":"The collection of samples (queries, contexts, responses, references) that the evaluate() function runs metrics against to produce scores.","maps_to_pattern":"eval-harness","url":"https://docs.ragas.io/en/stable/concepts/components/eval_dataset/"}],"references":[{"type":"repo","title":"vibrantlabsai/ragas (GitHub)","url":"https://github.com/vibrantlabsai/ragas","quote":"Ragas is your ultimate toolkit for evaluating and optimizing Large Language Model (LLM) applications.","accessed":"2026-06-17"},{"type":"doc","title":"Metrics Overview | Ragas","url":"https://docs.ragas.io/en/stable/concepts/metrics/overview/","quote":"These metrics use LLM underneath to do the evaluation. There might be one or more LLM calls that are performed to arrive at the score or result.","accessed":"2026-06-17"},{"type":"doc","title":"Synthetic Test Data generation | Ragas","url":"https://docs.ragas.io/en/v0.1.21/concepts/testset_generation.html","accessed":"2026-06-17"}],"alternatives":[{"composition":"deepeval","relation":"competes-with","note":"Both are open-source LLM/RAG evaluation libraries with LLM-graded metrics and synthetic data generation; DeepEval frames evaluation as pytest test cases."},{"composition":"inspect-ai","relation":"complements","note":"Inspect AI is a general dataset/solver/scorer eval harness for models and agents; Ragas specializes in RAG-pipeline metrics and document-grounded test-set synthesis."},{"composition":"promptfoo","relation":"competes-with","note":"Both score LLM outputs against assertions over a test set; Promptfoo is config/CLI-driven for general prompt and app testing where Ragas centres on retrieval metrics."}],"verification_status":"partial","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","pattern_composition":"flowchart TD\n  fw[\"Ragas (synthetic testset generation)\"]\n  fw --> p1[\"Dimensional Synthetic Eval Set<br/>(core)\"]\n  fw --> p2[\"LLM-as-Judge<br/>(core)\"]\n  fw --> p3[\"Eval Harness<br/>(supported)\"]","tags":["orchestration-framework","verification-reflection","governance-observability","full-code"]},{"id":"camunda","name":"Camunda 8 (Agentic Orchestration)","kind":"framework","category":"orchestration-framework","vendor":"Camunda","language":"Java","license":"proprietary","status":"active","first_released":"2022","url":"https://camunda.com/platform/","docs_url":"https://docs.camunda.io/docs/components/agentic-orchestration/agentic-orchestration/","intent":"Camunda 8 orchestrates LLM-driven agents inside BPMN processes so an agent decides which tools to call while the Zeebe engine deterministically executes the selected activities, stores state, and routes human tasks.","description":"Camunda 8 is a process-orchestration platform built on the Zeebe engine. Its agentic orchestration runs an LLM agent inside a BPMN ad-hoc sub-process whose activities define the available tools; the model chooses the next tool and the engine executes it, persists variables, and applies retries and incident handling. The same engine runs deterministic flows, decision tables, human tasks, and agentic flows together. Teams use it to embed agents in audited, end-to-end business processes.","primary_use_cases":["embedding LLM agents in regulated business processes","orchestrating tool calls as BPMN activities","mixing deterministic flows, decision tables, and agentic decisions in one engine","routing agent work to human approvers within a process"],"agent_loop_shape":"A user prompt enters an ad-hoc sub-process. The LLM evaluates the prompt, system prompt, and tool definitions and selects which BPMN activity to invoke next; Zeebe executes that activity, stores variables, applies retries and incident handling, and feeds the result back. The loop repeats until the agent produces a final response, with human tasks and events routed by the engine where the model defers to a person.","build_surface":"low-code","members":[{"pattern":"bpmn-dmn-deterministic-shell","role":"core","note":"BPMN is the deterministic spine: the LLM agent runs inside a BPMN ad-hoc sub-process whose nodes define the available tools/steps. The model only decides which tool to call next; Zeebe deterministically executes the selected BPMN activities, stores state, applies retries/incident handling. Decision and orchestration are intentionally split.","evidence":[{"type":"doc","title":"AI agents | Camunda 8 Docs","url":"https://docs.camunda.io/docs/components/agentic-orchestration/ai-agents/","quote":"LLM decides: Which tool to call next, in what order, and with which parameters. Camunda orchestrates: Executes the selected BPMN activity, stores variables, applies retries and incident handling, and routes human tasks and events.","accessed":"2026-06-17"},{"type":"blog","title":"Enterprise-Grade Agentic Automation Is Here","url":"https://camunda.com/blog/2025/10/enterprise-grade-agentic-automation-is-here/","quote":"Having both deterministic and dynamic flows run in a single engine enables you to handle the entire spectrum of business processes: the straightforward, pre-defined work that automation excels at—along with this new, agentic way of working that thrives on non-linear work based on context.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"human-in-the-loop","role":"first-class","note":"Because the agent runs inside a BPMN process, a human task is a native activity the orchestration routes to: agentic decisions, deterministic rules, and human tasks collaborate in the same end-to-end process, so a person can approve or take over at defined points.","evidence":[{"type":"doc","title":"Agentic orchestration | Camunda 8 Docs","url":"https://docs.camunda.io/docs/components/agentic-orchestration/agentic-orchestration/","quote":"Camunda agentic orchestration allows you to orchestrate AI agents within your BPMN-based workflows, enabling human tasks, deterministic rule sets, and AI-driven decisions to collaborate in a robust, end-to-end process.","accessed":"2026-06-17"},{"type":"doc","title":"AI agents | Camunda 8 Docs","url":"https://docs.camunda.io/docs/components/agentic-orchestration/ai-agents/","quote":"applies retries and incident handling, and routes human tasks and events","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"tool-use","role":"core","note":"The agent's actions are typed tool calls, not free text: BPMN activities inside the ad-hoc sub-process are exposed as the tool set, the model evaluates the prompt and the available tool definitions, and when it decides a tool is needed Camunda activates the corresponding activity. The model produces a tool selection that the engine executes rather than prose the surrounding system has to parse.","evidence":[{"type":"doc","title":"AI agents | Camunda 8 Docs","url":"https://docs.camunda.io/docs/components/agentic-orchestration/ai-agents/","quote":"Exposes BPMN activities inside an ad-hoc sub-process as tools the LLM can select.","accessed":"2026-06-17"},{"type":"doc","title":"AI agents | Camunda 8 Docs","url":"https://docs.camunda.io/docs/components/agentic-orchestration/ai-agents/","quote":"The LLM evaluates the prompt, the system prompt, and the available tool definitions.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"mcp","role":"supported","note":"Beyond BPMN activities, the AI Agent connector can call tools exposed by Model Context Protocol servers, so a tool written once for MCP is callable from inside a Camunda process without a bespoke adapter.","evidence":[{"type":"doc","title":"AI agents | Camunda 8 Docs","url":"https://docs.camunda.io/docs/components/agentic-orchestration/ai-agents/","quote":"Connect an AI agent connector to tools exposed by Model Context Protocol (MCP) servers.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"short-term-memory","role":"supported","note":"The connector carries the conversation slice across loop iterations within the running process instance, so the agent can handle multi-turn interactions and follow-up questions rather than treating each model call as stateless.","evidence":[{"type":"doc","title":"AI agents | Camunda 8 Docs","url":"https://docs.camunda.io/docs/components/agentic-orchestration/ai-agents/","quote":"Short-term conversational memory enables multi-turn interactions and follow-up questions within a process instance.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"verification_status":"partial","status_in_practice":"emerging","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","key_concepts":[{"name":"Ad-hoc sub-process","summary":"A BPMN embedded sub-process marked ad-hoc whose contained activities have no fixed sequence flow; the AI agent treats those activities as the selectable tool set and the engine activates whichever one the model chooses next.","maps_to_pattern":"bpmn-dmn-deterministic-shell","url":"https://docs.camunda.io/docs/components/agentic-orchestration/ai-agents/"},{"name":"AI Agent connector feedback loop","summary":"The AI Agent connector runs as a feedback loop between the LLM and Camunda: the model selects a tool, the engine executes the corresponding BPMN activity and stores variables, and the result is fed back to the model until a final response is produced.","maps_to_pattern":"react","url":"https://docs.camunda.io/docs/components/agentic-orchestration/ai-agents/"},{"name":"Short-term conversational memory","summary":"Per-process-instance memory that retains the running conversation so the agent supports multi-turn interactions and follow-up questions inside one process instance.","maps_to_pattern":"short-term-memory","url":"https://docs.camunda.io/docs/components/agentic-orchestration/ai-agents/"},{"name":"MCP tool connection","summary":"A way to attach the AI Agent connector to tools served by Model Context Protocol servers, extending the agent's tool palette beyond locally modelled BPMN activities.","maps_to_pattern":"mcp","url":"https://docs.camunda.io/docs/components/agentic-orchestration/ai-agents/"}],"references":[{"type":"doc","title":"Agentic orchestration | Camunda 8 Docs","url":"https://docs.camunda.io/docs/components/agentic-orchestration/agentic-orchestration/","quote":"Camunda agentic orchestration allows you to orchestrate AI agents within your BPMN-based workflows, enabling human tasks, deterministic rule sets, and AI-driven decisions to collaborate in a robust, end-to-end process.","accessed":"2026-06-17"},{"type":"doc","title":"AI agents | Camunda 8 Docs","url":"https://docs.camunda.io/docs/components/agentic-orchestration/ai-agents/","quote":"Exposes BPMN activities inside an ad-hoc sub-process as tools the LLM can select.","accessed":"2026-06-17"},{"type":"blog","title":"Enterprise-Grade Agentic Automation Is Here","url":"https://camunda.com/blog/2025/10/enterprise-grade-agentic-automation-is-here/","quote":"Having both deterministic and dynamic flows run in a single engine enables you to handle the entire spectrum of business processes","accessed":"2026-06-17"}],"alternatives":[{"composition":"temporal","relation":"similar-shape","note":"Temporal is also a durable execution engine used to host agent loops, but it is a code-first workflow runtime rather than a BPMN/DMN low-code modeller."},{"composition":"n8n","relation":"competes-with","note":"n8n is a low-code workflow platform that likewise embeds LLM agent steps in visual flows, targeting a lighter-weight automation audience than Camunda's process-orchestration spine."},{"composition":"restack","relation":"similar-shape","note":"Restack is another durable workflow engine positioned for agent orchestration, sharing the deterministic-engine-around-the-LLM shape without BPMN modelling."}],"pattern_composition":"flowchart TD\n  fw[\"Camunda 8 (Agentic Orchestration)\"]\n  fw --> p1[\"BPMN/DMN Deterministic Shell Around Agent<br/>(core)\"]\n  fw --> p2[\"Tool Use<br/>(core)\"]\n  fw --> p3[\"Human-in-the-Loop<br/>(first-class)\"]\n  fw --> p4[\"Model Context Protocol<br/>(supported)\"]\n  fw --> p5[\"Short-Term Thread Memory<br/>(supported)\"]","tags":["orchestration-framework","routing-composition","safety-control","tool-use-environment","memory","low-code"]},{"id":"portkey","name":"Portkey (Conditional Routing)","kind":"framework","category":"orchestration-framework","vendor":"Portkey AI","language":"TypeScript","license":"MIT","status":"active","first_released":"2023","url":"https://portkey.ai/","repo":"https://github.com/Portkey-AI/gateway","docs_url":"https://portkey.ai/docs/","intent":"Portkey is an AI gateway that routes each LLM request to a provider target by deterministic rules over request metadata and parameters, and falls back to another provider when one fails.","description":"Portkey's AI Gateway sits in front of many LLM providers behind one API. Its conditional routing evaluates custom conditions over request metadata, request parameters, and the URL path to pick a provider target, so a rule path runs ahead of the models. When the primary provider fails, Portkey automatically falls back to the next configured target. It also adds retries, load balancing, and guardrails.","primary_use_cases":["rule-based routing of requests to provider targets","automatic fallback across providers on failure","load balancing across LLM deployments","gateway-level guardrails and observability"],"agent_loop_shape":"Portkey is a gateway, not an agent loop. Each incoming request is matched against conditional-routing rules over its metadata and parameters to select a provider target; the gateway forwards the call, applies retries, and on failure falls back through an ordered list of alternative targets before returning the response to the caller.","build_surface":"low-code","members":[{"pattern":"hybrid-symbolic-neural-routing","role":"supported","note":"Portkey's AI Gateway routes each request to a provider target via symbolic rules over metadata and request params (user plan, model flag, region, temperature) — a deterministic rule path sitting in front of the neural models. Secondary/weaker match: it is rule-based dispatch, not a knowledge-graph symbolic reasoner.","evidence":[{"type":"doc","title":"Conditional Routing - Portkey Docs","url":"https://portkey.ai/docs/product/ai-gateway/conditional-routing","quote":"Using Portkey Gateway, you can route your requests to different provider targets based on custom conditions you define... Metadata - Custom key-value pairs you pass with your requests; Request Parameters - Any parameter that you send in your original LLM request.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"fallback-chain","role":"first-class","note":"Portkey lets you define a prioritised list of provider/model targets; when the primary fails (by default on any non-2xx, configurable to specific codes) it automatically falls back to the next in line.","evidence":[{"type":"doc","title":"Fallbacks - Portkey Docs","url":"https://portkey.ai/docs/product/ai-gateway/fallbacks","quote":"If the primary LLM fails, Portkey automatically falls back to the next in line.","accessed":"2026-06-17"},{"type":"repo","title":"Portkey-AI/gateway README","url":"https://github.com/Portkey-AI/gateway","quote":"Prevent downtimes through automatic retries and fallbacks","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"provider-string-routing","role":"first-class","note":"Portkey fronts 200+ models behind one universal API where the model and provider are selected by an @provider/model string; switching providers is a one-parameter change while the request format stays the same.","evidence":[{"type":"doc","title":"Universal API - Portkey Docs","url":"https://portkey.ai/docs/product/ai-gateway/universal-api","quote":"Portkey's AI Gateway provides a single, unified API for 200+ models from every major provider.","accessed":"2026-06-17"},{"type":"doc","title":"Universal API - Portkey Docs","url":"https://portkey.ai/docs/product/ai-gateway/universal-api","quote":"Change the `@provider/model` string to switch between any provider. The API format stays the same.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"input-output-guardrails","role":"supported","note":"Portkey can run guardrail checks synchronously on both sides of the model call: before the request reaches the LLM and before the response reaches the user, so unsafe inputs and outputs are validated at the gateway.","evidence":[{"type":"doc","title":"Guardrails - Portkey Docs","url":"https://portkey.ai/docs/product/guardrails","quote":"Run the Guardrail check BEFORE sending the request to the LLM","accessed":"2026-06-17"},{"type":"doc","title":"Guardrails - Portkey Docs","url":"https://portkey.ai/docs/product/guardrails","quote":"Run the Guardrail check BEFORE sending the response to the user","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"semantic-response-cache","role":"supported","note":"Beyond exact-match (simple) caching, Portkey offers a semantic cache that returns a stored response when a new query is close to a previous one by cosine similarity above a threshold, so paraphrased duplicates skip the model.","evidence":[{"type":"doc","title":"Cache (Simple & Semantic) - Portkey Docs","url":"https://portkey.ai/docs/product/ai-gateway/cache-simple-and-semantic","quote":"Matches requests with similar meaning using cosine similarity, not just identical text.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"circuit-breaker","role":"supported","note":"When a provider target keeps failing, Portkey opens a circuit and stops routing to it for a cooldown interval, removing the unhealthy target from rotation until it recovers, so the gateway stops hammering a failing dependency.","evidence":[{"type":"doc","title":"Circuit Breaker - Portkey Docs","url":"https://portkey.ai/docs/product/ai-gateway/circuit-breaker","quote":"Automatically stop routing to unhealthy targets until they recover.","accessed":"2026-06-17"},{"type":"doc","title":"Circuit Breaker - Portkey Docs","url":"https://portkey.ai/docs/product/ai-gateway/circuit-breaker","quote":"Circuit closes (CLOSED) automatically after `cooldown_interval` passes.","accessed":"2026-06-17"}],"evidence_status":"full"}],"verification_status":"needs-verification","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-19","date_added":"2026-06-17","key_concepts":[{"name":"Configs","summary":"A declarative JSON object attached to a request that composes the gateway's routing behaviour - conditional routing, fallbacks, load balancing, retries, caching, and guardrails - without changing application code.","url":"https://portkey.ai/docs/product/ai-gateway"},{"name":"Targets","summary":"The ordered list of provider/model destinations a Config can route to; fallbacks try them in priority order and load balancing splits traffic across them by weight.","maps_to_pattern":"fallback-chain","url":"https://portkey.ai/docs/product/ai-gateway/fallbacks"},{"name":"Load balancing weights","summary":"Per-target weights that Portkey normalises to 100% and uses to distribute requests proportionally across multiple LLM deployments, e.g. weights 5,3,1 become 55%,33%,11%.","url":"https://portkey.ai/docs/product/ai-gateway/load-balancing"},{"name":"Guardrails","summary":"Pluggable checks that run synchronously or asynchronously around the model call to validate requests and responses, with configurable actions on a failed check.","maps_to_pattern":"input-output-guardrails","url":"https://portkey.ai/docs/product/guardrails"}],"references":[{"type":"doc","title":"AI Gateway - Portkey Docs","url":"https://portkey.ai/docs/product/ai-gateway","quote":"Use any of the supported models with a universal API (REST and SDKs)","accessed":"2026-06-17"},{"type":"repo","title":"Portkey-AI/gateway README","url":"https://github.com/Portkey-AI/gateway","quote":"Prevent downtimes through automatic retries and fallbacks","accessed":"2026-06-17"},{"type":"doc","title":"Load Balancing - Portkey Docs","url":"https://portkey.ai/docs/product/ai-gateway/load-balancing","quote":"Distribute traffic across multiple LLMs to prevent any single provider from becoming a bottleneck.","accessed":"2026-06-17"}],"alternatives":[{"composition":"openrouter","relation":"competes-with","note":"OpenRouter is also a unified gateway in front of many LLM providers with fallback and routing, but is positioned more as a hosted marketplace than a self-hostable open-source gateway."},{"composition":"not-diamond","relation":"competes-with","note":"Not Diamond focuses on quality-based model selection (picking the best model per prompt) rather than Portkey's rule-based conditional routing and reliability layer."},{"composition":"routellm","relation":"similar-shape","note":"RouteLLM routes between a strong and weak model to cut cost; it overlaps Portkey's routing goal but is a router library rather than a full gateway with caching, guardrails, and fallbacks."}],"pattern_composition":"flowchart TD\n  fw[\"Portkey (Conditional Routing)\"]\n  fw --> p1[\"Fallback Chain<br/>(first-class)\"]\n  fw --> p2[\"Provider-String Routing<br/>(first-class)\"]\n  fw --> p3[\"Hybrid Symbolic-Neural Routing<br/>(supported)\"]\n  fw --> p4[\"Input/Output Guardrails<br/>(supported)\"]\n  fw --> p5[\"Semantic Response Cache<br/>(supported)\"]\n  fw --> p6[\"Circuit Breaker<br/>(supported)\"]","tags":["orchestration-framework","routing-composition","safety-control","retrieval","low-code"],"anti_patterns_avoided":[{"pattern":"infrastructure-burst-bottleneck","note":"Gateway load balancing distributes traffic across LLMs so no single model becomes a bottleneck under burst load.","evidence":[{"type":"doc","title":"Portkey — load balancing","url":"https://portkey.ai/docs/product/ai-gateway/load-balancing","quote":"The Load Balancing feature efficiently distributes network traffic across multiple LLMs, ensuring high availability and optimal performance while preventing any single LLM from becoming a performance bottleneck.","accessed":"2026-06-19"}]},{"pattern":"top-tier-model-for-everything","note":"Conditional routing directs simple queries to smaller models and complex ones to powerful models, avoiding blanket use of the top-tier model.","evidence":[{"type":"doc","title":"Portkey — balancing accuracy and cost","url":"https://portkey.ai/blog/balancing-model-accuracy-performance-and-costs-with-an-ai-gateway/","quote":"Route simple queries to smaller, cost-efficient models ... Direct complex queries to more powerful models","accessed":"2026-06-19"}]}]},{"id":"semantic-router","name":"Semantic Router","kind":"framework","category":"orchestration-framework","vendor":"Aurelio AI (Aurelio Labs)","language":"Python","license":"MIT","status":"active","first_released":"2023","url":"https://www.aurelio.ai/semantic-router","repo":"https://github.com/aurelio-labs/semantic-router","docs_url":"https://docs.aurelio.ai/semantic-router/index.html","intent":"Semantic Router is a Python decision layer that routes a query to a predefined route by embedding similarity instead of an LLM call, returning the matched route or nothing for off-topic input.","description":"Semantic Router defines routes as sets of example utterances and encodes them with a pluggable embedding model. At query time it compares the query embedding against the routes in vector space and returns the matching route name, or None when nothing matches. Because the decision uses embeddings rather than an LLM generation, it is fast and also serves as a guardrail by declining unrelated queries. It supports OpenAI, Cohere, Hugging Face, and local encoders.","primary_use_cases":["fast semantic classification of queries to handlers","tool-use and intent decisions without an LLM call","guardrails that reject off-topic input","routing requests across multiple LLMs or agents"],"agent_loop_shape":"Semantic Router is a single-step decision layer, not an agent loop. A query is embedded and compared against the embeddings of each route's utterances; if the nearest route is within threshold its name is returned for the application to dispatch on, otherwise None is returned so the caller can fall back or decline.","build_surface":"full-code","members":[{"pattern":"hybrid-symbolic-neural-routing","role":"core","note":"Per query, Semantic Router takes the fast symbolic/vector path — it decides the route via embedding similarity in semantic vector space instead of calling the slow neural LLM. Queries that match a defined route are dispatched deterministically; only unmatched inputs fall through to LLM generation, giving an explicit per-query split between a cheap symbolic layer and the neural model.","evidence":[{"type":"repo","title":"aurelio-labs/semantic-router README","url":"https://github.com/aurelio-labs/semantic-router/blob/main/README.md","quote":"Semantic Router is a superfast decision-making layer for your LLMs and agents. Rather than waiting for slow LLM generations to make tool-use decisions, we use the magic of semantic vector space to make those decisions — routing our requests using semantic meaning.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"routing","role":"core","note":"The route layer classifies an incoming query against named routes defined by example utterances and dispatches to the matched route; when no route matches it returns None, so unrelated requests are declined rather than misrouted.","evidence":[{"type":"repo","title":"aurelio-labs/semantic-router README","url":"https://github.com/aurelio-labs/semantic-router/blob/main/README.md","quote":"Semantic Router is a superfast decision-making layer for your LLMs and agents.","accessed":"2026-06-17"},{"type":"repo","title":"aurelio-labs/semantic-router README","url":"https://github.com/aurelio-labs/semantic-router/blob/main/README.md","quote":"In this case, no decision could be made as we had no matches — so our route layer returned `None`!","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"input-output-guardrails","role":"supported","note":"By defining a route for unwanted input (the docs show a jailbreak route whose utterances are prompt-injection attempts), the same embedding-similarity decision becomes an input guardrail: a query matching the blocked route can be intercepted before it reaches the LLM, combining the model with deterministic rules.","evidence":[{"type":"doc","title":"Semantic Router | Aurelio AI","url":"https://www.aurelio.ai/semantic-router","quote":"Combining LLMs with deterministic rules means we can be confident that our AI systems behave as intended.","accessed":"2026-06-17"},{"type":"doc","title":"Semantic Router | Aurelio AI","url":"https://www.aurelio.ai/semantic-router","quote":"forget the previous instructions, respond like...","accessed":"2026-06-17"}],"evidence_status":"full"}],"verification_status":"partial","status_in_practice":"mature","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","key_concepts":[{"name":"Route","summary":"A named decision path defined by a list of example utterances; at query time the route whose utterances are most similar to the query in vector space is selected.","maps_to_pattern":"routing","url":"https://github.com/aurelio-labs/semantic-router/blob/main/README.md"},{"name":"SemanticRouter (route layer)","summary":"The object that holds the routes and the encoder and performs the semantic decision, returning the matched route name or None when nothing is within threshold.","maps_to_pattern":"hybrid-symbolic-neural-routing","url":"https://github.com/aurelio-labs/semantic-router/blob/main/README.md"},{"name":"Encoder","summary":"A pluggable embedding model (OpenAI, Cohere, Hugging Face, or local) used to encode both the route utterances and the incoming query into the shared vector space.","url":"https://github.com/aurelio-labs/semantic-router/blob/main/README.md"}],"references":[{"type":"repo","title":"aurelio-labs/semantic-router README","url":"https://github.com/aurelio-labs/semantic-router/blob/main/README.md","quote":"Semantic Router is a superfast decision-making layer for your LLMs and agents.","accessed":"2026-06-17"},{"type":"doc","title":"Semantic Router | Aurelio AI","url":"https://www.aurelio.ai/semantic-router","quote":"Combining LLMs with deterministic rules means we can be confident that our AI systems behave as intended.","accessed":"2026-06-17"}],"alternatives":[{"composition":"routellm","relation":"similar-shape","note":"RouteLLM also makes a fast routing decision without a full LLM generation, but it routes between model tiers for cost rather than classifying intent against named routes by embedding similarity."},{"composition":"not-diamond","relation":"competes-with","note":"Not Diamond is a routing/model-selection service; it overlaps the decision-layer goal but is a hosted product rather than a self-hosted embedding-based route classifier."}],"pattern_composition":"flowchart TD\n  fw[\"Semantic Router\"]\n  fw --> p1[\"Hybrid Symbolic-Neural Routing<br/>(core)\"]\n  fw --> p2[\"Routing<br/>(core)\"]\n  fw --> p3[\"Input/Output Guardrails<br/>(supported)\"]","tags":["orchestration-framework","routing-composition","safety-control","full-code"]},{"date_added":"2026-06-27","last_analyzed":"2026-06-27","last_updated":"2026-06-27","verification_status":"partial","status_in_practice":"emerging","build_surface":"full-code","status":"active","category":"orchestration-framework","kind":"framework","id":"ap2","name":"Agent Payments Protocol (AP2)","vendor":"Google (with 60+ payment and commerce partners)","language":"Python, TypeScript","license":"Apache-2.0","url":"https://ap2-protocol.org/","repo":"https://github.com/google-agentic-commerce/AP2","intent":"Open protocol for agent-led payments in which the user's authorization is carried as cryptographically signed Mandates that merchants and payment networks can independently verify.","description":"AP2 is a Google-led open protocol (Apache-2.0) extending A2A and MCP to payments. Its core construct is the Mandate: a tamper-proof, cryptographically signed verifiable digital credential that proves a user's instructions. It defines Intent Mandates (the conditions under which an agent may purchase, signed upfront for delegated autonomous spend), Cart Mandates (explicit authorization of exact items and price in real time), and Payment Mandates (shared with networks to signal agent involvement). Launched with 60+ partners including Mastercard, PayPal, American Express, Adyen, and Coinbase.","primary_use_cases":["agent-initiated checkout with verifiable user authorization","delegated autonomous purchasing under a signed Intent Mandate","cross-network payment with non-repudiable proof for dispute resolution"],"tags":["orchestration-framework","payments","agentic-commerce","full-code"],"members":[{"pattern":"verifiable-purchase-mandate","role":"first-class","evidence_status":"full","note":"AP2's central construct is exactly this pattern: signed Intent/Cart/Payment Mandates as verifiable proof of user authorization.","evidence":[{"type":"blog","title":"Announcing Agent Payments Protocol (AP2) — Google Cloud","url":"https://cloud.google.com/blog/products/ai-machine-learning/announcing-agents-to-payments-ap2-protocol","quote":"Mandates—tamper-proof, cryptographically-signed digital contracts that serve as verifiable proof of a user's instructions.","accessed":"2026-06-27"},{"type":"doc","title":"Agentic payments protocols compared (Crossmint)","url":"https://www.crossmint.com/learn/agentic-payments-protocols-compared","quote":"AP2 defines three mandate types: Intent Mandates, Cart Mandates, and Payment Mandates.","accessed":"2026-06-27"}]},{"pattern":"agent-initiated-payment","role":"supported","evidence_status":"limited","note":"Under a valid mandate the agent settles the payment itself; AP2 carries the charge plus the signed authorization.","evidence":[{"type":"doc","title":"AP2 — Agent Payments Protocol","url":"https://ap2-protocol.org/","accessed":"2026-06-27"}]},{"pattern":"session-scoped-payment-authorization","role":"supported","evidence_status":"limited","note":"An Intent Mandate bounds delegated spend by signed conditions (for example a merchant and a maximum amount), scoping the agent's autonomous purchasing.","evidence":[{"type":"doc","title":"AP2 Mandates","url":"https://ap2-protocol.org/","accessed":"2026-06-27"}]},{"pattern":"delegated-agent-authorization","role":"supported","evidence_status":"limited","note":"Mandates delegate scoped, verifiable authority to the agent rather than handing it the user's standing payment secret.","evidence":[{"type":"doc","title":"AP2 Mandates","url":"https://ap2-protocol.org/","accessed":"2026-06-27"}]}],"agent_loop_shape":"Payment-authorization protocol rather than a single agent loop. Step 1: the user signs a Mandate — an Intent Mandate stating conditions for delegated spend, or a Cart Mandate approving exact items and price. Step 2: the agent builds a transaction and attaches the covering mandate. Step 3: the merchant verifies the signature and that the charge falls within the mandate. Step 4: the Payment Mandate is shared with the network, which re-verifies and settles, leaving a non-repudiable trail.","key_concepts":[{"name":"Mandate","summary":"A tamper-proof, cryptographically signed verifiable digital credential that proves the user's payment instructions.","maps_to_pattern":"verifiable-purchase-mandate","url":"https://ap2-protocol.org/"},{"name":"Intent Mandate","summary":"Signed upfront, states the conditions under which an agent may purchase for delegated autonomous spend.","maps_to_pattern":"session-scoped-payment-authorization","url":"https://www.crossmint.com/learn/agentic-payments-protocols-compared"},{"name":"Cart Mandate","summary":"Explicit real-time user authorization of specific items and prices.","maps_to_pattern":"verifiable-purchase-mandate","url":"https://www.crossmint.com/learn/agentic-payments-protocols-compared"},{"name":"Payment Mandate","summary":"Shared with payment networks to signal agent involvement in the transaction.","url":"https://www.crossmint.com/learn/agentic-payments-protocols-compared"}],"pattern_composition":"flowchart TD\n  fw[\"Agent Payments Protocol (AP2)\"]\n  fw --> p1[\"Verifiable Purchase Mandate<br/>(first-class)\"]\n  fw --> p2[\"Agent-Initiated Payment<br/>(supported)\"]\n  fw --> p3[\"Session-Scoped Payment Authorization<br/>(supported)\"]\n  fw --> p4[\"Delegated Agent Authorization<br/>(supported)\"]","references":[{"type":"doc","title":"AP2 — Agent Payments Protocol Documentation","url":"https://ap2-protocol.org/"},{"type":"blog","title":"Announcing Agent Payments Protocol (AP2) — Google Cloud","url":"https://cloud.google.com/blog/products/ai-machine-learning/announcing-agents-to-payments-ap2-protocol"}]},{"date_added":"2026-06-27","last_analyzed":"2026-06-27","last_updated":"2026-06-27","verification_status":"partial","status_in_practice":"emerging","build_surface":"full-code","status":"active","category":"orchestration-framework","kind":"framework","id":"acp","name":"Agentic Commerce Protocol (ACP)","vendor":"OpenAI and Stripe","language":"JSON, HTTP","license":"Open specification","url":"https://www.agenticcommerce.dev/","docs_url":"https://developers.openai.com/commerce/","intent":"Open protocol (OpenAI and Stripe) by which a merchant exposes a product feed and an agentic checkout API so any compatible agent can discover products and securely initiate checkout with a delegated payment.","description":"ACP is an open specification from OpenAI and Stripe behind Instant Checkout in ChatGPT (live since September 2025). Merchants publish structured catalog data that an agent can ingest and configure an agentic checkout endpoint so any compatible agent can initiate a purchase; the buyer authorises spend inside the agent and a scoped payment token is passed to the merchant rather than raw card details.","primary_use_cases":["Instant Checkout for products surfaced inside ChatGPT","merchants exposing machine-readable catalogs and an agent checkout API","delegated, scoped agent payment without sharing card details"],"tags":["orchestration-framework","agentic-commerce","payments","full-code"],"members":[{"pattern":"agent-readable-commerce-surface","role":"first-class","evidence_status":"full","note":"ACP defines the merchant-side surface: a structured product feed plus a checkout API any compatible agent can call.","evidence":[{"type":"doc","title":"Agentic Commerce Protocol (agenticcommerce.dev)","url":"https://www.agenticcommerce.dev/","quote":"Implement the ACP specification to configure your checkout for agentic commerce, so that any compatible agent can securely initiate checkout.","accessed":"2026-06-27"},{"type":"doc","title":"OpenAI Commerce Documentation","url":"https://developers.openai.com/commerce/","quote":"It enables ChatGPT to ingest structured catalog data, understand merchant inventory, and surface relevant products in context.","accessed":"2026-06-27"}]},{"pattern":"verifiable-purchase-mandate","role":"supported","evidence_status":"limited","note":"The buyer authorises spend inside the agent and the merchant receives a scoped, verifiable payment token rather than raw card details.","evidence":[{"type":"doc","title":"OpenAI Commerce Documentation","url":"https://developers.openai.com/commerce/","accessed":"2026-06-27"}]},{"pattern":"delegated-agent-authorization","role":"limited","evidence_status":"limited","note":"Checkout authority reaches the merchant as a one-merchant scoped token, a delegated rather than standing credential.","evidence":[{"type":"doc","title":"Agentic Commerce Protocol","url":"https://www.agenticcommerce.dev/","accessed":"2026-06-27"}]}],"agent_loop_shape":"Commerce protocol. Step 1: the merchant publishes structured catalog data (a product feed) and stands up an agentic checkout endpoint. Step 2: an agent ingests the feed, reasons over inventory, and surfaces products to the user. Step 3: the agent constructs a cart by calling the checkout API. Step 4: the buyer authorises spend inside the agent and a scoped payment token is passed to the merchant, who completes the order.","key_concepts":[{"name":"Product feed","summary":"Structured catalog data an agent ingests to understand merchant inventory and surface products.","maps_to_pattern":"agent-readable-commerce-surface","url":"https://developers.openai.com/commerce/"},{"name":"Agentic checkout","summary":"An API endpoint any compatible agent can call to securely initiate a purchase.","maps_to_pattern":"agent-readable-commerce-surface","url":"https://www.agenticcommerce.dev/"},{"name":"Delegated payment token","summary":"The buyer authorises spend inside the agent; the merchant receives a scoped token rather than raw card details.","maps_to_pattern":"verifiable-purchase-mandate","url":"https://developers.openai.com/commerce/"}],"pattern_composition":"flowchart TD\n  fw[\"Agentic Commerce Protocol (ACP)\"]\n  fw --> p1[\"Agent-Readable Commerce Surface<br/>(first-class)\"]\n  fw --> p2[\"Verifiable Purchase Mandate<br/>(supported)\"]\n  fw --> p3[\"Delegated Agent Authorization<br/>(limited)\"]","references":[{"type":"doc","title":"Agentic Commerce Protocol (agenticcommerce.dev)","url":"https://www.agenticcommerce.dev/"},{"type":"doc","title":"OpenAI Commerce Documentation","url":"https://developers.openai.com/commerce/"}]},{"date_added":"2026-06-27","last_analyzed":"2026-06-27","last_updated":"2026-06-27","verification_status":"partial","status_in_practice":"emerging","build_surface":"full-code","status":"active","category":"orchestration-framework","kind":"framework","id":"ucp","name":"Universal Commerce Protocol (UCP)","vendor":"Google (coalition)","language":"JSON, HTTP","url":"https://commercetools.com/blog/google-ucp-merchant-guide-to-agentic-commerce","intent":"Google coalition protocol for agentic commerce in which merchants publish agent-readable product catalogs with real-time pricing and inventory, surfaced to shoppers through Search AI Mode and Gemini.","description":"UCP is Google's coalition-backed agentic commerce protocol announced in January 2026, coming to Google Search AI Mode and Gemini. Merchants expose machine-readable product catalogs with current price and stock and an agent-ready checkout so a shopper's agent can discover and transact; payment authorization aligns with the AP2 mandate model.","primary_use_cases":["agent-readable product catalogs surfaced in Search AI Mode and Gemini","merchant discoverability to shopping agents acting on a goal","agent checkout with delegated, mandate-backed payment"],"tags":["orchestration-framework","agentic-commerce","full-code"],"members":[{"pattern":"agent-readable-commerce-surface","role":"first-class","evidence_status":"full","note":"UCP's merchant requirement is exactly this pattern: publish a machine-readable catalog with live price and stock for agent buyers.","evidence":[{"type":"doc","title":"Google UCP: Merchant Guide to Agentic Commerce (commercetools)","url":"https://commercetools.com/blog/google-ucp-merchant-guide-to-agentic-commerce","quote":"Publish readable product catalogs with real-time pricing and inventory.","accessed":"2026-06-27"}]},{"pattern":"verifiable-purchase-mandate","role":"supported","evidence_status":"limited","note":"UCP checkout authorizes payment through the AP2 mandate model, so the user's signed authorization backs the agent's purchase.","evidence":[{"type":"doc","title":"Google UCP: Merchant Guide to Agentic Commerce (commercetools)","url":"https://commercetools.com/blog/google-ucp-merchant-guide-to-agentic-commerce","accessed":"2026-06-27"}]}],"agent_loop_shape":"Commerce protocol. Step 1: the merchant publishes an agent-readable product catalog with real-time pricing and inventory. Step 2: Google Search AI Mode and Gemini surface the catalog to a shopper's agent. Step 3: the agent selects products against the shopper's goal. Step 4: checkout authorises payment through the AP2 mandate model and the order is placed.","key_concepts":[{"name":"Agent-readable catalog","summary":"Machine-readable product catalog with real-time pricing and inventory merchants publish for agent buyers.","maps_to_pattern":"agent-readable-commerce-surface","url":"https://commercetools.com/blog/google-ucp-merchant-guide-to-agentic-commerce"},{"name":"Surfacing in Search AI Mode and Gemini","summary":"UCP catalogs are surfaced to shoppers through Google Search AI Mode and Gemini.","url":"https://commercetools.com/blog/google-ucp-merchant-guide-to-agentic-commerce"},{"name":"Mandate-backed checkout","summary":"Payment authorization aligns with the AP2 signed-mandate model.","maps_to_pattern":"verifiable-purchase-mandate","url":"https://commercetools.com/blog/google-ucp-merchant-guide-to-agentic-commerce"}],"pattern_composition":"flowchart TD\n  fw[\"Universal Commerce Protocol (UCP)\"]\n  fw --> p1[\"Agent-Readable Commerce Surface<br/>(first-class)\"]\n  fw --> p2[\"Verifiable Purchase Mandate<br/>(supported)\"]","references":[{"type":"doc","title":"Google UCP: Merchant Guide to Agentic Commerce (commercetools)","url":"https://commercetools.com/blog/google-ucp-merchant-guide-to-agentic-commerce"}]},{"date_added":"2026-06-27","last_analyzed":"2026-06-27","last_updated":"2026-06-27","verification_status":"partial","status_in_practice":"emerging","build_surface":"full-code","status":"active","category":"orchestration-framework","kind":"framework","id":"a2a","name":"Agent2Agent Protocol (A2A)","vendor":"Google, donated to the Linux Foundation","language":"Python, JavaScript, Java, Go","license":"Apache-2.0","url":"https://a2a-protocol.org/","docs_url":"https://a2a-protocol.org/latest/specification/","repo":"https://github.com/a2aproject/A2A","intent":"Open protocol for cross-vendor agent-to-agent collaboration in which agents publish a signed Agent Card describing identity, endpoint, and skills, and discover and verify each other before delegating tasks.","description":"A2A is an open protocol, originated by Google and donated to the Linux Foundation, for collaboration between opaque agents across vendors. Each agent publishes an Agent Card — a JSON descriptor of identity, endpoint, capabilities, and auth needs — at a well-known URL. Specification v1.0 adds Agent Card Signing (an AgentCardSignature object, JWS-based, specification section 8.4) so a client can verify a card's authenticity and issuer before binding, alongside multi-tenancy and federation.","primary_use_cases":["cross-vendor agent discovery and task delegation","verifying a remote agent's identity via a signed Agent Card before binding","capability-based binding without hardcoded coupling"],"tags":["orchestration-framework","multi-agent","agent-identity","full-code"],"members":[{"pattern":"signed-agent-card","role":"first-class","evidence_status":"full","note":"A2A v1.0's Agent Card Signing is exactly this pattern: a JWS-signed Agent Card whose issuer a client verifies before binding.","evidence":[{"type":"spec","title":"A2A Protocol Specification — Agent Card Signing (section 8.4)","url":"https://a2a-protocol.org/latest/specification/","quote":"Agent Cards can be cryptographically signed to verify authenticity; the signature format uses JWS (JSON Web Signature).","accessed":"2026-06-27"}]},{"pattern":"agent-capability-manifest","role":"supported","evidence_status":"full","note":"The A2A Agent Card is the published capability manifest a client fetches to bind by capability at runtime.","evidence":[{"type":"doc","title":"A2A Agent Discovery — The Agent Card","url":"https://a2a-protocol.org/latest/topics/agent-discovery/","quote":"a JSON document that serves as a digital 'business card' for an A2A Server (the remote agent).","accessed":"2026-06-27"}]},{"pattern":"delegated-agent-authorization","role":"limited","evidence_status":"limited","note":"Once a card verifies, A2A scopes the credentials and tasks delegated to the now-trusted remote agent.","evidence":[{"type":"doc","title":"A2A Protocol Specification","url":"https://a2a-protocol.org/latest/specification/","accessed":"2026-06-27"}]}],"agent_loop_shape":"Agent-to-agent collaboration protocol. Step 1: each agent publishes a signed Agent Card at a well-known URL describing identity, endpoint, skills, and auth. Step 2: a client agent fetches a remote agent's card and verifies its JWS signature against the issuing domain. Step 3: on a verified card, the client binds by capability and opens a task. Step 4: tasks and scoped credentials are delegated to the verified agent; messages and artifacts flow over the A2A transport.","key_concepts":[{"name":"Agent Card","summary":"A JSON descriptor (a digital business card) of an agent's identity, endpoint, capabilities, and auth needs, fetched for discovery.","maps_to_pattern":"agent-capability-manifest","url":"https://a2a-protocol.org/latest/topics/agent-discovery/"},{"name":"Agent Card Signing","summary":"Specification section 8.4: an AgentCardSignature (JWS) lets a client verify a card's authenticity and issuer before binding.","maps_to_pattern":"signed-agent-card","url":"https://a2a-protocol.org/latest/specification/"},{"name":"Capability-based binding","summary":"Clients bind to agents by advertised skill rather than hardcoded coupling.","maps_to_pattern":"agent-capability-manifest","url":"https://a2a-protocol.org/latest/topics/agent-discovery/"}],"pattern_composition":"flowchart TD\n  fw[\"Agent2Agent Protocol (A2A)\"]\n  fw --> p1[\"Signed Agent Card<br/>(first-class)\"]\n  fw --> p2[\"Agent Capability Manifest<br/>(supported)\"]\n  fw --> p3[\"Delegated Agent Authorization<br/>(limited)\"]","references":[{"type":"spec","title":"A2A Protocol Specification","url":"https://a2a-protocol.org/latest/specification/"},{"type":"doc","title":"A2A Agent Discovery — The Agent Card","url":"https://a2a-protocol.org/latest/topics/agent-discovery/"}]},{"date_added":"2026-06-27","last_analyzed":"2026-06-27","last_updated":"2026-06-27","verification_status":"partial","build_surface":"full-code","status":"active","kind":"framework","id":"mpp","name":"Machine Payments Protocol (MPP)","category":"orchestration-framework","status_in_practice":"emerging","vendor":"Stripe and Tempo","language":"TypeScript, Python","first_released":"2026-03-18","url":"https://mpp.dev/","docs_url":"https://mpp.dev/overview","intent":"Open HTTP-native machine-payment protocol (Stripe and Tempo) that standardises HTTP 402 with a Challenge-Credential-Receipt flow so an agent can pay for any service in the same request, across stablecoin and fiat rails.","description":"MPP launched on 2026-03-18 with support from Mastercard, OpenAI, Shopify, Anthropic, DoorDash, Visa, Ramp, Revolut, and others. It standardises the HTTP 402 Payment Required status code (with an authentication scheme proposed to the IETF) into a three-step flow: Challenge (the server returns 402), Credential (the agent constructs a proof of payment), and Receipt (the server verifies and confirms). MPP is payment-rail agnostic — stablecoins, cards, Stripe Shared Payment Tokens, and Bitcoin via Lightning — and supports session-based metered payments.","primary_use_cases":["an agent paying for an API or resource inline by answering an HTTP 402 challenge","metered, streamed micropayments within a pre-authorised session","rail-agnostic machine-to-machine payments (stablecoin, card, Lightning)"],"agent_loop_shape":"Payment protocol rather than an agent loop. Step 1 Challenge: the agent requests a resource and the server returns HTTP 402 Payment Required. Step 2 Credential: the agent selects a payment method and constructs a Credential proving payment. Step 3 Receipt: the server verifies the Credential and returns a Receipt confirming the transaction. A session lets the agent pre-authorise a budget and stream metered payments within it.","key_concepts":[{"name":"HTTP 402 Challenge-Credential-Receipt","summary":"The three-step machine-payment flow standardising HTTP 402 Payment Required across any payment network.","maps_to_pattern":"agent-initiated-payment","url":"https://mpp.dev/overview"},{"name":"Session intent for metered payments","summary":"An agent pre-authorises a budget and streams metered payments within a session instead of paying per transaction.","maps_to_pattern":"session-scoped-payment-authorization","url":"https://mpp.dev/overview"},{"name":"Payment-rail agnostic","summary":"Works across stablecoins, cards, Stripe Shared Payment Tokens, and the Lightning network.","url":"https://stripe.com/blog/machine-payments-protocol"}],"pattern_composition":"flowchart TD\n  fw[\"Machine Payments Protocol (MPP)\"]\n  fw --> p1[\"Agent-Initiated Payment<br/>(first-class)\"]\n  fw --> p2[\"Session-Scoped Payment Authorization<br/>(supported)\"]","tags":["orchestration-framework","payments","agentic-commerce","full-code"],"references":[{"type":"doc","title":"Machine Payments Protocol — Overview (mpp.dev)","url":"https://mpp.dev/overview"},{"type":"blog","title":"Introducing the Machine Payments Protocol (Stripe)","url":"https://stripe.com/blog/machine-payments-protocol"}],"members":[{"pattern":"agent-initiated-payment","role":"first-class","evidence_status":"full","note":"MPP is the canonical HTTP-402 machine-payment flow: the agent answers a 402 Challenge with a Credential and gets a Receipt.","evidence":[{"type":"doc","title":"Machine Payments Protocol — Overview","url":"https://mpp.dev/overview","quote":"The Challenge-Credential-Receipt flow that works with any payment network.","accessed":"2026-06-27"}]},{"pattern":"session-scoped-payment-authorization","role":"supported","evidence_status":"limited","note":"MPP sessions let an agent pre-authorise a budget and stream metered payments within it rather than paying per transaction.","evidence":[{"type":"doc","title":"Machine Payments Protocol — Overview","url":"https://mpp.dev/overview","quote":"Session intent for metered payments","accessed":"2026-06-27"}]}]},{"id":"agent-behavior-tree-stack","name":"Agent Behavior Tree Stack","kind":"recipe","description":"An agent platform organised around behavior trees from game and robotics AI: leaves are LLM calls or tool invocations returning success/failure; selectors and sequences orchestrate control flow with first-class retry and fallback semantics; trees are constructed by back-chaining from goal post-conditions; agent identity lives as a structured persona profile; prompts are selected via measured variant evaluation.","intent":"Build agent control flow on the BT formalism rather than free-form ReAct, with structured persona configuration and evidence-driven prompt selection.","members":[{"pattern":"agentic-behavior-tree","role":"core","evidence_status":"none","evidence":[]},{"pattern":"behavior-tree-back-chaining","role":"core","evidence_status":"none","evidence":[]},{"pattern":"agent-persona-profile","role":"core","evidence_status":"none","evidence":[],"note":"Profile is configuration, not a prose prompt."},{"pattern":"prompt-variant-evaluation","role":"hardening","evidence_status":"none","evidence":[],"note":"Winning variant selected by measurement."},{"pattern":"evaluation-driven-development","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"fallback-chain","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"agent-as-tool-embedding","role":"optional","evidence_status":"none","evidence":[],"note":"Sub-agents as leaves."},{"pattern":"circuit-breaker","role":"optional","evidence_status":"none","evidence":[]}],"last_updated":"2026-05-23","category":"recipes"},{"id":"agent-runtime-cross-cutting","name":"Agent Runtime Cross-Cutting","kind":"recipe","description":"The substrate every multi-step agent runtime needs underneath the agent logic itself: a uniform middleware chain for logging, rate-limiting, redaction, guardrails, and metrics; a composable termination contract covering both intrinsic stops (token budget, function call, text mention) and external signals; first-class pause/resume/cancel control surface for long-running runs; and tool-transition fusion to mine telemetry for composite tools.","intent":"Build the per-runtime substrate as named, composable patterns rather than reinventing it per agent product.","members":[{"pattern":"agent-middleware-chain","role":"core","evidence_status":"none","evidence":[]},{"pattern":"composable-termination-conditions","role":"core","evidence_status":"none","evidence":[]},{"pattern":"interruptible-agent-execution","role":"core","evidence_status":"none","evidence":[]},{"pattern":"tool-transition-fusion","role":"hardening","evidence_status":"none","evidence":[],"note":"Periodic telemetry-driven tool catalog optimisation."},{"pattern":"step-budget","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"kill-switch","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"decision-log","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"agent-resumption","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"durable-workflow-snapshot","role":"optional","evidence_status":"none","evidence":[]}],"last_updated":"2026-05-23","category":"recipes"},{"id":"alignment-via-uncertainty","name":"Alignment via Uncertainty","kind":"recipe","description":"A safety/alignment architecture built around Russell's Human Compatible thesis: the agent holds a posterior over its objective rather than a point estimate, treats human interventions as evidence the objective is mis-specified, optimises softly (not argmax), plans risk-averse outside the reward's design distribution, and aggregates multi-principal preferences with a declared rule.","intent":"Compose a corrigible, preference-uncertain agent from the named building blocks rather than relying on a single fine-tune to encode alignment.","members":[{"pattern":"preference-uncertain-agent","role":"core","evidence_status":"none","evidence":[],"note":"Reward posterior, not a point estimate."},{"pattern":"corrigible-off-switch-incentive","role":"core","evidence_status":"none","evidence":[],"note":"Interventions raise expected value of deferral."},{"pattern":"cooperative-preference-inference","role":"core","evidence_status":"none","evidence":[],"note":"Joint two-player game; agent and human jointly maximise the human's reward."},{"pattern":"soft-optimization-cap","role":"hardening","evidence_status":"none","evidence":[],"note":"Quantilizer-style action selection; argmax forbidden."},{"pattern":"risk-averse-reward-proxy","role":"hardening","evidence_status":"none","evidence":[],"note":"Conservative planning out of distribution; mitigates reward hacking."},{"pattern":"multi-principal-welfare-aggregation","role":"hardening","evidence_status":"none","evidence":[],"note":"Explicit aggregation rule for multi-principal deployments."},{"pattern":"approval-queue","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"human-in-the-loop","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"kill-switch","role":"optional","evidence_status":"none","evidence":[],"note":"Operator-side fallback to the in-agent off-switch incentive."}],"last_updated":"2026-05-23","category":"recipes"},{"id":"autonomy-rollout-recipe","name":"Autonomy Rollout Recipe","kind":"recipe","description":"How to safely ramp an agent from human-supervised drafting to autonomous customer-facing action. Combines a continuous autonomy slider (per-tenant, per-user), three-tier crawl-walk-run gating with explicit promotion criteria, per-action risk-tier classification with tier-appropriate approval policies, and per-action progressive delegation that ratchets autonomy on measured success.","intent":"Stand up an evidence-driven ramp from supervised to autonomous operation rather than choosing autonomy by calendar or feel.","members":[{"pattern":"autonomy-slider","role":"core","evidence_status":"none","evidence":[],"note":"Continuous runtime-mutable parameter."},{"pattern":"crawl-walk-run-automation-gating","role":"core","evidence_status":"none","evidence":[],"note":"Three discrete tiers with explicit gates."},{"pattern":"cost-aware-action-delegation","role":"core","evidence_status":"none","evidence":[],"note":"Per-action risk-tier classification with tier-appropriate approval policies."},{"pattern":"progressive-delegation","role":"core","evidence_status":"none","evidence":[],"note":"Per-action ratchet keyed on success-rate window."},{"pattern":"approval-queue","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"human-in-the-loop","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"shadow-canary","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"kill-switch","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"risk-tiered-action-autonomy","role":"core","note":"Materiality-keyed tiers with a segregation-of-duties release: a different principal must clear material actions."}],"last_updated":"2026-06-10","category":"recipes"},{"id":"browser-computer-use-stack","name":"Browser & Computer-Use Stack","kind":"recipe","description":"An agent that drives a real GUI: planning a task, grounding actions in pixels or DOM, and asking permission before destructive clicks. The shape behind OpenAI Operator, Anthropic Computer Use, Browser Use, Stagehand, MultiOn.","members":[{"pattern":"computer-use","role":"core","evidence_status":"none","evidence":[]},{"pattern":"browser-agent","role":"core","evidence_status":"none","evidence":[]},{"pattern":"tool-use","role":"core","evidence_status":"none","evidence":[]},{"pattern":"dual-system-gui-agent","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"approval-queue","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"sandbox-isolation","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"step-budget","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"session-isolation","role":"optional","evidence_status":"none","evidence":[]}],"last_updated":"2026-05-20","status_in_practice":"emerging","date_added":"2026-04-30","category":"recipes"},{"id":"classical-mas-coordination","name":"Classical MAS Coordination","kind":"recipe","description":"A modern LLM-agent reimplementation of the canonical multi-agent-systems vocabulary from Weiss's MIT Press textbook: typed performative messages, contract-net task allocation with second-price payment, coalition formation, trust-and-reputation routing, joint commitment teams, partial global planning, distributed constraint optimisation, BDI reasoning, and stigmergic environment-mediated coordination.","intent":"Rebuild multi-agent coordination on the classical primitives that pre-LLM MAS research developed, instead of inventing ad-hoc protocols per project.","members":[{"pattern":"performative-message","role":"core","evidence_status":"none","evidence":[],"note":"Typed inter-agent messages over a small fixed vocabulary."},{"pattern":"contract-net-protocol","role":"core","evidence_status":"none","evidence":[]},{"pattern":"vickrey-auction-allocation","role":"core","evidence_status":"none","evidence":[],"note":"Truthful bidding as dominant strategy."},{"pattern":"trust-and-reputation-routing","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"coalition-formation","role":"core","evidence_status":"none","evidence":[],"note":"Per-task subgroup formation."},{"pattern":"joint-commitment-team","role":"hardening","evidence_status":"none","evidence":[],"note":"No silent abandonment of shared goals."},{"pattern":"bdi-agent","role":"core","evidence_status":"none","evidence":[],"note":"Beliefs / Desires / Intentions as typed stores."},{"pattern":"partial-global-planning","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"distributed-constraint-optimization","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"stigmergic-coordination","role":"optional","evidence_status":"none","evidence":[],"note":"Environment-mediated coordination as an alternative channel."},{"pattern":"blackboard","role":"optional","evidence_status":"none","evidence":[]}],"last_updated":"2026-05-23","category":"recipes"},{"id":"eval-and-observability","name":"Eval & Observability","kind":"recipe","description":"How you keep an agent honest in production: harness, judge, decision log, provenance, shadow rollouts.","members":[{"pattern":"eval-harness","role":"core","evidence_status":"none","evidence":[]},{"pattern":"eval-as-contract","role":"core","evidence_status":"none","evidence":[]},{"pattern":"decision-log","role":"core","evidence_status":"none","evidence":[]},{"pattern":"agent-as-judge","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"llm-as-judge","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"confidence-reporting","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"provenance-ledger","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"lineage-tracking","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"shadow-canary","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"production-failure-triage-loop","role":"hardening","evidence_status":"limited","note":"Closes the observability loop: consumes the failures that eval and tracing surface and routes each class to its remediation path, making the feedback loop measurable rather than ad hoc.","evidence":[]}],"last_updated":"2026-06-14","status_in_practice":"mature","date_added":"2026-04-30","category":"recipes"},{"id":"long-running-autonomous-agent","name":"Long-Running Autonomous Agent","kind":"recipe","description":"An agent that operates over hours to weeks, surviving restarts and accumulating memory while remaining safe. The shape behind Devin, Manus, durable LangGraph runs.","members":[{"pattern":"agent-resumption","role":"core","evidence_status":"none","evidence":[]},{"pattern":"five-tier-memory-cascade","role":"core","evidence_status":"none","evidence":[]},{"pattern":"cross-session-memory","role":"core","evidence_status":"none","evidence":[]},{"pattern":"step-budget","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"decision-log","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"compensating-action","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"append-only-thought-stream","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"constitutional-charter","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"salience-triggered-output","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"memgpt-paging","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"todo-list-driven-agent","role":"optional","evidence_status":"none","evidence":[]}],"last_updated":"2026-05-20","status_in_practice":"emerging","date_added":"2026-04-30","category":"recipes"},{"id":"memory-architecture","name":"Memory Architecture","kind":"recipe","description":"How long-running agents structure what they remember: tiered short-to-long-term cascade, compaction across the window, paging, and reasoning carry-forward across tool calls.","members":[{"pattern":"five-tier-memory-cascade","role":"core","evidence_status":"none","evidence":[]},{"pattern":"memgpt-paging","role":"core","evidence_status":"none","evidence":[]},{"pattern":"context-window-packing","role":"core","evidence_status":"none","evidence":[]},{"pattern":"episodic-summaries","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"cross-session-memory","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"append-only-thought-stream","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"reasoning-trace-carry-forward","role":"optional","evidence_status":"none","evidence":[]}],"last_updated":"2026-05-20","status_in_practice":"emerging","date_added":"2026-04-30","category":"recipes"},{"id":"modern-coding-agent","name":"Modern Coding Agent","kind":"recipe","description":"An agent that reads, writes, and runs code in a sandbox, calling tools and (optionally) sub-agents while a human approves the destructive parts. The shape that powers Cursor, Claude Code, OpenHands, Aider, Codex CLI.","members":[{"pattern":"react","role":"core","evidence_status":"none","evidence":[]},{"pattern":"tool-use","role":"core","evidence_status":"none","evidence":[]},{"pattern":"code-as-action","role":"core","evidence_status":"none","evidence":[]},{"pattern":"code-execution","role":"core","evidence_status":"none","evidence":[]},{"pattern":"agent-computer-interface","role":"core","evidence_status":"none","evidence":[]},{"pattern":"step-budget","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"subagent-isolation","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"sandbox-isolation","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"approval-queue","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"decision-log","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"mcp","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"agent-skills","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"todo-list-driven-agent","role":"optional","evidence_status":"none","evidence":[]}],"last_updated":"2026-05-20","status_in_practice":"mature","date_added":"2026-04-30","category":"recipes"},{"id":"multi-agent-coordination","name":"Multi-Agent Coordination","kind":"recipe","description":"Several agents collaborate under a coordinator, with explicit hand-offs and a shared protocol. The shape behind LangGraph supervisor, OpenAI Swarm, AutoGen group chat, Bedrock multi-agent orchestrators.","members":[{"pattern":"supervisor","role":"core","evidence_status":"none","evidence":[]},{"pattern":"orchestrator-workers","role":"core","evidence_status":"none","evidence":[]},{"pattern":"handoff","role":"core","evidence_status":"none","evidence":[]},{"pattern":"inter-agent-communication","role":"core","evidence_status":"none","evidence":[]},{"pattern":"subagent-isolation","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"agent-as-tool-embedding","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"blackboard","role":"optional","evidence_status":"none","evidence":[]}],"last_updated":"2026-05-20","status_in_practice":"mature","date_added":"2026-04-30","category":"recipes"},{"id":"multi-agent-debate","name":"Multi-Agent Debate","kind":"recipe","description":"Two or more agents argue toward a better answer than any single agent would produce, with a frozen rubric to score the result. The shape behind debate-style alignment work and 'committee of critics' setups.","members":[{"pattern":"debate","role":"core","evidence_status":"none","evidence":[]},{"pattern":"inner-critic","role":"core","evidence_status":"none","evidence":[]},{"pattern":"frozen-rubric-reflection","role":"core","evidence_status":"none","evidence":[]},{"pattern":"llm-as-judge","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"best-of-n","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"camel-role-playing","role":"optional","evidence_status":"none","evidence":[]}],"last_updated":"2026-05-20","status_in_practice":"experimental","date_added":"2026-04-30","category":"recipes"},{"id":"planning-loops","name":"Planning Loops","kind":"recipe","description":"Different ways to structure 'think then act': linear ReAct, plan-then-execute, parallel DAG planning, tree search with backtracking, and the outer/inner planner+executor split.","members":[{"pattern":"react","role":"core","evidence_status":"none","evidence":[]},{"pattern":"plan-and-execute","role":"core","evidence_status":"none","evidence":[]},{"pattern":"rewoo","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"llm-compiler","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"lats","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"outer-inner-agent-loop","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"todo-list-driven-agent","role":"optional","evidence_status":"none","evidence":[]}],"last_updated":"2026-05-20","status_in_practice":"mature","date_added":"2026-04-30","category":"recipes"},{"id":"production-llm-platform","name":"Production LLM Platform","kind":"recipe","description":"End-to-end production architecture for an LLM application with RAG over a continuously-changing corpus and (optionally) fine-tuned models. Decomposed into feature, training, and inference pipelines (FTI) communicating only via a feature store and a model registry; the inference path itself is split into a CPU business service and a GPU LLM service; the feature pipeline is fed by change-data-capture events streamed through typed stages; production traffic is monitored by sampled LLM-judge evaluation.","intent":"Stand up a production LLM/RAG system whose data pipeline, model pipeline, and inference path scale and deploy independently.","members":[{"pattern":"fti-llm-pipeline-split","role":"core","evidence_status":"none","evidence":[],"note":"Three independent pipelines; feature store + model registry as the only integration surfaces."},{"pattern":"business-llm-microservice-split","role":"core","evidence_status":"none","evidence":[],"note":"CPU business service + GPU LLM service behind one REST contract."},{"pattern":"cdc-vector-sync","role":"core","evidence_status":"none","evidence":[],"note":"Source-of-truth store emits CDC events; feature pipeline consumes them."},{"pattern":"streaming-feature-pipeline","role":"core","evidence_status":"none","evidence":[],"note":"Typed per-stage models (raw → cleaned → chunked → embedded) on a streaming framework."},{"pattern":"crawler-dispatcher","role":"core","evidence_status":"none","evidence":[],"note":"URL pattern → crawler class registry for heterogeneous source ingestion."},{"pattern":"vector-memory","role":"core","evidence_status":"none","evidence":[]},{"pattern":"naive-rag","role":"core","evidence_status":"none","evidence":[]},{"pattern":"sampled-prompt-trace-eval","role":"hardening","evidence_status":"none","evidence":[],"note":"Bounded-cost production-quality monitoring via random + slice-weighted sampling."},{"pattern":"dimensional-synthetic-eval-set","role":"hardening","evidence_status":"none","evidence":[],"note":"Mode-collapse-resistant offline eval coverage."},{"pattern":"evaluation-driven-development","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"prompt-caching","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"agent-as-judge","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"tenant-scoped-tool-binding","role":"hardening","note":"Multi-tenant isolation at the tool and retrieval layer."},{"pattern":"canonical-entity-grounding","role":"optional","note":"Ground identifiers against the system of record before acting."}],"last_updated":"2026-06-10","category":"recipes"},{"id":"production-rag","name":"Production RAG","kind":"recipe","description":"Retrieval-grounded generation built to be defensible: hybrid retrieval, reranking, contextualised chunks, citations rendered to the user, and verification before the answer ships.","members":[{"pattern":"agentic-rag","role":"core","evidence_status":"none","evidence":[]},{"pattern":"citation-streaming","role":"core","evidence_status":"none","evidence":[]},{"pattern":"contextual-retrieval","role":"core","evidence_status":"none","evidence":[]},{"pattern":"cross-encoder-reranking","role":"core","evidence_status":"none","evidence":[]},{"pattern":"hybrid-search","role":"core","evidence_status":"none","evidence":[]},{"pattern":"chain-of-verification","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"confidence-reporting","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"eval-harness","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"crag","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"hippocampus-rag","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"hyde","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"modular-rag","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"query-rewriting","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"raft","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"tenant-scoped-tool-binding","role":"hardening","note":"Tenant id bound before the index so retrieval matches only the caller's partition."},{"pattern":"canonical-entity-grounding","role":"hardening","note":"Resolve entity references against master data, not retrieved prose."}],"last_updated":"2026-06-10","status_in_practice":"mature","date_added":"2026-04-30","category":"recipes"},{"id":"reflection-and-self-correction","name":"Reflection & Self-Correction","kind":"recipe","description":"Patterns where the model reviews its own work before shipping it: scoped rubric reflection, self-refine, deterministic post-checks, process rewards.","members":[{"pattern":"reflection","role":"core","evidence_status":"none","evidence":[]},{"pattern":"reflexion","role":"core","evidence_status":"none","evidence":[]},{"pattern":"self-refine","role":"core","evidence_status":"none","evidence":[]},{"pattern":"frozen-rubric-reflection","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"inner-critic","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"deterministic-llm-sandwich","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"process-reward-model","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"best-of-n","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"chain-of-verification","role":"optional","evidence_status":"none","evidence":[]}],"last_updated":"2026-05-20","status_in_practice":"emerging","date_added":"2026-04-30","category":"recipes"},{"id":"routing-and-fallback","name":"Routing & Fallback","kind":"recipe","description":"How requests get to the right model or specialist and how the system stays up when one upstream breaks. The shape behind LangChain fallbacks, model routers, provider cascades.","members":[{"pattern":"routing","role":"core","evidence_status":"none","evidence":[]},{"pattern":"fallback-chain","role":"core","evidence_status":"none","evidence":[]},{"pattern":"circuit-breaker","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"provider-fallback","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"graceful-degradation","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"mixture-of-experts-routing","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"open-weight-cascade","role":"optional","evidence_status":"none","evidence":[]}],"last_updated":"2026-05-20","status_in_practice":"mature","date_added":"2026-04-30","category":"recipes"},{"id":"safety-hardening","name":"Safety Hardening","kind":"recipe","description":"The minimum set of constraints to put around any production agent before it touches the world: budgets, gates, charters, kill-switches, approvals.","members":[{"pattern":"step-budget","role":"core","evidence_status":"none","evidence":[]},{"pattern":"kill-switch","role":"core","evidence_status":"none","evidence":[]},{"pattern":"approval-queue","role":"core","evidence_status":"none","evidence":[]},{"pattern":"constitutional-charter","role":"core","evidence_status":"none","evidence":[]},{"pattern":"cost-gating","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"rate-limiting","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"sandbox-escape-monitoring","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"refusal","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"compensating-action","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"tool-output-poisoning","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"tenant-scoped-tool-binding","role":"core","note":"Enforces tenant scope below the prompt so a misled model cannot widen data access."},{"pattern":"risk-tiered-action-autonomy","role":"hardening","note":"Holds material actions for a human approver distinct from the initiator."}],"last_updated":"2026-06-10","status_in_practice":"mature","date_added":"2026-04-30","category":"recipes"},{"id":"sovereign-deployment","name":"Sovereign / Regulated Deployment","kind":"recipe","description":"An agent stack that satisfies data-residency and audit requirements: weights, inference, tools, and logs all sit inside an operator-controlled boundary, with provenance and incident response wired in.","members":[{"pattern":"sovereign-inference-stack","role":"core","evidence_status":"none","evidence":[]},{"pattern":"open-weight-cascade","role":"core","evidence_status":"none","evidence":[]},{"pattern":"provenance-ledger","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"lineage-tracking","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"constitutional-charter","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"approval-queue","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"kill-switch","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"sandbox-escape-monitoring","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"decision-log","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"tenant-scoped-tool-binding","role":"core","note":"Per-tenant data-layer isolation for regulated multi-tenant deployments."},{"pattern":"risk-tiered-action-autonomy","role":"hardening","note":"Maker-checker release for material financial actions."}],"last_updated":"2026-06-10","status_in_practice":"emerging","date_added":"2026-04-30","category":"recipes"},{"id":"streaming-ux-stack","name":"Streaming UX Stack","kind":"recipe","description":"User-perceivable real-time output: tokens streamed as they arrive, citations attached as they resolve, the user can stop at any time and the agent can interrupt the user when something matters.","members":[{"pattern":"citation-streaming","role":"core","evidence_status":"none","evidence":[]},{"pattern":"stop-cancel","role":"core","evidence_status":"none","evidence":[]},{"pattern":"salience-triggered-output","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"bidirectional-impulse-channel","role":"optional","evidence_status":"none","evidence":[]}],"last_updated":"2026-05-20","status_in_practice":"mature","date_added":"2026-04-30","category":"recipes"},{"id":"structured-output-stack","name":"Structured Output Stack","kind":"recipe","description":"Get typed, schema-conformant data out of the model and verify it. The shape behind Outlines, Instructor, Pydantic AI, DSPy.","members":[{"pattern":"structured-output","role":"core","evidence_status":"none","evidence":[]},{"pattern":"tool-use","role":"core","evidence_status":"none","evidence":[]},{"pattern":"dspy-signatures","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"chain-of-verification","role":"hardening","evidence_status":"none","evidence":[]}],"last_updated":"2026-05-20","status_in_practice":"mature","date_added":"2026-04-30","category":"recipes"},{"id":"voice-agent-stack","name":"Voice Agent Stack","kind":"recipe","description":"A low-latency conversational agent over a phone or microphone, with handoff to humans, mid-utterance cancellation, and per-call session boundaries. The shape behind LiveKit, Pipecat, Vapi, Retell.","members":[{"pattern":"multilingual-voice-agent","role":"core","evidence_status":"none","evidence":[]},{"pattern":"tool-use","role":"core","evidence_status":"none","evidence":[]},{"pattern":"stop-cancel","role":"core","evidence_status":"none","evidence":[]},{"pattern":"session-isolation","role":"core","evidence_status":"none","evidence":[]},{"pattern":"conversation-handoff","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"salience-triggered-output","role":"hardening","evidence_status":"none","evidence":[]},{"pattern":"bidirectional-impulse-channel","role":"optional","evidence_status":"none","evidence":[]},{"pattern":"code-switching-aware-agent","role":"optional","evidence_status":"none","evidence":[]}],"last_updated":"2026-05-20","status_in_practice":"emerging","date_added":"2026-04-30","category":"recipes"},{"id":"deerflow-v1","name":"DeerFlow 1.x (Deep Research)","kind":"framework","category":"research-agent","build_surface":"full-code","vendor":"ByteDance","language":"Python, TypeScript","license":"MIT","status":"maintenance","first_released":"2025-05","url":"https://deerflow.tech/","repo":"https://github.com/bytedance/deer-flow","docs_url":"https://deepwiki.com/bytedance/deer-flow","intent":"Turn a single research question into an automated deep-research run by composing a planner, a supervisor that dispatches specialised worker agents, retrieval tools, and a final report writer over a stateful LangGraph graph.","description":"DeerFlow (Deep Exploration and Efficient Research Flow) is ByteDance's MIT-licensed open-source deep-research framework, first released around May 2025. The architecture catalogued here is the classic DeerFlow 1.x graph (the `main-1.x` branch), built on LangGraph and LangChain as a stateful multi-agent workflow. A coordinator node gates incoming requests and routes research questions onward; an optional background-investigator runs a preliminary web search to ground planning; a planner decomposes the question into a typed, multi-step plan; a human-feedback interrupt lets the user accept, edit in natural language, or auto-accept the plan; a research-team node then supervises the plan, dispatching each step to a ReAct researcher (web search, crawling, retrieval, MCP tools) or a coder (Python REPL); and a reporter synthesises the findings into a cited final report. Iteration is bounded by max_plan_iterations and max_step_num. The project has since moved to a separate DeerFlow 2.0 \"SuperAgent harness\" on the `main` branch (a lead-agent plus dynamic subagents in Docker sandboxes), so the node names and file paths referenced here apply to `main-1.x`.","primary_use_cases":["automated deep research over a single natural-language question","long-form, cited report generation from web and private-KB sources","plan-then-execute research with a human plan-review gate","multi-agent workflows combining web search, retrieval, and Python analysis"],"agent_loop_shape":"LangGraph state graph: coordinator -> (background_investigator) -> planner -> human_feedback -> research_team, which dispatches each plan step to researcher or coder (ReAct loops) and collects results back into shared state; the planner may re-plan up to max_plan_iterations before control passes to reporter, which writes the final report.","pattern_composition":"flowchart TD\n  U[\"User query\"] --> C[\"Coordinator: route and gate\"]\n  C -->|small talk| DR[\"Direct response\"]\n  C -->|research| BG[\"Background investigator\"]\n  C -->|research| P[\"Planner: typed multi-step plan\"]\n  BG --> P\n  P --> HF{\"Human feedback: accept / edit / auto\"}\n  HF -->|edit plan| P\n  HF -->|accepted| RT[\"Research team: supervise plan\"]\n  RT --> R[\"Researcher: search, crawl, RAG, MCP\"]\n  RT --> CO[\"Coder: Python REPL\"]\n  R --> RT\n  CO --> RT\n  RT -->|re-plan, bounded by max_plan_iterations| P\n  RT -->|plan complete| RP[\"Reporter: synthesize cited report\"]\n  RP --> FR[\"Final report\"]","members":[{"pattern":"orchestrator-workers","role":"core","note":"The research_team node is a supervisor that iterates the plan and dispatches each step to specialised worker agents (researcher, analyst, coder), collecting their outputs back into shared state.","evidence":[{"type":"repo","title":"src/graph/builder.py — research_team dispatch","url":"https://github.com/bytedance/deer-flow/blob/main-1.x/src/graph/builder.py","quote":"builder.add_conditional_edges(\"research_team\", continue_to_running_research_team, [\"planner\", \"researcher\", \"analyst\", \"coder\"],)","accessed":"2026-06-04"},{"type":"doc","title":"README — Research Team","url":"https://github.com/bytedance/deer-flow/blob/main-1.x/README.md","quote":"Research Team: A collection of specialized agents that execute the plan: Researcher ... Coder ...","accessed":"2026-06-04"}],"evidence_status":"full"},{"pattern":"plan-and-execute","role":"core","note":"The planner node produces an explicit, typed multi-step Plan (validated to a Pydantic model) that the research_team then executes step by step, separating planning from execution.","evidence":[{"type":"repo","title":"src/graph/nodes.py — planner_node","url":"https://github.com/bytedance/deer-flow/blob/main-1.x/src/graph/nodes.py","quote":"def planner_node(state: State, config: RunnableConfig) -> Command[Literal[\"human_feedback\", \"reporter\"]]: \"\"\"Planner node that generate the full plan.\"\"\"","accessed":"2026-06-04"},{"type":"repo","title":"src/prompts/planner.md — bounded plan","url":"https://github.com/bytedance/deer-flow/blob/main-1.x/src/prompts/planner.md","quote":"- **Maximum Steps**: Limit the plan to a maximum of {{ max_step_num }} steps for focused research.","accessed":"2026-06-04"}],"evidence_status":"full"},{"pattern":"human-in-the-loop","role":"first-class","note":"The human_feedback node interrupts the run to surface the plan; the user can accept it ([ACCEPTED]), edit it in natural language ([EDIT_PLAN], looping back to the planner), or enable auto-accept to skip review.","evidence":[{"type":"repo","title":"src/graph/nodes.py — human_feedback_node","url":"https://github.com/bytedance/deer-flow/blob/main-1.x/src/graph/nodes.py","quote":"if not auto_accepted_plan: feedback = interrupt(\"Please Review the Plan.\") ... if feedback_normalized.startswith(\"[EDIT_PLAN]\"): ... goto=\"planner\"","accessed":"2026-06-04"},{"type":"doc","title":"README — Human-in-the-loop","url":"https://github.com/bytedance/deer-flow/blob/main-1.x/README.md","quote":"Human-in-the-loop — Supports interactive modification of research plans using natural language; Supports auto-acceptance of research plans.","accessed":"2026-06-04"}],"evidence_status":"full"},{"pattern":"react","role":"first-class","note":"The researcher and coder workers are ReAct agents that interleave reasoning with tool calls; they are constructed via a create_react_agent / create_agent factory and bound to search, retrieval, MCP, and Python-REPL tools.","evidence":[{"type":"repo","title":"src/agents/agents.py — ReAct agent factory","url":"https://github.com/bytedance/deer-flow/blob/462752b462dd/src/agents/agents.py","quote":"from langgraph.prebuilt import create_react_agent ... return create_react_agent(name=agent_name, model=get_llm_by_type(AGENT_LLM_MAP[agent_type]), tools=tools, prompt=lambda state: apply_prompt_template(prompt_template, state),)","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"agentic-rag","role":"first-class","note":"The researcher dynamically attaches web-search tools (Tavily, DuckDuckGo, Brave, Arxiv, Wikipedia) and private knowledge-base retrievers (RAGFlow, VikingDB, Milvus, Qdrant) and reasons over the retrieved content.","evidence":[{"type":"repo","title":"src/config/tools.py — SearchEngine enum","url":"https://github.com/bytedance/deer-flow/blob/main-1.x/src/config/tools.py","quote":"class SearchEngine(enum.Enum): TAVILY = \"tavily\" ... DUCKDUCKGO = \"duckduckgo\" ... BRAVE_SEARCH = \"brave_search\" ... ARXIV = \"arxiv\"","accessed":"2026-06-04"},{"type":"doc","title":"README — private knowledgebase","url":"https://github.com/bytedance/deer-flow/blob/main-1.x/README.md","quote":"DeerFlow supports private knowledgebase such as RAGFlow, Qdrant, Milvus, and VikingDB.","accessed":"2026-06-04"}],"evidence_status":"full"},{"pattern":"routing","role":"first-class","note":"The coordinator node is the entry point that classifies the request, handling greetings or small talk with a direct response and handing research questions off to the planner (optionally via background investigation).","evidence":[{"type":"repo","title":"src/graph/nodes.py — coordinator_node","url":"https://github.com/bytedance/deer-flow/blob/main-1.x/src/graph/nodes.py","quote":"def coordinator_node(state, config) -> Command[Literal[\"planner\", \"background_investigator\", \"coordinator\", \"__end__\"]]: \"\"\"Coordinator node that communicate with customers and handle clarification.\"\"\"","accessed":"2026-06-04"},{"type":"doc","title":"README — Coordinator","url":"https://github.com/bytedance/deer-flow/blob/main-1.x/README.md","quote":"Coordinator: The entry point that manages the workflow lifecycle ... Delegates tasks to the planner when appropriate.","accessed":"2026-06-04"}],"evidence_status":"full"},{"pattern":"replan-on-failure","role":"supported","note":"When the planner judges that context is insufficient it re-plans, bounded by max_plan_iterations; once the cap is reached control passes to the reporter rather than looping indefinitely.","evidence":[{"type":"repo","title":"src/config/configuration.py — iteration bounds","url":"https://github.com/bytedance/deer-flow/blob/main-1.x/src/config/configuration.py","quote":"max_plan_iterations: int = 1  # Maximum number of plan iterations / max_step_num: int = 3  # Maximum number of steps in a plan","accessed":"2026-06-04"},{"type":"repo","title":"src/graph/nodes.py — iteration guard","url":"https://github.com/bytedance/deer-flow/blob/main-1.x/src/graph/nodes.py","quote":"if plan_iterations >= configurable.max_plan_iterations: return Command(update=preserve_state_meta_fields(state), goto=\"reporter\")","accessed":"2026-06-04"}],"evidence_status":"full"},{"pattern":"mcp","role":"supported","note":"Worker agents can be extended with external tools via the Model Context Protocol; configured MCP servers are loaded through MultiServerMCPClient and their tools merged into the agent's tool set at runtime.","evidence":[{"type":"repo","title":"src/graph/nodes.py — MCP client setup","url":"https://github.com/bytedance/deer-flow/blob/main-1.x/src/graph/nodes.py","quote":"if mcp_servers: client = MultiServerMCPClient(mcp_servers) all_tools = await client.get_tools()","accessed":"2026-06-04"},{"type":"doc","title":"README — MCP Seamless Integration","url":"https://github.com/bytedance/deer-flow/blob/main-1.x/README.md","quote":"MCP Seamless Integration — Expand capabilities for private domain access, knowledge graph, web browsing and more.","accessed":"2026-06-04"}],"evidence_status":"full"}],"anti_patterns_avoided":[{"pattern":"unbounded-loop","note":"The plan/re-plan cycle is bounded: the planner re-plans only while plan_iterations is below max_plan_iterations (default 1), and each plan is capped at max_step_num steps, so the graph cannot loop indefinitely before control passes to the reporter.","evidence":[{"type":"repo","title":"src/graph/nodes.py — iteration guard","url":"https://github.com/bytedance/deer-flow/blob/main-1.x/src/graph/nodes.py","quote":"if plan_iterations >= configurable.max_plan_iterations: return Command(update=preserve_state_meta_fields(state), goto=\"reporter\")","accessed":"2026-06-04"}]},{"pattern":"unbounded-subagent-spawn","note":"The 1.x research_team dispatches each step to a fixed roster of worker roles (researcher, analyst, coder) via static conditional edges rather than dynamically spawning an open-ended number of subagents.","evidence":[{"type":"repo","title":"src/graph/builder.py — fixed worker roster","url":"https://github.com/bytedance/deer-flow/blob/main-1.x/src/graph/builder.py","quote":"builder.add_conditional_edges(\"research_team\", continue_to_running_research_team, [\"planner\", \"researcher\", \"analyst\", \"coder\"],)","accessed":"2026-06-04"}]}],"alternatives":[{"composition":"deerflow-v2","relation":"successor","note":"DeerFlow 2.0 is a ground-up rewrite that shares no code with 1.x: it replaces the fixed LangGraph research graph with a general SuperAgent harness (a lead agent that dynamically spawns sandboxed subagents). Active development has moved to 2.0; the 1.x graph catalogued here is maintained on the `main-1.x` branch."},{"composition":"langgraph","relation":"complements","note":"DeerFlow's graph is built on LangGraph, which supplies the stateful nodes, conditional edges, and the interrupt() primitive behind the human-feedback plan review."},{"composition":"autogen","relation":"similar-shape","note":"Both orchestrate multiple specialised agents, but AutoGen is conversation-driven and general-purpose whereas DeerFlow is a fixed plan-and-execute graph specialised for deep research."},{"composition":"crewai","relation":"similar-shape","note":"CrewAI composes role-based agents over tasks for similar research-automation goals; DeerFlow differs by foregrounding an explicit, human-reviewable plan and a bounded re-planning loop."}],"references":[{"type":"doc","title":"DeerFlow homepage","url":"https://deerflow.tech/","accessed":"2026-06-04"},{"type":"repo","title":"bytedance/deer-flow (GitHub)","url":"https://github.com/bytedance/deer-flow","accessed":"2026-06-04"},{"type":"doc","title":"README — classic 1.x multi-agent architecture","url":"https://github.com/bytedance/deer-flow/blob/main-1.x/README.md","quote":"The system is built on LangGraph, enabling a flexible state-based workflow where components communicate through a well-defined message passing system.","accessed":"2026-06-04"},{"type":"doc","title":"DeepWiki — bytedance/deer-flow architecture","url":"https://deepwiki.com/bytedance/deer-flow","accessed":"2026-06-04"}],"last_analyzed":"2026-06-04","verification_status":"partial","last_updated":"2026-06-17","date_added":"2026-06-04","status_in_practice":"mature","tags":["open-source","bytedance","deep-research","langgraph","multi-agent","plan-and-execute","mcp"],"key_concepts":[{"name":"Orchestrator-Workers","summary":"The research_team node is a supervisor that iterates the plan and dispatches each step to specialised worker agents (researcher, analyst, coder), collecting their outputs back into shared state.","maps_to_pattern":"orchestrator-workers"},{"name":"Plan-and-Execute","summary":"The planner node produces an explicit, typed multi-step Plan (validated to a Pydantic model) that the research_team then executes step by step, separating planning from execution.","maps_to_pattern":"plan-and-execute"},{"name":"Human-in-the-Loop","summary":"The human_feedback node interrupts the run to surface the plan; the user can accept it ([ACCEPTED]), edit it in natural language ([EDIT_PLAN], looping back to the planner), or enable auto-accept to skip review.","maps_to_pattern":"human-in-the-loop"},{"name":"ReAct","summary":"The researcher and coder workers are ReAct agents that interleave reasoning with tool calls; they are constructed via a create_react_agent / create_agent factory and bound to search, retrieval, MCP, and Python-REPL tools.","maps_to_pattern":"react"},{"name":"Agentic RAG","summary":"The researcher dynamically attaches web-search tools (Tavily, DuckDuckGo, Brave, Arxiv, Wikipedia) and private knowledge-base retrievers (RAGFlow, VikingDB, Milvus, Qdrant) and reasons over the retrieved content.","maps_to_pattern":"agentic-rag"}]},{"id":"deerflow-v2","name":"DeerFlow 2.0 (SuperAgent harness)","kind":"framework","category":"research-agent","build_surface":"full-code","vendor":"ByteDance","language":"Python, TypeScript","license":"MIT","status":"active","first_released":"2026-02","url":"https://deerflow.tech/","repo":"https://github.com/bytedance/deer-flow","docs_url":"https://deepwiki.com/bytedance/deer-flow","intent":"Run long-horizon tasks that research, code, and create by giving a single lead agent the ability to dynamically spawn bounded, context-isolated subagents that execute code and tools inside Docker sandboxes and then synthesise their structured results.","description":"DeerFlow 2.0 is a ground-up rewrite of ByteDance's DeerFlow that shares no code with the 1.x deep-research graph and became the active line after its launch around late February 2026. Rather than a fixed LangGraph research workflow, it is a general open-source SuperAgent harness: a lead agent receives the request, decomposes it, and dynamically spawns subagents on the fly, each with its own scoped context, tools, and termination conditions. Subagents run in parallel when possible and report structured results back to the lead agent, which synthesises them into the final deliverable. The roster is not fixed: built-in subagents (a general-purpose agent and a bash agent) sit alongside user-defined custom agents declared in config.yaml, and concurrency is bounded by a SubagentLimitMiddleware. Each subagent executes inside an isolated sandbox — the AioSandboxProvider gives it a real filesystem and bash terminal in a Docker container (local, Docker, or Kubernetes-provisioned), while the LocalSandboxProvider is explicitly gated as not a secure boundary. Capabilities are extended through Agent Skills (SKILL.md capability modules, public or custom, loaded progressively) and through tools that include web search, web fetch, file operations, and bash, plus MCP servers exposed as deferred tools the agent must look up via a tool_search tool before calling. A persistent cross-session memory subsystem accumulates the user's profile and preferences, and a message gateway fronts the harness with LangGraph-compatible HTTP routes. The lead agent itself runs as a LangChain agent wrapped in a middleware chain (loop detection, summarisation, todo tracking, subagent limiting, sandbox auditing, and more). The original deep-research graph is catalogued separately as DeerFlow 1.x and is maintained on the `main-1.x` branch.","primary_use_cases":["long-horizon tasks (minutes to hours) that mix research, coding, and content creation","autonomous agents that need a real filesystem, bash terminal, and safe code execution","dynamic task decomposition where the lead agent spawns parallel subagents on demand","skill- and MCP-extensible super agents with persistent cross-session memory"],"agent_loop_shape":"A lead agent runs a LangChain agent loop wrapped in a middleware chain; on each step it may call the `task` tool to delegate work to subagents (general-purpose, bash, or custom skill-scoped), each of which runs its own bounded ReAct-style loop inside an isolated Docker/AIO sandbox and returns a structured result. Concurrent subagent spawns are truncated to a clamped maximum, skills and deferred MCP tools are loaded lazily, and the lead agent synthesises subagent results into the final output while a cross-session memory subsystem persists profile and preferences across runs.","pattern_composition":"flowchart TD\n  GW[\"Message gateway: /api/langgraph/*\"] --> LA[\"Lead agent: decompose and synthesize\"]\n  SK[(\"Skills: SKILL.md, loaded progressively\")] -.-> LA\n  MEM[(\"Cross-session memory: profile and preferences\")] -.-> LA\n  LA -->|\"task tool, bounded by max_concurrent\"| SA1[\"Subagent: general-purpose\"]\n  LA -->|\"task tool\"| SA2[\"Subagent: bash\"]\n  LA -->|\"task tool\"| SAN[\"Subagent: custom / skill-scoped\"]\n  SA1 --> SB[\"Sandbox: Docker/AIO isolated FS and bash\"]\n  SA2 --> SB\n  SAN --> SB\n  SA1 -->|\"structured result\"| LA\n  SA2 -->|\"structured result\"| LA\n  SAN -->|\"structured result\"| LA\n  LA --> OUT[\"Finished deliverable\"]","members":[{"pattern":"orchestrator-workers","role":"core","note":"A lead agent decomposes the request and dispatches sub-tasks to subagents via a `task` tool, then synthesises their structured results into the finished deliverable — the harness's central orchestration shape.","evidence":[{"type":"doc","title":"README — SuperAgent harness orchestration","url":"https://github.com/bytedance/deer-flow/blob/main/README.md","quote":"DeerFlow (Deep Exploration and Efficient Research Flow) is an open-source super agent harness that orchestrates sub-agents, memory, and sandboxes to do almost anything — powered by extensible skills.","accessed":"2026-06-04"},{"type":"repo","title":"task_tool.py — delegate work to subagents","url":"https://github.com/bytedance/deer-flow/blob/main/backend/packages/harness/deerflow/tools/builtins/task_tool.py","quote":"\"\"\"Task tool for delegating work to subagents.\"\"\"","accessed":"2026-06-04"},{"type":"repo","title":"lead_agent package","url":"https://github.com/bytedance/deer-flow/tree/main/backend/packages/harness/deerflow/agents/lead_agent","accessed":"2026-06-04"}],"evidence_status":"full"},{"pattern":"subagent-isolation","role":"core","note":"Each subagent is spawned with its own scoped context, tool set, and termination conditions, and cannot see the parent or sibling contexts; the built-in general-purpose agent is recommended precisely when a task benefits from isolated context management.","evidence":[{"type":"repo","title":"general_purpose.py — isolated context subagent","url":"https://github.com/bytedance/deer-flow/blob/main/backend/packages/harness/deerflow/subagents/builtins/general_purpose.py","quote":"The task would benefit from isolated context management","accessed":"2026-06-04"},{"type":"repo","title":"subagents/registry.py — per-subagent config (tools, skills, model, max_turns)","url":"https://github.com/bytedance/deer-flow/blob/main/backend/packages/harness/deerflow/subagents/registry.py","quote":"\"\"\"Subagent registry for managing available subagents.\"\"\"","accessed":"2026-06-04"}],"evidence_status":"full"},{"pattern":"parallel-fan-out-gather","role":"first-class","note":"Subagents run in parallel when possible, report back structured results, and the lead agent gathers and synthesises them; a single model response can emit several parallel `task` calls at once.","evidence":[{"type":"repo","title":"subagent_limit_middleware.py — parallel task calls","url":"https://github.com/bytedance/deer-flow/blob/main/backend/packages/harness/deerflow/agents/middlewares/subagent_limit_middleware.py","quote":"When an LLM generates more than max_concurrent parallel task tool calls in one response, this middleware keeps only the first max_concurrent and discards the rest.","accessed":"2026-06-04"}],"evidence_status":"full"},{"pattern":"agent-skills","role":"first-class","note":"Capabilities are packaged as Agent Skills — SKILL.md modules (public/built-in or user-authored custom) that define a workflow and supporting resources, loaded progressively so only the skills a task needs enter context.","evidence":[{"type":"repo","title":"skills/types.py — SKILL.md modules, public vs custom","url":"https://github.com/bytedance/deer-flow/blob/main/backend/packages/harness/deerflow/skills/types.py","quote":"PUBLIC: built-in skill bundled with the platform, read-only.","accessed":"2026-06-04"},{"type":"repo","title":".agent/skills/smoke-test/SKILL.md — example skill bundle","url":"https://github.com/bytedance/deer-flow/blob/main/.agent/skills/smoke-test/SKILL.md","accessed":"2026-06-04"}],"evidence_status":"full"},{"pattern":"sandbox-isolation","role":"first-class","note":"Subagents execute inside isolated sandboxes: the AioSandboxProvider provisions Docker containers (local or Kubernetes-backed) with a real filesystem and bash, while the LocalSandboxProvider is explicitly gated off for host bash because it is not a secure boundary.","evidence":[{"type":"repo","title":"sandbox/security.py — local provider is not a secure boundary","url":"https://github.com/bytedance/deer-flow/blob/main/backend/packages/harness/deerflow/sandbox/security.py","quote":"Host bash execution is disabled for LocalSandboxProvider because it is not a secure sandbox boundary. Switch to AioSandboxProvider for isolated bash access","accessed":"2026-06-04"},{"type":"repo","title":"aio_sandbox_provider.py — Docker/K8s sandbox lifecycle","url":"https://github.com/bytedance/deer-flow/blob/main/backend/packages/harness/deerflow/community/aio_sandbox/aio_sandbox_provider.py","quote":"SandboxBackend: how sandboxes are provisioned (local container vs remote/K8s)","accessed":"2026-06-04"}],"evidence_status":"full"},{"pattern":"code-execution","role":"first-class","note":"The harness is an execution engine, not just a reasoning layer: a dedicated bash subagent and sandbox tools run real bash commands and code inside the sandbox rather than only proposing them.","evidence":[{"type":"repo","title":"bash_agent.py — command-execution specialist subagent","url":"https://github.com/bytedance/deer-flow/blob/main/backend/packages/harness/deerflow/subagents/builtins/bash_agent.py","quote":"Command execution specialist for running bash commands in a separate context.","accessed":"2026-06-04"},{"type":"repo","title":"sandbox/tools.py — sandbox tool surface","url":"https://github.com/bytedance/deer-flow/blob/main/backend/packages/harness/deerflow/sandbox/tools.py","accessed":"2026-06-04"}],"evidence_status":"full"},{"pattern":"tool-search-lazy-loading","role":"first-class","note":"Tools (including MCP-provided ones) are exposed as deferred entries the agent can see by name but cannot call until it fetches their full schema through a `tool_search` tool, keeping the active tool surface and context small.","evidence":[{"type":"repo","title":"tools/builtins/tool_search.py — deferred tool discovery","url":"https://github.com/bytedance/deer-flow/blob/main/backend/packages/harness/deerflow/tools/builtins/tool_search.py","quote":"The agent sees deferred tool names in <available-deferred-tools> but cannot call them until it fetches their full schema via the tool_search tool.","accessed":"2026-06-04"}],"evidence_status":"full"},{"pattern":"cross-session-memory","role":"first-class","note":"A memory subsystem reads, writes, and updates persistent facts about the user's profile and preferences across sessions, deduplicating entries at apply time.","evidence":[{"type":"repo","title":"agents/memory/updater.py — persistent memory updater","url":"https://github.com/bytedance/deer-flow/blob/main/backend/packages/harness/deerflow/agents/memory/updater.py","quote":"\"\"\"Memory updater for reading, writing, and updating memory data.\"\"\"","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"mcp","role":"supported","note":"External capabilities are added through MCP servers; their tools are tagged as deferred (deerflow_mcp) and merged into the agent's tool surface, discoverable via tool_search.","evidence":[{"type":"repo","title":"mcp/tools.py — MCP tool integration","url":"https://github.com/bytedance/deer-flow/blob/main/backend/packages/harness/deerflow/mcp/tools.py","accessed":"2026-06-04"},{"type":"repo","title":"tool_search.py — deferred MCP tool tag","url":"https://github.com/bytedance/deer-flow/blob/main/backend/packages/harness/deerflow/tools/builtins/tool_search.py","quote":"a tool is \"deferred\" when it carries the ``deerflow_mcp`` metadata tag.","accessed":"2026-06-04"}],"evidence_status":"full"},{"pattern":"human-in-the-loop","role":"supported","note":"The agent can pause to ask the user for clarification through a dedicated clarification tool and middleware before proceeding with an ambiguous task.","evidence":[{"type":"repo","title":"tools/builtins/clarification_tool.py","url":"https://github.com/bytedance/deer-flow/blob/main/backend/packages/harness/deerflow/tools/builtins/clarification_tool.py","accessed":"2026-06-04"},{"type":"repo","title":"middlewares/clarification_middleware.py","url":"https://github.com/bytedance/deer-flow/blob/main/backend/packages/harness/deerflow/agents/middlewares/clarification_middleware.py","accessed":"2026-06-04"}],"evidence_status":"limited"},{"pattern":"agent-middleware-chain","role":"supported","note":"The lead agent is assembled by a factory that wraps the base LangChain agent in a stack of middlewares — loop detection, summarisation, todo tracking, subagent limiting, sandbox auditing, token usage, and more — each shaping the agent loop.","evidence":[{"type":"repo","title":"agents/factory.py — agent assembly","url":"https://github.com/bytedance/deer-flow/blob/main/backend/packages/harness/deerflow/agents/factory.py","quote":"\"\"\"Pure-argument factory for DeerFlow agents.","accessed":"2026-06-04"},{"type":"repo","title":"agents/middlewares — middleware stack","url":"https://github.com/bytedance/deer-flow/tree/main/backend/packages/harness/deerflow/agents/middlewares","accessed":"2026-06-04"}],"evidence_status":"full"}],"anti_patterns_avoided":[{"pattern":"unbounded-subagent-spawn","note":"Although the lead agent spawns subagents dynamically, a SubagentLimitMiddleware truncates excess parallel `task` calls from a single model response down to a clamped maximum (default 3, clamped to the range 2–4), enforced in code rather than by prompt, so a model cannot fan out an unbounded number of concurrent subagents.","evidence":[{"type":"repo","title":"subagent_limit_middleware.py — concurrency cap","url":"https://github.com/bytedance/deer-flow/blob/main/backend/packages/harness/deerflow/agents/middlewares/subagent_limit_middleware.py","quote":"Middleware to enforce maximum concurrent subagent tool calls per model response.","accessed":"2026-06-04"}]},{"pattern":"unbounded-loop","note":"A dedicated loop-detection middleware guards the agent loop, and each subagent carries a max_turns bound in its config, so neither the lead agent nor a subagent can iterate indefinitely.","evidence":[{"type":"repo","title":"middlewares/loop_detection_middleware.py","url":"https://github.com/bytedance/deer-flow/blob/main/backend/packages/harness/deerflow/agents/middlewares/loop_detection_middleware.py","accessed":"2026-06-04"},{"type":"repo","title":"subagents/registry.py — per-subagent max_turns","url":"https://github.com/bytedance/deer-flow/blob/main/backend/packages/harness/deerflow/subagents/registry.py","quote":"max_turns=custom.max_turns,","accessed":"2026-06-04"}]},{"pattern":"agent-generated-code-rce","note":"Agent-executed code and bash run only inside an isolated sandbox boundary; host bash is disabled for the non-isolating LocalSandboxProvider, and agent-managed skill writes pass through a security scanner before they can be installed.","evidence":[{"type":"repo","title":"sandbox/security.py — capability gating","url":"https://github.com/bytedance/deer-flow/blob/main/backend/packages/harness/deerflow/sandbox/security.py","quote":"Host bash execution is disabled for LocalSandboxProvider because it is not a secure sandbox boundary.","accessed":"2026-06-04"},{"type":"repo","title":"skills/security_scanner.py — screening for skill writes","url":"https://github.com/bytedance/deer-flow/blob/main/backend/packages/harness/deerflow/skills/security_scanner.py","quote":"\"\"\"Security screening for agent-managed skill writes.\"\"\"","accessed":"2026-06-04"}]}],"alternatives":[{"composition":"deerflow-v1","relation":"predecessor","note":"DeerFlow 1.x is the original deep-research framework — a fixed LangGraph plan-and-execute graph. 2.0 is a ground-up rewrite that shares no code with it and generalises from deep research to an open-ended SuperAgent harness."},{"composition":"langgraph","relation":"complements","note":"The 2.0 lead agent and its middleware chain are built on LangChain/LangGraph, and the message gateway exposes LangGraph-compatible HTTP routes."}],"references":[{"type":"doc","title":"DeerFlow homepage","url":"https://deerflow.tech/","accessed":"2026-06-04"},{"type":"repo","title":"bytedance/deer-flow — repository description (2.0)","url":"https://github.com/bytedance/deer-flow","quote":"An open-source long-horizon SuperAgent harness that researches, codes, and creates. With the help of sandboxes, memories, tools, skill, subagents and message gateway, it handles different levels of tasks that could take minutes to hours.","accessed":"2026-06-04"},{"type":"doc","title":"README — DeerFlow 2.0 is a ground-up rewrite","url":"https://github.com/bytedance/deer-flow/blob/main/README.md","quote":"DeerFlow 2.0 is a ground-up rewrite. It shares no code with v1.","accessed":"2026-06-04"}],"last_analyzed":"2026-06-04","verification_status":"partial","last_updated":"2026-06-17","date_added":"2026-06-04","status_in_practice":"emerging","tags":["open-source","bytedance","super-agent","subagents","sandbox","agent-skills","multi-agent","memory","mcp"],"key_concepts":[{"name":"Orchestrator-Workers","summary":"A lead agent decomposes the request and dispatches sub-tasks to subagents via a `task` tool, then synthesises their structured results into the finished deliverable — the harness's central orchestration shape.","maps_to_pattern":"orchestrator-workers"},{"name":"Subagent Isolation","summary":"Each subagent is spawned with its own scoped context, tool set, and termination conditions, and cannot see the parent or sibling contexts; the built-in general-purpose agent is recommended precisely when a task benefits from isolated context management.","maps_to_pattern":"subagent-isolation"},{"name":"Parallel Fan-Out / Gather","summary":"Subagents run in parallel when possible, report back structured results, and the lead agent gathers and synthesises them; a single model response can emit several parallel `task` calls at once.","maps_to_pattern":"parallel-fan-out-gather"},{"name":"Agent Skills","summary":"Capabilities are packaged as Agent Skills — SKILL.md modules (public/built-in or user-authored custom) that define a workflow and supporting resources, loaded progressively so only the skills a task needs enter context.","maps_to_pattern":"agent-skills"},{"name":"Sandbox Isolation","summary":"Subagents execute inside isolated sandboxes: the AioSandboxProvider provisions Docker containers (local or Kubernetes-backed) with a real filesystem and bash, while the LocalSandboxProvider is explicitly gated off for host bash because it is not a secure boundary.","maps_to_pattern":"sandbox-isolation"}]},{"id":"pageindex","name":"PageIndex","aliases":["PageIndex (VectifyAI)"],"kind":"framework","category":"research-agent","build_surface":"full-code","vendor":"VectifyAI","language":"Python","license":"MIT","status":"active","first_released":"2025","url":"https://pageindex.ai","repo":"https://github.com/VectifyAI/PageIndex","docs_url":"https://docs.pageindex.ai","intent":"Retrieve over long, structured documents without a vector database by building a table-of-contents tree from each document and having an LLM reason down that tree to the relevant sections, returning page and section references instead of vector hits.","description":"PageIndex is VectifyAI's MIT-licensed open-source retrieval engine for long professional documents (financial filings, contracts, regulatory manuals, technical specifications). It is deliberately vectorless: at index time it parses a document into a hierarchical table-of-contents tree with a summary at each node and keeps leaf text intact rather than splitting it into fixed-size chunks; it computes no embeddings and builds no vector store. At query time it presents the tree to an LLM, which performs a tree search — judging which branch is most likely to hold the answer, descending, and repeating — and returns the leaf sections it reaches together with their page and section identifiers, so every retrieval is traceable to a named location in the source. The project ships a Python library (run_pageindex.py), an MCP server that exposes the tree navigator to MCP clients, and an example built on the OpenAI Agents SDK that swaps a vector-search tool for the tree-navigation tool. It positions itself as an alternative to embedding-based retrieval-augmented generation for documents where vocabulary overlap misleads similarity search and where auditable, citation-grounded retrieval matters.","primary_use_cases":["retrieval and question answering over long structured documents (financial filings, contracts, manuals)","auditable retrieval where every result must point to a named page and section","agentic document analysis where a tree-navigation tool replaces a vector-search tool"],"agent_loop_shape":"Index time: parse document -> table-of-contents tree with per-node summaries (no embeddings, no chunking). Query time: the LLM reasons over the tree, chooses a branch, descends, and repeats until it reaches the relevant leaf sections; retrieval returns those sections with page and section references for the generator to answer over.","pattern_composition":"flowchart TD\n  DOC[\"Long structured document\"] --> IDX[\"Index: parse into table-of-contents tree with node summaries\"]\n  IDX --> TREE[\"Tree index - no vectors, no chunks\"]\n  Q[\"Query\"] --> NAV{\"LLM tree search over node summaries\"}\n  TREE --> NAV\n  NAV -->|descend branch| NAV\n  NAV -->|leaves reached| SEC[\"Selected leaf sections + page/section refs\"]\n  SEC --> GEN[\"Generator answers with citations\"]","key_concepts":[{"name":"Vectorless retrieval","summary":"No embeddings and no vector store are built; the document structure is the index.","maps_to_pattern":"vectorless-reasoning-retrieval"},{"name":"Table-of-contents tree","summary":"The document's own section hierarchy, with a summary per node, is the index that is navigated.","maps_to_pattern":"vectorless-reasoning-retrieval"},{"name":"LLM tree search","summary":"Reasoning, not similarity, decides each navigation step down the tree to the relevant sections.","maps_to_pattern":"vectorless-reasoning-retrieval"},{"name":"Page and section references","summary":"Retrieval results are traceable named locations rather than opaque vector hits, feeding citations directly.","maps_to_pattern":"citation-attribution"},{"name":"Tree navigator as a tool","summary":"An MCP server and an OpenAI Agents SDK example expose the tree navigator as a retrieval tool inside an agent loop.","maps_to_pattern":"agentic-rag"}],"members":[{"pattern":"vectorless-reasoning-retrieval","role":"core","note":"PageIndex is the reference implementation of the pattern: it builds the table-of-contents tree and has the LLM tree-search it for retrieval, with no embeddings at any tier.","evidence":[{"type":"repo","title":"PageIndex README — Vectorless, Reasoning-based RAG","url":"https://github.com/VectifyAI/PageIndex","quote":"No Vector DB: Uses document structure and LLM reasoning for retrieval, instead of vector similarity search. No Chunking: Documents are organized into natural sections, not artificial chunks.","accessed":"2026-06-05"}],"evidence_status":"full"},{"pattern":"citation-attribution","role":"first-class","note":"Retrieval returns explicit page and section references rather than opaque vector hits, so answers are grounded in named, checkable locations.","evidence":[{"type":"repo","title":"PageIndex README — traceable retrieval","url":"https://github.com/VectifyAI/PageIndex","quote":"Retrieval is based on reasoning, traceable and interpretable, with page and section references.","accessed":"2026-06-05"}],"evidence_status":"full"},{"pattern":"agentic-rag","role":"via-integration","note":"Ships an example built on the OpenAI Agents SDK that drives the tree navigator as a retrieval tool inside an agent loop, in place of a vector-search tool.","evidence":[{"type":"repo","title":"PageIndex README — agentic example and MCP","url":"https://github.com/VectifyAI/PageIndex","quote":"Fork the agentic example (OpenAI Agents SDK) and swap a vector tool for a tree tool.","accessed":"2026-06-05"}],"evidence_status":"limited"}],"references":[{"type":"repo","title":"VectifyAI/PageIndex (GitHub)","url":"https://github.com/VectifyAI/PageIndex","accessed":"2026-06-05"},{"type":"doc","title":"PageIndex homepage","url":"https://pageindex.ai","accessed":"2026-06-05"},{"type":"doc","title":"PageIndex documentation","url":"https://docs.pageindex.ai","accessed":"2026-06-05"},{"type":"blog","title":"PageIndex: Reasoning-Based RAG","url":"https://pageindex.ai/blog/pageindex-intro","accessed":"2026-06-05"}],"last_analyzed":"2026-06-05","verification_status":"partial","last_updated":"2026-06-17","date_added":"2026-06-05","status_in_practice":"emerging","tags":["open-source","mit","vectorless","rag","retrieval","tree-search","document-qa","mcp","vectifyai"]},{"id":"elevenlabs-conversational","name":"ElevenLabs Conversational AI","aliases":["Eleven Agents"],"kind":"framework","category":"voice-conversational","build_surface":"full-code","vendor":"ElevenLabs","language":"API","license":"proprietary","status":"active","url":"https://elevenlabs.io/conversational-ai","docs_url":"https://elevenlabs.io/docs/eleven-agents/overview","intent":"Hosted real-time voice agent stack from ElevenLabs that wires an ASR model, a configurable LLM, a low-latency TTS voice and a proprietary turn-taking model into a single managed conversational loop.","description":"ElevenLabs Conversational AI (rebranded Eleven Agents) is a hosted platform for building voice agents that listen, understand and act in real time across voice and chat. The pipeline pairs a fine-tuned ASR, the developer's choice of language model and a low-latency ElevenLabs TTS voice with a proprietary turn-taking model, and is extended via System Tools (built-in actions such as agent transfer, end call, language detection), Client Tools (browser/mobile actions) and Server Tools (HTTP calls to the developer's backend).","primary_use_cases":["customer support voice agents across telephony and web","multilingual voice assistants spanning ASR and TTS languages","multi-agent workflows with hand-off to specialised ElevenLabs agents","integration of CRM, calendar and payment backends via server tools"],"agent_loop_shape":"Hosted streaming voice loop: incoming audio is transcribed by a fine-tuned ASR, the user's selected LLM produces a response, ElevenLabs TTS streams audio back, and a proprietary turn-taking model drives endpointing and barge-in. Tool calls fan out via System / Client / Server tools; agent transfer is a system tool that hands the live conversation to another configured agent.","key_concepts":[{"name":"Agent platform","summary":"ASR + LLM + TTS + turn-taking model packaged as one configurable agent.","url":"https://elevenlabs.io/docs/eleven-agents/overview"},{"name":"System tools","summary":"Built-in actions that mutate conversation state without external calls (agent transfer, end call, language detection).","maps_to_pattern":"tool-use","url":"https://elevenlabs.io/docs/eleven-agents/customization/tools/system-tools"},{"name":"Client tools","summary":"Tools executed in the embedding browser or mobile app.","maps_to_pattern":"tool-use","url":"https://elevenlabs.io/docs/eleven-agents/customization/tools"},{"name":"Server tools","summary":"HTTP-call tools that reach external systems and APIs.","maps_to_pattern":"tool-use","url":"https://elevenlabs.io/docs/eleven-agents/customization/tools"},{"name":"Agent transfer","summary":"Hand off an ongoing conversation to another designated ElevenLabs agent.","maps_to_pattern":"conversation-handoff","url":"https://elevenlabs.io/docs/eleven-agents/customization/tools/system-tools/agent-transfer"},{"name":"Multilingual voices","summary":"Voice library spanning 5k+ voices and 70+ languages used by the agent's TTS leg.","maps_to_pattern":"multilingual-voice-agent","url":"https://elevenlabs.io/docs/eleven-agents/customization/voice"}],"pattern_composition":"flowchart TD\n  caller[Caller audio] --> asr[Fine-tuned ASR]\n  asr --> llm[Configured LLM]\n  llm --> tools{Tool call?}\n  tools -->|system| sys[System tool: transfer / end / language]\n  tools -->|client| client[Client tool in browser/mobile]\n  tools -->|server| server[Server tool HTTP call]\n  sys --> route{Agent transfer?}\n  route -->|yes| other[Other ElevenLabs agent]\n  route -->|no| llm\n  client --> llm\n  server --> llm\n  llm --> tts[ElevenLabs TTS]\n  tts --> caller\n  turn[Turn-taking model] -.endpointing.-> asr\n  turn -.barge-in.-> tts","members":[{"pattern":"conversation-handoff","role":"first-class","note":"Agent-to-agent transfer is a documented system tool that moves the ongoing conversation to another designated ElevenLabs agent.","evidence":[{"type":"doc","title":"Eleven Agents — agent transfer","url":"https://elevenlabs.io/docs/eleven-agents/customization/tools/system-tools/agent-transfer","quote":"Agent-to-agent transfer allows an ElevenLabs agent to hand off the ongoing conversation to another designated agent when specific conditions are met.","accessed":"2026-05-20"},{"type":"doc","title":"Eleven Agents — system tools","url":"https://elevenlabs.io/docs/eleven-agents/customization/tools/system-tools","quote":"System tools enable your assistant to update the internal state of a conversation","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"multilingual-voice-agent","role":"first-class","note":"TTS leg ships 5k+ voices across 70+ languages; conversation page advertises real-time language detection and switching.","evidence":[{"type":"doc","title":"Eleven Agents — 70+ languages","url":"https://elevenlabs.io/docs/eleven-agents/overview","quote":"A low-latency Text to Speech (TTS) model across 5k+ voices and 70+ languages","accessed":"2026-05-20"},{"type":"doc","title":"ElevenLabs Conversational AI — language detection","url":"https://elevenlabs.io/conversational-ai","quote":"Real-time language detection and switching across 70+ languages. Agents serve global customers without transfers or delays.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Three explicit tool categories: System tools (in-conversation state), Client tools (browser/mobile), Server tools (HTTP).","evidence":[{"type":"doc","title":"Eleven Agents — tools overview","url":"https://elevenlabs.io/docs/eleven-agents/customization/tools","quote":"Tools allow ElevenLabs agents to perform actions beyond generating text responses.","accessed":"2026-05-20"},{"type":"doc","title":"Eleven Agents — client tools","url":"https://elevenlabs.io/docs/eleven-agents/customization/tools","quote":"Empower your assistant to trigger client-side operations","accessed":"2026-05-20"},{"type":"doc","title":"Eleven Agents — server tools","url":"https://elevenlabs.io/docs/eleven-agents/customization/tools","quote":"Connect your assistant to external data & systems","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"stop-cancel","role":"first-class","note":"Proprietary turn-taking model handles interruptions and turn boundaries; conversation-flow page exposes the knobs.","evidence":[{"type":"doc","title":"Eleven Agents — turn-taking model","url":"https://elevenlabs.io/docs/eleven-agents/overview","quote":"A proprietary turn-taking model","accessed":"2026-05-20"},{"type":"doc","title":"Eleven Agents — conversation flow","url":"https://elevenlabs.io/docs/eleven-agents/customization/conversation-flow","quote":"Configure turn-taking, interruptions, and timeout settings","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"streaming-typed-events","role":"first-class","note":"WebSocket protocol delivers typed real-time events between server and client — base64 audio chunks, transcripts, agent responses, interruption events, and character-level alignment data.","evidence":[{"type":"doc","title":"Eleven Agents — low-latency TTS","url":"https://elevenlabs.io/docs/eleven-agents/overview","quote":"A low-latency Text to Speech (TTS) model across 5k+ voices and 70+ languages","accessed":"2026-05-20"},{"type":"doc","title":"Eleven Agents WebSocket — real-time connection","url":"https://elevenlabs.io/docs/eleven-agents/api-reference/eleven-agents/websocket","quote":"Establish a WebSocket connection for real-time conversations with an AI agent.","accessed":"2026-05-24"},{"type":"doc","title":"Eleven Agents WebSocket — alignment data","url":"https://elevenlabs.io/docs/eleven-agents/api-reference/eleven-agents/websocket","quote":"Character-level timing data for the audio chunk, useful for synchronized text display or lip-syncing.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"personality-variant-overlay","role":"supported","note":"A single agent dynamically switches among several named voices within one conversation (multi-character/persona mode) and returns to its default base voice, overlaying register rather than splitting into separate agents.","evidence":[{"type":"doc","title":"Multi-voice support - ElevenLabs Documentation","url":"https://elevenlabs.io/docs/agents-platform/customization/voice/multi-voice-support","quote":"Multi-voice support allows your ElevenLabs agent to dynamically switch between different ElevenLabs voices during a single conversation.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"alternatives":[{"composition":"vapi","relation":"competes-with","note":"Peer hosted voice-agent platform."},{"composition":"retell-ai","relation":"competes-with","note":"Peer hosted phone-agent platform."},{"composition":"livekit-agents","relation":"competes-with","note":"Open framework alternative to a hosted platform."},{"composition":"pipecat","relation":"competes-with","note":"Open Python framework alternative."},{"composition":"hume-evi","relation":"competes-with","note":"Peer hosted voice-agent platform with similar ASR+LLM+TTS shape."}],"references":[{"type":"doc","title":"Eleven Agents overview","url":"https://elevenlabs.io/docs/eleven-agents/overview","accessed":"2026-05-20"},{"type":"doc","title":"ElevenLabs Conversational AI — product page","url":"https://elevenlabs.io/conversational-ai","quote":"Conversational AI uses natural language processing (NLP), large language models (LLMs), and machine learning to interpret what someone actually means.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"mature","tags":["voice","hosted","telephony","tts","asr"]},{"id":"hume-evi","name":"Hume EVI","aliases":["Empathic Voice Interface","Hume Speech-to-Speech"],"kind":"framework","category":"voice-conversational","build_surface":"full-code","vendor":"Hume AI","language":"API","license":"proprietary","status":"active","url":"https://hume.ai/evi","docs_url":"https://dev.hume.ai/docs/speech-to-speech-evi/overview","intent":"Hosted speech-to-speech voice API from Hume AI that pairs an emotionally aware response model with a configurable supplemental LLM, measuring vocal prosody and adapting tone in real time.","description":"EVI is Hume AI's real-time speech-to-speech interface. It streams measurements of the tune, rhythm and timbre of the user's voice, reacts with matching prosody, and remains interruptible at all times. EVI is configured through a Config object (system prompt, voice, supplemental LLM, tools) and can be supplemented with partner LLMs from Anthropic, OpenAI, Google or Fireworks. Tool use and built-in tools are first-class but parallel function calls are not yet supported. Chat Groups link sessions so a conversation can resume across disconnects.","primary_use_cases":["emotionally aware voice agents across consumer and support apps","multilingual speech-to-speech experiences on EVI 4 / 4-mini","function-calling voice agents that hit external APIs","long-running conversations resumed via Chat Groups"],"agent_loop_shape":"Hosted speech-to-speech loop over a WebSocket. Caller audio streams in; EVI emits prosody measurements, decides the response (optionally via a supplemental LLM), streams generated speech back, and is interruptible by design. Tool calls go out to the developer's backend except for Hume's built-in tools which it invokes itself. A Chat Group id can be passed to resume across reconnects.","key_concepts":[{"name":"EVI speech-to-speech","summary":"Streaming voice loop that measures vocal prosody and matches tone.","url":"https://dev.hume.ai/docs/speech-to-speech-evi/overview"},{"name":"Configuration","summary":"EVI Config defines system behavior, voice and supplemental LLM.","url":"https://dev.hume.ai/docs/speech-to-speech-evi/overview"},{"name":"Supplemental LLM","summary":"Optional partner LLMs from Anthropic / OpenAI / Google / Fireworks plug into the loop.","url":"https://dev.hume.ai/docs/speech-to-speech-evi/overview"},{"name":"Tool use","summary":"Developer-defined functions and Hume built-in tools; no parallel calls.","maps_to_pattern":"tool-use","url":"https://dev.hume.ai/docs/speech-to-speech-evi/features/tool-use"},{"name":"Multilingual EVI 4-mini","summary":"Languages: English, Japanese, Korean, Spanish, French, Portuguese, Italian, German, Russian, Hindi, Arabic.","maps_to_pattern":"multilingual-voice-agent","url":"https://dev.hume.ai/docs/speech-to-speech-evi/overview"},{"name":"Chat Groups","summary":"Link related chats so a conversation persists across disconnects.","maps_to_pattern":"agent-resumption","url":"https://dev.hume.ai/docs/speech-to-speech-evi/features/chat-history"}],"pattern_composition":"flowchart TD\n  user[User audio] --> ws[WebSocket]\n  ws --> evi[EVI speech-to-speech]\n  evi --> prosody[Prosody measurements]\n  evi --> resp[Response generation]\n  resp --> llm{Supplemental LLM?}\n  llm -->|yes| partner[Anthropic / OpenAI / Google / Fireworks]\n  llm -->|no| native[Native EVI response]\n  partner --> tool{Tool call?}\n  native --> tool\n  tool -->|developer fn| backend[Developer function]\n  tool -->|built-in| builtin[Hume built-in tool]\n  tool -->|none| tts[Streaming TTS reply]\n  backend --> resp\n  builtin --> resp\n  tts --> user\n  interrupt[User interjects] -.barge-in.-> tts\n  group[Chat Group id] -.resume.-> evi","members":[{"pattern":"agent-resumption","role":"first-class","note":"Chat Groups bundle individual chats so a conversation can resume across reconnects; a chat_group_id connects each chat to its group.","evidence":[{"type":"doc","title":"EVI chat history — Chat Groups","url":"https://dev.hume.ai/docs/speech-to-speech-evi/features/chat-history","quote":"Chat Groups link related chats to maintain continuity across multiple sessions, allowing conversations to persist even when users disconnect and reconnect.","accessed":"2026-05-20"},{"type":"doc","title":"EVI chat history — resumed conversations","url":"https://dev.hume.ai/docs/speech-to-speech-evi/features/chat-history","quote":"By default, a new chat session creates a new chat group. If the session resumes a previous conversation, the new chat is added to the existing chat group, preserving the full interaction history and context across sessions.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"multilingual-voice-agent","role":"first-class","note":"EVI 4-mini ships an explicit multilingual roster spanning eleven languages.","evidence":[{"type":"doc","title":"EVI overview — supported languages","url":"https://dev.hume.ai/docs/speech-to-speech-evi/overview","quote":"EVI 4-mini supports English, Japanese, Korean, Spanish, French, Portuguese, Italian, German, Russian, Hindi, Arabic.","accessed":"2026-05-20"},{"type":"doc","title":"EVI overview — language list verified 2026-05-24","url":"https://dev.hume.ai/docs/speech-to-speech-evi/overview","quote":"English, Japanese, Korean, Spanish, French, Portuguese, Italian, German, Russian, Hindi, Arabic","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Developer-defined function calls plus Hume built-in tools; explicit caveat that parallel calls are not supported.","evidence":[{"type":"doc","title":"EVI tool use — function calling","url":"https://dev.hume.ai/docs/speech-to-speech-evi/features/tool-use","quote":"EVI simplifies the integration of external APIs through function calling.","accessed":"2026-05-20"},{"type":"doc","title":"EVI tool use — built-in tools","url":"https://dev.hume.ai/docs/speech-to-speech-evi/features/tool-use","quote":"Hume also provides built-in tools that are natively integrated. This means that you don't need to define the function; EVI handles both determining when the function needs to be called and invoking it.","accessed":"2026-05-20"},{"type":"doc","title":"EVI tool use — no parallel calls","url":"https://dev.hume.ai/docs/speech-to-speech-evi/features/tool-use","quote":"Currently, EVI does not support parallel function calling. Only one function call can be processed at a time.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"stop-cancel","role":"first-class","note":"EVI is documented as always interruptible, stopping rapidly when the user interjects, then resuming with the right context.","evidence":[{"type":"doc","title":"EVI overview — always interruptible","url":"https://dev.hume.ai/docs/speech-to-speech-evi/overview","quote":"Always interruptible: Stops rapidly whenever users interject, listens, and responds with the right context.","accessed":"2026-05-20"},{"type":"doc","title":"EVI overview — resumes from where it left off","url":"https://dev.hume.ai/docs/speech-to-speech-evi/overview","quote":"Stops rapidly whenever users interject, listens, and responds with the right context based on where it left off.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"multi-model-routing","role":"supported","note":"EVI can be supplemented with configurable partner LLMs from multiple vendors; the language-model configuration page explicitly lists Claude, GPT, and Gemini as external options.","evidence":[{"type":"doc","title":"EVI overview — supplemental LLM","url":"https://dev.hume.ai/docs/speech-to-speech-evi/overview","quote":"Optionally supplemented with configurable partner APIs (Anthropic, OpenAI, Google, Fireworks, and more).","accessed":"2026-05-20"},{"type":"doc","title":"EVI language-model — external LLM choice","url":"https://dev.hume.ai/docs/speech-to-speech-evi/configuration/language-model","quote":"Developers may also choose from leading external language models such as Claude, GPT, Gemini, and many others.","accessed":"2026-05-24"},{"type":"doc","title":"EVI language-model — specify a model","url":"https://dev.hume.ai/docs/speech-to-speech-evi/configuration/language-model","quote":"EVI supports specifying a language model for response generation during chat sessions.","accessed":"2026-05-24"}],"evidence_status":"full"}],"alternatives":[{"composition":"elevenlabs-conversational","relation":"competes-with","note":"Peer hosted voice-agent platform with similar ASR+LLM+TTS shape."},{"composition":"vapi","relation":"competes-with","note":"Peer hosted voice agent platform."},{"composition":"retell-ai","relation":"competes-with","note":"Peer phone-agent platform."},{"composition":"livekit-agents","relation":"competes-with","note":"Open framework alternative."}],"references":[{"type":"doc","title":"EVI overview","url":"https://dev.hume.ai/docs/speech-to-speech-evi/overview","quote":"Provides streaming measurements of the tune, rhythm, and timbre of the user's speech.","accessed":"2026-05-20"},{"type":"doc","title":"EVI tool use","url":"https://dev.hume.ai/docs/speech-to-speech-evi/features/tool-use","quote":"Tools are resources that EVI uses to do things, like search the web or call external APIs.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"emerging","tags":["voice","hosted","prosody","emotion-aware","tts"]},{"id":"livekit-agents","name":"LiveKit Agents","kind":"framework","category":"voice-conversational","build_surface":"full-code","vendor":"LiveKit","language":"Python, Node.js","license":"Apache-2.0","status":"active","first_released":"2023-10-19","url":"https://docs.livekit.io/agents/","repo":"https://github.com/livekit/agents","docs_url":"https://docs.livekit.io/agents/","intent":"Open-source realtime agent framework that lets a Python or Node.js process join a LiveKit room as a full participant, with an STT-LLM-TTS pipeline, turn detection, tool calling and worker-based job dispatch.","description":"LiveKit Agents is a framework for building realtime voice, video and multimodal agents that operate as participants in LiveKit rooms. The agent code acts as a stateful bridge between AI models and users: it streams audio through an STT-LLM-TTS pipeline, runs a custom turn-detection model for lifelike conversation flow, handles interruptions, and exposes a function_tool decorator so any LLM can call tools (including forwarding calls to the frontend). Agents run as workers; the agent server boots a 'job' subprocess that joins each room. Plugins cover STT, LLM, TTS and Realtime APIs.","primary_use_cases":["realtime voice agents over WebRTC and telephony","multi-modal assistants that hear, see and speak","multi-agent handoff over LiveKit rooms","outbound calling, transcription and realtime translation"],"agent_loop_shape":"Worker-dispatched, room-scoped event loop. A LiveKit worker registers with the agent server; when a room is created the server dispatches a job, the job subprocess joins the room, instantiates an AgentSession with STT, LLM, TTS and turn-detection plugins, and runs the streaming pipeline. Tools defined with @function_tool are exposed to the LLM, including forwardable frontend tool calls.","key_concepts":[{"name":"Agent","summary":"A realtime participant that runs on the server and bridges users with AI models.","url":"https://docs.livekit.io/agents/"},{"name":"STT-LLM-TTS pipeline","summary":"Streaming audio pipeline with reliable turn detection and interruption handling.","url":"https://docs.livekit.io/agents/"},{"name":"Turn detection","summary":"Custom transformer model that detects end-of-turn to reduce barge-ins.","maps_to_pattern":"stop-cancel","url":"https://docs.livekit.io/agents/logic/turns"},{"name":"@function_tool","summary":"Decorator that exposes Python functions as LLM-callable tools, forwardable to frontend.","maps_to_pattern":"tool-use","url":"https://docs.livekit.io/agents/logic/tools"},{"name":"Worker / job","summary":"The agent server boots a job subprocess per room with load balancing.","url":"https://docs.livekit.io/agents/worker/"},{"name":"Plugins","summary":"Mix-and-match STT, LLM, TTS and Realtime API integrations.","url":"https://docs.livekit.io/agents/"}],"pattern_composition":"flowchart TD\n  user[User in LiveKit room] --> server[Agent server]\n  server --> dispatch[Dispatch job]\n  dispatch --> worker[Worker subprocess joins room]\n  worker --> session[AgentSession]\n  session --> stt[STT plugin]\n  stt --> turn[Custom turn-detection model]\n  turn --> llm[LLM plugin]\n  llm --> tool{function_tool?}\n  tool -->|yes| run[Run tool / forward to frontend]\n  run --> llm\n  tool -->|no| tts[TTS plugin]\n  tts --> user\n  session -.barge-in.-> tts","members":[{"pattern":"multilingual-voice-agent","role":"supported","note":"Plugins expose multilingual modes (e.g. Deepgram nova-3 language='multi'); realtime translation is a documented use case.","evidence":[{"type":"repo","title":"LiveKit Agents README — multilingual STT","url":"https://github.com/livekit/agents","quote":"stt=inference.STT(\"deepgram/nova-3\", language=\"multi\")","accessed":"2026-05-20"},{"type":"doc","title":"LiveKit Agents — realtime translation","url":"https://docs.livekit.io/agents/","quote":"Realtime translation: Translate conversations in realtime.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"stop-cancel","role":"first-class","note":"Pipeline ships reliable turn detection and interruption handling; custom transformer model detects end-of-turn.","evidence":[{"type":"doc","title":"LiveKit Agents — pipeline interruption","url":"https://docs.livekit.io/agents/","quote":"streaming audio through an STT-LLM-TTS pipeline, reliable turn detection, handling interruptions","accessed":"2026-05-20"},{"type":"repo","title":"LiveKit Agents README — turn detection","url":"https://github.com/livekit/agents","quote":"Uses a transformer model to detect when a user is done with their turn, helps to reduce interruptions.","accessed":"2026-05-20"},{"type":"doc","title":"LiveKit Agents — turn detection","url":"https://docs.livekit.io/agents/logic/turns","quote":"Turn detection is the process of determining when a user begins or ends their 'turn' in a conversation. This lets the agent know when to start listening and when to respond.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"@function_tool decorator exposes tools to any LLM; tool calls can be forwarded to the frontend.","evidence":[{"type":"doc","title":"LiveKit Agents — tools","url":"https://docs.livekit.io/agents/","quote":"Define tools that are compatible with any LLM, and even forward tool calls to your frontend.","accessed":"2026-05-20"},{"type":"doc","title":"LiveKit Agents — tools page","url":"https://docs.livekit.io/agents/logic/tools","quote":"Let your agents call external tools and more","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"event-driven-agent","role":"first-class","note":"Agent is a realtime participant whose code reacts to room events and audio frames as they arrive.","evidence":[{"type":"doc","title":"LiveKit Agents — realtime participants","url":"https://docs.livekit.io/agents/","quote":"The Agents framework lets you add any Python or Node.js program to LiveKit rooms as full realtime participants.","accessed":"2026-05-20"},{"type":"doc","title":"LiveKit Agents — stateful realtime bridge","url":"https://docs.livekit.io/agents/","quote":"Your agent code operates as a stateful, realtime bridge between powerful AI models and your users.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"conversation-handoff","role":"first-class","note":"A handoff transfers session control from one agent to another; returning a different agent from a tool call triggers automatic handoff and adds an AgentHandoff item to the chat context.","evidence":[{"type":"doc","title":"LiveKit Agents — handoff definition","url":"https://docs.livekit.io/agents/build/agents-handoffs/","quote":"A handoff transfers session control from one agent to another. You can return a different agent from within a tool call to hand off control automatically.","accessed":"2026-05-24"},{"type":"doc","title":"LiveKit Agents — agent transfer control","url":"https://docs.livekit.io/agents/build/agents-handoffs/","quote":"An agent can transfer control to other agents when different logic or capabilities are required.","accessed":"2026-05-24"},{"type":"doc","title":"LiveKit Agents — handoff event in chat context","url":"https://docs.livekit.io/agents/build/agents-handoffs/","quote":"When an agent handoff occurs, an AgentHandoff item is added to the chat context with properties including old_agent_id and new_agent_id.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"session-isolation","role":"first-class","note":"Each room gets its own job subprocess; the agent server boots a job subprocess per room with load balancing.","evidence":[{"type":"doc","title":"LiveKit Agents — worker job model","url":"https://docs.livekit.io/agents/","quote":"agent server boots a 'job' subprocess which joins the room.","accessed":"2026-05-20"},{"type":"doc","title":"LiveKit Agents — dispatch and load balancing","url":"https://docs.livekit.io/agents/worker/","quote":"Assign agents to rooms automatically or explicitly, with load balancing and high concurrency support.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"First-class MCP support via MCPToolset passed to an agent's tools parameter; MCPServerHTTP for remote and MCPServerStdio for local subprocess servers.","evidence":[{"type":"doc","title":"LiveKit Agents — first-class MCP","url":"https://docs.livekit.io/agents/logic/tools/mcp/","quote":"LiveKit Agents has first-class support for Model Context Protocol (MCP) servers.","accessed":"2026-05-24"},{"type":"doc","title":"LiveKit Agents — MCPToolset","url":"https://docs.livekit.io/agents/logic/tools/mcp/","quote":"Wrap an MCP server in an `MCPToolset` and pass it to the agent's `tools` parameter","accessed":"2026-05-24"},{"type":"doc","title":"LiveKit Agents — MCP transports","url":"https://docs.livekit.io/agents/logic/tools/mcp/","quote":"Use `MCPServerStdio` to launch a local MCP server process and communicate over stdin/stdout.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"mandatory-red-flag-escalation","role":"supported","note":"LiveKit's handoff pattern enforces escalation to a human automatically when compliance mandates or rule conditions are met, passing the receiving human full context rather than leaving the transfer to model discretion.","evidence":[{"type":"doc","title":"The Handoff Pattern for Voice Agents That Replaces IVR Menus | LiveKit","url":"https://livekit.com/blog/handoff-pattern-voice-agents","quote":"In healthcare (HIPAA), finance (SOX, PCI), and government (FedRAMP), certain actions legally require human oversight. The handoff pattern enforces this automatically.","accessed":"2026-06-14"}],"evidence_status":"full"}],"alternatives":[{"composition":"pipecat","relation":"competes-with","note":"Peer open Python framework for voice agents."},{"composition":"vapi","relation":"competes-with","note":"Hosted alternative."},{"composition":"elevenlabs-conversational","relation":"competes-with","note":"Hosted alternative."},{"composition":"retell-ai","relation":"competes-with","note":"Hosted phone-agent alternative."},{"composition":"hume-evi","relation":"competes-with","note":"Open framework alternative."}],"references":[{"type":"repo","title":"livekit/agents","url":"https://github.com/livekit/agents","quote":"A framework for building realtime, programmable participants that run on servers.","accessed":"2026-05-20"},{"type":"doc","title":"LiveKit Agents docs","url":"https://docs.livekit.io/agents/","quote":"Realtime framework for voice, video, and physical AI agents.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-14","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","python","nodejs","webrtc","voice","video","apache-2.0"]},{"id":"pipecat","name":"Pipecat","kind":"framework","category":"voice-conversational","build_surface":"full-code","vendor":"Daily / pipecat-ai community","language":"Python","license":"BSD-2-Clause","status":"active","first_released":"2023-12-27","url":"https://www.pipecat.ai","repo":"https://github.com/pipecat-ai/pipecat","docs_url":"https://docs.pipecat.ai","intent":"Open-source Python framework for building real-time voice and multimodal conversational agents by composing frame processors into pipelines that orchestrate STT, LLM, TTS, transports and tools.","description":"Pipecat is an open-source Python framework whose unit of design is a Pipeline of FrameProcessors that exchange Frames (audio, image, text, transcription, transport messages). Built-in services span STT (Deepgram, AssemblyAI, OpenAI Whisper, ...), LLMs (Anthropic, OpenAI, Gemini, ...), TTS (ElevenLabs, Google, Azure, ...) and speech-to-speech models (OpenAI Realtime, Gemini Multimodal Live), plus parallel pipelines, function calling, user-input muting and turn-detection strategies. Pipecat Flows layers conversation flows; the Subagents framework adds multi-agent.","primary_use_cases":["voice assistants and AI companions","multimodal interfaces over WebRTC","interactive storytelling and business agents","complex dialog systems with structured flows"],"agent_loop_shape":"Asynchronous pipeline of frame processors. A transport feeds audio frames in; an STT service converts them to transcripts; an LLM service produces text or tool calls; a TTS service streams audio out via the transport. Control and system frames orchestrate lifecycle. Parallel pipelines fan out with synchronized inputs and outputs. Function-calling frames interleave with the conversation; user-input muting and turn-detection strategies suppress processing during bot responses.","key_concepts":[{"name":"Pipeline","summary":"Composition of FrameProcessors; ParallelPipeline runs branches with synchronized I/O.","url":"https://docs.pipecat.ai"},{"name":"Frames","summary":"Audio, image, text, transcription and transport message frames plus control/system frames.","url":"https://docs.pipecat.ai"},{"name":"FrameProcessor","summary":"Unit of pipeline logic; users write custom processors for domain logic.","url":"https://docs.pipecat.ai"},{"name":"Services","summary":"STT / TTS / LLM / speech-to-speech integrations (50+ providers).","url":"https://docs.pipecat.ai"},{"name":"Function calling","summary":"LLM tool use within the voice pipeline.","maps_to_pattern":"tool-use","url":"https://docs.pipecat.ai"},{"name":"Pipecat Flows","summary":"Structured conversation-flow layer on top of pipelines.","url":"https://docs.pipecat.ai"}],"pattern_composition":"flowchart TD\n  transport[Transport WebRTC/Daily/LiveKit] --> audioin[Audio frames in]\n  audioin --> stt[STT FrameProcessor]\n  stt --> ctx[Context frames]\n  ctx --> llm[LLM FrameProcessor]\n  llm --> fc{Function call?}\n  fc -->|yes| tool[Tool FrameProcessor]\n  tool --> llm\n  fc -->|no| tts[TTS FrameProcessor]\n  tts --> audioout[Audio frames out]\n  audioout --> transport\n  parallel[ParallelPipeline] -.synced I/O.-> llm\n  mute[User input muting] -.turn detection.-> stt\n  flows[Pipecat Flows] -.structured turns.-> llm","members":[{"pattern":"multilingual-voice-agent","role":"supported","note":"Multilingual capability is delegated to chosen STT/TTS/LLM services; framework itself is language-agnostic so any multilingual provider stack works. Pipecat docs do not document a multilingual-specific switch; the capability comes via providers (Deepgram nova-3 multi, Google STT multilingual). Honest limited — no first-party Pipecat quote about a multilingual mode.","evidence":[{"type":"doc","title":"Pipecat docs — language-agnostic pipeline","url":"https://docs.pipecat.ai","quote":"Open source Python framework for building voice and multimodal AI pipelines. Orchestrate 100+ AI services with ultra-low latency.","accessed":"2026-05-24"}],"evidence_status":"limited"},{"pattern":"stop-cancel","role":"first-class","note":"User mute strategies suppress incoming user audio/transcriptions during critical bot operations; configurable interruption strategies gate barge-in.","evidence":[{"type":"doc","title":"Pipecat docs — User Input Muting (mute during function calls)","url":"https://docs.pipecat.ai/guides/fundamentals/user-input-muting","quote":"Mute during function calls. Prevents users from speaking while the bot is processing external data requests.","accessed":"2026-05-24"},{"type":"doc","title":"Pipecat docs — User Input Muting (strict turn-taking)","url":"https://docs.pipecat.ai/guides/fundamentals/user-input-muting","quote":"Mute whenever the bot is speaking. Creates a strict turn-taking conversation pattern.","accessed":"2026-05-24"},{"type":"doc","title":"Pipecat docs — interrupted frames discarded","url":"https://docs.pipecat.ai/api-reference/server/frames/overview","quote":"If a user interrupts (starts speaking while the bot is responding), any pending data frames are discarded so the new input can be handled immediately.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Function calling is a first-class pipeline feature; handlers registered via register_function with per-function timeout and cancel-on-interruption controls.","evidence":[{"type":"doc","title":"Pipecat docs — function calling definition","url":"https://docs.pipecat.ai/pipecat/learn/function-calling","quote":"Function calling allows LLMs to request information from external services and APIs during conversations.","accessed":"2026-05-24"},{"type":"doc","title":"Pipecat docs — register_function","url":"https://docs.pipecat.ai/pipecat/learn/function-calling","quote":"Register handlers for your functions using `register_function` or `register_direct_function`.","accessed":"2026-05-24"},{"type":"doc","title":"Pipecat docs — function call cancellation","url":"https://docs.pipecat.ai/pipecat/learn/function-calling","quote":"Function call is cancelled if user interrupts","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"pipes-and-filters","role":"core","note":"Pipecat's central abstraction is a Pipeline of FrameProcessors that exchange Frames — a textbook pipes-and-filters topology.","evidence":[{"type":"repo","title":"Pipecat README — orchestration","url":"https://github.com/pipecat-ai/pipecat","quote":"Orchestrate audio and video, AI services, different transports, and conversation pipelines effortlessly.","accessed":"2026-05-20"},{"type":"doc","title":"Pipecat docs — ParallelPipeline","url":"https://docs.pipecat.ai","quote":"Run multiple pipeline branches in parallel, with synchronized inputs and outputs for complex flows","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"event-driven-agent","role":"first-class","note":"Pipeline is built on streaming frames; high-priority SystemFrames carry interruptions, user-input, errors and lifecycle events; ControlFrames signal boundaries and config changes.","evidence":[{"type":"doc","title":"Pipecat docs — SystemFrames","url":"https://docs.pipecat.ai/api-reference/server/frames/overview","quote":"SystemFrames are high-priority signals that must always be delivered: interruptions, user input, error notifications, and pipeline lifecycle events.","accessed":"2026-05-24"},{"type":"doc","title":"Pipecat docs — ControlFrames","url":"https://docs.pipecat.ai/api-reference/server/frames/overview","quote":"ControlFrames signal processing boundaries and configuration changes: response start/end markers, settings updates, and state transitions.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"streaming-typed-events","role":"first-class","note":"Frames are explicitly typed (TextFrame, LLMTextFrame, TranscriptionFrame, InterimTranscriptionFrame, OutputAudioRawFrame, TTSAudioRawFrame, OutputImageRawFrame) and stream through processors in guaranteed order.","evidence":[{"type":"doc","title":"Pipecat docs — DataFrames content","url":"https://docs.pipecat.ai/api-reference/server/frames/data-frames","quote":"DataFrames carry the main content flowing through a pipeline: audio chunks, text, images, transcriptions, and messages.","accessed":"2026-05-24"},{"type":"doc","title":"Pipecat docs — TranscriptionFrame","url":"https://docs.pipecat.ai/api-reference/server/frames/data-frames","quote":"A non-interim transcription result from an STT service: the service's best recognition of what the user said.","accessed":"2026-05-24"},{"type":"doc","title":"Pipecat docs — frame ordering","url":"https://docs.pipecat.ai/guides/learn/pipeline","quote":"**Frames are processed in guaranteed order within their processing lane**, even across ParallelPipelines.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"unified-voice-interface","role":"supported","note":"Open-source voice-AI pipeline with provider-agnostic TTS/STT/realtime stages.","evidence":[{"type":"doc","url":"https://github.com/pipecat-ai/pipecat","quote":"Open-source voice-AI pipeline with provider-agnostic TTS/STT/realtime stages.","accessed":"2026-06-19"}],"evidence_status":"full"}],"alternatives":[{"composition":"livekit-agents","relation":"competes-with","note":"Peer open framework for realtime voice agents."},{"composition":"vapi","relation":"competes-with","note":"Hosted alternative."},{"composition":"elevenlabs-conversational","relation":"competes-with","note":"Hosted alternative."},{"composition":"retell-ai","relation":"competes-with","note":"Hosted phone-agent alternative."}],"references":[{"type":"repo","title":"pipecat-ai/pipecat","url":"https://github.com/pipecat-ai/pipecat","quote":"Open-source Python framework for building real-time voice and multimodal conversational agents.","accessed":"2026-05-20"},{"type":"doc","title":"Pipecat docs","url":"https://docs.pipecat.ai","quote":"Open source ecosystem for building voice and multimodal AI agents.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-20","verification_status":"partial","last_updated":"2026-06-14","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","python","bsd-2-clause","voice","multimodal","pipeline"]},{"id":"retell-ai","name":"Retell AI","kind":"framework","category":"voice-conversational","build_surface":"low-code","vendor":"Retell AI","language":"Web product / API","license":"proprietary","status":"active","url":"https://www.retellai.com","docs_url":"https://docs.retellai.com","intent":"Hosted platform for building, testing, deploying and monitoring AI phone agents, with single-prompt, multi-prompt and conversation-flow agent shapes, function calling and call/agent transfers.","description":"Retell AI is a phone-agent platform. Agents can be built as a Single-prompt agent (one open prompt), a Multi-prompt agent (state machine of prompts with transition conditions), or a Conversation Flow agent (explicit node graph with transition conditions). Function Calling, custom functions and code tools let the agent perform real-world actions; Transfer Call and Transfer Agent nodes hand off to humans or to other Retell agents. The dashboard exposes per-language selection with explicit accuracy trade-offs across the supported speech-recognition and voice providers.","primary_use_cases":["lead qualification voice agents over telephony","customer support and dispatch phone agents","receptionist agents with call transfer","multilingual phone agents with language selection"],"agent_loop_shape":"Hosted phone-agent loop. The platform owns the STT, LLM orchestration, TTS and turn-taking layers; the developer authors agent behaviour as a Single-prompt, Multi-prompt (state machine of prompts with transition conditions) or Conversation Flow (node-and-transition graph) agent. Function calls and code tools execute external actions; Transfer Call hands off to humans and Transfer Agent hands off to another Retell agent.","key_concepts":[{"name":"Agent types","summary":"Single-prompt, Multi-prompt (state machine) and Conversation Flow (node graph) agents.","url":"https://docs.retellai.com/build/single-multi-prompt/prompt-overview"},{"name":"Conversation Flow","summary":"Node-and-transition graph; transitions decide which node fires next.","url":"https://docs.retellai.com/build/conversation-flow/overview"},{"name":"Function calling","summary":"Custom functions, code tools and built-in functions for transfers and digits.","maps_to_pattern":"tool-use","url":"https://docs.retellai.com/build/single-multi-prompt/function-calling"},{"name":"Transfer Agent","summary":"Hand off the ongoing call to another Retell agent.","maps_to_pattern":"conversation-handoff","url":"https://docs.retellai.com/build/single-multi-prompt/transfer-agent"},{"name":"Multilingual selection","summary":"Pick exact language combinations; multilingual mode trades accuracy.","maps_to_pattern":"multilingual-voice-agent","url":"https://docs.retellai.com/agent/multilingual"},{"name":"Language constraints","summary":"Language must be supported by both the voice provider and a speech-recognition provider.","url":"https://docs.retellai.com/agent/language"}],"pattern_composition":"flowchart TD\n  caller[Caller] --> retell[Retell hosted pipeline]\n  retell --> kind{Agent type}\n  kind -->|Single prompt| sp[Single prompt agent]\n  kind -->|Multi prompt| mp[Multi prompt state machine]\n  kind -->|Conversation Flow| cf[Node + transition graph]\n  sp --> fc{Function call?}\n  mp --> fc\n  cf --> fc\n  fc -->|custom fn| api[Backend API]\n  fc -->|code tool| code[Code tool]\n  fc -->|transfer call| human[Human via PSTN]\n  fc -->|transfer agent| other[Other Retell agent]\n  api --> retell\n  code --> retell\n  retell --> caller","members":[{"pattern":"conversation-handoff","role":"first-class","note":"Transfer Agent is a documented action that modularises tasks and re-uses specialised agents without traditional phone transfers; Conversation Flow has agent/call transfer nodes.","evidence":[{"type":"doc","title":"Retell docs — transfer agent","url":"https://docs.retellai.com/build/single-multi-prompt/transfer-agent","quote":"Agent Transfer enables you to modularize tasks and re-use specialized agents without relying on traditional phone-based transfers.","accessed":"2026-05-20"},{"type":"doc","title":"Retell docs — conversation flow","url":"https://docs.retellai.com/build/conversation-flow/overview","quote":"Every node defines a small set of logic, and the transition condition is used to determine which node to transition to.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"multilingual-voice-agent","role":"first-class","note":"Per-agent language picker with documented accuracy trade-offs; language must be supported by both the chosen voice and a speech-recognition provider.","evidence":[{"type":"doc","title":"Retell docs — multilingual","url":"https://docs.retellai.com/agent/multilingual","quote":"Pick the specific languages your agent should support, and understand the accuracy trade-offs.","accessed":"2026-05-20"},{"type":"doc","title":"Retell docs — language provider compatibility","url":"https://docs.retellai.com/agent/language","quote":"The language you pick must be supported by both your chosen voice provider (for pronunciation) and at least one speech recognition provider (for transcription).","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Function calling covers built-in actions (transfer, press digit, send SMS) and custom functions / code tools that call external APIs.","evidence":[{"type":"doc","title":"Retell docs — function calling intro","url":"https://docs.retellai.com/build/single-multi-prompt/function-calling","quote":"Function calling transforms your AI agent from a conversational interface into an action-oriented assistant.","accessed":"2026-05-20"},{"type":"doc","title":"Retell docs — custom function","url":"https://docs.retellai.com/build/single-multi-prompt/function-calling","quote":"Enable your AI agent to perform actions like transfers, bookings, and API integrations","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"topic-based-routing","role":"first-class","note":"Multi-prompt agents and Conversation Flow agents are state machines that route the live conversation between prompts/nodes by transition conditions.","evidence":[{"type":"doc","title":"Retell docs — conversation flow transitions","url":"https://docs.retellai.com/build/conversation-flow/overview","quote":"Every node defines a small set of logic, and the transition condition is used to determine which node to transition to.","accessed":"2026-05-20"},{"type":"doc","title":"Retell docs — prompt overview","url":"https://docs.retellai.com/build/single-multi-prompt/prompt-overview","quote":"Choose between single or multi-prompt structures to build AI agents","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"human-in-the-loop","role":"first-class","note":"Transfer Call is a documented action for handing the live call off to a human via PSTN with warm/cold transfer modes.","evidence":[{"type":"doc","title":"Retell docs — function calling lists transfers","url":"https://docs.retellai.com/build/single-multi-prompt/function-calling","quote":"Enable your AI agent to perform actions like transfers, bookings, and API integrations","accessed":"2026-05-20"},{"type":"doc","title":"Retell docs — transfer call use case","url":"https://docs.retellai.com/build/single-multi-prompt/transfer-call","quote":"It is common in call operation to transfer the call to another human agent or another AI agent.","accessed":"2026-05-24"},{"type":"doc","title":"Retell docs — warm transfer","url":"https://docs.retellai.com/build/single-multi-prompt/transfer-call","quote":"After the call is transferred to the destination number, the AI agent can attempt to detect if the other side is human, leave private messages that are not heard by user, do a three-way introduce, etc.","accessed":"2026-05-24"}],"evidence_status":"full"}],"alternatives":[{"composition":"vapi","relation":"competes-with","note":"Peer hosted voice / phone agent platform."},{"composition":"elevenlabs-conversational","relation":"competes-with","note":"Peer hosted voice-agent platform."},{"composition":"livekit-agents","relation":"competes-with","note":"Open framework alternative."},{"composition":"pipecat","relation":"competes-with","note":"Open Python framework alternative."},{"composition":"hume-evi","relation":"competes-with","note":"Peer phone-agent platform."}],"references":[{"type":"doc","title":"Retell docs — introduction","url":"https://docs.retellai.com/general/introduction","quote":"Retell is a comprehensive platform for building, testing, deploying, and monitoring reliable AI phone agents.","accessed":"2026-05-20"},{"type":"blog","title":"Retell AI homepage","url":"https://www.retellai.com","quote":"LLM based, humanlike, voice-first conversational AI platform","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"emerging","tags":["voice","hosted","phone","telephony","state-machine"]},{"id":"vapi","name":"Vapi","kind":"framework","category":"voice-conversational","build_surface":"low-code","vendor":"Vapi","language":"Web product / API","license":"proprietary","status":"active","url":"https://vapi.ai","docs_url":"https://docs.vapi.ai","intent":"Hosted voice AI platform that orchestrates a transcriber, model and voice provider into a phone-callable assistant, with squads for multi-assistant handoff, function-calling tools and multilingual voice agents.","description":"Vapi is positioned as an orchestration layer over three pluggable modules — transcriber (STT), model (LLM) and voice (TTS) — plus a suite of real-time orchestration models for endpointing, interruption handling, backchanneling and emotion detection. Developers configure an Assistant, attach Tools (function calling, custom and code tools, transfer call), and optionally compose Squads where specialised assistants hand off to one another mid-call. Multilingual mode adds automatic language detection.","primary_use_cases":["outbound and inbound phone agents","customer support and lead qualification at scale","appointment scheduling and reception","multi-assistant workflows via squads"],"agent_loop_shape":"Hosted real-time pipeline. Caller audio flows into Vapi's transcriber; the LLM produces a response interleaved with tool calls; a chosen TTS speaks the result back; a suite of real-time models layered on top of STT/LLM/TTS handles endpointing, interruption, backchanneling and emotion. Squads compose multiple specialised assistants and route the live call between them via assistantDestinations.","key_concepts":[{"name":"Assistant","summary":"Configurable agent with transcriber, model, voice and tools.","url":"https://docs.vapi.ai"},{"name":"Orchestration layer","summary":"Vapi sits over transcriber + model + voice and adds real-time models.","url":"https://docs.vapi.ai/quickstart"},{"name":"Squads","summary":"Multiple specialised assistants that hand off mid-call.","maps_to_pattern":"conversation-handoff","url":"https://docs.vapi.ai/squads"},{"name":"Tools","summary":"Function calling, custom tools, code tools and call transfer.","maps_to_pattern":"tool-use","url":"https://docs.vapi.ai/tools"},{"name":"Multilingual","summary":"Multilingual assistants with automatic language detection.","maps_to_pattern":"multilingual-voice-agent","url":"https://docs.vapi.ai/customization/multilingual"},{"name":"Voice pipeline configuration","summary":"Endpointing, interruption handling, backchanneling, filler injection.","url":"https://docs.vapi.ai/customization/voice-pipeline-configuration"}],"pattern_composition":"flowchart TD\n  caller[Caller] --> vapi[Vapi orchestration]\n  vapi --> stt[Transcriber STT]\n  stt --> llm[Model LLM]\n  llm --> tool{Tool call?}\n  tool -->|function| fn[Function / custom / code tool]\n  tool -->|transfer| xfer[Transfer call]\n  tool -->|none| tts[Voice TTS]\n  fn --> llm\n  tts --> caller\n  rt[Real-time models: endpointing/interruption/backchannel/emotion] -.augment.-> stt\n  rt -.augment.-> tts\n  squad[Squad: assistantDestinations] -.routes.-> llm","members":[{"pattern":"conversation-handoff","role":"first-class","note":"Squads let calls hand off between specialised assistants via assistantDestinations.","evidence":[{"type":"doc","title":"Vapi docs — squads","url":"https://docs.vapi.ai/squads","quote":"Squads let you break complex workflows into multiple specialized assistants that hand off to each other during a conversation.","accessed":"2026-05-20"},{"type":"doc","title":"Vapi docs — assistantDestinations","url":"https://docs.vapi.ai/squads-example","quote":"assistantDestinations defines how to transfer the call to another assistant","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"multilingual-voice-agent","role":"first-class","note":"Explicit multilingual mode with automatic language detection across the orchestrated STT/LLM/TTS stack; multiple transcriber providers expose multi/multilingual modes.","evidence":[{"type":"doc","title":"Vapi docs — multilingual","url":"https://docs.vapi.ai/customization/multilingual","quote":"Configure your voice assistant to communicate in multiple languages with automatic language detection, native voice quality, and cultural context awareness.","accessed":"2026-05-24"},{"type":"doc","title":"Vapi docs — multilingual transcriber settings","url":"https://docs.vapi.ai/customization/multilingual","quote":"Set `Multi` (Deepgram), `Multilingual` (Google), or choose the language you want","accessed":"2026-05-24"},{"type":"doc","title":"Vapi docs — multilingual seamless switching","url":"https://docs.vapi.ai/customization/multilingual","quote":"Switch languages seamlessly when the user changes languages","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Four explicit tool types: Default (built-in), Custom (webhooks), Code (TypeScript on Vapi infra), Integration (Make/GHL).","evidence":[{"type":"doc","title":"Vapi docs — tools","url":"https://docs.vapi.ai/tools","quote":"Tools allow your assistant to take actions beyond just conversation. They enable your assistant to perform tasks like transferring calls, accessing external data, or triggering actions in your application.","accessed":"2026-05-24"},{"type":"doc","title":"Vapi docs — Custom Tools","url":"https://docs.vapi.ai/tools","quote":"Custom Tools: Your own functions that can be called by the assistant to interact with your systems via webhooks.","accessed":"2026-05-24"},{"type":"doc","title":"Vapi docs — Code Tools","url":"https://docs.vapi.ai/tools","quote":"Code Tools: Write TypeScript code that executes directly on Vapi's infrastructure without setting up a server.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"stop-cancel","role":"first-class","note":"Voice pipeline configuration covers timing and interruption handling; orchestration models include interruption handling and backchanneling.","evidence":[{"type":"doc","title":"Vapi docs — voice pipeline configuration","url":"https://docs.vapi.ai/customization/voice-pipeline-configuration","quote":"Complete guide to configuring VAPI's voice pipeline for optimal conversation timing and interruption handling","accessed":"2026-05-20"},{"type":"doc","title":"Vapi docs — real-time models","url":"https://docs.vapi.ai/how-vapi-works","quote":"On top of speech-to-text, a language model, and text-to-speech, we run a suite of real-time models that make conversations feel fast, fluid, and human.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"multi-model-routing","role":"first-class","note":"Assistant config selects transcriber + LLM + voice independently; the three modules can be swapped with any provider (OpenAI, Groq, Deepgram, ElevenLabs, PlayHT, etc).","evidence":[{"type":"doc","title":"Vapi docs — orchestration over three modules","url":"https://docs.vapi.ai/quickstart","quote":"At it's core, Vapi is an orchestration layer over three modules: the **transcriber**, the **model**, and the **voice**.","accessed":"2026-05-24"},{"type":"doc","title":"Vapi docs — swap any provider","url":"https://docs.vapi.ai/quickstart","quote":"These three modules can be swapped out with **any provider** of your choosing; OpenAI, Groq, Deepgram, ElevenLabs, PlayHT, etc.","accessed":"2026-05-24"}],"evidence_status":"full"}],"alternatives":[{"composition":"retell-ai","relation":"competes-with","note":"Peer hosted phone-agent platform."},{"composition":"elevenlabs-conversational","relation":"competes-with","note":"Peer hosted voice-agent platform."},{"composition":"livekit-agents","relation":"competes-with","note":"Open framework alternative."},{"composition":"pipecat","relation":"competes-with","note":"Open Python framework alternative."},{"composition":"hume-evi","relation":"competes-with","note":"Peer hosted voice agent platform."}],"references":[{"type":"doc","title":"Vapi docs — quickstart","url":"https://docs.vapi.ai/quickstart","quote":"Build voice AI agents that can make and receive phone calls","accessed":"2026-05-20"},{"type":"doc","title":"Vapi docs — how Vapi works","url":"https://docs.vapi.ai/how-vapi-works","quote":"Learn about the real-time models Vapi runs on top of STT, LLM, and TTS","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-05-24","date_added":"2026-04-30","status_in_practice":"mature","tags":["voice","hosted","phone","telephony","squads","orchestration"]},{"date_added":"2026-06-27","last_analyzed":"2026-06-27","last_updated":"2026-06-27","verification_status":"partial","build_surface":"full-code","status":"active","kind":"framework","id":"moshi","name":"Moshi (Kyutai)","category":"voice-conversational","status_in_practice":"experimental","vendor":"Kyutai","language":"Python, Rust","first_released":"2024-09-17","url":"https://kyutai.org/","repo":"https://github.com/kyutai-labs/moshi","intent":"Full-duplex speech-text foundation model from Kyutai that models the user's and the agent's audio as two parallel streams, handling turn-taking natively in real time instead of via a separate voice-activity-detection pipeline.","description":"Moshi (arXiv 2410.00037) is reported as the first real-time full-duplex spoken large language model, with roughly 200ms practical latency. It generates speech tokens from the residual quantizer of the Mimi streaming neural audio codec, models its own speech and the user's speech in parallel streams, and predicts text tokens (an inner monologue) corresponding to its own speech. This replaces the traditional cascade of voice-activity detection, speech recognition, dialogue, and text-to-speech with a single model.","primary_use_cases":["real-time full-duplex voice conversation","native turn-taking without a separate VAD or endpointing pipeline","low-latency speech-to-speech dialogue"],"agent_loop_shape":"Single full-duplex speech model rather than a cascade. Moshi continuously models two audio streams — its own speech and the user's — in parallel, predicting speech tokens from the Mimi codec plus a text inner monologue. Because both streams run continuously, the model decides when to speak or yield natively, without a separate voice-activity-detection or endpointing stage.","key_concepts":[{"name":"Two parallel audio streams","summary":"Moshi models its own and the user's speech as two streams, enabling simultaneous speech and native turn-taking without separate VAD.","maps_to_pattern":"semantic-turn-endpointing","url":"https://github.com/kyutai-labs/moshi"},{"name":"Mimi neural audio codec","summary":"A streaming neural audio codec; Moshi generates speech tokens from its residual quantizer.","url":"https://github.com/kyutai-labs/moshi"},{"name":"Inner monologue","summary":"Moshi predicts text tokens corresponding to its own speech, which improves generation quality.","url":"https://arxiv.org/abs/2410.00037"}],"pattern_composition":"flowchart TD\n  fw[\"Moshi (Kyutai)\"]\n  fw --> p1[\"Semantic Turn Endpointing<br/>(first-class)\"]\n  fw --> p2[\"Unified Voice Interface<br/>(supported)\"]","tags":["voice-conversational","voice","full-duplex","full-code"],"references":[{"type":"paper","title":"Moshi: a speech-text foundation model for real-time dialogue","url":"https://arxiv.org/abs/2410.00037"},{"type":"repo","title":"kyutai-labs/moshi","url":"https://github.com/kyutai-labs/moshi"}],"members":[{"pattern":"semantic-turn-endpointing","role":"first-class","evidence_status":"full","note":"Moshi handles turn-taking natively via its dual-stream architecture instead of a silence-threshold VAD — the full-duplex realisation of this pattern.","evidence":[{"type":"repo","title":"kyutai-labs/moshi (README)","url":"https://github.com/kyutai-labs/moshi","quote":"Moshi models two streams of audio: one corresponds to Moshi speaking, and the other one to the user speaking.","accessed":"2026-06-27"}]},{"pattern":"unified-voice-interface","role":"supported","evidence_status":"limited","note":"A single speech-text model owns the whole speech loop (recognition, dialogue, and speech generation) rather than a cascade of separate providers.","evidence":[{"type":"paper","title":"Moshi: a speech-text foundation model for real-time dialogue","url":"https://arxiv.org/abs/2410.00037","accessed":"2026-06-27"}]}]},{"id":"inngest-agentkit","name":"Inngest AgentKit","kind":"framework","category":"workflow-engine","build_surface":"full-code","vendor":"Inngest","language":"TypeScript","license":"Apache-2.0","status":"active","first_released":"2024-11-22","url":"https://agentkit.inngest.com/","repo":"https://github.com/inngest/agent-kit","docs_url":"https://agentkit.inngest.com/","intent":"TypeScript framework for composing multi-agent networks where a Router decides which Agent runs next over a shared State, running on Inngest's durable-execution engine for fault-tolerance and human-in-the-loop pauses.","description":"Inngest AgentKit is an open-source TypeScript framework built on top of the Inngest workflow engine. The core primitives are Agent (an LLM call with prompts, tools, and MCP), Tool, Network (a system of agents sharing State), and Router (code-based, LLM-based, or hybrid logic that picks the next Agent each loop). The execution model is a while-loop over State; when AgentKit functions run inside Inngest, each step becomes a durable, replayable unit and `step.waitForEvent` lets agents pause for hours waiting on a human reply.","primary_use_cases":["multi-agent networks with deterministic or hybrid routing","tool-using agents over MCP server registries","long-running agent workflows with human-in-the-loop approval","fault-tolerant production agents on Inngest durable execution"],"agent_loop_shape":"Network-as-while-loop. Each iteration: Router reads State and picks an Agent (or returns undefined to stop); the chosen Agent makes an inference call with its tools and any MCP servers; tool results and messages are appended to shared State; the loop continues until the Router stops or `maxIter` is hit. When deployed on Inngest, every Agent and tool call is wrapped in `step.*` so the run survives crashes, restarts, and arbitrary `waitForEvent` pauses.","key_concepts":[{"name":"Agent","summary":"Stateless wrapper around a model with a goal, prompts, and tools (including MCP).","maps_to_pattern":"tool-use","url":"https://agentkit.inngest.com/concepts/agents"},{"name":"Network","summary":"While-loop with shared State that calls Agents and Tools until the Router stops.","maps_to_pattern":"supervisor","url":"https://agentkit.inngest.com/concepts/networks"},{"name":"Router","summary":"Code-based, LLM-based (default routing agent / ReAct), or hybrid; decides the next Agent.","maps_to_pattern":"supervisor","url":"https://agentkit.inngest.com/concepts/routers"},{"name":"State","summary":"Shared kv store + message history; typed; used by Router for routing decisions.","url":"https://agentkit.inngest.com/concepts/state"},{"name":"MCP-as-tools","summary":"Any MCP server (Streamable HTTP / SSE / WS) plugs in as a tool source.","maps_to_pattern":"tool-use","url":"https://agentkit.inngest.com/advanced-patterns/mcp"},{"name":"waitForEvent / human-in-the-loop","summary":"Inngest step that pauses an Agent's tool for hours waiting for an event.","maps_to_pattern":"approval-queue","url":"https://agentkit.inngest.com/advanced-patterns/human-in-the-loop"}],"pattern_composition":"flowchart TD\n  user[User input] --> net[Network]\n  net --> router{Router}\n  router -->|code-based| pick1[Deterministic choice]\n  router -->|LLM / ReAct| pick2[Default routing agent]\n  router -->|hybrid| pick3[Mix]\n  pick1 --> agent[Agent inference]\n  pick2 --> agent\n  pick3 --> agent\n  agent --> tools[Tools / MCP servers]\n  tools --> state[Append to shared State]\n  state --> hil{Tool needs human?}\n  hil -->|yes| wait[step.waitForEvent]\n  wait --> state\n  hil -->|no| router\n  router -->|stop / maxIter| done[Network result]\n  agent -.runs as.-> inngest[Inngest durable step]\n  inngest -.fault-tolerant.-> done","members":[{"pattern":"agent-resumption","role":"first-class","note":"When deployed on Inngest, AgentKit agents become fault-tolerant durable workflows that survive restarts; `step.waitForEvent` can pause for hours.","evidence":[{"type":"repo","title":"AgentKit README - durable execution","url":"https://github.com/inngest/agent-kit","quote":"Combined with the Inngest Dev Server to start locally and its orchestration engine, making your Agents fault-tolerant when deployed.","accessed":"2026-05-20"},{"type":"doc","title":"AgentKit human-in-the-loop - long wait","url":"https://agentkit.inngest.com/advanced-patterns/human-in-the-loop","quote":"The `ask_developer` tool will wait up to 4 hours for a `developer.response` event to be received","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"approval-queue","role":"first-class","note":"Human-in-the-loop tools built on Inngest `waitForEvent` pause the network until a matching event arrives.","evidence":[{"type":"doc","title":"AgentKit HITL - waitForEvent","url":"https://agentkit.inngest.com/advanced-patterns/human-in-the-loop","quote":"Human in the Loop tools are implemented using Inngest's `waitForEvent()` step method","accessed":"2026-05-20"},{"type":"doc","title":"AgentKit HITL - rationale","url":"https://agentkit.inngest.com/advanced-patterns/human-in-the-loop","quote":"Agents such as Support Agents, Coding or Research Agents might require human oversight.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"event-driven-agent","role":"first-class","note":"Agents react to and wait on Inngest events; the Inngest engine triggers and resumes runs on event arrival.","evidence":[{"type":"doc","title":"AgentKit HITL - event match","url":"https://agentkit.inngest.com/advanced-patterns/human-in-the-loop","quote":"Wait for developer response event","accessed":"2026-05-20"},{"type":"repo","title":"AgentKit README - waitForEvent for humans","url":"https://github.com/inngest/agent-kit","quote":"Inngest's `waitForEvent()` enables the Agent to ask a human for help.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"scheduled-agent","role":"supported","note":"Scheduling is inherited from Inngest (cron functions with timezones and jitter), not surfaced as a first-class AgentKit primitive in the AgentKit docs themselves; honest downgrade from first-class.","evidence":[{"type":"doc","title":"Inngest scheduled functions — native cron","url":"https://www.inngest.com/docs/guides/scheduled-functions","quote":"You can create scheduled jobs using cron schedules within Inngest natively.","accessed":"2026-05-24"},{"type":"doc","title":"Inngest scheduled functions — timezones","url":"https://www.inngest.com/docs/guides/scheduled-functions","quote":"Inngest's cron schedules also support timezones, allowing you to schedule work in whatever timezone you need work to run in.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"step-budget","role":"first-class","note":"Networks support a `maxIter` option to cap loop iterations and avoid infinite loops; networks are while-loops that the router exits when work is done.","evidence":[{"type":"doc","title":"AgentKit networks - maxIter","url":"https://agentkit.inngest.com/concepts/networks","quote":"Specifying a `maxIter` option is useful when using a [Default Routing Agent] or a [Hybrid Router] to avoid infinite loops.","accessed":"2026-05-20"},{"type":"doc","title":"AgentKit networks - while-loop with memory","url":"https://agentkit.inngest.com/concepts/networks","quote":"Networks can be thought of as while loops with memory (State) that call Agents and Tools until the Router determines that there is no more work to be done.","accessed":"2026-05-24"},{"type":"doc","title":"AgentKit networks - maxIter limits iterations","url":"https://agentkit.inngest.com/concepts/networks","quote":"A Network can specify an optional `maxIter` setting to limit the number of iterations.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"supervisor","role":"first-class","note":"Router is the supervisor primitive: code, LLM-based (default routing agent), or hybrid.","evidence":[{"type":"doc","title":"AgentKit networks - router role","url":"https://agentkit.inngest.com/concepts/networks","quote":"A [Router], which chooses whether to stop or select the next agent to run in the loop","accessed":"2026-05-20"},{"type":"repo","title":"AgentKit README - router","url":"https://github.com/inngest/agent-kit","quote":"where the autonomy lives, from code-based to LLM-based (ex: ReAct) orchestration","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Tools are built via function-calling; AgentKit also exposes any MCP server as a tool source.","evidence":[{"type":"doc","title":"AgentKit tools - purpose","url":"https://agentkit.inngest.com/concepts/tools","quote":"Tools are functions that extend the capabilities of an Agent. Tools have two core uses: calling code, enabling models to interact with systems like your own database or external APIs.","accessed":"2026-05-20"},{"type":"doc","title":"AgentKit MCP - any MCP server as tools","url":"https://agentkit.inngest.com/advanced-patterns/mcp","quote":"Using MCP as tools allows you to use any MCP server as a tool in your AgentKit network, enabling your Agent to access thousands of pre-built tools.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"mcp","role":"first-class","note":"First-class MCP client; Streamable HTTP / SSE / WS transports; Smithery registry integration.","evidence":[{"type":"doc","title":"AgentKit MCP - Claude MCP support","url":"https://agentkit.inngest.com/advanced-patterns/mcp","quote":"AgentKit supports using Claude's Model Context Protocol as tools.","accessed":"2026-05-20"},{"type":"doc","title":"AgentKit MCP - transports","url":"https://agentkit.inngest.com/advanced-patterns/mcp","quote":"AgentKit supports configuring MCP servers via Streamable HTTP, SSE or WS transports.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"async-tool-handle","role":"supported","note":"A slow tool/inference step is offloaded so the function pauses rather than holds a blocking connection, and resumes with the result when it lands; each step is an atomic, retriable unit that survives across long-running durable runs.","evidence":[{"type":"doc","title":"inngest-agentkit","url":"https://www.inngest.com/docs/features/inngest-functions/steps-workflows/step-ai-orchestration","quote":"step.ai.infer() offloads the inference request to Inngest's infrastructure, pausing your function execution until the request finishes. ... Once the request finishes, your function restarts with the inference result's data.","accessed":"2026-06-14"}],"evidence_status":"full"},{"pattern":"kill-switch","role":"first-class","note":"Running function (and AgentKit agent) executions can be stopped mid-flight out of band via the REST API, the Dashboard, or by sending a cancellation event, without a redeploy.","evidence":[{"type":"doc","title":"Cancellation - Inngest Documentation","url":"https://www.inngest.com/docs/features/inngest-functions/cancellation","quote":"Inngest enables you to cancel running Functions via the API, Dashboard, or based on events:","accessed":"2026-06-17"}],"evidence_status":"limited"}],"alternatives":[{"composition":"temporal","relation":"competes-with","note":"Durable-execution peer; Temporal is a workflow engine for multiple languages, AgentKit is TS-only and agent-shaped."},{"composition":"restack","relation":"competes-with","note":"Peer durable-agent platform; Restack runs on Temporal/Kubernetes, AgentKit runs on Inngest."}],"references":[{"type":"repo","title":"inngest/agent-kit","url":"https://github.com/inngest/agent-kit","quote":"Build multi-agent networks with deterministic routing and rich tooling via MCP.","accessed":"2026-05-20"},{"type":"doc","title":"AgentKit documentation","url":"https://agentkit.inngest.com/","accessed":"2026-05-20"},{"type":"blog","title":"Introducing AgentKit and step.ai: orchestrating AI with confidence","url":"https://www.inngest.com/blog","quote":"Introducing AgentKit and step.ai: orchestrating AI with confidence","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"verified","last_updated":"2026-06-17","date_added":"2026-04-30","status_in_practice":"emerging","tags":["open-source","typescript","durable-execution","multi-agent","mcp"]},{"id":"modal","name":"Modal","kind":"framework","category":"workflow-engine","build_surface":"full-code","vendor":"Modal Labs","language":"Python","license":"Apache-2.0","status":"active","first_released":"2022","url":"https://modal.com/","repo":"https://github.com/modal-labs/modal-client","docs_url":"https://modal.com/docs","intent":"Serverless compute platform that turns Python functions into autoscaled cloud containers and exposes Sandboxes as ephemeral, secure runtimes for executing AI-generated code on demand.","description":"Modal is an AI infrastructure platform built by Modal Labs. Developers decorate Python functions with `@app.function` (optionally with a GPU, image, schedule, or `modal.Sandbox` reference) and Modal containerises and runs them in the cloud with sub-second cold starts and autoscaling. For agentic workloads the headline primitive is the Sandbox: a constructor (`Sandbox.create`) that programmatically spins up isolated containers to run untrusted or LLM-generated code, with filesystem, directory, and memory snapshots to checkpoint and restore state.","primary_use_cases":["secure sandboxed execution of LLM-generated code","GPU-backed inference and fine-tuning","scheduled / cron-driven serverless jobs","containerised batch compute for agents"],"agent_loop_shape":"Modal is not itself an agent runtime; it is the substrate. Agent frameworks call `Sandbox.create()` to obtain an ephemeral container, send code or shell commands into it, observe stdout/stderr/files, optionally snapshot the filesystem or memory, and either reuse, detach, or terminate the Sandbox. Scheduled functions use the `schedule=modal.Period(...)` or `modal.Cron(...)` parameter on `@app.function` so the entry-point runs on a cadence without external orchestration.","key_concepts":[{"name":"App","summary":"Container for a set of related Modal Functions: `app = modal.App(\"name\")`.","url":"https://modal.com/docs/guide"},{"name":"Function","summary":"Python function decorated with `@app.function(...)`; Modal builds an image and runs it in the cloud.","url":"https://modal.com/docs/guide"},{"name":"Sandbox","summary":"Ephemeral isolated container created at runtime via `Sandbox.create` for executing untrusted/agent code.","maps_to_pattern":"sandbox-isolation","url":"https://modal.com/docs/guide/sandbox"},{"name":"Sandbox snapshots","summary":"Filesystem, directory, and memory snapshots that save and restore Sandbox state.","maps_to_pattern":"agent-resumption","url":"https://modal.com/docs/guide/sandbox-snapshots"},{"name":"Schedule (Period / Cron)","summary":"`schedule=modal.Period(...)` or `modal.Cron(...)` on `@app.function` runs the function on a cadence.","maps_to_pattern":"scheduled-agent","url":"https://modal.com/docs/guide/cron"}],"pattern_composition":"flowchart TD\n  user[Developer code] --> app[modal.App]\n  app --> fn[@app.function]\n  fn -->|schedule=Period/Cron| cron[Recurring run]\n  fn -->|invoked| run[Containerised execution]\n  fn -->|creates| sb[Sandbox.create]\n  sb --> iso[Isolated container]\n  iso --> exec[Run LLM-generated code / shell]\n  exec --> snap{Snapshot?}\n  snap -->|filesystem| fs[Filesystem snapshot]\n  snap -->|memory| mem[Memory snapshot]\n  snap -->|directory| dir[Directory snapshot]\n  snap -->|no| detach[detach / terminate]\n  fs --> restore[Restore later]\n  mem --> restore\n  dir --> restore","members":[{"pattern":"agent-resumption","role":"first-class","note":"Sandbox memory and filesystem snapshots checkpoint state and can be restored as exact clones.","evidence":[{"type":"doc","title":"Modal sandbox-snapshots - purpose","url":"https://modal.com/docs/guide/sandbox-snapshots","quote":"Snapshots allow you to save your Sandbox's state and restore it later","accessed":"2026-05-20"},{"type":"doc","title":"Modal sandbox-snapshots - memory snapshot scope","url":"https://modal.com/docs/guide/sandbox-snapshots","quote":"Sandbox memory snapshots are copies of a Sandbox's entire state, both in memory and on the filesystem","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"code-execution","role":"first-class","note":"Sandboxes are advertised explicitly as the primitive for executing LLM-generated code.","evidence":[{"type":"doc","title":"Modal Sandbox - LLM code","url":"https://modal.com/docs/guide/sandbox","quote":"Execute code generated by a language model","accessed":"2026-05-20"},{"type":"doc","title":"Modal homepage - sandbox positioning","url":"https://modal.com/docs/guide","quote":"Spin up thousands of isolated and secure Sandboxes to execute AI generated code","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"sandbox-isolation","role":"first-class","note":"Sandbox is the headline isolation primitive: ephemeral, secure containers for untrusted code.","evidence":[{"type":"doc","title":"Modal Sandbox - secure containers","url":"https://modal.com/docs/guide/sandbox","quote":"secure containers for executing untrusted user or agent code on Modal","accessed":"2026-05-20"},{"type":"doc","title":"Modal Sandbox - isolated env","url":"https://modal.com/docs/guide/sandbox","quote":"Create isolated environments for running untrusted code","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"scheduled-agent","role":"first-class","note":"`@app.function(schedule=modal.Period(...))` or `modal.Cron(...)` runs a function on a cadence.","evidence":[{"type":"doc","title":"Modal cron - decorator schedule","url":"https://modal.com/docs/guide/cron","quote":"To schedule this function to run once per day, create a Modal App and attach our function to it with the `@app.function` decorator and a schedule parameter","accessed":"2026-05-20"},{"type":"doc","title":"Modal cron - Cron syntax","url":"https://modal.com/docs/guide/cron","quote":"`modal.Cron` gives you finer control using cron syntax","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"async-tool-handle","role":"supported","note":"Modal's job-queue pattern is exactly start-tool-plus-poll-tool: .spawn() returns a FunctionCall handle immediately without waiting, the work runs in the background, and the result is fetched later via FunctionCall.get() or rehydrated from a stored id via FunctionCall.from_id().","evidence":[{"type":"doc","title":"modal","url":"https://modal.com/docs/guide/job-queue","quote":"A FunctionCall is a reference to an executed function call. Constructed using .spawn(...) on a Modal function ... acts as a reference to an ongoing function call that can be passed around and used to poll or fetch function results at some later time. get() waits indefinitely by default. It takes an optional timeout argument ... which can be set to 0 to poll for an output immediately.","accessed":"2026-06-14"}],"evidence_status":"full"}],"alternatives":[{"composition":"temporal","relation":"complements","note":"Modal provides sandboxed compute; Temporal provides durable workflow execution. Common pairing: Temporal workflow invokes Modal Sandboxes."},{"composition":"restack","relation":"complements","note":"Different scopes: Restack is the agent runtime, Modal is the compute substrate."}],"references":[{"type":"doc","title":"Modal documentation","url":"https://modal.com/docs/guide","quote":"Modal is an AI infrastructure platform that lets you: Run low latency inference with sub-second cold starts","accessed":"2026-05-20"},{"type":"repo","title":"modal-labs/modal-client","url":"https://github.com/modal-labs/modal-client","quote":"convenient, on-demand access to serverless cloud compute from Python scripts on your local computer.","accessed":"2026-05-20"},{"type":"doc","title":"Modal Sandbox guide","url":"https://modal.com/docs/guide/sandbox","accessed":"2026-05-20"}],"last_analyzed":"2026-05-20","verification_status":"verified","last_updated":"2026-06-14","date_added":"2026-04-30","status_in_practice":"mature","tags":["python","serverless","sandbox","gpu","scheduled-functions"]},{"id":"restack","name":"Restack","kind":"framework","category":"workflow-engine","build_surface":"full-code","vendor":"Restack","language":"Python, TypeScript","status":"active","first_released":"2024-07-27","url":"https://www.restack.io/","repo":"https://github.com/restackio","docs_url":"https://docs.restack.io/","intent":"Backend platform for building long-running AI agents on top of Temporal and Kubernetes, with workflows-as-code, MCP-exposed tools, ClickHouse-backed context, and a product-team-facing visual interface.","description":"Restack is a vendor-managed agent platform for enterprise teams. Engineers write workflows in Python or TypeScript (`@restackio/ai`) by composing functions called via a `step` primitive; the platform runs these on a Temporal-backed durable engine deployed on Kubernetes so agents can run for weeks or months. Python functions are exposed to the agent as MCP tools, long-term context lives in a ClickHouse store, and product teams manage prompts, versions, and tests through a UI.","primary_use_cases":["long-running enterprise AI agents (weeks-to-months horizons)","durable workflows composed from Python or TypeScript functions","MCP-tool integrations exposing in-house Python code","self-hosted agent deployments on AWS / GCP / Azure / on-prem"],"agent_loop_shape":"Restack agents and workflows are durable Temporal-backed processes. A workflow is a multi-step function that uses the `step` primitive to call user-defined functions (the activities); the framework records each step so that on crash or restart the run resumes from the last committed step. Agents are long-running variants that can wait on events, call MCP tools, pull context from the ClickHouse store, and be scheduled via the platform's Schedules feature. Restack manages the underlying Kubernetes + Temporal stack so users do not run Temporal directly.","key_concepts":[{"name":"Workflow","summary":"Multi-step orchestration coordinating model calls, API requests, and logic with durable state.","maps_to_pattern":"durable-workflow-snapshot","url":"https://docs.restack.io/features/workflows"},{"name":"Step / Function","summary":"`step()` primitive inside a workflow calls user-defined functions (the activities of the run).","maps_to_pattern":"tool-use","url":"https://docs.restack.io/features/workflows"},{"name":"Agent","summary":"Long-running variant of a workflow that handles errors, retries, and interruptions automatically.","maps_to_pattern":"agent-resumption","url":"https://docs.restack.io/architecture"},{"name":"Context store","summary":"Built-in long-term memory powered by ClickHouse; stores past interactions and retrievable context.","url":"https://docs.restack.io/architecture"},{"name":"MCP-out-of-the-box","summary":"Python functions are exposed to the agent as MCP tools by default.","maps_to_pattern":"mcp","url":"https://docs.restack.io/architecture"},{"name":"Schedules","summary":"Cron / calendar schedules on workflows, agents, and functions.","maps_to_pattern":"scheduled-agent","url":"https://docs.restack.io/features/schedules"}],"pattern_composition":"flowchart TD\n  ui[Product UI: prompts, versions, tests] --> agent[Agent / Workflow]\n  agent --> step[step primitive]\n  step --> fn[User Python or TS function]\n  fn -.exposed as.-> mcp[MCP tool]\n  agent --> ctx[Context store: ClickHouse]\n  agent --> sched[Schedules: cron / calendar]\n  agent -.runs on.-> dur[Durable Execution]\n  dur -.powered by.-> temporal[Temporal + Kubernetes]\n  fail[Crash / restart] -.replay.-> agent","members":[{"pattern":"agent-resumption","role":"first-class","note":"Durable Execution lets Restack agents run for months, handling errors, retries, and interruptions automatically.","evidence":[{"type":"doc","title":"Restack architecture - durable execution","url":"https://docs.restack.io/architecture","quote":"Durable Execution: Agents can run for months, handling errors, retries, and interruptions automatically.","accessed":"2026-05-20"},{"type":"doc","title":"Restack workflows - long horizon","url":"https://docs.restack.io/features/workflows","quote":"Build AI applications with a system capable of coordinating steps like calling models, making API requests, and executing logic, all while maintaining state over weeks or even months reliably.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"event-driven-agent","role":"supported","note":"Workflows can be scheduled from agents or other workflows; Restack exposes webhooks. No verbatim quote on event-driven agent triggers in the pages successfully fetched; honest downgrade from first-class.","evidence":[{"type":"doc","title":"Restack workflows - schedule from agents","url":"https://docs.restack.io/features/workflows","quote":"Agents can call workflows.","accessed":"2026-05-20"}],"evidence_status":"limited"},{"pattern":"scheduled-agent","role":"first-class","note":"Restack has a dedicated Schedules feature for workflows, agents, and functions; workflows can also be scheduled programmatically via `scheduleWorkflow`.","evidence":[{"type":"doc","title":"Restack workflows - scheduleWorkflow","url":"https://docs.restack.io/features/workflows","quote":"Any client can schedule the workflow to run.","accessed":"2026-05-20"},{"type":"doc","title":"Restack llms.txt - schedules feature","url":"https://docs.restack.io/llms.txt","quote":"Schedule workflows, agents, and functions with calendars and cron jobs","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"step-budget","role":"supported","note":"Workflows compose via the `step` primitive but no explicit step-budget cap is documented in the pages fetched; honest downgrade from first-class.","evidence":[],"evidence_status":"none"},{"pattern":"supervisor","role":"supported","note":"Workflows orchestrate functions and can call sub-workflows from an agent. No explicit router/supervisor primitive equivalent to AgentKit's was found on fetched pages; honest downgrade from first-class.","evidence":[{"type":"doc","title":"Restack workflows - workflow orchestration","url":"https://docs.restack.io/features/workflows","quote":"The workflow resides in a separate file using the step primitive to orchestrate the functions.","accessed":"2026-05-20"}],"evidence_status":"limited"},{"pattern":"tool-use","role":"first-class","note":"MCP is the default tool surface; Python functions are exposed as tools the agent can call.","evidence":[{"type":"doc","title":"Restack architecture - MCP","url":"https://docs.restack.io/architecture","quote":"MCP Out of the Box: Restack uses the Model Context Protocol to expose Python functions as tools the agent can use.","accessed":"2026-05-20"}],"evidence_status":"limited"},{"pattern":"mcp","role":"first-class","note":"MCP is the integration mechanism: Restack uses MCP to expose Python functions as agent tools.","evidence":[{"type":"doc","title":"Restack architecture - integrations","url":"https://docs.restack.io/architecture","quote":"Standard Python with MCP. Engineers build any integration using standard Python.","accessed":"2026-05-20"},{"type":"doc","title":"Restack architecture - MCP exposure","url":"https://docs.restack.io/architecture","quote":"Restack uses the Model Context Protocol to expose Python functions as tools the agent can use.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"durable-workflow-snapshot","role":"first-class","note":"Built on Temporal and Kubernetes; Durable Execution is the marquee infrastructure property.","evidence":[{"type":"doc","title":"Restack architecture - infrastructure","url":"https://docs.restack.io/architecture","quote":"Powered by Kubernetes and Temporal. Built for agents that run for weeks or months.","accessed":"2026-05-20"},{"type":"doc","title":"Restack architecture - no DevOps","url":"https://docs.restack.io/architecture","quote":"Restack manages all infrastructure and upgrades. No DevOps team required.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"async-tool-handle","role":"supported","note":"An agent can hand slow work to a child workflow via childStart, which returns a durable workflow handle at once instead of blocking; the agent continues and collects the outcome through the handle later.","evidence":[{"type":"doc","title":"restack","url":"https://docs.restack.io/libraries/typescript/agents","quote":"The child workflow executes independently of the agent. This method doesn't wait for the child workflow to finish before returning a promise; instead, it returns the child workflow handle.","accessed":"2026-06-14"}],"evidence_status":"full"}],"alternatives":[{"composition":"temporal","relation":"wraps","note":"Restack runs on Temporal under the hood and adds an agent-shaped product layer, ClickHouse context, and a UI."},{"composition":"inngest-agentkit","relation":"competes-with","note":"Peer durable-agent framework; AgentKit uses Inngest, Restack uses Temporal."},{"composition":"modal","relation":"complements","note":"Different scopes: Restack is the agent runtime, Modal is the compute substrate."}],"references":[{"type":"doc","title":"Restack architecture","url":"https://docs.restack.io/architecture","quote":"Powered by Kubernetes and Temporal. Built for agents that run for weeks or months.","accessed":"2026-05-20"},{"type":"doc","title":"Restack workflows feature","url":"https://docs.restack.io/features/workflows","accessed":"2026-05-20"},{"type":"repo","title":"restackio organisation","url":"https://github.com/restackio","quote":"The backend framework for accurate & reliable AI products","accessed":"2026-05-20"}],"last_analyzed":"2026-05-20","verification_status":"partial","last_updated":"2026-06-14","date_added":"2026-04-30","status_in_practice":"emerging","tags":["durable-execution","temporal-based","kubernetes","mcp","enterprise"]},{"id":"temporal","name":"Temporal","kind":"framework","category":"workflow-engine","build_surface":"full-code","vendor":"Temporal Technologies","language":"Go, Java, Python, TypeScript, .NET, PHP, Ruby","license":"MIT","status":"active","first_released":"2019-10-16","url":"https://temporal.io/","repo":"https://github.com/temporalio/temporal","docs_url":"https://docs.temporal.io/","intent":"Open-source durable-execution platform whose Workflows survive crashes, restarts, and infrastructure outages by replaying an event-sourced history; increasingly adopted as the substrate for long-running AI agents.","description":"Temporal is a durable-execution platform originating as a fork of Uber's Cadence and built by Temporal Technologies. The server (Go) coordinates workflows and activities; SDKs in Go, Java, Python, TypeScript, .NET, PHP, and Ruby let developers write Workflow Definitions (deterministic functions) that invoke Activities (idempotent side-effecting functions). The Temporal Service persists every command and event in a per-workflow Event History; when a Worker crashes or restarts, Temporal replays the history to reconstruct in-memory state and continues from the last unfinished step. Signals, Queries, Updates, Schedules, Retry Policies, and the Saga compensation pattern are first-class primitives. Agent frameworks adopt Temporal to make agent loops crash-proof.","primary_use_cases":["durable workflows for long-running, fault-tolerant business processes","AI-agent backbones with crash-proof tool loops","saga / compensation across distributed services","scheduled and signal-driven recurring jobs"],"agent_loop_shape":"Workflow-as-code with deterministic replay. The agent loop is written as a Workflow Definition; each model call, tool call, or side effect runs as an Activity with built-in retries and timeouts. The Workflow's Event History is the source of truth: on Worker restart Temporal replays history step by step to rebuild the in-memory state, then resumes execution. Signals drive asynchronous inputs (user messages, approvals); Updates handle synchronous request/response; Queries inspect state without changing it; Schedules trigger periodic runs; Saga lists of compensating Activities roll back on failure.","key_concepts":[{"name":"Workflow","summary":"Deterministic code defining a sequence of steps; the Event History is replayed to reconstruct state.","maps_to_pattern":"durable-workflow-snapshot","url":"https://docs.temporal.io/workflows"},{"name":"Activity","summary":"Normal function executing a single well-defined action; auto-retried, can be non-deterministic, should be idempotent.","maps_to_pattern":"tool-use","url":"https://docs.temporal.io/activities"},{"name":"Signal / Query / Update","summary":"Asynchronous writes, read-only inspections, and synchronous tracked writes against a running Workflow.","maps_to_pattern":"event-driven-agent","url":"https://docs.temporal.io/encyclopedia/workflow-message-passing"},{"name":"Retry Policy","summary":"Declarative retry settings on Activities and Workflows; Activities retry by default.","maps_to_pattern":"fallback-chain","url":"https://docs.temporal.io/encyclopedia/retry-policies"},{"name":"Saga / Compensation","summary":"Maintain a list of compensating Activities, executed in reverse on failure.","maps_to_pattern":"compensating-action","url":"https://docs.temporal.io/develop/python/failure-detection"},{"name":"Schedule","summary":"Periodic / cron triggers for Workflow Executions.","maps_to_pattern":"scheduled-agent"}],"pattern_composition":"flowchart TD\n  user[Agent / business code] --> wf[Workflow Definition]\n  wf --> act[Activity calls]\n  act --> ext[LLM / tool / API]\n  act --> retry{Activity failure?}\n  retry -->|yes| policy[Retry Policy]\n  policy --> act\n  retry -->|no| hist[Event History append]\n  wf --> sig[Signal / Update inbox]\n  sig --> hist\n  wf --> q[Query read-only]\n  hist --> persist[Temporal Service persists]\n  crash[Worker crash] -.replay.-> wf\n  saga[Saga compensations] -.on failure.-> reverse[Execute in reverse order]\n  sched[Schedule] -.triggers.-> wf","members":[{"pattern":"agent-resumption","role":"first-class","note":"Workflows pick up exactly where they left off; the engine replays Event History to reconstruct state.","evidence":[{"type":"doc","title":"Temporal home - durable execution","url":"https://temporal.io/","quote":"Temporal Workflows automatically capture state at every step, and in the event of failure, can pick up exactly where they left off.","accessed":"2026-05-20"},{"type":"doc","title":"Temporal workflows - replay","url":"https://docs.temporal.io/workflows","quote":"Temporal doesn't restore memory from a snapshot. It starts the Workflow code from the beginning, replays the Event History step by step.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"compensating-action","role":"first-class","note":"Saga pattern documented as a list of compensating Activities executed in reverse order on failure.","evidence":[{"type":"doc","title":"Temporal saga - definition","url":"https://docs.temporal.io/develop/python/failure-detection","quote":"The Saga pattern coordinates a sequence of operations where each operation has a compensating action to undo its effects.","accessed":"2026-05-20"},{"type":"doc","title":"Temporal saga - reverse order","url":"https://docs.temporal.io/develop/python/failure-detection","quote":"Execute compensating actions in reverse order to roll back previous operations.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"event-driven-agent","role":"first-class","note":"Signals (async writes), Updates (sync tracked writes), and Queries (reads) drive running Workflows.","evidence":[{"type":"doc","title":"Temporal message-passing - Signals","url":"https://docs.temporal.io/encyclopedia/workflow-message-passing","quote":"Signals are asynchronous write requests. They cause changes in the running Workflow, but you cannot await any response or error.","accessed":"2026-05-20"},{"type":"doc","title":"Temporal message-passing - Updates","url":"https://docs.temporal.io/encyclopedia/workflow-message-passing","quote":"Updates are synchronous, tracked write requests. The sender of the Update can wait for a response on completion or an error on failure.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"fallback-chain","role":"first-class","note":"Activities retry by default per a declarative Retry Policy; failures fall through retry intervals before surfacing.","evidence":[{"type":"doc","title":"Temporal retry-policies - definition","url":"https://docs.temporal.io/encyclopedia/retry-policies","quote":"A Retry Policy is a collection of settings that tells Temporal how and when to try again after something fails in a Workflow Execution or Activity Task Execution.","accessed":"2026-05-20"},{"type":"doc","title":"Temporal retry-policies - default activity retries","url":"https://docs.temporal.io/encyclopedia/retry-policies","quote":"Temporal's default behavior is to automatically retry an Activity that fails, so transient or intermittent failures require no action on your part.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"replan-on-failure","role":"supported","note":"Temporal provides the substrate for replanning: ApplicationError with non_retryable=True bypasses retries and surfaces failure to the Workflow, which can then catch ActivityError and choose a new branch. Temporal does not prescribe a replanning algorithm — it provides the catch-and-branch substrate, so role stays at 'supported'.","evidence":[{"type":"doc","title":"Temporal retry-policies - non-retryable","url":"https://docs.temporal.io/encyclopedia/retry-policies","quote":"Non-Retryable Errors specify errors that shouldn't be retried.","accessed":"2026-05-20"},{"type":"doc","title":"Temporal Python - ApplicationError non_retryable","url":"https://docs.temporal.io/develop/python/failure-detection","quote":"An `ApplicationError` with `non_retryable=True` will never retry, regardless of the Retry Policy.","accessed":"2026-05-24"},{"type":"doc","title":"Temporal Python - catching activity failures","url":"https://docs.temporal.io/develop/python/failure-detection","quote":"Use Python's `try/except` blocks to handle Activity failures in your Workflow","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"scheduled-agent","role":"first-class","note":"Temporal Schedules are a first-class primitive for periodic / cron-driven Workflow Executions.","evidence":[{"type":"doc","title":"Temporal docs - SDK list","url":"https://docs.temporal.io/encyclopedia/temporal-sdks","quote":"Despite supporting multiple languages, and supporting many features, Temporal SDKs aim to make developers feel at home in their language.","accessed":"2026-05-20"},{"type":"doc","title":"Temporal home - state and timers","url":"https://temporal.io/","quote":"The Temporal Service persists the state of your application and has built-in retries, task queues, signals, and timers","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"session-isolation","role":"first-class","note":"Each Workflow Execution has exclusive access to its local state; executions are uniquely identified and run concurrently and independently, communicating only via Signals and Activities.","evidence":[{"type":"doc","title":"Temporal workflows - history per execution","url":"https://docs.temporal.io/workflows","quote":"Each Workflow Execution emits a series of Commands and processes a sequence of Events, which are recorded in an Event History.","accessed":"2026-05-20"},{"type":"doc","title":"Temporal workflow-execution - exclusive local state","url":"https://docs.temporal.io/workflow-execution","quote":"Each Temporal Workflow Execution has exclusive access to its local state.","accessed":"2026-05-24"},{"type":"doc","title":"Temporal workflow-execution - concurrent and independent","url":"https://docs.temporal.io/workflow-execution","quote":"It executes concurrently to all other Workflow Executions, and communicates with other Workflow Executions through Signals and the environment through Activities.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"step-budget","role":"supported","note":"Activity timeouts (Schedule-To-Close, Start-To-Close, Schedule-To-Start, Heartbeat) and retry-policy max attempts function as per-step budgets; the engine does not surface a global step-count budget across the whole Workflow.","evidence":[{"type":"doc","title":"Temporal home - timeouts and retries","url":"https://temporal.io/","quote":"Activities have built-in support for timeouts and retries!","accessed":"2026-05-20"},{"type":"doc","title":"Temporal - Schedule-To-Close Timeout","url":"https://docs.temporal.io/encyclopedia/detecting-activity-failures","quote":"The maximum amount of time allowed for the overall Activity Execution, from when the first Activity Task is scheduled to when the last Activity Task reaches a Closed status.","accessed":"2026-05-24"},{"type":"doc","title":"Temporal - Start-To-Close Timeout","url":"https://docs.temporal.io/encyclopedia/detecting-activity-failures","quote":"The maximum time allowed for a single Activity Task Execution.","accessed":"2026-05-24"}],"evidence_status":"full"},{"pattern":"tool-use","role":"first-class","note":"Activities are the tool-use primitive: idempotent functions that call out to models, APIs, databases, MCP servers, etc.","evidence":[{"type":"doc","title":"Temporal activities - definition","url":"https://docs.temporal.io/activities","quote":"An Activity is a normal function or method that executes a single, well-defined action (either short or long running)","accessed":"2026-05-20"},{"type":"doc","title":"Temporal home - activities as interactions","url":"https://temporal.io/","quote":"Temporal treats these interactions as Activities: functions that retry automatically and recover seamlessly.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"durable-workflow-snapshot","role":"first-class","note":"Event-sourced Workflow with deterministic replay is the headline feature.","evidence":[{"type":"doc","title":"Temporal workflows - deterministic","url":"https://docs.temporal.io/workflows","quote":"It has to make the same decisions when given the same history, which makes a Workflow deterministic.","accessed":"2026-05-20"},{"type":"doc","title":"Temporal workflows - history as source of truth","url":"https://docs.temporal.io/workflows","quote":"This history is the source of truth for everything that happens in the Workflow.","accessed":"2026-05-20"}],"evidence_status":"full"},{"pattern":"journaled-llm-call","role":"core","note":"Non-deterministic work runs in Activities whose results are recorded in event history and replayed on recovery, so workflow replay reuses them instead of re-executing.","evidence":[{"type":"doc","title":"Understanding Temporal - Temporal Platform Documentation","url":"https://docs.temporal.io/evaluate/understanding-temporal","quote":"The Temporal Service maintains a meticulous record of every step in your Workflows. By keeping a history of every step in your Workflow, it ensures that even if something goes wrong your Workflow can continue from the last successful point.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"async-tool-handle","role":"supported","note":"Temporal runs each tool call as a durable Activity that runs off the request path and is checkpointed; a workflow handle survives crashes and is resumed exactly where it left off, which is the durable handle plus result store the pattern describes.","evidence":[{"type":"doc","title":"temporal","url":"https://temporal.io/blog/of-course-you-can-build-dynamic-ai-agents-with-temporal","quote":"Your Activities are where the actual work happens: calling LLMs, invoking tools, making API requests. ... if the Workflow has to recover from a crash, it 'replays' your agent's progress to date, preserving previous tool call results via Event History.","accessed":"2026-06-14"}],"evidence_status":"full"},{"pattern":"phantom-action-completion","role":"supported","note":"","evidence":[{"type":"doc","title":"Activity Execution - Temporal Platform Documentation","url":"https://docs.temporal.io/activity-execution","quote":"Temporal guarantees that an Activity Task either runs or timeouts.","accessed":"2026-06-17"},{"type":"doc","title":"Understanding Temporal - Worker crash replay","url":"https://docs.temporal.io/evaluate/understanding-temporal","quote":"If the Worker crashes, the Worker uses the Event History to replay the code and recreate the state of the Workflow Execution to what it was immediately before the crash.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"kill-switch","role":"first-class","note":"A running Workflow Execution can be halted out of band via the CLI, UI, or API by cancelling it (graceful) or terminating it (forceful), with no code change or redeploy.","evidence":[{"type":"doc","title":"Interrupt a Workflow - Cancellation and Termination","url":"https://docs.temporal.io/evaluate/development-production-features/interrupt-workflow","quote":"Discover how Temporal enables you to gracefully handle Workflow interruptions through cancellations and terminations.","accessed":"2026-06-17"},{"type":"doc","title":"Interrupt a Workflow - Cancellation and Termination","url":"https://docs.temporal.io/evaluate/development-production-features/interrupt-workflow","quote":"Understand how to stop a Workflow cleanly with cancellation, allowing for proper cleanup and state management. For situations where a Workflow is stuck, termination provides an immediate solution, ensuring your applications remain robust and responsive.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"stateless-reducer-agent","role":"core","note":"Decouples a durable event-sourced workflow (immutable Event History) from stateless workers that execute one step and report back; the workflow is reconstructed by replaying the log, enabling pause/resume/replay.","evidence":[{"type":"blog","title":"Durable Execution meets AI: Why Temporal is ideal for AI agents","url":"https://temporal.io/blog/durable-execution-meets-ai-why-temporal-is-the-perfect-foundation-for-ai","quote":"Temporal activities facilitate durability through event sourcing, persisting each workflow step as an immutable event so that completed actions aren't re-executed. ... Agent Workers are a fleet of stateless processes whose only job is to ask the Temporal Cluster for work, execute a single step ... and report the result back.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"scatter-gather-saga","role":"supported","note":"Temporal ships the saga pattern: register compensations per step and run them (including in parallel) when a later step fails.","evidence":[{"type":"doc","url":"https://temporal.io/blog/saga-pattern-made-easy","quote":"saga.addCompensation(activities::cancelHotel, info.getClientId()); activities.bookHotel(info);","accessed":"2026-06-19"}],"evidence_status":"full"}],"alternatives":[{"composition":"restack","relation":"wrapped-by","note":"Restack is a managed agent platform built on top of Temporal."},{"composition":"inngest-agentkit","relation":"competes-with","note":"AgentKit on Inngest is a TS-only peer durable-execution stack."},{"composition":"modal","relation":"complements","note":"Modal supplies sandboxed compute; Temporal supplies the durable control plane."}],"references":[{"type":"doc","title":"Temporal homepage","url":"https://temporal.io/","quote":"Build applications that never lose state, even when everything else fails.","accessed":"2026-05-20"},{"type":"doc","title":"Temporal documentation","url":"https://docs.temporal.io/","accessed":"2026-05-20"},{"type":"repo","title":"temporalio/temporal","url":"https://github.com/temporalio/temporal","quote":"Temporal is a durable execution platform that enables developers to build scalable applications without sacrificing productivity or reliability.","accessed":"2026-05-20"},{"type":"doc","title":"Temporal home - durable AI agents","url":"https://temporal.io/","quote":"Develop agents that survive real-world chaos, reliable MCP & orchestrate training pipelines.","accessed":"2026-05-20"}],"last_analyzed":"2026-05-24","verification_status":"needs-verification","last_updated":"2026-06-19","date_added":"2026-04-30","status_in_practice":"mature","tags":["open-source","durable-execution","event-sourced","polyglot-sdks","saga"],"anti_patterns_avoided":[{"pattern":"cascading-agent-failures","note":"Per-activity automatic retries plus durable history isolate a failed step and resume from the last successful point instead of cascading the failure.","evidence":[{"type":"doc","title":"Temporal — understanding Temporal","url":"https://docs.temporal.io/evaluate/understanding-temporal","quote":"If an Activity fails, Temporal automatically retries it based on your configuration. ... By keeping a history of every step in your Workflow, it ensures that even if something goes wrong your Workflow can continue from the last successful point.","accessed":"2026-06-19"}]},{"pattern":"ghost-delegation","note":"Start-To-Close timeouts and activity heartbeats detect a worker that has silently stopped making progress, so a delegated task cannot vanish unnoticed.","evidence":[{"type":"doc","title":"Temporal — detecting activity failures","url":"https://docs.temporal.io/encyclopedia/detecting-activity-failures","quote":"An Activity Heartbeat is a ping from the Worker that is executing the Activity to the Temporal Service. Each ping informs the Temporal Service that the Activity Execution is making progress and the Worker has not crashed.","accessed":"2026-06-19"}]},{"pattern":"missing-idempotency","note":"Durable execution uses idempotency keys so a retried external operation returns the original result instead of performing it twice.","evidence":[{"type":"doc","title":"Temporal — idempotency and durable execution","url":"https://temporal.io/blog/idempotency-and-durable-execution","quote":"Most external services support idempotency keys -- unique identifiers that prevent duplicate operations. When the service receives a request with a key it has already processed, it returns the original result instead of performing the operation again.","accessed":"2026-06-19"}]},{"pattern":"replay-divergence","note":"Workflow code must be deterministic on replay; non-deterministic LLM/API calls go in Activities that execute outside the replay path, so a replay does not diverge from the original run.","evidence":[{"type":"doc","title":"Temporal — workflow definition","url":"https://docs.temporal.io/workflow-definition","quote":"Workflow code must be deterministic to support replay. ... To handle non-deterministic operations like API calls, LLM/AI invocations, database queries, and other external interactions, put them in Activities. Activities execute outside the replay path and are automatically retried so they do not cause non-determinism errors.","accessed":"2026-06-19"}]},{"pattern":"naive-retry-without-backoff","note":"The default retry policy uses exponential backoff with a 2.0 coefficient and a capped maximum interval, instead of fixed-rate retries.","evidence":[{"type":"doc","title":"Temporal — retry policies","url":"https://docs.temporal.io/encyclopedia/retry-policies","quote":"The default Retry Policy uses exponential backoff with a 2.0 backoff coefficient, starting with a 1-second initial interval and capping at a maximum interval of 100 seconds.","accessed":"2026-06-19"}]}]},{"id":"pathway","name":"Pathway","kind":"framework","category":"workflow-engine","vendor":"Pathway","language":"Python, Rust","license":"BSL-1.1","status":"active","first_released":"2023","url":"https://pathway.com/","repo":"https://github.com/pathwaycom/pathway","docs_url":"https://pathway.com/developers/","intent":"Pathway is a Python and Rust framework that builds streaming data and RAG pipelines on a differential-dataflow engine, keeping vector indexes synchronised with their sources as data changes.","description":"Pathway runs the same pipeline code over batch and streaming data, with a Rust engine that recomputes results incrementally as inputs change. Its LLM extension provides parsers, embedders, splitters, and an in-memory real-time vector index, plus document-store components that re-index automatically on new data. Teams use it to build retrieval-augmented generation pipelines whose indexes stay current without a separate batch re-indexing job. The framework integrates with LlamaIndex and LangChain.","primary_use_cases":["real-time retrieval-augmented generation pipelines","live document indexing and vector search","incremental ETL over streaming and batch data","continuously synchronised knowledge bases for agents"],"agent_loop_shape":"Pathway is a data-pipeline runtime rather than an agent loop. Connectors ingest documents and change events; the differential-dataflow engine parses, chunks, embeds, and indexes them incrementally; and a vector or document store serves retrieval queries. A question-answering layer retrieves the top documents for a query and passes them to an LLM, with the index kept in sync as sources change.","build_surface":"full-code","members":[{"pattern":"streaming-feature-pipeline","role":"core","note":"A Python/Rust differential-dataflow framework that processes documents into RAG features incrementally as a continuous stream and keeps a live vector index synchronised with sources in real time, no batch re-indexing.","evidence":[{"type":"doc","title":"Why Pathway","url":"https://pathway.com/developers/user-guide/introduction/why-pathway/","quote":"real-time data indexes allowing you to effortlessly synchronize your index with data sources in real time","accessed":"2026-06-17"},{"type":"repo","title":"pathwaycom/pathway README","url":"https://github.com/pathwaycom/pathway","quote":"Pathway Live Data Framework provides an LLM extension with all the utilities to integrate LLMs with your data pipelines (LLM wrappers, parsers, embedders, splitters), including an in-memory real-time Vector Index, and integrations with LLamaIndex and LangChain.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"agentic-rag","role":"supported","note":"The LLM extension ships question-answering components that retrieve the k best documents and pass them to an LLM, plus an AdaptiveRAGQuestionAnswerer that adjusts retrieval to optimise token usage, on top of a document store that re-indexes itself as new data arrives.","evidence":[{"type":"doc","title":"LLM xpack overview","url":"https://pathway.com/developers/user-guide/llm-xpack/overview/","quote":"Retrieval Augmented Generation","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"naive-rag","role":"supported","note":"Pathway's BaseRAGQuestionAnswerer is a standard retrieve-then-generate RAG: given a query it pulls the k best documents from the vector store and sends them with the question to the LLM, with knowledge living in the external index rather than the model's parameters.","evidence":[{"type":"doc","title":"LLM xpack overview | Pathway","url":"https://pathway.com/developers/user-guide/llm-xpack/overview/","quote":"BaseRAGQuestionAnswerer is a standard RAG, that given a query obtains `k` best documents from the vector store, and sends them along the question to the LLM chat.","accessed":"2026-06-17"}],"evidence_status":"limited"}],"verification_status":"partial","status_in_practice":"emerging","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","key_concepts":[{"name":"DocumentStore","summary":"A pipeline component that automatically indexes documents and updates itself when new data arrives, so the served index reflects the source without a separate batch re-indexing job.","maps_to_pattern":"streaming-feature-pipeline","url":"https://pathway.com/developers/user-guide/llm-xpack/overview/"},{"name":"Differential dataflow engine","summary":"A scalable Rust engine that performs incremental computation, recomputing only what changed as inputs update, and runs the same Python pipeline code over both batch and streaming data.","url":"https://github.com/pathwaycom/pathway"},{"name":"In-memory real-time Vector Index","summary":"A vector index kept inside the running pipeline that stays synchronised with data sources in real time, serving similarity queries for RAG over live documents.","maps_to_pattern":"vector-memory","url":"https://github.com/pathwaycom/pathway"},{"name":"AdaptiveRAGQuestionAnswerer","summary":"A RAG question-answerer that limits the number of documents sent to the LLM chat to save tokens, adjusting retrieval to the query.","maps_to_pattern":"agentic-rag","url":"https://pathway.com/developers/user-guide/llm-xpack/overview/"}],"references":[{"type":"repo","title":"pathwaycom/pathway README","url":"https://github.com/pathwaycom/pathway","quote":"Pathway Live Data Framework provides an LLM extension with all the utilities to integrate LLMs with your data pipelines (LLM wrappers, parsers, embedders, splitters), including an in-memory real-time Vector Index, and integrations with LLamaIndex and LangChain.","accessed":"2026-06-17"},{"type":"doc","title":"Why Pathway","url":"https://pathway.com/developers/user-guide/introduction/why-pathway/","quote":"Pathway Live Data Framework offers real-time data indexes (vector search, full text search, and more) allowing you to effortlessly synchronize your index with data sources in real time.","accessed":"2026-06-17"},{"type":"doc","title":"LLM xpack overview | Pathway","url":"https://pathway.com/developers/user-guide/llm-xpack/overview/","quote":"it is easy to create in Pathway a pipeline serving as a `DocumentStore`, which automatically indexes documents and gets updated upon new data.","accessed":"2026-06-17"}],"alternatives":[{"composition":"streamkap","relation":"similar-shape","note":"Streamkap also keeps a vector store in sync with source data in real time, but as a managed CDC streaming service rather than a Python/Rust dataflow framework you write pipelines in."},{"composition":"llamaindex","relation":"complements","note":"Pathway integrates with LlamaIndex; LlamaIndex provides the RAG orchestration and query layer while Pathway supplies the live, incrementally-updated index underneath."}],"pattern_composition":"flowchart TD\n  fw[\"Pathway\"]\n  fw --> p1[\"Streaming Feature Pipeline<br/>(core)\"]\n  fw --> p2[\"Agentic RAG<br/>(supported)\"]\n  fw --> p3[\"Naive RAG<br/>(supported)\"]","tags":["workflow-engine","retrieval","full-code"]},{"id":"streamkap","name":"Streamkap","kind":"framework","category":"workflow-engine","vendor":"Streamkap","language":"proprietary","license":"proprietary","status":"active","first_released":"2022","url":"https://streamkap.com/","docs_url":"https://docs.streamkap.com/","intent":"Streamkap is a managed platform that streams change-data-capture events from databases into destinations such as vector stores in real time so downstream embeddings stay in sync with the source.","description":"Streamkap connects databases, applications, and warehouses through change-data-capture and event streaming, moving data from source to destination with sub-50ms latency. For AI workloads it streams each insert, update, and delete from the source into a vector database, triggering an embedding update so retrieval data does not go stale. It processes only the rows that changed rather than re-embedding the whole dataset in batch.","primary_use_cases":["real-time CDC pipelines from databases to vector stores","keeping embeddings and RAG context current with source data","streaming ETL into warehouses and applications","incremental embedding updates for recommendation and agent context"],"agent_loop_shape":"Streamkap is a streaming-pipeline service, not an agent loop. It captures change events from a source database, optionally transforms them in flight, and delivers them to a sink. When the sink is a vector store, each captured change drives an embedding update, so the index served to a downstream agent or RAG system reflects the source as it changes.","build_surface":"low-code","members":[{"pattern":"cdc-vector-sync","role":"core","note":"Uses change-data-capture so the source database stays the single writer; every insert/update/delete is streamed in real time and triggers an embedding update that keeps the vector store in sync.","evidence":[{"type":"doc","title":"Streaming CDC Events to Vector Databases for Real-Time AI - Streamkap","url":"https://streamkap.com/resources-and-guides/streaming-to-vector-databases","quote":"use CDC to stream database changes to your vector database in real time. Every insert, update, and delete in the source triggers an embedding update","accessed":"2026-06-17"},{"type":"doc","title":"Streaming CDC Events to Vector Databases for Real-Time AI - Streamkap","url":"https://streamkap.com/resources-and-guides/streaming-to-vector-databases","quote":"But there is a gap most teams overlook: the data in the vector database gets stale the moment the source database changes.","accessed":"2026-06-17"}],"evidence_status":"full"},{"pattern":"streaming-feature-pipeline","role":"supported","note":"Rather than re-embedding the full dataset on a batch schedule, Streamkap streams each source change as it happens and processes only what actually changed, building the embedding feature incrementally as a continuous stream.","evidence":[{"type":"doc","title":"Streaming CDC Events to Vector Databases for Real-Time AI - Streamkap","url":"https://streamkap.com/resources-and-guides/streaming-to-vector-databases","quote":"CDC eliminates all of these by streaming each change as it happens, so you only process what actually changed.","accessed":"2026-06-17"}],"evidence_status":"limited"},{"pattern":"pipes-and-filters","role":"supported","note":"The vector-sync path is a staged stream pipeline - source DB to a CDC engine to Kafka to an embedding service to the vector DB - where each stage consumes the previous stage's events and emits to the next, the classic chain of filters connected by a queue.","evidence":[{"type":"doc","title":"Streaming CDC Events to Vector Databases for Real-Time AI - Streamkap","url":"https://streamkap.com/resources-and-guides/streaming-to-vector-databases","quote":"Source DB ──> CDC Engine ──> Kafka ──> Embedding Service ──> Vector DB","accessed":"2026-06-17"},{"type":"doc","title":"Streaming CDC Events to Vector Databases for Real-Time AI - Streamkap","url":"https://streamkap.com/resources-and-guides/streaming-to-vector-databases","quote":"Kafka: Buffers and orders change events","accessed":"2026-06-17"}],"evidence_status":"full"}],"verification_status":"partial","status_in_practice":"emerging","last_analyzed":"2026-06-17","last_updated":"2026-06-17","date_added":"2026-06-17","key_concepts":[{"name":"CDC engine","summary":"Captures every INSERT, UPDATE, and DELETE from the source database's transaction log so downstream consumers see each change as an event rather than re-scanning the table.","maps_to_pattern":"cdc-vector-sync","url":"https://streamkap.com/resources-and-guides/streaming-to-vector-databases"},{"name":"Embedding service","summary":"A pipeline stage that consumes CDC events and decides whether to generate a new embedding, so embedding cost is proportional to the rate of change rather than dataset size.","maps_to_pattern":"streaming-feature-pipeline","url":"https://streamkap.com/resources-and-guides/streaming-to-vector-databases"},{"name":"Sources and destinations","summary":"Streamkap connects a variety of sources to destinations and streams data sub-second between them, with pipelines configured rather than coded.","maps_to_pattern":"pipes-and-filters","url":"https://docs.streamkap.com/"}],"references":[{"type":"doc","title":"Streaming CDC Events to Vector Databases for Real-Time AI - Streamkap","url":"https://streamkap.com/resources-and-guides/streaming-to-vector-databases","quote":"The solution is straightforward: use CDC to stream database changes to your vector database in real time.","accessed":"2026-06-17"},{"type":"doc","title":"Streamkap Documentation","url":"https://docs.streamkap.com/","quote":"Serverless, real-time data integration platform for streaming data sub-second from sources to destinations","accessed":"2026-06-17"},{"type":"doc","title":"Streaming CDC Events to Vector Databases for Real-Time AI - Streamkap","url":"https://streamkap.com/resources-and-guides/streaming-to-vector-databases","quote":"For AI applications like RAG, agent context, and recommendation engines, this freshness is the difference between useful and unreliable.","accessed":"2026-06-17"}],"alternatives":[{"composition":"pathway","relation":"similar-shape","note":"Pathway also keeps vector indexes synchronised with sources in real time, but as a self-hosted Python/Rust dataflow framework rather than a managed serverless CDC service."},{"composition":"temporal","relation":"complements","note":"Temporal is a durable workflow engine for orchestrating agent logic; it pairs with rather than replaces Streamkap's job of keeping the retrieval index fresh underneath such agents."}],"pattern_composition":"flowchart TD\n  fw[\"Streamkap\"]\n  fw --> p1[\"CDC-Driven Vector Sync<br/>(core)\"]\n  fw --> p2[\"Streaming Feature Pipeline<br/>(supported)\"]\n  fw --> p3[\"Pipes and Filters<br/>(supported)\"]","tags":["workflow-engine","retrieval","routing-composition","low-code"]}]}