Haystack
Type: full-code · Vendor: deepset · Language: Python · License: Apache-2.0 · Status: active · Status in practice: mature
Build production-ready LLM applications — RAG, agents, and multimodal search — as explicit pipelines of typed Components with a tool-using Agent at the top.
Description. Haystack is deepset's open-source orchestration framework for retrieval-augmented and agentic LLM applications. Its core abstractions are Components (typed building blocks) wired into Pipelines, with an Agent component that runs a tool-calling loop over chat-based LLMs. Tools include Tool, ComponentTool, PipelineTool, MCPTool, and MCPToolset, so a Haystack agent can call other pipelines, MCP servers, or arbitrary Python functions. Haystack ships first-class hybrid retrieval (BM25 + dense) backends for Weaviate, OpenSearch, Azure AI Search, and an in-memory store; cross-encoder rerankers; and Hayhooks for exposing pipelines as REST or MCP servers.
Agent loop shape. Pipeline-of-components topped by a loop-based Agent. The Agent component takes a chat-based LLM and a list of Tools (Tool, ComponentTool, PipelineTool, MCPTool, Toolset, MCPToolset, SearchableToolset) and iterates: call LLM, dispatch any tool calls, update state, repeat until an exit condition (text reply with no tool call, or a named tool being executed) or max_agent_steps is reached.
Primary use cases
- production RAG over millions of documents
- tool-using agents with explicit retrieval and routing
- hybrid (BM25 + dense) search pipelines with reranking
- agents exposed as MCP servers via Hayhooks
Key concepts
- Component / Pipeline (docs) — Components are typed building blocks; Pipelines connect their outputs to compatible inputs to form a graph.
- Agent → react (docs) — Tool-using loop component that drives a chat-based LLM through tool calls until an exit condition fires or max_agent_steps is hit.
- MCPTool / MCPToolset → mcp (docs) — Connects to external MCP servers over Streamable HTTP, SSE, or StdIO and exposes them to the Agent as tools.
- Hybrid retrievers → hybrid-search (docs) — First-class BM25 + dense embedding retrievers for Weaviate, OpenSearch, Azure AI Search, and in-memory stores; results merged via RRF.
- max_agent_steps + exit conditions → step-budget (docs) — Agent iteration is bounded by max_agent_steps (default 100) and a list of exit conditions (e.g. 'text' or a tool name).
- Hayhooks (docs) — Serves Haystack pipelines as REST APIs or MCP servers.
Patterns this full-code implements —
- ★★ReAct
Haystack's Agent component is a tool-calling loop over a chat LLM; it iterates LLM call → tool dispatch → state update until an exit condition fires.
- ★★Tool Use
Agent accepts Tool, ComponentTool, PipelineTool, MCPTool, Toolset, MCPToolset, and SearchableToolset instances.
- ★★Step Budget
Agent runs are bounded by max_agent_steps (default 100) and explicit exit conditions; stops when the LLM replies without a tool call or when a named tool is executed.
- ★★Model Context Protocol
MCPTool and MCPToolset integrate external MCP servers over Streamable HTTP, SSE, or StdIO; Hayhooks also serves Haystack pipelines as MCP servers.
- ★★Hybrid Search
Multiple built-in hybrid retrievers combine BM25 keyword search with dense embedding retrieval; Azure AI Search merges via RRF.
- ★★Cross-Encoder Reranking
Haystack ships rerankers (SentenceTransformersDiversityRanker, LostInTheMiddleRanker, transformers cross-encoder rerankers) as Components that slot into a pipeline downstream of retrievers.
- ★★Agentic RAG
The Agent + retriever + reranker + generator pipeline is Haystack's canonical agentic-RAG shape; deepset explicitly markets RAG as a primary use case.
- ★★Streaming Typed Events
Agent streams token-level outputs from the LLM, surfaced as typed events during the run.
- ★★Human-in-the-Loop
Haystack documents a Human in the Loop pattern for intercepting tool calls for human review; not the headline feature but explicitly supported.
- ★★Structured Output
Haystack supports state schemas and validates runtime state, but vendor docs frame structured output through state schemas rather than a first-class JSON-schema response API.
Neighbourhood
Click any neighbour to follow the lineage. Scroll to zoom, drag to pan.
Instantiates
Listed as alternative by (2)
References
deepset-ai/haystack
“open-source AI orchestration framework for building context-engineered, production-ready LLM applications”
“Haystack structures agents and applications as explicit, modular pipelines composed of retrievers, routers, memory layers, tools, evaluators, and generators.”
Provenance
- Last analyzed:
- Last updated:
- Verification status: verified