IX · Routing & CompositionMature★★

MRKL Systems (Modular Neuro-Symbolic)

also known as Modular Reasoning Knowledge and Language, Neuro-Symbolic Router

Route each request through an LLM dispatcher to specialized symbolic or neural expert modules (calculator, knowledge base, code executor) rather than asking one LLM to do everything; integrate the modules' results for the final response.

This pattern helps complete certain larger patterns —

  • specialisesHybrid Symbolic-Neural Routing★— Per query, route between a symbolic path (rule engine, knowledge graph) and a neural path (LLM), using the LLM for interpretation and the symbolic layer for exact constraints.

Context

An agent faces tasks that combine reasoning (good for LLMs) with operations LLMs are notoriously bad at (exact arithmetic, structured database queries, deterministic computation). Asking the LLM to do all of it produces well-known failures: arithmetic mistakes, table hallucinations, code that doesn't compile.

Problem

Single-LLM 'do it all' wastes the model on tasks symbolic systems do better, and inherits the LLM's failures on those tasks (calculation errors, fabricated DB facts). Yet rejecting the LLM throws out its reasoning value.

Forces

  • Router design adds an upstream component.
  • Expert modules must have callable interfaces.
  • Result integration logic is non-trivial when expert outputs are structured.

Example

A research agent gets 'What was Tesla's Q3 2024 revenue, and what's the year-over-year growth rate?' MRKL router dispatches: SQL expert pulls revenue figures from financial DB; calculator expert computes growth rate; LLM integrates and produces 'Tesla's Q3 2024 revenue was $25.18B, growing 7.8% year-over-year.' Asking the LLM alone would risk hallucinating the figures or miscalculating the growth.

Diagram

Solution

Therefore:

Karpas et al. 2022 — MRKL architecture. (1) Router LLM receives the request, identifies relevant expert modules. (2) Dispatch to each module with structured inputs. (3) Integrate module outputs back into the LLM's reasoning. Expert modules can be calculator (Wolfram Alpha), knowledge base (SQL, vector DB), code executor (Python sandbox), specialist models. Precursor to modern tool-using agents. Pair with tool-use, function-calling, augmented-llm, multi-model-routing, hybrid-symbolic-neural-routing.

What this pattern forbids. The LLM does not perform tasks the expert modules can perform; dispatch is mandatory for those task classes.

And the patterns that stand alongside it, or against it —

  • complementsTool Use★★— Let the LLM produce typed calls against an external toolkit instead of producing free-form text the surrounding system has to parse.
  • complementsAugmented LLM★★— Build the foundational agent block as an LLM augmented with retrieval, tools, and memory that the model actively chooses to use, rather than a bare-model call.
  • complementsMulti-Model Routing★★— Send each request to the cheapest model that can handle it well.
  • complementsToolformer✕— Train the model to learn when and how to call tools through self-supervised data, without human annotation.

Neighbourhood

Click any neighbour to follow the language. Scroll to zoom, drag to pan.

References

Provenance