VI · Multi-AgentEmerging★

Topic-Based Routing

also known as Agent Pub/Sub, Topic and Subscription, Subject-Based Routing

Route inter-agent messages through named topics that agents subscribe to, instead of having senders address each other by id.

This pattern helps complete certain larger patterns —

  • specialisesInter-Agent Communication★— Define a protocol for agents to exchange tasks, capabilities, and results across process or vendor boundaries.
  • used-byHierarchical Retrieval★★— Route a query through a multi-level cascade — coarse source or index selection, then per-source narrower retrieval, then chunk-level — so each retrieval decision is pushed to the cheapest tier that can answer it.

Context

A team is building a multi-agent system in which a message produced by one agent is potentially of interest to several others, and the set of interested agents may change over time. The sender does not know — and should not need to know — exactly which agents will care about its message, and new subscribers should be able to join the system without forcing changes to anyone who is already publishing.

Problem

Direct agent-to-agent addressing, where a sender names each receiver explicitly, creates a dense web of dependencies in which every sender carries knowledge about every receiver it might want to reach. Adding a new participant then requires editing every sender that should be able to reach it, and removing one leaves dangling references everywhere. The team needs a routing mechanism where senders publish to named topics and interested agents subscribe to those topics, so that sender and receiver are decoupled and the wiring can change without touching either end.

Forces

  • Decoupling sender from receiver is the central benefit of pub/sub.
  • Topic semantics — wildcards, ordering guarantees, durability — change the failure modes substantially.
  • Broadcast traffic on a busy topic can overwhelm slow subscribers without back-pressure.
  • Debugging is harder when nobody owns the addressing decision.

Example

An incident-response platform has many agents. A monitor agent publishes to `telemetry.alert`; a triage agent subscribes to it and may publish to `incident.opened`; an audit agent subscribes to both topics; a paging agent subscribes only to `incident.opened`. Adding a new compliance-export agent that needs every incident is a one-line subscription on `incident.opened` — none of the existing publishers change. Topic schemas are versioned (`incident.opened.v1`) and subscribers declare which versions they accept.

Diagram

Solution

Therefore:

Define a small set of typed Topics (`telemetry.parsed`, `incident.opened`, `plan.proposed`). Agents publish to topics; agents that care subscribe to topics. The runtime fans messages out to all subscribers of a topic, applies back-pressure on slow consumers, and provides delivery guarantees appropriate to the topic class. Pair with actor-model-agents to keep each subscriber's processing isolated, and with event-driven-agent when the topic carries external events. Topic schemas are first-class artefacts; subscribers depend on the schema, not on the publisher.

What this pattern forbids. Senders do not address receivers by id; cross-agent messaging must go through named topics with explicit subscriptions, and topic schemas are not allowed to mutate without versioning.

And the patterns that stand alongside it, or against it —

  • complementsActor-Model Agents★— Implement each agent as an independent actor with its own mailbox, processing asynchronous messages one at a time and never sharing mutable state with peers.
  • complementsEvent-Driven Agent★★— Trigger the agent on external events (webhooks, message queues, file changes) instead of user requests or schedules.
  • alternative-toBlackboard·— Give multiple agents a shared, queryable workspace they can read from and write to as they collaborate.
  • alternative-toPipes and Filters★★— Compose stream-shaped processing as a chain of small filters connected by pipes.
  • complementsComplexity-Based Routing★— Estimate a request's difficulty up front and bind it to the cheapest model tier that can answer well, using an explicit complexity classifier as the routing key.

Neighbourhood

Click any neighbour to follow the language. Scroll to zoom, drag to pan.

Used in frameworks

References

Provenance