Semantic Router
Semantic Router is a Python decision layer that routes a query to a predefined route by embedding similarity instead of an LLM call, returning the matched route or nothing for off-topic input.
Description
Semantic Router defines routes as sets of example utterances and encodes them with a pluggable embedding model. At query time it compares the query embedding against the routes in vector space and returns the matching route name, or None when nothing matches. Because the decision uses embeddings rather than an LLM generation, it is fast and also serves as a guardrail by declining unrelated queries. It supports OpenAI, Cohere, Hugging Face, and local encoders.
Solution
Semantic Router is a single-step decision layer, not an agent loop. A query is embedded and compared against the embeddings of each route's utterances; if the nearest route is within threshold its name is returned for the application to dispatch on, otherwise None is returned so the caller can fall back or decline.
Primary use cases
- fast semantic classification of queries to handlers
- tool-use and intent decisions without an LLM call
- guardrails that reject off-topic input
- routing requests across multiple LLMs or agents
Open the full interactive page →
Diagram, neighbourhood map, code examples, related patterns and full provenance.