Semantic Router
Type: full-code · Vendor: Aurelio AI (Aurelio Labs) · Language: Python · License: MIT · Status: active · Status in practice: mature · First released: 2023
Semantic Router is a Python decision layer that routes a query to a predefined route by embedding similarity instead of an LLM call, returning the matched route or nothing for off-topic input.
Description. Semantic Router defines routes as sets of example utterances and encodes them with a pluggable embedding model. At query time it compares the query embedding against the routes in vector space and returns the matching route name, or None when nothing matches. Because the decision uses embeddings rather than an LLM generation, it is fast and also serves as a guardrail by declining unrelated queries. It supports OpenAI, Cohere, Hugging Face, and local encoders.
Agent loop shape. Semantic Router is a single-step decision layer, not an agent loop. A query is embedded and compared against the embeddings of each route's utterances; if the nearest route is within threshold its name is returned for the application to dispatch on, otherwise None is returned so the caller can fall back or decline.
Primary use cases
- fast semantic classification of queries to handlers
- tool-use and intent decisions without an LLM call
- guardrails that reject off-topic input
- routing requests across multiple LLMs or agents
Key concepts
- Route → routing (docs) — A named decision path defined by a list of example utterances; at query time the route whose utterances are most similar to the query in vector space is selected.
- SemanticRouter (route layer) → hybrid-symbolic-neural-routing (docs) — The object that holds the routes and the encoder and performs the semantic decision, returning the matched route name or None when nothing is within threshold.
- Encoder (docs) — A pluggable embedding model (OpenAI, Cohere, Hugging Face, or local) used to encode both the route utterances and the incoming query into the shared vector space.
Patterns this full-code implements —
- ★Hybrid Symbolic-Neural Routing
Per query, Semantic Router takes the fast symbolic/vector path — it decides the route via embedding similarity in semantic vector space instead of calling the slow neural LLM. Queries that match a de…
- ★★Routing
The route layer classifies an incoming query against named routes defined by example utterances and dispatches to the matched route; when no route matches it returns None, so unrelated requests are d…
- ★★Input/Output Guardrails
By defining a route for unwanted input (the docs show a jailbreak route whose utterances are prompt-injection attempts), the same embedding-similarity decision becomes an input guardrail: a query mat…
Neighbourhood
Click any neighbour to follow the lineage. Scroll to zoom, drag to pan.
Alternatives & relatives
- full-code · framework
RouteLLM
RouteLLM also makes a fast routing decision without a full LLM generation, but it routes between model tiers for cost rather than classifying intent against named routes by embedding similarity.
- app · closed product
Not Diamond
Not Diamond is a routing/model-selection service; it overlaps the decision-layer goal but is a hosted product rather than a self-hosted embedding-based route classifier.
References
Provenance
- Last analyzed:
- Last updated:
- Verification status: partial