Semantic Router operates by embedding text or data into vector representations and making routing decisions based on semantic similarity within a high-dimensional space.
Vector Embeddings
Semantic Router converts text or other input data into numerical representations called embeddings. These embeddings capture the semantic meaning of the data.
Semantic Space
All embeddings are placed in a high-dimensional semantic space. In this space, data points with similar meanings are located closer to each other.
Routing Decisions
When a new request arrives, its embedding is calculated and compared with embeddings of predefined routes. The route with the closest semantic match determines the action or response.
Speed
Semantic Router makes routing decisions in milliseconds, significantly reducing LLM response times.
Efficiency
By avoiding unnecessary LLM generations for routing decisions, it optimizes compute and resource usage.
Scalability
It supports a large number of routes and complex decision-making scenarios.
Accuracy
Routing decisions are based on semantic similarity, ensuring high accuracy.
Flexibility
Semantic Router integrates with multiple LLM providers and vector database systems.
AI Agents
Routes user requests to the correct tools or actions efficiently.
Chatbots
Determines user intent quickly and accurately.
Recommendation Systems
Recommends products or content based on semantic user preferences.
Content Moderation
Classifies and filters content according to predefined semantic categories.
The following example demonstrates how to define routes and initialize a Semantic Router using Python.
from semantic_router import Route
import os
from getpass import getpass
from semantic_router.encoders import OpenAIEncoder
from semantic_router.layer import RouteLayer
os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY")
encoder = OpenAIEncoder()
politics = Route(
name="politics",
utterances=[
"isn't politics the best thing ever",
"why don't you tell me about your political opinions",
"don't you just love the president",
"don't you just hate the president",
"they're going to destroy this country!",
"they will save the country!",
],
)
chitchat = Route(
name="chitchat",
utterances=[
"how's the weather today?",
"how are things going?",
"lovely weather today",
"the weather is horrendous",
"let's go to the chippy",
],
)
routes = [politics, chitchat]
rl = RouteLayer(encoder=encoder, routes=routes)
Testing with a politics-related query:
rl("don't you love politics?")
Output:
RouteChoice(name='politics', function_call=None, similarity_score=None)
Testing with a chitchat-related query:
rl("how's the weather today?")
Output:
RouteChoice(name='chitchat', function_call=None, similarity_score=None)
Testing with an unrelated query:
rl("I'm interested in learning about llama 2")
Output:
RouteChoice(name=None, function_call=None, similarity_score=None)
Semantic Router is a powerful tool for building efficient and intelligent AI systems. By leveraging semantic understanding rather than rule-based logic, it significantly accelerates decision-making and improves overall system performance.
GitHub Repository: aurelio-labs/semantic-router
Another Get-Rich-Quick Scheme Using LLMs, RAG, Semantic Routing & Prompts Driven Feedback
Smarter Chatbots: How Semantic Routing Prevents the Unwanted
