PHASE 7Intermediate ~22 min· topic 6 of 9

Topic 7.6

Routing

In one line

Not every question belongs in the handbook index. Routing sends a question to the right place (documents, a database, or nowhere): clear rules first, then a small classifier, and a safe default when nothing is sure.

Think of it like this

A hospital front desk. "Where is my appointment letter?" goes to the records office, "Where do I park?" goes to the signs outside, and "hello" gets a smile. A good receptionist does not send everyone to the same room.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Router: Code that decides where a question goes.
Route: One possible destination: the documents, a database, or nothing.
Rule: An exact pattern that decides a route, such as an order id format.
Classifier: A small model or scoring function that picks a route from the question's text.
Threshold: The minimum evidence needed before leaving the default route.
Default route: The destination used when nothing is sure.

Step by step

01Step 1: list the destinations

docs for the handbook, orders for the user's bookings, none for small talk and off-topic. Write down what each can answer.

02Step 2: write rules for the certain cases

An order id has a fixed shape. If any word looks like lb followed by digits, route to orders and record why.

route.pywhole filepython

for word in words:  # rule first: an order id looks like lb1042
    if word.startswith("lb") and word[2:].isdigit():
        return "orders", "rule: order id " + word

03Step 3: score the fuzzy cases

Count how many of each route's words are in the question. A route needs its minimum number of matches before it can win.

04Step 4: fall back to the default

If no route clears its threshold, the answer is docs. Make the default the route that fails gently.

Step 4: fall back to the defaultdiagram

Rendering diagram…

05Step 5: read the trace

The example prints the route and the reason for six questions. The two surprises are the instructive ones: the weather question is routed to none because of one off-topic word, and the booking-refund question stays with the documents.

06Step 6: keep a test set of routes

Write twenty questions with their correct routes and run the router on them whenever you change a word list. A router is code, and it needs tests like any other code.

Try it yourself

1
Add a route
Add a route pricing with words price, prices, cost and a threshold of 1. Ask "What does the monthly pass cost?". Is that what you want? What about "cost of my last invoice"?
2
Find a wrong route
Write three questions that the router sends to the wrong place. Which threshold or word would fix each, without breaking the six that already work?
3
Raise a threshold
Set the none threshold to 2. Which of the six questions change? What happens to a plain "Hello!"?

Code & diagrams

A router: rules, a small classifier, a default New tab

The orders route is a stand-in: a real one would look the booking up in a database.

Expected output

Where is my booking LB1042?                 orders  rule: order id lb1042
show my last invoice                        orders  classifier score 2
Hello!                                      none    classifier score 1
Do shops open on holidays?                  docs    default
What is the weather tomorrow?               none    classifier score 1
Can I cancel my booking and get a refund?   docs    default

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

A threshold of one for orders

In the Python tab change the threshold for the orders route: replace THRESHOLD = {"orders": 2, "none": 1} with THRESHOLD = {"orders": 1, "none": 1}. Run.

terminal

$ Run

── what you'll see ──

Do shops open on holidays? docs default

What is the weather tomorrow? none classifier score 1

Can I cancel my booking and get a refund? orders classifier score 1

Myth vs fact

Myth

Routing needs a language model.

Fact

Rules and keyword scores cover many cases and are easier to test. Add a model for the cases they cannot decide.

Myth

A router only has to be right on average.

Fact

Each wrong route is a failed answer. Keep the reason with every decision so wrong routes are easy to find.

Myth

More routes are always better.

Fact

Every route is another place to be wrong and another thing to maintain. Add one when a real question needs it.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
Log the route, the reason and the scores. When the router drifts after a word-list change, the log shows which questions moved.
▸
A router is also a safety layer: send questions that ask for something the product must not do to a refusal route before any retrieval happens.
▸
If you route with a model, give it route descriptions, not just names, require one of the allowed route names as output, and validate the reply like any other model output.

Remember this

1
Why routing exists. AskDocs so far has one source: the handbook. Real products grow more: a database of the user's bookings, a pricing table, a second index of policies, or no source at all (small talk, off-topic). Sending every question to the handbook retrieval returns the wrong thing for most of these. A router picks the destination first.
2
Rules first. Some questions are decided by their shape. An order id like LB1042 can only be asked about the orders system, so a rule catches it with certainty: the example routes "Where is my booking LB1042?" to orders with the reason "rule: order id lb1042". Rules are exact, instant, free and easy to explain to a user and to a support engineer.
3
A small classifier for the rest. For fuzzier cases the example scores each route by the number of its keywords in the question, with a minimum per route: orders needs 2 matches (booking, invoice, last...), none needs 1. "show my last invoice" scores 2 for orders. "Hello!" and "What is the weather tomorrow?" score 1 for none. A classifier can also be a model or an embedding comparison to route descriptions, but start with something you can read.
4
The safe default. If nothing clears its threshold, go to the main route (here docs). "Do shops open on holidays?" matches no route words and goes to the documents. "Can I cancel my booking and get a refund?" mentions booking, but only one orders word, below the threshold of 2, so it also goes to the documents. That is the right answer: it is a policy question. The break-it lowers the threshold to 1 and sends it to the orders database, which cannot answer it.
5
Routing wrongly costs more than routing nowhere. A question wrongly sent to a database returns nothing or something irrelevant. A question wrongly sent to the handbook usually returns "I can't find that". Set thresholds so that uncertain questions fall back to the route that fails gently.
6
What the destinations are. In this course the orders route is a stand-in: a real one would call a database or an API with the parameters read from the question. Calling tools is the subject of the coming AI Agents course. Here the router only decides *where*. Two indexes (for example current policies and archived ones) are also a routing choice, and a filter (Topic 7.7) is a lighter alternative to a second index.
7
Measure the router. Keep a set of questions with the route they should take, and count correct routes. A router that is 95 percent right still sends one in twenty questions to the wrong place, so log the route and the reason with every request. The reason string is what lets you fix a wrong route in minutes.

Explain it without notes

Why use rules before a classifier?

What should the default route be, and why?

Why does the example send "Can I cancel my booking and get a refund?" to the documents?

How would you test a router?

Practice

Write route(question) that returns orders if any word matches lb followed by digits (use re.fullmatch), else docs. Print the route for three questions.

Write classify(scores, minimum, default) that returns the name with the highest score only when it reaches minimum and no other name has the same score. Otherwise return default. Print the result for three score dicts.

Trade-offs

↔
Rules: exact and cheap, but they cover only what you thought of.
↔
A classifier: handles variety, but can be wrong without an obvious reason.
↔
A low threshold: fewer defaults and more mistakes. A high threshold: safer, and more questions fall to the default.

Done when you can

Done when you can write rules, a keyword classifier and a default route.
Done when you can explain why the default should be the gentlest route.
Done when you can attach a reason to every routing decision.
Done when you can test a router with labelled questions.

Command Palette

Words you'll meet

Step by step

01Step 1: list the destinations

02Step 2: write rules for the certain cases

03Step 3: score the fuzzy cases

04Step 4: fall back to the default

05Step 5: read the trace

06Step 6: keep a test set of routes

Try it yourself

Add a route

Find a wrong route

Raise a threshold

Code & diagrams

Break it on purpose

A threshold of one for orders

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can