A cheap-first router and the cost it saves
Example prices held in variables; the small model's confidence decides who answers. Check real prices with your provider.
Not every request needs your most expensive model. A router sends easy requests to a cheap, fast model and escalates hard ones to a stronger model, using a confidence or validation signal. Measured on your own examples, it can cut cost a lot, but only if the escalation signal is trustworthy.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 6 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write route(confidence, threshold) returning "small" when the confidence is at least the threshold and "large" otherwise, and print the route for the confidences 0.9, 0.7 and 0.5 with a threshold of 0.7.
Write expected_cost(small, large, p) returning small + p * large and print it for p of 0.1 and 0.3 with small 0.001 and large 0.02 (format with 4 decimals).
Explain it without notes
What is a model cascade and when does it save money?
What signals can trigger escalation, and which are most reliable?
How do you evaluate a router before using it in production?
What new failure modes does routing add?
Expected output
t1: confidence 0.90 -> small, cost 0.000040
t2: confidence 0.50 -> small -> large, cost 0.001302
t3: confidence 0.95 -> small, cost 0.000031
t4: confidence 0.80 -> small, cost 0.000049
t5: confidence 0.60 -> small -> large, cost 0.001680
t6: confidence 0.85 -> small, cost 0.000030
escalated 2 of 6
routed 0.003132 dollars, always large 0.005840 dollars
saved 46.4 percent