Recipe: the cheapest model that clears the quality bar
Scripted replies stand in for real calls. Prices are made up. The loop is the same with real calls.
Five recipes for choosing and changing models with evidence: gates, the cheapest model that clears your quality bar, a weighted scorecard, a swap guard that compares models case by case, and configuration that makes switching a one-line change.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 6 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write pass_percent(replies, expected) that returns the whole-number percentage of replies equal to the expected value after strip() and upper(), and print it for three replies with one failure.
Write regressions(before, after) where each is a list of booleans (passed or not, per case); it returns the indexes where before is true and after is false. Print the result for [True, True, False, True] and [True, False, True, True].
Explain it without notes
Describe a repeatable process for choosing a model for a new feature.
Why can two models with the same pass rate still not be interchangeable, and how do you protect against it?
How should model names and providers appear in your code?
Expected output
small-open: 4 of 6 passed (66 percent), cost 0.40 per 1k calls
failed: Bharat -> India
failed: U.S.A. -> US.
mid-hosted: 5 of 6 passed (83 percent), cost 2.00 per 1k calls
failed: Nippon -> NP
large-hosted: 6 of 6 passed (100 percent), cost 9.00 per 1k calls
chosen: mid-hosted
the most expensive candidate would cost 4.5 times as much per call