Recipe: the swap guard
Both models pass 5 of 6, but the candidate regresses a critical case. Same average, different behaviour.
Five recipes for choosing and changing models with evidence: gates, the cheapest model that clears your quality bar, a weighted scorecard, a swap guard that compares models case by case, and configuration that makes switching a one-line change.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 6 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write pass_percent(replies, expected) that returns the whole-number percentage of replies equal to the expected value after strip() and upper(), and print it for three replies with one failure.
Write regressions(before, after) where each is a list of booleans (passed or not, per case); it returns the indexes where before is true and after is false. Print the result for [True, True, False, True] and [True, False, True, True].
Explain it without notes
Describe a repeatable process for choosing a model for a new feature.
Why can two models with the same pass rate still not be interchangeable, and how do you protect against it?
How should model names and providers appear in your code?
Expected output
current: 5 of 6, candidate: 5 of 6
improved: Nippon
regressed: United Kingdom
verdict: BLOCK (a critical case regressed: United Kingdom)