Recipe: a per-feature cost and latency report
Twelve made-up log lines. Prices and alert thresholds are example numbers: set your own in configuration.
The phase's recipes in one place: a client that wraps any model with a cache, a rate limiter, retries with jitter, a fallback chain and a cost guard, and a per-feature report that turns your call log into costs, p95 latency and alerts.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 6 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write ask_with_fallback(models, prompt) where models is a list of functions that take a prompt and either return text or raise RuntimeError. Return (index, text) for the first one that works, or (-1, "canned") if all fail. Test with a first function that fails and a second that works.
Write report_line(feature, latencies, costs) returning "<feature> calls N p95 X ms cost Y" where p95 uses the nearest-rank method and cost is the sum of costs. Print it for "chat", [100, 200, 300, 400] and [5, 5, 5, 5].
Explain it without notes
Walk through the order of the layers in the resilient client and justify it.
Why translate errors into RateLimited and ServerBusy at the edge?
What does the per-feature report tell you that the provider's bill does not?
List the values that belong in configuration rather than code, and say why.
Expected output
chat calls 4 errors 0 p50 640 ms p95 700 ms cost 3280 micro-dollars
search calls 4 errors 1 p50 380 ms p95 2900 ms cost 3170 micro-dollars
ALERT search: p95 above 2000 ms
ALERT search: 1 of 4 calls failed
summary calls 4 errors 1 p50 1600 ms p95 6100 ms cost 10880 micro-dollars
ALERT summary: p95 above 2000 ms
ALERT summary: 1 of 4 calls failed