Recipe: the resilient client
Cache, budget guard, limiter, retry with full jitter, fallback chain and cost meter around a fake model. Prices, limits and times are example numbers on a simulated clock.
The phase's recipes in one place: a client that wraps any model with a cache, a rate limiter, retries with jitter, a fallback chain and a cost guard, and a per-feature report that turns your call log into costs, p95 latency and alerts.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 6 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write ask_with_fallback(models, prompt) where models is a list of functions that take a prompt and either return text or raise RuntimeError. Return (index, text) for the first one that works, or (-1, "canned") if all fail. Test with a first function that fails and a second that works.
Write report_line(feature, latencies, costs) returning "<feature> calls N p95 X ms cost Y" where p95 uses the nearest-rank method and cost is the sum of costs. Print it for "chat", [100, 200, 300, 400] and [5, 5, 5, 5].
Explain it without notes
Walk through the order of the layers in the resilient client and justify it.
Why translate errors into RateLimited and ServerBusy at the edge?
What does the per-feature report tell you that the provider's bill does not?
List the values that belong in configuration rather than code, and say why.
Expected output
request 1: from primary at t=100
request 2: from cache at t=1000
request 3: from backup at t=2268
request 4: from backup at t=3200
request 5: refused, the budget is used up (spent 420)
spent 420 of 500 micro-dollars, primary called 6 times, backup 2