Recipe: a resilient model client
The transport replays scripted responses. Waits are recorded, not slept, so the example is instant. Jitter uses the course's seeded generator.
Two recipes to copy: a model client with a key from the environment, a timeout on every call, error classification and retries with jitter, and a wrapper that writes one safe log line per call with tokens, latency and cost. Both sit behind the same complete interface.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 6 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write backoff_cap(attempt, base_ms=500, max_ms=8000) that returns min(max_ms, base_ms * 2**attempt) and print it for attempts 0 to 5.
Write log_line(model, outcome, tokens_in, tokens_out, latency_ms) that returns the compact JSON text of a dictionary with those five fields in that order.
Explain it without notes
Walk through what the resilient client does for a 429 with Retry-After, a 401 and a timeout.
Why does the logging wrapper write its line in a finally block and leave out the prompt text?
Why add full jitter to exponential backoff?
Expected output
limit, overload, then ok -> Hello! finish=stop attempts=3 waits_ms=[2000, 357]
wrong key -> ModelError HTTP 401 retryable=no waits_ms=[]
timeouts forever -> ModelError HTTP 408 retryable=yes waits_ms=[321, 94, 1370]
cut-off reply -> Hel finish=length attempts=1 waits_ms=[]
no key in the environment -> ModelError MODEL_API_KEY is not set