A chat endpoint as a plain function
No web server: the handler maps a raw body to a status and a JSON body, which is what makes it easy to test.
Your service is an API other programs depend on, so its request, response and error shapes are a contract. Design them small and strict: validate everything, clamp what costs money, return one error shape, support streaming without changing the fields, and version the path so you can change it later.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 6 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Add the check temperature (optional, number from 0 to 2) to a small handler validate(body) that returns None when valid or a tuple (422, "temperature") when not. Test it with 0.7, 3 and the text hot.
Write clamp(value, low, high) and use it to turn the requested max_tokens values [10, 600, -5] into applied values between 1 and 512.
Explain it without notes
What belongs in an LLM endpoint's request, and which fields must you clamp or allow-list?
Why use one error envelope for every failure?
How should streaming and non-streaming responses relate?
How do you change an API without breaking existing callers?
Expected output
200 {"id":"req_1","model":"small","reply":"You said: hello there","finish_reason":"stop","max_tokens_used":256,"usage":{"input":2,"output":4}}
400 {"error":{"type":"invalid_json","message":"The body is not valid JSON.","param":null},"request_id":"req_2"}
422 {"error":{"type":"invalid_request","message":"messages must be a non-empty list.","param":"messages"},"request_id":"req_3"}
422 {"error":{"type":"invalid_request","message":"Unknown model.","param":"model"},"request_id":"req_4"}
200 {"id":"req_5","model":"large","reply":"You said: explain retries","finish_reason":"stop","max_tokens_used":512,"usage":{"input":2,"output":4}}