A request through the layers
One process, five separate parts. Each log line shows the layers the request reached.
A production LLM service is a normal web backend with a few new layers: a gateway that checks who is calling, a prompt layer, a model client, optional retrieval and tools, storage, and observability around all of it. Knowing the layers tells you where each earlier phase plugs in.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 6 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Add a moderation(prompt) layer to the service that returns False for any question containing banana. Make handle return status 400 for blocked questions and log the layers reached. Show the log for two questions, one blocked and one allowed.
Write layer_report(log_lines) that takes the log lines of the example and returns how many requests stopped at each last layer. Print the result for the five example requests.
Explain it without notes
Name the layers of an LLM service and say what each is responsible for.
Why should the gateway run before the prompt layer and the model call?
What does it mean for a service to be stateless, and why does it matter for LLM backends?
Why keep the SDK behind a model-client module instead of calling it from request handlers?
Expected output
status=200 tenant=acme layers=gateway>prompt>model>storage tokens=12+7
status=401 tenant=- layers=gateway tokens=0+0
status=200 tenant=acme layers=gateway>prompt>model>storage tokens=10+5
status=429 tenant=acme layers=gateway tokens=0+0
status=200 tenant=globex layers=gateway>prompt>model>storage tokens=8+3
stored: acme=2 globex=1