A conversation store with a budget and a summary
Words stand in for tokens and a few kept words stand in for a model-written summary.
The model remembers nothing, so your service must store each conversation and send the right part of it every time. That means a database table, a token budget that trims old messages, an optional summary of what was trimmed, strict separation between users, and a way to delete data on request.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 6 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write build_context(messages, budget) where messages is a list of (role, text) and the budget counts words. Keep the newest messages that fit, in original order, and always keep the last message even if it alone is over the budget. Test it with a budget of 6.
Write can_read(conversation, user) that returns True only if conversation["owner"] == user, and show it blocks a different user.
Explain it without notes
Why must an LLM service store conversations itself?
How do you decide which messages to send to the model on each call?
What are the risks of summarising old messages, and how do you limit them?
How do you keep one user's conversation from leaking to another user?
Expected output
after user: messages=1 total=6
after assistant: messages=2 total=12
after user: messages=2 total=14
after assistant: messages=1 total=14
after user: messages=1 total=12
context sent to the model:
[summary] still dropping after then update the
[user] firmware updated and it works now
ben sees 1 line, ana sees 2
forget ana: yes, again: no