Trimming a conversation to a token budget
Token counts are estimates (4 characters per token plus overhead). Real counts come from the reply's usage (Topic 2.3).
A chat is a list of messages, each with a role: system (rules), user (the person) and assistant (the model). The model remembers nothing between calls, so your program stores the list and sends all of it every time, trimming it when it grows too long.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 6 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write add_turn(history, user_text, reply_text) that appends a user and an assistant message to history and returns it.
Write last_user_message(messages) that returns the content of the newest user message, or None if there is none.
Explain it without notes
Why must an application re-send the conversation on every call?
What are the roles in a chat request and who writes each?
How would you stop a long conversation from overflowing the context window?
Expected output
budget 200: kept 6 of 6 (76 tokens) -> system, user, assistant, user, assistant, user
budget 60: kept 4 of 6 (50 tokens) -> system, user, assistant, user
budget 40: kept 2 of 6 (22 tokens) -> system, user
budget 25: kept 2 of 6 (22 tokens) -> system, user