Token-budget history trimmer, with tests
A stand-in tokenizer (about 4 characters per token plus 4 per message). In production, pass the model's real counter.
Four small coding-round problems taken from real LLM plumbing: a token-bucket rate limiter, retry with full jitter, a streaming line parser that survives split chunks, and a history trimmer that keeps the system message. Each has tests that print PASS or FAIL, in three languages.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write fixed_window(times_ms, limit, window_ms) that returns "yes" or "no" for each request time, allowing at most limit requests per window. Show it on two lists, and explain from the output why a token bucket is better.
Write ceiling(attempt, base_ms=200, cap_ms=10_000) that gives the largest wait for a retry attempt, and print the ceilings for attempts 0 to 7.
Explain it without notes
Why use a token bucket instead of a fixed-window counter for rate limiting?
Why does retry need jitter, and why a cap?
Why does a streaming parser need to keep a buffer between chunks, and which edge cases do you test?
In the history trimmer, why keep the system message, drop the oldest messages first and avoid gaps?
Expected output
whole history: 77 tokens
PASS everything fits: nothing is dropped
budget 40 keeps: system,user
PASS the system message survives
PASS it fits the budget
PASS the newest message is kept
PASS order is preserved and the history starts with a user message
budget 49 keeps: system,user,assistant,user
PASS a bigger budget keeps more, still starting with a user message
PASS a system message bigger than the budget is an error
PASS a budget too small for the newest message is an error
8 of 8 passed