Rate limiter: a token bucket with tests
Time is an argument and the level is kept in thousandths of a token, so the tests are exact.
Four small coding-round problems taken from real LLM plumbing: a token-bucket rate limiter, retry with full jitter, a streaming line parser that survives split chunks, and a history trimmer that keeps the system message. Each has tests that print PASS or FAIL, in three languages.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write fixed_window(times_ms, limit, window_ms) that returns "yes" or "no" for each request time, allowing at most limit requests per window. Show it on two lists, and explain from the output why a token bucket is better.
Write ceiling(attempt, base_ms=200, cap_ms=10_000) that gives the largest wait for a retry attempt, and print the ceilings for attempts 0 to 7.
Explain it without notes
Why use a token bucket instead of a fixed-window counter for rate limiting?
Why does retry need jitter, and why a cap?
Why does a streaming parser need to keep a buffer between chunks, and which edge cases do you test?
In the history trimmer, why keep the system message, drop the oldest messages first and avoid gaps?
Expected output
PASS a burst of 5 is allowed, the 6th is denied
PASS half a second later one token is back
PASS after a long pause only `capacity` is allowed
PASS a request bigger than the bucket is never allowed
PASS an earlier timestamp does not crash or refill
5 of 5 passed