Support bot estimate: tokens, cost and peak load
Example prices and traffic. The loop re-sends earlier turns, which is why input tokens dominate.
Design a support assistant that answers from help articles, looks up orders and hands over to a human, with numbers for traffic, tokens, cost and storage. You will practise the interview flow: requirements, estimates, architecture, safety, evaluation, operations.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
A shop's bot gets 30,000 calls a day, the busiest minute is 4 times the average, and each call takes 2.5 seconds. Print the average and peak requests per second and how many calls are in flight at the peak.
Each stored message averages 150 tokens at 4 bytes per token plus 200 bytes of row overhead. For 1 million messages a day kept 30 days, print the storage per day and in total in gigabytes.
Explain it without notes
Why do you start a system design answer with questions and numbers instead of drawing boxes?
Why do input tokens grow faster than the number of turns in a chat, and how do you control it?
Which parts of the support bot are deterministic code, and which are the model, and why does the split matter?
How would you roll out the assistant safely?
Expected output
tokens per day: 888000000 in, 54000000 out
cost per day: 525 dollars, per month: 15750 dollars
requests per second: average 3.47, peak 17.36
calls in flight at peak: 53
input tokens per minute at peak: 3081360
what-if, 1000 retrieved tokens instead of 1500: saves 2250 dollars per month