Support bot estimate: storing the chat history
Bytes per token, row overhead and retention are assumptions you say out loud in the interview.
Design a support assistant that answers from help articles, looks up orders and hands over to a human, with numbers for traffic, tokens, cost and storage. You will practise the interview flow: requirements, estimates, architecture, safety, evaluation, operations.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
A shop's bot gets 30,000 calls a day, the busiest minute is 4 times the average, and each call takes 2.5 seconds. Print the average and peak requests per second and how many calls are in flight at the peak.
Each stored message averages 150 tokens at 4 bytes per token plus 200 bytes of row overhead. For 1 million messages a day kept 30 days, print the storage per day and in total in gigabytes.
Explain it without notes
Why do you start a system design answer with questions and numbers instead of drawing boxes?
Why do input tokens grow faster than the number of turns in a chat, and how do you control it?
Which parts of the support bot are deterministic code, and which are the model, and why does the split matter?
How would you roll out the assistant safely?
Expected output
messages per day: 600000
stored per day: 0.5 GB (messages 0.4, turn logs 0.1)
kept 90 days: 44.8 GB, with 3 replicas: 134.5 GB
if every turn also stored its retrieved text: 206.8 GB for 90 days
a page of 50 messages read back: about 34 KB