A JSON reply is useless until the last brace
Streaming a reply that a program must parse gains nothing: it only becomes valid at the end.
Streaming does not make the model faster. It shows the first words after the model starts writing instead of after it finishes, which is what the reader feels. It does nothing for programs that need the whole reply, such as JSON for code.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write streaming_gain(prefill_ms, ms_per_token, tokens) that returns how many times sooner the first token arrives than the whole reply, as an integer (round down).
Write should_stream(for_human, reply_tokens, must_check_first) that returns True only when a human reads the reply as it arrives, it has more than 30 tokens, and it does not need checking first.
Explain it without notes
What problem does streaming solve, and what does it not change?
What is the difference between time to first token and total time, and what drives each?
Give three situations where streaming does not help, and say why.
Why can a streaming reasoning model still feel slow to start?
Expected output
after piece 1: {"city" not valid yet
after piece 2: {"city":"Pune" not valid yet
after piece 3: {"city":"Pune","temp" not valid yet
after piece 4: {"city":"Pune","temp":31 not valid yet
after piece 5: {"city":"Pune","temp":31} valid JSON
first usable JSON: piece 5 of 5