Command Palette

Search for a command to run...

Hectal
Phase 2Beginner3 of 16 in AI Frontend

Streaming Fundamentals

How a streamed answer really arrives: bytes, text, Server-Sent Events and the chunk format, then rendering and scrolling it smoothly. Mitra learns to stream.

Phase 0 read a stream with a few lines and trusted them. This phase opens the box. You will see what a response body really is (a ReadableStream of bytes), why bytes must be turned into text with care (an emoji can arrive cut in half), what a Server-Sent Events message looks like on the wire, and how to read the OpenAI-compatible chunk format that most providers speak.

Then the screen side: how to show hundreds of tokens without making the page stutter, and how to scroll while text arrives without fighting a reader who has scrolled up to re-read something.

Everything here uses plain fetch, with no SDK, against a scripted stand-in model inside the page. In Phase 5 an SDK will do this work for you. You will then know exactly what it is doing, and what to look at when something breaks. By the end, Mitra streams its answers with a parser that survives every split.

0/8 · 0%
8 topics ~4 h 21 code blocks & diagrams
Start with the first topic
1
2.1

Why Streaming

Streaming does not make a model finish sooner. It lets the user start reading after the first few tokens instead of staring at a blank box until the last one. It costs more code and creates partial results to handle, and it does not suit short answers or JSON you cannot show half-finished.

25 min 2 code practice

2
2.2

ReadableStream and the Reader

A response body is a ReadableStream: a queue of byte chunks that arrive over time. You take chunks out with a reader, looping on read() until done is true. A chunk is whatever the network handed over, not a word or a line, and a stream can be read only once by one reader at a time.

25 min 3 code practice

3
2.3

Bytes to Text

The network carries bytes, and some characters take several bytes. A chunk boundary can fall inside a character, so decoding each chunk on its own produces broken text. Use one decoder for the whole stream with stream: true, or TextDecoderStream, which does it for you.

25 min 3 code practice

4
2.4

Server-Sent Events

Server-Sent Events (SSE) is a plain-text format for a server to push messages over one open HTTP response: lines like data: …, with a blank line ending each message. Chat apps read it with fetch, because EventSource cannot send a POST body, and they buffer text across network packets because a line can be cut anywhere.

28 min 3 code practice

5
2.5

The OpenAI-Compatible Stream Format

Most providers stream answers as SSE events whose data is a chat.completion.chunk JSON object: a role-only first chunk, content deltas, a final chunk with finish_reason (and often usage), then the literal text [DONE]. A parser must survive every split and every odd chunk: empty choices, error events, a missing [DONE].

29 min 3 code practice

6
2.6

Rendering Tokens Without Jank

Tokens can arrive faster than a screen can usefully redraw. Keep the update cheap: change only the one message that is growing, draw at most once per frame, never leave the last piece unshown, and draw the cursor with CSS so it is not part of the text.

25 min 3 code practice

7
2.7

Scrolling While Text Arrives

Follow new text only while the reader is at the bottom. If they scroll up, stop following and offer a Jump to latest button. Use a small threshold, scroll before paint, and plan for phone keyboards and for browsers that scroll for you.

24 min 2 code practice

8
2.8

Build: Mitra Streams

Mitra now reads the real chat stream format with a parser that survives every split, shows the answer growing at most once per frame, scrolls politely, and tells apart a finished answer, a cut-short one, a broken one and an empty one.

38 min 2 code practice

Practise this phase

Hands-on exercises: write the code, see your page live, and press Check to have it tested automatically.

1

Decode Without Breaking Characters

The route /api/ask sends the text Chai ☕ चाय 🙂 as bytes in packets of 3 bytes, so many characters are cut in two.

Beginner
2

Buffer the Events

The route /api/chat streams the answer Chai is tea. in the chat format, cut into small packets so lines arrive in pieces.

Beginner
3

Say Why the Answer Ended

readSSE is ready: it yields { type: "text", text } and { type: "finish", reason } parts. Two buttons ask the route a short and a long question.

Intermediate
4

Follow Only the Reader Who Is at the Bottom

#add adds a line of text to the box #box. Make the box follow new lines (scroll to the bottom) only when the reader is already within 24 pixels of the bottom.

Intermediate