PHASE 5Intermediate ~24 min· topic 2 of 8

Topic 5.2

Server-Sent Events from First Principles

In one line

Streaming APIs send server-sent events: lines that start with data:, separated by blank lines, ending with [DONE]. Network chunks can cut a line anywhere, so a parser must buffer until the blank line.

Think of it like this

A friend reads you a long message over a bad phone line. You hear "I will be there at se" ... "ven tomorrow". You cannot act on the first half. You keep the half-sentence in your head until the rest arrives and the pause tells you the sentence is over. An SSE parser does the same: it keeps the unfinished text until a blank line says the event is complete.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Server-sent events (SSE): A simple way for a server to push a stream of text events over one HTTP response, separated by blank lines.
Event: One message in the stream: one or more lines followed by a blank line.
Chunk: A piece of bytes the network delivers. It can end anywhere, not only at the end of a line.
Buffer: A piece of memory where you keep unfinished text until the rest arrives.
[DONE]: The last data line of an OpenAI-style stream. It is plain text, not JSON.
Keep-alive comment: A line starting with a colon (: ping) that keeps the connection busy so proxies do not close it. Readers ignore it.
Delta: The new part of the reply in this event, for example the next few characters, not the whole reply so far.

Step by step

01Step 1: look at the raw stream

This is what a chat API sends with stream: true. Every event is a data: line and a blank line. The first event here is a keep-alive comment. The reply text is spread over delta.content fields, and the stream ends with the non-JSON line data: [DONE].

stream.txtwhole filetext

: ping

data: {"choices":[{"index":0,"delta":{"content":"Hel"},"finish_reason":null}]}

data: {"choices":[{"index":0,"delta":{"content":"lo"},"finish_reason":null}]}

data: {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

02Step 2: the naive parser

The first example splits the stream by lines inside each chunk. It works when the whole stream arrives as one chunk (size 1000). It loses data for chunk sizes 170 and 100, because a chunk boundary cuts a data: line into two halves and neither half is a whole line.

Step 2: the naive parserdiagram

Rendering diagram…

03Step 3: keep a buffer

The fix is a class with one string, the buffer. feed(chunk) appends the chunk, cuts complete events off the front at each blank line, and keeps the unfinished tail. It returns the list of complete events found so far (possibly empty).

sse.pywhole filepython

buffer = ""
for chunk in chunks:
    buffer += chunk
    while "\n\n" in buffer:
        event, buffer = buffer.split("\n\n", 1)
        handle(event)  # a complete event
# whatever is left in buffer is unfinished

04Step 4: read the data field

Inside one event, look at each line: ignore lines starting with :, and for lines starting with data: remove the prefix and one optional space. If an event has several data: lines, join them with a newline (the SSE rule). Chat APIs send one JSON object per event.

05Step 5: stop at [DONE]

Check for the literal [DONE] before parsing JSON, and stop reading. Then run the stream through chunk sizes 1000, 60, 20 and 1. All four must give the same text. If they do not, the parser still depends on chunk boundaries.

06Step 6: see the real thing

The lab sends a real request with stream: true and shows the raw lines. Find the first chunk (it often carries only the role), the chunks with content, the last chunk with finish_reason (Groq puts usage numbers inside it, under x_groq; Topic 5.3 explains where usage lives), and the final [DONE]. Then read the shown Python, JavaScript and Java code that does the same with a real connection.

Try it yourself

1
Predict the damage
In the first example, change the chunk sizes to (1000, 45, 7). Before running, guess whether the text still shows Hel, lo and !. Then run and look at the unreadable-lines count. Does a smaller chunk always lose more?
2
Add a second ping
In the parser example, put ": another ping\n\n" between the second and third payload in RAW. Run it. The text must stay Hello!: the comment is ignored at every chunk size.
3
Test the one-character case
Look at the output for chunk size 1: the parser was called once per character. Explain why the answer is still Hello! by finding the line in feed that keeps the unfinished tail.
4
Compare with the lab
In the lab, look at the raw stream and find one data: line. Copy its JSON into any JSON viewer and find delta. What other keys does the real chunk carry that the course example leaves out?

Code & diagrams

The naive parser loses data when chunks cut lines New tab

The same stream is cut into chunks of 1000, 170 and 100 characters. Only the first size survives a parser that assumes whole lines.

Expected output

chunk size 1000:  1 chunks, text: Hello!, unreadable lines: 0
chunk size  170:  2 chunks, text: Hello, unreadable lines: 0
chunk size  100:  4 chunks, text: Hel, unreadable lines: 3

A buffering SSE parser gives the same text for every chunk size New tab

Feed chunks, cut at each blank line, keep the unfinished tail. A keep-alive comment is ignored and [DONE] stops the reader.

Expected output

chunk size 1000:   1 chunks, text: Hello!
chunk size   60:   6 chunks, text: Hello!
chunk size   20:  17 chunks, text: Hello!
chunk size    1: 326 chunks, text: Hello!

Reading a real SSE responsepython

Not run here: it needs a network call and your own GROQ_API_KEY. Plain HTTP, so it works with any OpenAI-compatible provider. Each language joins chunks into lines for you.

# pip install httpx      (plain HTTP: this is what any SDK does for you)
import json
import os

import httpx

url = "https://api.groq.com/openai/v1/chat/completions"
headers = {"Authorization": "Bearer " + os.environ["GROQ_API_KEY"]}  # the key comes from the environment
body = {
    "model": "qwen/qwen3.8-27b",
    "stream": True,
    "messages": [{"role": "user", "content": "Count from 1 to 5."}],
}

with httpx.stream("POST", url, headers=headers, json=body, timeout=30) as response:
    response.raise_for_status()
    for line in response.iter_lines():  # httpx already joins the network chunks into whole lines
        if not line.startswith("data:"):
            continue  # blank separator lines and ": comments"
        data = line[5:].strip()
        if data == "[DONE]":
            break
        choices = json.loads(data)["choices"]  # can be empty on a usage-only chunk
        if choices:
            print(choices[0]["delta"].get("content") or "", end="", flush=True)  # content can be null

See the raw SSE linesreal model

Try: Send with streaming on and open the raw stream. Find: the first chunk (does it have content?), a chunk with a piece of the reply, the chunk with finish_reason set, any usage numbers, and the final data: [DONE] line. How many content chunks were there for a five-line answer?

System

user

Modelmid-size, also reads imagesTemperature · 0.0Max tokens (1–1024) Stream (SSE) JSON mode

Goals

The reply finished on its own
The reply contains the numbers 1 to 5

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Forget the buffer

In the parser example, change self.buffer += chunk so the buffer is reset on every call: self.buffer = chunk (in JavaScript this.buffer = chunk, in Java buffer = chunk).

terminal

$ Run

── what you'll see ──

chunk size 1000: 1 chunks, text: Hello!

chunk size 60: 6 chunks, text:

chunk size 20: 17 chunks, text:

chunk size 1: 326 chunks, text:

Myth vs fact

Myth

One network chunk is one SSE event.

Fact

Chunks are cut by the network and by proxies, not by the server's events. One chunk can hold three events, or half of one.

Myth

Every data line is JSON.

Fact

The last line of an OpenAI-style stream is data: [DONE], which is not JSON. Comments (: ping) are not data at all.

Myth

SSE is a special protocol that needs special servers.

Fact

It is plain text over an ordinary HTTP response that stays open. Any server that can write a response in pieces can send it.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
Test the parser the way this topic does: the same input cut at every chunk size from 1 up to the whole stream must produce the same events. This one test catches most streaming bugs, and it needs no network.
▸
Accept \r\n as well as \n, and ignore unknown fields (event:, id:, retry:). Some providers send an event: line before each data: line (the Anthropic API does); a parser that only handles data: lines still works if it ignores the others, but you may want the event type. Anthropic's stream also has no [DONE]: it ends with a message_stop event, sends ping events along the way, and reports failures as an event: error event.
▸
Browsers have EventSource, but it can only send GET requests without a body or custom headers. Chat APIs use POST with a JSON body and an Authorization header, so browser code uses fetch and reads the body stream itself, as the JavaScript example shows.
▸
A reader must treat the end of the connection without [DONE] as a failure, not a success. Topic 5.7 covers detecting it.

Remember this

1
Why this exists. HTTP was built for one request and one response. To send a reply in pieces, the server keeps the response open and writes small messages onto it. A plain stream of bytes needs a convention for where one message ends. Server-sent events (SSE) is the simplest widely used convention: text, in lines, with a blank line ending each event. Browsers support it natively, and the chat APIs of OpenAI-compatible providers use it when you set stream: true.
2
The format. The response has Content-Type: text/event-stream. An event is one or more lines, then a blank line. A line data: {...} carries the payload (one JSON object per event for chat APIs). A line starting with : is a comment, used as a keep-alive so proxies do not close an idle connection. Other fields exist in the SSE standard (event:, id:, retry:) and chat APIs mostly use only data:. The last event of an OpenAI-style stream is the literal text data: [DONE], which is not JSON: check for it before you parse.
3
The trap: chunks are not lines. The network delivers bytes in pieces of any size, cut anywhere: between events, in the middle of a line, even in the middle of a multi-byte character. A chunk is not an event. The first example splits the stream into chunks of 170 and 100 characters and feeds each chunk to a parser that assumes whole lines. At 100 it loses most of the reply and counts unreadable lines. At 170 it loses the last piece (!) without any error at all. With a chunk size of 1000 (everything in one piece) it looks fine, which is why this bug passes testing on a fast local network and fails in production.
4
The fix: a buffer. Append every chunk to a buffer. While the buffer contains a blank line (\n\n), cut off everything before it as one complete event and keep the rest. Whatever is left after the last blank line is an unfinished event and waits for the next chunk. The second example does exactly this and prints Hello! for chunk sizes 1000, 60, 20 and even 1. Real servers may also use \r\n line endings: robust parsers accept both (the SSE standard allows \n, \r\n and \r).
5
Decode bytes carefully. Real network chunks are bytes, not text. A character such as an accented letter or an emoji is several bytes, and a chunk boundary can fall in the middle. Decode with a streaming decoder that keeps the incomplete bytes for the next chunk (TextDecoder with { stream: true } in JavaScript, an incremental UTF-8 decoder in Python, a reader that works on lines in Java). Decoding each chunk on its own produces garbled characters. The course examples use ASCII, so the bytes-versus-text issue is described, not run.
6
You rarely write this in production, because SDKs and line-based readers do it for you (iter_lines, BodyHandlers.ofLines(), the browser's EventSource). But you must understand it: when a proxy re-chunks the stream, when a library hides a chunk bug, when you write the relay for your own server (Topic 5.4), or when you see strange half-lines in logs. The real-model lab below shows the raw lines of a real stream.

Explain it without notes

Describe the SSE wire format used by chat completion streams.

Why does a parser that splits each chunk by newline fail, and how does a buffer fix it?

Why is [DONE] special, and what happens if you pass it to a JSON parser?

Why must a streaming decoder be used for bytes, and what goes wrong without it?

Practice

Write parse_event(block) that takes the text of one event (without the final blank line) and returns the joined data of its data: lines, or None if there are none (for example a comment).

Write chunk_everywhere(text) that returns the text cut into every possible 2-piece split ([(text[:i], text[i:]) ...] for i from 1 to len(text) - 1) so you can test a parser at every cut point.

Trade-offs

↔
SSE is simple text and works through ordinary HTTP infrastructure, but it only flows server to client; a second channel (a normal request) is needed to cancel or send more.
↔
A hand-written parser teaches you the format, but production code is safer on a well-tested library reader unless you control both ends.
↔
Small events give smooth typing but cost more bytes in framing: each token carries a whole JSON object. Some servers batch several tokens per event.

Done when you can

Done when you can describe the data: line format, blank-line separators, comments and [DONE].
Done when you can explain why a chunk is not an event.
Done when you can write a buffering parser that gives the same output at every chunk size.
Done when you can read the raw lines of a real stream in the lab and name the parts.

Command Palette

Words you'll meet

Step by step

01Step 1: look at the raw stream

02Step 2: the naive parser

03Step 3: keep a buffer

04Step 4: read the data field

05Step 5: stop at [DONE]

06Step 6: see the real thing

Try it yourself

Predict the damage

Add a second ping

Test the one-character case

Compare with the lab

Code & diagrams

Break it on purpose

Forget the buffer

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can