PHASE 3Intermediate ~27 min· topic 3 of 7

Topic 3.3

When the Stream Breaks

In one line

Once the first bytes arrive the status is already 200, so a failure after that does not look like an HTTP error. A stream can break by throwing, by carrying an error message inside it, by closing early, or by sending data you cannot read. Detect each, keep the text, and offer a fresh try.

Think of it like this

A phone call that goes silent halfway through a sentence. The call did connect, so you cannot say it "failed to connect". You have to notice that the voice stopped, decide whether the person hung up or the line dropped, and call back. A broken stream is that call: it started fine, and you have to notice the ending was wrong.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Mid-stream failure: A failure that happens after the response has started, so the status is already 200.
Error event: A message about an error that is carried inside the stream as data, instead of as an HTTP status.
Truncated stream: A stream that closed before its normal ending. No error is raised, so you must check for the ending yourself.
Normal ending: The last chunk with a finish reason, followed by data: [DONE]. Seeing it proves the answer is complete.
Keep-alive comment: A line starting with : that the server sends only to keep the connection open. It carries no data and is ignored.
Serverless timeout: A hosting limit on how long one function may run. If an answer takes longer, the platform cuts the response.
Resume: Continuing a broken generation from where it stopped. Most providers do not support it, so a retry starts again.

Step by step

01Four breaks, one status

The first live example opens five streams and reads each with the same parser. Every one answers HTTP 200, including the four that break. The parser never throws: it returns the text and an outcome that says how the stream ended.

Four breaks, one statusdiagram

Rendering diagram…

02Never trust done on its own

In the loop, done: true only says the connection closed. The parser sets a flag when it reads [DONE] and decides at the end: flag set means complete, flag missing means "ended early". This is the whole defence against truncated streams, and it is three lines.

the rulewhole filejs

let complete = false;
// inside the line loop:
if (line.slice(6) === "[DONE]") complete = true;
// after the loop:
return { text, outcome: complete ? "complete" : "ended early, no [DONE]" };

03Keep the text, say the truth

The second example is Mitra's screen. The first request breaks after "You can return an unopened item within". The page keeps that text inside a block that says "The answer was cut off", and shows a Try again button. When the second request starts, the old partial stays visible; it is replaced only when the first new token arrives.

04Failure before any text

If the break happens before the first token there is no partial text, and the block says "Mitra could not answer" instead. The same component handles both because it checks whether cutText is empty.

05Where these come from in production

Phones switch networks. Laptops sleep with a tab open. A reverse proxy has an idle limit. A serverless function reaches its maximum duration. The provider restarts a node. None of these are bugs in your code, and all of them happen every day at scale, which is why this path needs real code and not a comment.

Try it yourself

1
Make the break come before any text
In the second example, change the first scripted reply to { text: "", error: "connection lost" } and run it.
The log says cut off, kept: "" and the page shows "Mitra could not answer." with no italic text above it. One component, two messages, chosen by whether any text arrived.
2
Ignore the error event
In the first example, delete the line that starts if (chunk.error) and run it again.
The error-event line now says connection lost (Cannot read properties of undefined (reading '0')) (the wording is Chrome's): a parser bug is reported as a network problem. Compare it with the correct line to see why a dedicated check is worth three lines.
3
Add a keep-alive
In the first example, add a line to the serve handler: if (mode === "ping") return sse([": ping\n\n", delta("Hi"), "data: [DONE]\n\n"]); and put "ping" in the list of modes at the bottom.
The new line prints ping | HTTP 200 | kept: "Hi" | outcome: complete. The parser skips the comment line (it does not start with data: ) and still completes. A server may send these every few seconds to stop a proxy from closing an idle connection.

Code & diagrams

Four ways a 200 response goes wrong

All five say HTTP 200. Only reading the stream tells them apart. The wording of the connection-lost message is Chrome's ("network error"); other browsers differ, so real code branches on the error type, not the text.

Preview

Press Run to see the page.

Console

Nothing logged yet.

Expected console output

healthy     | HTTP 200 | kept: "Orders ship in 1 to 2 working days." | outcome: complete
dropped     | HTTP 200 | kept: "Orders ship in 1 to 2" | outcome: connection lost (network error)
error-event | HTTP 200 | kept: "Orders ship in 1 to 2" | outcome: error event: The model is overloaded
truncated   | HTTP 200 | kept: "Orders ship in 1 to 2" | outcome: ended early, no [DONE]
garbled     | HTTP 200 | kept: "Orders ship " | outcome: unreadable data in the stream

Keep the partial answer, say it was cut off, try again

The visitor presses Try again as soon as the notice appears. While the second request waits for its first token the cut-off text is still on screen, so the page never goes blank.

Preview

Press Run to see the page.

Console

Nothing logged yet.

Expected console output

cut off, kept: "You can return an unopened item within"
done: "You can return an unopened item within 7 days of delivery."

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Parse an error event as if it were a token

In the first example, delete the line if (chunk.error) return { text, outcome: "error event: " + chunk.error.message };.

terminal

$ The console, the error-event line

── what you'll see ──

error-event | HTTP 200 | kept: "Orders ship in 1 to 2" | outcome: connection lost (Cannot read properties of undefined (reading '0'))

Myth vs fact

Myth

If res.ok was true, the answer will arrive in full.

Fact

res.ok describes only the start. The stream can still break, carry an error, or close early, all after a 200.

Myth

A stream that ends normally (done: true) is complete.

Fact

A clean close can be a timeout in a proxy or a function limit. Only the normal ending (finish_reason then [DONE]) proves completeness.

Myth

The browser tells you the stream was truncated.

Fact

It does not. An early close looks like any other close, which is why your own code must check for the ending.

Myth

The right retry resumes exactly where the stream stopped.

Fact

Most providers cannot resume a generation. A retry is a new request that starts from the beginning.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
Some providers send errors as a named SSE event (event: error followed by a data line) instead of a bare data line. A robust parser handles both: track the current event: name while reading lines, and treat error as a break (Topic 2.4 covers the format).
▸
If the end of the stream is cut by a hosting time limit, the symptom is a cluster of answers that all stop at about the same age. Plot the length or time of "ended early" answers: a spike at one value points to a limit you can raise, or to a long answer you should shorten.
▸
A user who loses the connection in the middle of an answer often comes back to a page that shows a half answer and a spinner that never ends. Always leave the streaming state through a path that runs on every exit (finally), and listen for the offline event if you want to explain a lost connection.
▸
Do not retry automatically on a broken stream by default. The model has already used tokens, the user may have read half of it, and a silent retry can show a different answer under their eyes. Let the user decide, or retry only when nothing has been shown yet.

Remember this

1
The problem this solves: In Topic 0.4 and Topic 1.3 you checked res.ok and the status code. That only protects the start of a request. By the time the first token arrives, the server has already sent 200 OK, and a status line cannot be taken back. Everything that goes wrong after that has to be found by reading the stream carefully. This topic lists the four ways a stream breaks and how each one looks in code.
2
Break 1: the connection drops. The network disappears (a phone moves from Wi-Fi to mobile data, a laptop sleeps, the server process is killed). The pending reader.read() rejects with a TypeError. Its message depends on the browser (Chrome says network error, Firefox and Safari use their own words), so branch on the error type, not the text. Whatever you already appended is safe in your variable.
3
Break 2: an error inside a 200 response. The server can still write an error message into the stream, because it cannot change the status any more. In the OpenAI-compatible format it is a data line whose JSON has an error key instead of choices. Providers do this when the model is overloaded or a limit is hit after the answer started. If your parser assumes every data line has choices[0], it crashes with a confusing Cannot read properties of undefined. Check for chunk.error first.
4
Break 3: the stream ends early, without an error. The connection closes cleanly after some tokens, but the normal ending (finish_reason, then data: [DONE]) never came. This is what a timeout in a proxy or a hosting platform's maximum function time looks like: the response is cut and the browser sees an ordinary end. A reader that only stops at done would call that answer finished. The rule is: a stream is complete only if you saw the normal ending. No [DONE] means unfinished.
5
Break 4: data you cannot read. A line that is not valid JSON (a half-written line, a proxy injecting HTML, a gateway error page) makes JSON.parse throw. Do not let it surface as a vague error from deep inside the loop, and do not swallow it silently. Catch it, stop reading, keep the text so far, and report the stream as broken. Lines starting with : are comments (keep-alives) and are skipped, which is a different thing (Topic 2.4).
6
What to keep, what to say, how to retry. Keep the text that arrived. Say what happened in words ("The answer was cut off", Topic 3.2) and never show a raw TypeError. Offer Try again, which sends a new request. Most providers cannot resume a half-finished generation, so the retry produces a new answer from the start; keep the old partial visible until the first token of the new one arrives, so the screen never goes blank (Topic 3.4). In production, count these endings: a rising "ended early" rate is how you find a proxy or timeout problem that individual users never report.

Explain it without notes

Why can't res.ok protect you from a stream that fails halfway?

How do you detect a stream that was truncated without an error?

What does an error inside a 200 response look like, and what happens if the parser does not expect it?

What should Mitra do on screen when the stream breaks after some text?

Practice

Add a sixth mode to the first example: slow-garbage, which sends one valid token and then a line data: <html>Bad gateway</html>. What outcome does the parser report?

Write a function endingOf(result) that turns { text, outcome } from the first example into one of "complete", "cut" or "failed" (failed meaning nothing arrived at all).

Trade-offs

↔
Being strict (anything without [DONE] is unfinished) will occasionally label a good answer as cut off if a provider forgets the marker, but the opposite choice silently shows truncated answers as complete. Strict is the safer default.
↔
Keeping the broken partial on screen while retrying avoids a blank flash, at the cost of a moment where two answers could be confused. Marking the old one clearly is what makes that safe.

Done when you can

I can list the four ways a stream breaks after a 200 status.
I can read a stream so that it never throws and always reports how it ended.
I can detect an early close by checking for the normal ending.
I can recognise an error event inside a stream and handle it before reading choices.
I can keep the partial text, label it, and offer a retry.

Command Palette

Words you'll meet

Step by step

01Four breaks, one status

02Never trust done on its own

03Keep the text, say the truth

04Failure before any text

05Where these come from in production

Try it yourself

Make the break come before any text

Ignore the error event

Add a keep-alive

Code & diagrams

Break it on purpose

Parse an error event as if it were a token

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can