PHASE 5Intermediate ~21 min· topic 6 of 8

Topic 5.6

Streaming Structured Output

In one line

Half of a JSON document is not valid JSON, so a normal parser fails until the end. A partial-JSON reader closes the open string and brackets so you can show fields as they arrive, for display only, never for decisions.

Think of it like this

A form being filled in by a person in front of you. You can already read the name they wrote while they are still typing the address. You must not mail the form yet: the address is not finished, and the number they are writing might still gain another digit.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Partial JSON: The beginning of a JSON document that has not finished arriving. It is usually not valid on its own.
Stack: A list where you add and remove from the end. Here it holds the brackets that are still open.
Live view: The best guess of the data so far, including the field currently being written.
Stable view: Only the fields that are certainly finished.
JSON Lines: A format with one complete JSON object per line, so each line can be read on its own.
Dangling: Left hanging at the end, such as a comma or a colon with nothing after it.

Step by step

01Step 1: try the normal parser and watch it fail

Take the reply {"name":"Asha Rao","skills":["python","sql"],"years":12} and cut it after 30 characters. json.loads raises an error, because the string, the list and the object are all still open.

main.pywhole filepython

import json

text = '{"name":"Asha Rao","skills":["pyt'
try:
    json.loads(text)
except json.JSONDecodeError as error:
    print('failed:', error)

02Step 2: scan once, remember what is open

scan() walks the characters. Inside a string it only watches for the closing quote (and skips an escaped one, \"). Outside a string it pushes } or ] when it sees { or [, and pops on a closing bracket. It also remembers the position of the last comma.

Step 2: scan once, remember what is opendiagram

Rendering diagram…

03Step 3: close the text

close() appends what is missing and then calls the normal parser. {"name":"Asha R becomes {"name":"Asha R"}, and ["python", loses its comma and gets ].

04Step 4: fall back to a whole field

When closing still fails (the prefix ends inside a key, for example {"na), cut the text at the last comma and close that. The result is always the fields that were complete before. If there is no comma yet, show an empty object.

05Step 5: build the stable view

The stable view cuts at the last comma directly inside the outer object, so a half-written value is never included. When the whole text parses, both views are equal. Run the example and compare the two columns line by line.

06Step 6: decide at the end

Use the preview for the screen only. When the stream finishes, parse the full text strictly, validate it and then act (Topics 4.5 and 4.6). If you need to act earlier, stream JSON Lines so every line is complete.

Try it yourself

1
Find the lie
Run the example and find the line where years is 1. What will it be at the end? Write down the rule for when a partial number can be trusted (only after a comma or the closing brace follows it).
2
Change the chunk size
Change the step 6 to 5, then 7. Does the live view ever show a key without a value? Look for lines that show {} and explain why they appear.
3
Add a nested object
Change the reply to include "address":{"city":"Pune","pin":"411001"} before years. Does the live view handle the nested object? Does the stable view show address before the comma after it arrives?

Code & diagrams

A partial JSON reader: live and stable views New tab

The same reply is read after every 6 characters. The Java version uses the course's Json helper; real Java projects use Jackson.

Expected output

 6 chars | live: {} | stable: {}
12 chars | live: {"name":"Ash"} | stable: {}
18 chars | live: {"name":"Asha Rao"} | stable: {}
24 chars | live: {"name":"Asha Rao"} | stable: {"name":"Asha Rao"}
30 chars | live: {"name":"Asha Rao","skills":[""]} | stable: {"name":"Asha Rao"}
36 chars | live: {"name":"Asha Rao","skills":["python"]} | stable: {"name":"Asha Rao"}
42 chars | live: {"name":"Asha Rao","skills":["python","sql"]} | stable: {"name":"Asha Rao"}
48 chars | live: {"name":"Asha Rao","skills":["python","sql"]} | stable: {"name":"Asha Rao","skills":["python","sql"]}
54 chars | live: {"name":"Asha Rao","skills":["python","sql"],"years":1} | stable: {"name":"Asha Rao","skills":["python","sql"]}
56 chars | live: {"name":"Asha Rao","skills":["python","sql"],"years":12} | stable: {"name":"Asha Rao","skills":["python","sql"],"years":12}

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Forget the closing quote

In close, stop closing an open string: replace ('"' if in_str else "") with "" (in JavaScript (inStr ? '"' : "") with "").

terminal

$ Run

── what you'll see ──

6 chars | live: {} | stable: {}

12 chars | live: {} | stable: {}

18 chars | live: {"name":"Asha Rao"} | stable: {}

24 chars | live: {"name":"Asha Rao"} | stable: {"name":"Asha Rao"}

30 chars | live: {"name":"Asha Rao"} | stable: {"name":"Asha Rao"}

36 chars | live: {"name":"Asha Rao"} | stable: {"name":"Asha Rao"}

42 chars | live: {"name":"Asha Rao","skills":["python"]} | stable: {"name":"Asha Rao"}

48 chars | live: {"name":"Asha Rao","skills":["python","sql"]} | stable: {"name":"Asha Rao","skills":["python","sql"]}

54 chars | live: {"name":"Asha Rao","skills":["python","sql"],"years":1} | stable: {"name":"Asha Rao","skills":["python","sql"]}

56 chars | live: {"name":"Asha Rao","skills":["python","sql"],"years":12} | stable: {"name":"Asha Rao","skills":["python","sql"],"years":12}

Myth vs fact

Myth

A partial-JSON reader gives me the data early, so I can act early.

Fact

It gives a guess for display. A number can grow, a string can continue, a field can still change. Act only on the final, validated document.

Myth

JSON mode means the stream is always valid JSON.

Fact

Only the final text is valid, and even that can be cut short by max_tokens. Prefixes are never valid until the end.

Myth

Closing brackets by hand is enough for any JSON.

Fact

Strings with escapes, keys without values and half-written numbers or keywords need care. Use a tested library when you can.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
Test the reader exhaustively with the same trick as the SSE parser: take a known valid document, cut it at every position, and assert that the function never raises and that the live value is always a prefix of the final one (apart from the numbers).
▸
A stable view needs a rule per shape. For arrays of objects (a list of results), the usual rule is that all items before the last comma are finished, and the last item is previewed greyed out.
▸
Reasoning models and tool calls have the same problem: tool arguments are partial JSON until the stream ends. You can preview arguments, but run the tool only after the full text validates (Topic 7.4).
▸
A schema-constrained decoder (Topic 4.3) makes the final text valid, not the prefixes. Streaming and constrained output work together, but partial parsing is still needed on the client.

Remember this

1
Why this exists. Topic 5.1 showed that a JSON reply is useless to a program until the last brace. But a screen can do better than a spinner: a card that fills in name, then skills, then years as the model writes them feels fast and alive. For that the screen needs the data so far, even though the text so far is not valid JSON. A partial JSON reader produces a best-guess value from an unfinished prefix.
2
The idea: close what is open. Walk the text once and keep a stack of open brackets ({ and [) and a flag for being inside a string. To read the prefix, add a closing quote if a string is open, drop a dangling comma, turn a dangling "key": into null, and append the closing brackets in reverse order. Then use the normal parser. The example's close() does exactly that.
3
What can still go wrong. A prefix can end in the middle of a key ({"na), a number ("years":1, which later becomes 12), or a keyword (tru). Closing it gives invalid JSON or a wrong value. The example's live() tries the closed text first, and falls back to the text before the last comma, which is always a whole field. Its output also shows two hazards: for a moment skills is [""] (an empty placeholder string before python arrives), and for a moment years is 1. A partial number can be wrong, so a partial value is for display only.
4
Two views: live and stable. The live view shows everything including the field being written (a name growing letter by letter). The stable view shows only fields that are certainly finished: everything before the last comma directly inside the outer object. Use live for typing effects, and stable for anything that triggers an action, such as enabling a button, filling a form field the user may submit, or calling a tool.
5
Never decide on partial data. Do not validate against the schema, save to a database, run a tool or send an email from a partial value. The decision point is the end of the stream: parse the whole text with the strict parser, validate it (Topic 4.5), and only then act. If the final text does not validate, drop the preview and use the retry loop (Topic 4.6).
6
Alternatives. Libraries do this for you: Pydantic can parse partial JSON. pydantic_core.from_json(text, allow_partial=True) returns what is complete and drops a half-written trailing value, so a name still being typed is missing until it ends. The Vercel AI SDK can stream a partially filled object too (its structured-output streaming; older versions call it streamObject, so check your version's docs). Another design avoids the problem. Ask the model for one complete JSON object per line (JSON Lines), so every line is valid on its own. Or stream the explanation as text and request the structured data in a second non-streamed call. Choose the simplest one that gives the screen what it needs.

Explain it without notes

Why is a prefix of a JSON document usually not valid JSON, and how does a partial reader deal with it?

What is the difference between a live and a stable view, and when do you use each?

Why should you never act on partial JSON?

Practice

Write close_brackets(text) that, for text with no strings (only {, }, [, ] and other characters), returns the closing brackets needed, in the right order. For {[1,{ it returns }]}.

Write last_complete_prefix(text) that returns the text before the last comma, or an empty string if there is no comma.

Trade-offs

↔
A live preview feels fast but can show wrong intermediate values; a stable view is safer and updates in larger steps.
↔
JSON Lines gives clean, complete pieces, but asks the model for a slightly unusual format and loses the single-document structure.
↔
A tested library saves you the edge cases and adds a dependency; a hand-written reader is short and teaches the idea.

Done when you can

Done when you can explain why a JSON prefix does not parse.
Done when you can write a reader that closes the open string and brackets.
Done when you can explain live versus stable views and when to use each.
Done when you can say why partial data must not trigger actions.

Command Palette

Words you'll meet

Step by step

01Step 1: try the normal parser and watch it fail

02Step 2: scan once, remember what is open

03Step 3: close the text

04Step 4: fall back to a whole field

05Step 5: build the stable view

06Step 6: decide at the end

Try it yourself

Find the lie

Change the chunk size

Add a nested object

Code & diagrams

Break it on purpose

Forget the closing quote

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can