PHASE 4Intermediate ~20 min· topic 2 of 8

Topic 4.2

Asking for JSON (and How Parsing Fails)

In one line

You can ask any model for JSON in the prompt, and it usually complies, but replies arrive in code fences, wrapped in chatter, with trailing commas, single quotes or cut off. A tolerant extractor handles the common cases, and knowing its limits tells you when you need more.

Think of it like this

Asking a friend to write your address on a card, with nothing else. Most of the time you get the address. Sometimes you get "Sure! Here you go:" before it and "Hope that helps!" after it, sometimes the card is in a fancy frame, sometimes the pen ran out halfway through the postcode. You can still use most of these cards if you know where the address starts and ends.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Code fence: Three backticks around a block of text in Markdown. Models often put JSON inside one.
Trailing comma: A comma after the last item, as in {"a": 1,}. Allowed in some languages, not in JSON.
Truncated: Cut short before the end, for example because the reply hit max_tokens.
Brace: The characters { and } that open and close a JSON object.
Extractor: Code that finds the JSON part inside a longer piece of text.
finish_reason: A field in the response that says why the model stopped: stop (finished) or length (ran out of tokens).

Step by step

01Step 1: the prompt

The prompt asks for JSON and names the keys: Reply with only a JSON object with the keys "name" and "age". No other text. This is the baseline, and it works on a clean day.

02Step 2: the six replies

The example scripts six replies a real model might produce for the same request: clean, in a code fence, with chatter, with a trailing comma, with single quotes, and cut off. Each is something people report in real projects.

03Step 3: a brace-counting extractor

find_object walks the text from the first {, counts { up and } down, and returns the slice when the count returns to zero. It tracks whether it is inside a string so a brace in "a } b" does not confuse it. This handles fences and chatter, because both are outside the braces.

main.pywhole filepython

elif ch == "{":
    depth += 1
elif ch == "}":
    depth -= 1
    if depth == 0:
        return text[start : i + 1]

Step 3: a brace-counting extractordiagram

Rendering diagram…

04Step 4: read the results

Three of the six replies (clean, fence, chatter) are read. The trailing comma and the single quotes are found as objects but rejected by the parser as invalid JSON. The cut-off reply never closes, so no object is found. The program reports each case in plain words instead of crashing.

05Step 5: why not repair everything?

You could delete trailing commas and swap quotes. That works for those two and breaks on the next dialect, or changes a string that really contained a comma. A repair is a guess. Safer: detect, send the problem back to the model (Topic 4.6), and keep the strict parser.

Try it yourself

1
Predict each line
Before running, write down for each of the six replies whether the extractor finds an object and whether the parser accepts it. Then run and compare.
2
Two objects
Change the chatter reply to Sure! Use {"name":"Ada","age":36} or {"name":"Bob","age":41}. Which one does the extractor return? Is that the right one, and could code know?
3
A brace inside a string
Add a reply {"name":"Ada {the first}","age":36}. Does the extractor still read it? Then try deleting the string handling (the in_string lines) and see what breaks.
4
The lab
In the lab, run it three times without changing the prompt. Count how many replies passed both goals. Then tighten the prompt and count again.

Code & diagrams

A brace-counting extractor on six kinds of reply New tab

Backticks are built with a repeat so the example needs no special quoting. Java uses the course's Json helper.

Expected output

clean           ok name=Ada age=36
code fence      ok name=Ada age=36
chatter         ok name=Ada age=36
trailing comma  invalid JSON
single quotes   invalid JSON
cut off         no complete object (cut off?)

Ask for JSON in the prompt, with JSON mode offreal model

Try: Press Send. Look at the raw reply: does it start with a brace, or with a code fence or a sentence? Edit the prompt (for example "Reply with only the JSON object, no Markdown, no other text") until both goals turn green, then Send a few times to see if it stays that way.

System

user

Modelmid-size, also reads imagesTemperature · 1.0Max tokens (1–1024) Stream (SSE) JSON mode

Goals

The whole reply is valid JSON
No Markdown code fence in the reply

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Two objects in one reply

Change the chatter reply to 'Sure! Use {"name":"Ada","age":36} or {"name":"Bob","age":41}' and run the example.

terminal

$ Run

── what you'll see ──

chatter ok name=Ada age=36

Myth vs fact

Myth

Saying "only JSON, no other text" in the prompt prevents extra text.

Fact

It makes extra text rarer, not impossible. Always handle the case.

Myth

The text between the first { and the last } is always the JSON.

Fact

Not if the reply has two objects, or braces in the chatter. Count matching braces instead.

Myth

A repair function that fixes every malformed reply is the right design.

Fact

Each repair is a guess about what the model meant. Detect and retry is safer than guess and continue.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
Check finish_reason before parsing. A length reply is the cause of most "mysterious" parse errors, and the cure is a larger max_tokens (and for reasoning models, room for hidden thinking, Topic 1.8; reading finish_reason is Topic 2.3).
▸
Log the raw reply whenever parsing fails, with a request id. Without the raw text you cannot tell a prompt problem from a model change.
▸
If the model's own reply is often fenced, you can add the fence to the prompt as the expected format and strip it deliberately. A stable known wrapper is easier than a random one.

Remember this

1
Why this exists. A model is not a JSON serializer. It writes the characters of JSON because it saw a lot of JSON, not because anything checks the result. The usual prompt ("Reply with only a JSON object with keys name and age") works most of the time and fails in a small set of repeating ways. Knowing the set lets you handle them with a small piece of code.
2
Code fences. Models are trained on chat and Markdown, so they often wrap JSON in triple backticks, sometimes with the word json after them. json.loads of that text fails at the first backtick. Fix: look for the first { instead of assuming the text starts with it.
3
Chatter before and after. "Sure! Here is the JSON:" and "Hope this helps!" are polite and break parsers. Fix: take only the part from the first { to its matching }. Matching is the key word: "the last } in the text" is wrong when the reply contains two objects or when text after the JSON contains a brace. The extractor below counts open and close braces and ignores the ones inside strings.
4
Not valid JSON. Models trained on code write what looks like JSON in other dialects: a trailing comma before }, single quotes, comments (// like this), unquoted keys, True/None from Python. A strict parser rejects all of them. You can add repairs one by one, but every repair is a guess, so prefer to detect, report and retry (Topic 4.6).
5
Cut off. If the reply hits max_tokens it stops in the middle (finish_reason: "length", Topic 2.3), leaving an object that never closes. No extractor can fix that, because the missing part does not exist. Fix: check finish_reason first, give max_tokens enough room, and retry.
6
Limits of tolerance. The extractor is useful, but it takes the first object it finds, which may be the wrong one if the model writes an example before the answer. And prompts that say "only JSON" reduce the chatter without removing it. For stronger guarantees you need the provider's JSON mode or schema-constrained output (Topic 4.3), and validation of the values (Topic 4.5).

Explain it without notes

Name five ways a model's reply can break a JSON parser, and the fix for each.

Why is "take the text from the first { to the last }" a weaker extractor than counting braces?

Why not just repair trailing commas and single quotes automatically?

A parse error shows up only on long answers. What is the likely cause and how do you confirm it?

Practice

Write strip_fence(text) that removes a leading line starting with three backticks and a trailing line of three backticks, and returns the inside. If there is no fence it returns the text unchanged.

Write is_cut_off(text) that returns True when the first { in the text never gets a matching } (count braces, ignoring strings is not needed for this version).

Trade-offs

↔
A tolerant extractor accepts more replies but can hide model drift; count how often it needed to work.
↔
Strict parsing plus a retry is simpler and safer than clever repairs, at the price of an extra call now and then.
↔
A larger max_tokens prevents truncation and raises the worst-case cost and time.

Done when you can

Done when you can list the usual ways a prompt-only JSON reply breaks.
Done when you can write a brace-counting extractor that skips strings.
Done when you can tell truncation from malformed JSON by looking at finish_reason.
Done when you can explain why you detect and retry instead of repairing silently.

Command Palette

Words you'll meet

Step by step

01Step 1: the prompt

02Step 2: the six replies

03Step 3: a brace-counting extractor

04Step 4: read the results

05Step 5: why not repair everything?

Try it yourself

Predict each line

Two objects

A brace inside a string

The lab

Code & diagrams

Break it on purpose

Two objects in one reply

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can