PHASE 4Intermediate ~19 min· topic 1 of 8

Topic 4.1

Why Free Text Breaks Programs

In one line

People read prose; programs need fields. When a program depends on a model's free-text answer, every small change in wording becomes a bug. Structured output is how you turn the model's text into data your code can trust.

Think of it like this

A paper form vs a handwritten letter. A clerk handling forms looks in box 3 for the date and box 5 for the amount, and can process a thousand an hour. If people send letters instead, one writes "I paid last Tuesday", another "$40 on the 3rd", a third forgets the amount. A human can cope. A machine that expects box 5 cannot. Structured output is giving the model a form to fill in.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Structured output: A model reply in a fixed, machine-readable shape (usually JSON) instead of free prose.
Free text: Normal sentences with no fixed layout. Easy for people, hard for programs.
Parse: Turn text into data your program can use, such as a dictionary or an object.
JSON: A text format for data: objects {"a": 1}, lists [1, 2], strings, numbers, true, false and null.
Field: One named value inside a JSON object, such as urgency.
Contract: The agreement between two pieces of code about the exact shape of the data passed between them.
Quiet failure: The call succeeds but the content is wrong for your program, so nothing alerts you.

Step by step

01Step 1: the program wants a label

A support tool must put each ticket into a queue by urgency. The model's answer has to become one of three words. A sentence is not good enough.

02Step 2: parse prose

The first rule that comes to mind is "the reply starts with Urgency: ". The example runs it on four replies a real model could give for the same ticket. Only one starts that way, so the parser reads 1 of 4. The other three are perfectly good answers to a person.

main.pywhole filepython

def read_prose(reply):
    if reply.startswith("Urgency: "):
        return reply[len("Urgency: "):]
    return None

03Step 3: add more rules, and watch them multiply

You could add "or contains the word high", "or starts with High", "or says urgent". Each rule fixes one reply and invites a wrong match ("not high priority" contains "high"). This is the road to a pile of regular expressions nobody dares to touch.

04Step 4: ask for fields instead

Tell the model to reply with a JSON object with the keys urgency and reason. Now the contract is the key name. Reading it is json.loads(reply)["urgency"], and all four replies work.

Step 4: ask for fields insteaddiagram

Rendering diagram…

05Step 5: structure is not a guarantee

The JSON replies in the example are all clean because the example scripted them. A real model can still wrap JSON in chatter, leave a field out, or write "urgency": "urgent". The next topics deal with exactly those failures: parsing (4.2), provider features that help (4.3), describing the shape (4.4), checking it (4.5), repairing (4.6) and saying "unknown" (4.7).

Try it yourself

1
Predict first
Before running, count how many of the four prose replies start with Urgency: . Then run and compare with the first section.
2
Teach the parser one more phrasing
Make read_prose also accept replies that contain the word high (use in). Run it. How many are read now? Then add the reply This is not high priority. to the list and see what the parser says.
3
Compare languages
Run the example in all three languages. The only differences are how each language splits a string and reads JSON; the output is identical.

Code & diagrams

The same four answers: prose vs JSON New tab

The replies are scripted, so the example is repeatable. The Java version uses the course's Json helper (real projects use Jackson).

Expected output

-- prose replies --
reply 1: urgency=high
reply 2: cannot read it
reply 3: cannot read it
reply 4: cannot read it
read 1 of 4
-- JSON replies --
reply 1: urgency=high
reply 2: urgency=high
reply 3: urgency=medium
reply 4: urgency=high
read 4 of 4

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Change the capital letter

In the prose list, change the first reply from Urgency: high to urgency: high (a lowercase u) and run the first section.

terminal

$ Run

── what you'll see ──

-- prose replies --

reply 1: cannot read it

reply 2: cannot read it

reply 3: cannot read it

reply 4: cannot read it

read 0 of 4

Myth vs fact

Myth

If I write the format in the prompt, the model will always follow it.

Fact

It follows it most of the time. "Most of the time" at a million calls means thousands of broken replies. You still need to check every reply.

Myth

Structured output means the answer is correct.

Fact

It means the answer has the right shape. A well-formed {"urgency": "low"} can still be the wrong urgency.

Myth

Regular expressions are enough to read model text.

Fact

They work until the wording shifts. A fixed JSON contract is far more stable than any pattern over prose.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
Define the contract first, as a schema or a type, and write the prompt from it. When the prompt and the parser are written separately they drift apart.
▸
Measure the parse failure rate in production (Topic 11.4). A rate that creeps up after a model update is an early warning, long before users complain.
▸
Keep a reply that humans read apart from the data your code reads. Two fields (answer_for_user, route) beat one blob you both display and parse.

Remember this

1
Why this exists. An LLM's job is to continue text, so asked "how urgent is this ticket?" it answers the way a person would, in a sentence. The sentence changes from call to call, because the model picks its words with some randomness. Your program is not a person. It wants urgency to be exactly one of low, medium, high, so it can route the ticket. Anything else is an error you have to handle.
2
"Just ask nicely" works in the demo. You write "Reply in the format Urgency: <level>" and it works ten times. On the eleventh, the model writes "I would say this is high urgency." Nothing crashes at the model: the call succeeded (Topic 0.4 called this a quiet failure). The crash or the wrong behaviour happens later, in your parser, or worse, nowhere: the ticket silently goes to the wrong queue.
3
Parsing prose is a guessing game. To read a free-text answer you write string searches, regular expressions or keyword lists. Each one encodes a guess about the model's wording. Every new phrasing ("High - the customer cannot log in.") needs a new rule, rules start to clash, and a model update changes the phrasing again.
4
Structure moves the work to where it is cheap. Ask for JSON with named fields. Parsing becomes one library call (json.loads, JSON.parse), and the field names are the contract between the model and your code. What can still go wrong is smaller and checkable: the text may not be valid JSON, the fields may be missing or wrong, the values may be outside the allowed list. The rest of this phase handles those, one at a time.
5
JSON is the common choice. It is text, so a model can write it; every language can read it; and it has just enough types (strings, numbers, booleans, null, lists, objects) for most data. Other formats (XML, YAML, a custom key: value layout) work, but JSON is what providers build features around, so it is where this course focuses. A note on Java: the JDK has no JSON library. Real Java projects use Jackson (or Gson). This course gives Java examples a tiny helper class called Json so they run in your browser; Topic 4.5 shows the Jackson version.
6
The goal of the phase. By Topic 4.8 you will have one function in all three languages that takes a prompt and returns either a checked, typed value or a safe fallback. It also tells you which one you got. The example below is the smallest taste: the same four answers read as prose (one is understood) and as JSON (all four are).

Explain it without notes

Why is parsing free-text model output fragile?

What does "structured output" give you, and what does it not give you?

Why is a prose reply that your parser cannot read often worse than a crash?

Practice

Write read_urgency(reply) that parses a JSON reply and returns the value of urgency, or "unknown" if the reply is not valid JSON or has no urgency key.

Write count_readable(replies, reader) that returns how many replies a reader function could read (a reader returns None when it cannot).

Trade-offs

↔
Structured output costs a few extra tokens and some prompt text, and buys a parser you can trust.
↔
A strict format is easy for code and a little less natural for the model; keep the fields few and clearly named.
↔
Prose is still right for text people read. Use structure for what code consumes.

Done when you can

Done when you can explain why a parser based on wording breaks.
Done when you can name what structure gives you and what it does not.
Done when you can turn a prose question ("how urgent is this?") into a JSON contract with named fields.
Done when you can say why a silent misread is worse than a crash.

Command Palette

Words you'll meet

Step by step

01Step 1: the program wants a label

02Step 2: parse prose

03Step 3: add more rules, and watch them multiply

04Step 4: ask for fields instead

05Step 5: structure is not a guarantee

Try it yourself

Predict first

Teach the parser one more phrasing

Compare languages

Code & diagrams

Break it on purpose

Change the capital letter

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can