Command Palette

Search for a command to run...

Hectal
Phase 4Intermediate5 of 15 in LLM Engineering

Structured Output

Make a model return data a program can trust: JSON mode, schemas, validation, repair loops, honest unknowns, and one typed-call function in three languages.

A chat window can show any text. A program cannot. It needs fields: a label from a fixed list, a number in a range, a date, a yes or a no. This phase is about getting those from a model reliably, which is harder than it sounds. A model writes text one piece at a time. Nothing forces that text to be valid JSON, let alone the JSON your code expects.

You will start with the failures: replies wrapped in chatter, cut off in the middle, or using the wrong field names. Then you build each defence by hand, in order. First a tolerant extractor, then a validator that writes errors a model can act on, then a repair loop that sends those errors back, and last an honest way to say "I don't know". Only after that do you meet the library versions (Pydantic, Jackson with Bean Validation, Zod) and the provider features (JSON mode and schema-constrained output).

Every example runs on this page in Python, Java and JavaScript with the same output, using scripted fake models. Two real-model labs let you see JSON mode and a null-instead-of-invention extraction on a free hosted model. The phase ends with the playbook: one typed-call function you can copy.

0/8 · 0%
8 topics ~3 h 18 code blocks & diagrams
Start with the first topic
1
4.1

Why Free Text Breaks Programs

People read prose; programs need fields. When a program depends on a model's free-text answer, every small change in wording becomes a bug. Structured output is how you turn the model's text into data your code can trust.

19 min 1 code practice

2
4.2

Asking for JSON (and How Parsing Fails)

You can ask any model for JSON in the prompt, and it usually complies, but replies arrive in code fences, wrapped in chatter, with trailing commas, single quotes or cut off. A tolerant extractor handles the common cases, and knowing its limits tells you when you need more.

20 min 2 code practice

3
4.3

JSON Mode vs Schema-Constrained Output

Prompting only asks for JSON. Provider features make the decoder enforce it: JSON mode guarantees valid syntax, and schema-constrained output also forces the shape and the allowed values. Neither guarantees the answer is right.

21 min 3 code practice

4
4.4

Describing the Shape with JSON Schema

JSON Schema is a standard way to write down the shape of JSON: which keys exist, which are required, what type each has, and which values are allowed. The same schema can steer the model, drive a provider's constrained decoding and check the reply.

19 min 2 code practice

5
4.5

Validating Output

Parsing tells you the text is JSON. Validation tells you it has the right shape and allowed values. Write a small validator by hand first so you know what libraries do, then use Pydantic, Jackson with Bean Validation or Zod, and write errors a model can act on.

21 min 2 code practice

6
4.6

Repair and Retry Loops

When a reply fails validation, tell the model what was wrong and ask again, up to a fixed number of attempts, then fall back to a safe value. This fixes most failures at the cost of extra calls, so count the attempts and the money.

21 min 2 code practice

7
4.7

Extraction and Classification

Extraction pulls fields out of text and classification picks one label from a list. Both go wrong the same way: a model asked to fill every field will invent a value. Give it an honest way out (null or unknown), and check values against the source.

21 min 3 code practice

8
4.8

Playbook: Typed LLM Calls

The recipes of the whole phase in one place: parse a reply into a value or a list of problems, wrap it in one typed-call function that repairs and falls back, and count what happens. Copy these into any LLM feature.

23 min 3 code practice

Practise this phase

Hands-on exercises: write the code, run it, and press Check to have it tested automatically.

1

Build a JSON Schema from a field list

Write build_schema(fields). fields is a list of (name, kind, required) tuples. kind is either a type name such as "string" or "integer", or a list of allowed strings, which becomes {"type": "string", "enum": [...]}.

Beginner
2

Validate a reply and say what is wrong

Write validate(data, spec) where data is a dictionary and spec maps field names to "string", "integer" or "boolean". Return a list of problems, in the order of spec.

Beginner
3

Extract the first JSON object from chatty text

Write extract_object(text) that finds the first complete {...} in text by counting braces, ignoring braces inside double-quoted strings (a backslash escapes the next character inside a string), and returns it parsed with json.loads.

Intermediate
4

Drop values the source text does not support

Write keep_only_evidence(data, source). data is a dictionary of extracted fields and source is the original text. Return (kept, removed).

Intermediate
5

Classify with unknown and a confidence threshold

Write decide(reply, labels, threshold). reply is the model's text, labels is the list of allowed labels and threshold is an integer.

Intermediate
6

Write the repair loop

Write ask_with_repair(model, prompt, check, max_attempts). model(messages) returns the reply text, and messages is a list of (role, text) tuples. check(text) returns None if the reply is fine, or a sentence that says what is wrong.

Advanced