PHASE 16Advanced ~27 min· topic 6 of 14

Topic 16.6

Structured Output: JSON, Schemas and Pydantic

In one line

Programs need data, not prose, so you ask a model for JSON and then treat its answer as untrusted input: parse it, validate it against a schema with Pydantic, and if it is invalid send the error back and retry a limited number of times.

Think of it like this

A form at the post office. A clerk will not accept a sentence like "send it to my aunt in Pune, she lives near the market". The form has boxes: name, street, city, postcode. If a box is empty or wrong, the clerk hands the form back and says which box. Structured output is the form; validation is the clerk.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

JSON: A text format for dictionaries, lists, strings, numbers, true, false and null.
Parsing: Turning text into Python objects. json.loads does it for JSON.
Validation: Checking that data has the right fields, types and allowed values.
Schema: A description of the shape your data must have.
JSON Schema: A standard, written as JSON, for describing the shape of JSON data.
Pydantic: A library that turns type-hinted classes into validators and schemas.
Literal: A type hint that allows only the exact values listed.
Retry loop: Trying again after a failure, with a limit on how many times.

Step by step

01Why JSON fails in practice

The first example feeds json.loads four replies a model might plausibly give: correct JSON, a trailing comma, single quotes (valid Python, not JSON), and JSON wrapped in a polite sentence. Only the first parses. The error messages come from the standard library and say what went wrong.

02Extract carefully, never guess

A small helper that takes the text between the first { and the last } rescues the chatty case. It still calls json.loads afterwards, so garbage inside the braces still fails. Rescuing formatting noise is fine. Inventing missing data is not.

03Valid JSON is not valid data

A successful parse does not mean the data is right. In the break-it example the JSON is perfect but lacks a field, so data["priority"] raises KeyError later, far from the cause. Validate at the boundary so errors appear at the boundary.

04Define the shape once with Pydantic

The second example defines a Ticket class. priority may only be one of three words, estimate_hours must be above zero and at most 40, and title has a minimum length. Ticket.model_validate_json(raw) either returns a real Ticket object or raises.

main.pywhole filepython

from typing import Literal
from pydantic import BaseModel, Field

class Ticket(BaseModel):
    title: str = Field(min_length=3)
    priority: Literal["low", "medium", "high"]
    estimate_hours: float = Field(gt=0, le=40)

05The retry loop with feedback

Here the fake model is useful for a special reason: a real model fails unpredictably, but you need a test that fails in exactly the ways you want to handle. So it is scripted to fail twice: first it chats instead of giving JSON, then it uses the priority "urgent". Each time the code appends the bad reply and an error message ("priority: Input should be 'low', 'medium' or 'high'") and asks again. The third reply is valid. The loop is bounded by max_attempts.

The retry loop with feedbackdiagram

Rendering diagram…

06Schema from the class

The third example prints Ticket.model_json_schema(). Notice enum for the Literal, minLength and exclusiveMinimum from the constraints, and required for fields without a default. The text goes in the prompt (or the provider's schema option), so the model sees the same rules your validator enforces.

Provider features that constrain output to a schema make bad JSON rarer, but they do not remove the need to validate: check the current documentation for what your provider guarantees.

07Where this goes wrong

Silent coercion: Pydantic by default turns the string "2" into the number 2. This is called lax mode, and it is friendly but can hide mistakes. Decide whether that is acceptable. Another trap: asking for too many fields at once makes models drop some. And always keep a failure path, because some inputs will never produce valid output.

Try it yourself

1
Add a field
Add assignee: str | None = None to Ticket in the second example. Make the last fake reply include it. Does validation still pass without it? Why?
2
Reduce the attempts
Call get_ticket with max_attempts=2. Predict what happens at the end of the loop, then run it.

Code & diagrams

Four replies, one parser New tab

Only the first reply is acceptable JSON. The last line shows rescuing a chatty reply.

Expected output

good -> {'title': 'Fix login', 'priority': 'high'}
trailing comma -> JSONDecodeError: Illegal trailing comma before end of object
single quotes -> JSONDecodeError: Expecting property name enclosed in double quotes
chatty -> JSONDecodeError: Expecting value
extracted: {'title': 'Fix login'}

Validate with Pydantic and retry on invalid New tab

The fake model is passed in as a parameter, so the same function works with a real client later.

Expected output

attempt 1 rejected -> JSON: Invalid JSON: expected value at line 1 column 1
attempt 2 rejected -> priority: Input should be 'low', 'medium' or 'high'
accepted on attempt 3 -> title='Fix login' priority='high' estimate_hours=2.0
{'title': 'Fix login', 'priority': 'high', 'estimate_hours': 2.0}

The JSON Schema Pydantic generates New tab

Expected output

{
  "properties": {
    "title": {
      "description": "Short summary",
      "minLength": 3,
      "title": "Title",
      "type": "string"
    },
    "priority": {
      "enum": [
        "low",
        "medium",
        "high"
      ],
      "title": "Priority",
      "type": "string"
    },
    "estimate_hours": {
      "exclusiveMinimum": 0,
      "maximum": 40,
      "title": "Estimate Hours",
      "type": "number"
    }
  },
  "required": [
    "title",
    "priority",
    "estimate_hours"
  ],
  "title": "Ticket",
  "type": "object"
}

Asking a real model for a schema (not runnable here)python

schema = Ticket.model_json_schema()
prompt = (
    "Return ONLY a JSON object that matches this JSON Schema, with no other text.\n"
    + json.dumps(schema)
    + "\nTask: create a ticket for: login page is broken"
)
# send prompt through your ask() function from Topic 16.3, then:
# ticket = Ticket.model_validate_json(reply_text)

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Trust parsed JSON without validating

Read a field that the model forgot to include.

main.pywhole filepython

import json

reply = '{"title": "Fix login"}'   # valid JSON, but the model forgot a field
data = json.loads(reply)
print("parsed fine:", data)
print("priority is", data["priority"])

terminal

$ python main.py

── what you'll see ──

parsed fine: {'title': 'Fix login'}

Traceback (most recent call last):

File "main.py", line 6, in <module>

print("priority is", data["priority"])

~~~~^^^^^^^^^^^^

KeyError: 'priority'

Myth vs fact

Myth

If I ask for JSON, I get JSON.

Fact

Usually, not always. Models add commentary, wrap in fences, truncate when they run out of tokens, or break rules. Validate every time.

Myth

Retrying on invalid output is always safe and cheap.

Fact

Each retry costs tokens and time, and a model that cannot meet the schema will fail again. Limit attempts and have a fallback.

Myth

Pydantic fixes bad data.

Fact

It rejects bad data, and in lax mode makes safe conversions like "2" to 2. It never guesses missing values.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
Truncation looks like invalid JSON: if the reply hit the output token limit, the closing brace is missing. Check the response's stop reason and raise the limit or shorten the task instead of retrying blindly.
▸
Prefer small schemas and enums over free text. A field the model can only fill from five allowed values is easy to validate and to evaluate (Topic 16.11). Add a "reason" text field if you want an explanation to log.
▸
Pydantic's strict mode refuses conversions such as "2" to 2. Use it for data that must be exact. Keep validation errors out of user-facing messages, since they may echo model text.
▸
Know the three levels of "structured". A plain request in the prompt guarantees nothing. A provider's JSON mode (where offered) aims at syntactically valid JSON but does not enforce your schema. A schema-constrained mode (where offered) restricts generation to your schema. Constrained modes still cannot enforce business rules (a ticket id that exists, a date in the future), may support only a subset of JSON Schema keywords, and can be truncated by the token cap, so semantic validation in Python stays. Read your provider's current documentation for what it supports.
▸
Set model_config = ConfigDict(extra="forbid") on schemas for model output when an unexpected key should be an error instead of being silently dropped. Validate semantic rules with @field_validator or @model_validator, and make the feedback specific: field path plus what was wrong, not just "invalid".

Remember this

1
Language models write text. Your code needs fields it can use: a category, a number, a list of items. So you ask the model to answer in JSON (the text format for dictionaries and lists from Phase 7) matching a shape you describe. The written description of that shape is called a schema. Many providers also offer features that constrain the model to a schema. Tool calling (letting the model ask your code to run a function; Topic 16.10) is another route. The Python-side defence is the same either way.
2
Never trust the model's output. Even with a schema, it can return invalid JSON, wrap JSON in chatty text or code fences (the triple-backtick blocks chat models like to put code in), use a wrong type, invent a value outside your allowed list, or leave out a field. Parse and validate every reply exactly as you would validate input from a user.
3
Parsing turns text into Python objects with json.loads. It raises json.JSONDecodeError on trailing commas, single quotes, or surrounding prose. A careful helper may first strip code fences or take the text between the first { and last }, but never silently accept "close enough".
4
Validation checks the parsed data against the shape you want. Pydantic (Topic 15.4) does this from a class with type hints: BaseModel fields with types, Literal[...] for allowed values (a fixed list of choices, also called an enum), and Field(gt=0, le=40, min_length=3) for constraints. Model.model_validate_json(text) parses and validates in one step and raises a ValidationError whose .errors() lists each problem with its location.
5
Retry on invalid. If validation fails, send the model its own bad answer plus a short message listing exactly what is wrong, and ask again. Models often fix themselves on the second try. Always limit the attempts (two or three), because a model that cannot do it will loop forever and spend your money, and decide what the caller gets when the limit is reached (an error, a default, a human review queue).
6
You can also generate the schema from the model class: Ticket.model_json_schema() returns a JSON Schema dictionary you can put into the prompt or into a provider's schema or tool parameter field. One class then defines the prompt, the parser and the validator, so they never drift apart. This is why almost every AI library is built on Pydantic.

Explain it without notes

Why must you validate a model's JSON even if you asked for a specific schema?

Describe a retry-on-invalid loop and its safety limits.

How does Pydantic help with both prompting and validating?

Your service must extract an invoice into a typed object from a model. Walk through the design from the request to a trusted object, including what you do when it keeps failing.

Practice

Write safe_parse(text) that removes code fences, parses JSON and returns a dict, or None if parsing fails or the value is not a dict.

Write a Pydantic model Review with rating (an integer from 1 to 5) and sentiment (positive, neutral or negative). Validate one good and one bad reply.

Without Pydantic, write validate_ticket(data) that returns a list of problem strings (empty means valid).

Trade-offs

↔
Strict schema versus flexible text. Strict fields are easy to validate and use, but force the model into boxes and may drop nuance. Add an optional free-text "notes" field for what does not fit.
↔
Retry versus repair in code. Retrying costs a model call but handles any mistake. Repairing in code (stripping fences) is free but only for known patterns. Use code for known noise and retries for the rest.

Done when you can

Done when you can list three ways model JSON can be wrong.
Done when you can parse and validate a reply with a Pydantic model.
Done when you can write a bounded retry loop that sends validation errors back.
Done when you can generate a JSON Schema from a Pydantic class and explain each part.
Done when you know a full Hectal course on LLM engineering is planned, with structured output and tool use in depth.

Command Palette

Words you'll meet

Step by step

01Why JSON fails in practice

02Extract carefully, never guess

03Valid JSON is not valid data

04Define the shape once with Pydantic

05The retry loop with feedback

06Schema from the class

07Where this goes wrong

Try it yourself

Add a field

Reduce the attempts

Code & diagrams

Break it on purpose

Trust parsed JSON without validating

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can