PHASE 4Intermediate ~21 min· topic 3 of 8

Topic 4.3

JSON Mode vs Schema-Constrained Output

In one line

Prompting only asks for JSON. Provider features make the decoder enforce it: JSON mode guarantees valid syntax, and schema-constrained output also forces the shape and the allowed values. Neither guarantees the answer is right.

Think of it like this

A vending machine keypad. A blank notepad lets you write any order, including a snack the machine doesn't have. A keypad only lets you press A1 to D4. Constrained decoding is a keypad for the model: at each step, the keys that would break the format simply can't be pressed.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Token: A small piece of text (a word, part of a word, or a symbol) that a model reads and writes one at a time.
Score (logit): A number the model gives each possible next token. Higher means more likely.
Mask: A rule that removes some tokens from the choices at a step so they cannot be picked.
Constrained decoding: Writing text with a mask at every step so the result always follows a format or grammar.
JSON mode: A provider setting that guarantees valid JSON syntax, without checking your fields.
JSON Schema: A JSON document that describes the shape of other JSON: keys, types, allowed values.
Strict: A flag in schema mode that makes the provider enforce the schema exactly.

Step by step

01Step 1: the unconstrained model

The example's fake model scores the pieces it could write at each of three steps. With no limits it picks the top-scoring piece each time, and its favourite first piece is the chatty Sure, . The result is Sure, mixed!, which is not JSON.

02Step 2: JSON mode masks the syntax

In JSON mode the first piece must open the object and the last must close it, so the chatty pieces are removed. The middle step is still free, so the top-scoring value mixed wins. The result is valid JSON with a value you never allowed.

03Step 3: a schema masks the values too

With a schema, the middle step may only choose from the enum positive, negative, neutral. mixed is masked, and the best allowed value, positive, is chosen. Shape and values are now right by construction.

Step 3: a schema masks the values toodiagram

Rendering diagram…

04Step 4: the toy is simpler than the real thing

Real decoders work on tokens that are often fragments of words, and they track a grammar built from your schema so they know which tokens are legal at each position. The example uses whole pieces to show the idea. Do not read it as how a particular provider implements it.

05Step 5: try it on a real model

The lab below sends a real request in JSON mode (response_format set to json_object) to a free hosted model. Read the raw HTTP request to see the field, then reduce your prompt until the reply loses a key, and add the key back. Goals check that the reply is valid JSON and has the keys you asked for.

06Step 6: the schema request in code

The last block shows the plain-HTTP shape of a schema-constrained request in all three languages. It is not run here because it needs a key. It is the same body you would send from any language.

Try it yourself

1
Predict the three lines
Before running, work out by hand which piece wins at each step in each mode (the highest score among the allowed pieces). Then run and compare.
2
Change the scores
Give positive a score of 50 in the middle step. Which modes change their answer, and why does JSON mode not change?
3
Remove a piece from the mask
Change the schema mode so the last step allows ! too. What does the verdict say, and what does it tell you about a mask that is too loose?
4
The lab
Send the lab three times. Do the replies differ? Open Raw HTTP and find response_format. Then remove the word JSON from the system message and send again. Does the provider complain, or does the reply change? Providers differ here, so read what comes back.

Code & diagrams

Three modes of the same fake model New tab

A toy: whole pieces of text instead of real tokens. It shows what a mask does, not how any provider implements it.

Expected output

no limits  Sure, mixed!  =>  not JSON
JSON mode  {"sentiment":"mixed"}  =>  JSON, value not allowed
schema     {"sentiment":"positive"}  =>  JSON, value ok

JSON mode on a real modelreal model

Try: Send as it is: JSON mode is on (see response_format in the raw request). Then change the system message so it no longer asks for the key "reason" and send again. What does the reply have now? Put the key back, then add "or sarcastic" to the list of sentiments and see whether the model respects the new word.

System

user

Modelsmall, fast, reasons firstTemperature · 1.0Max tokens (1–1024)Reasoning effort Stream (SSE) JSON mode

Goals

The reply is valid JSON
It has the keys sentiment and reason
sentiment is one of the three allowed words

The same idea with a schema, as plain HTTPpython

Not run here: it needs your own GROQ_API_KEY and the network. In strict mode Groq rejects a schema unless every object sets additionalProperties to false and lists every property in required (checked against the live API on 2026-10-10). A field that may be missing is declared as a type that allows null.

# pip install httpx      (plain HTTP: this shape works with any OpenAI-compatible endpoint)
import os

import httpx

body = {
    "model": "openai/gpt-oss-20b",
    "messages": [{"role": "user", "content": "Classify the sentiment of: Great service, slow delivery."}],
    "max_tokens": 500,
    "response_format": {
        "type": "json_schema",
        "json_schema": {
            "name": "sentiment",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {"sentiment": {"type": "string", "enum": ["positive", "negative", "neutral"]}},
                "required": ["sentiment"],
                "additionalProperties": False,
            },
        },
    },
}
r = httpx.post(
    "https://api.groq.com/openai/v1/chat/completions",
    headers={"Authorization": "Bearer " + os.environ["GROQ_API_KEY"]},
    json=body,
    timeout=30,
)
r.raise_for_status()
print(r.json()["choices"][0]["message"]["content"])  # a JSON string that matches the schema

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Allow a bad value in the enum

In the Python example add "mixed" to VALUES (a fourth allowed value) and run it.

terminal

$ Run

── what you'll see ──

no limits Sure, mixed! => not JSON

JSON mode {"sentiment":"mixed"} => JSON, value ok

schema {"sentiment":"mixed"} => JSON, value ok

Myth vs fact

Myth

JSON mode means the reply matches my format.

Fact

It means the reply is valid JSON. The keys and values can be anything.

Myth

With strict schemas I no longer need validation.

Fact

Truncation, unsupported keywords, provider changes and wrong-but-allowed values still exist. Validate anyway; it costs microseconds.

Myth

Constrained output makes the model smarter.

Fact

It only limits what can be written. A wrong answer in the right shape is still wrong.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
Constrained decoding changes the probabilities the model samples from, which can subtly hurt quality on hard tasks. If accuracy drops, test an unconstrained prompt plus a validator on your own examples (Phase 9).
▸
Put reasoning before the answer in the schema (for example a reason field first, then label), because the model writes fields in order and thinking first can improve the label.
▸
Pin the schema in version control with the prompt. A silent change to an enum is a breaking change for every consumer of that field.

Remember this

1
Why this exists. Topic 4.2 showed that asking for JSON works most of the time. At a million calls, "most of the time" is thousands of failures, each costing a retry. If the provider's own decoder refuses to write anything invalid, those failures mostly disappear, and you don't pay for the repair loop. This is a feature of the API, not of your prompt.
2
How a model writes, in plain words. A model produces text one token (a small piece of text) at a time. At each step it computes a score for every token it knows, turns the scores into probabilities and picks one (Topic 1.6). Recall from Topic 1.1: a token is a small piece of text. Constrained decoding adds one step: before picking, the decoder sets the score of every token that would break the rules to "impossible" (a mask). The model can only choose among valid continuations. The example below does exactly this with three steps and a handful of tokens.
3
JSON mode guarantees the text is valid JSON syntax: braces close, strings are quoted, no trailing commas. It does not know your fields, so {"sentiment": "mixed"} is perfectly valid JSON and may not be a value you allow. On Groq (and OpenAI-compatible APIs) you turn it on with response_format: {"type": "json_object"}, and the docs ask you to say you want JSON in the prompt.
4
Schema-constrained output (often called structured outputs) takes a JSON Schema (Topic 4.4) and forces the reply to match it: the right keys, the right types, enum values from your list. On Groq the request uses response_format: {"type": "json_schema", "json_schema": {"name": ..., "strict": true, "schema": ...}}. In Groq's documentation (checked 2026-10-10), strict schema mode is listed for openai/gpt-oss-20b, openai/gpt-oss-120b and qwen/qwen3.8-27b, while JSON mode is listed as available for all models. Support changes with models and providers, so check the provider's page for the model you use.
5
What constraints cannot do. They cannot make the answer true: "sentiment": "positive" for an angry review is perfectly valid. They cannot stop a reply being cut off by max_tokens, which leaves invalid JSON even in these modes (check finish_reason). And providers support only part of JSON Schema, so a keyword they don't support may be rejected or ignored. Keep validating on your side (Topic 4.5) and keep the repair loop (Topic 4.6). Constrained output cuts the failures; it doesn't make them impossible.
6
Which to use. Schema-constrained output when the model and provider support it (best guarantee for shape). JSON mode when only that is available, plus a validator. Prompt-only plus extraction when you use a model with neither. In all cases the prompt should still describe the fields in words, because the schema controls what can be written, and the prompt tells the model what each field means.

Explain it without notes

What does JSON mode guarantee, and what does it not?

Explain constrained decoding in plain words.

Give three reasons you should still validate when a provider offers strict schema output.

When would you choose prompt-only JSON plus an extractor over a provider feature?

Practice

Write pick(scores, allowed) that returns the highest-scoring key of the dictionary scores, considering only keys in the list allowed (all keys if allowed is None).

Write in_enum(text, field, values) that parses text as JSON and returns True only if it is a JSON object whose field is in values. Invalid JSON gives False.

Trade-offs

↔
Schema-constrained output gives the strongest shape guarantee, but ties your code to a provider feature and a schema subset.
↔
JSON mode is widely available and simple; you still write the validator.
↔
Strict constraints can slightly lower quality on hard reasoning tasks; measure on your own examples.

Done when you can

Done when you can explain what a mask does at each step of generation.
Done when you can tell what JSON mode guarantees and what it leaves to you.
Done when you can write the request body for JSON mode and for a strict schema.
Done when you can name three failures that remain even with strict schemas.
Done when you can check a provider's docs for which models support which mode.

Command Palette

Words you'll meet

Step by step

01Step 1: the unconstrained model

02Step 2: JSON mode masks the syntax

03Step 3: a schema masks the values too

04Step 4: the toy is simpler than the real thing

05Step 5: try it on a real model

06Step 6: the schema request in code

Try it yourself

Predict the three lines

Change the scores

Remove a piece from the mask

The lab

Code & diagrams

Break it on purpose

Allow a bad value in the enum

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can