Validating a JSON answer with citations
Replies 1 to 5 are scripted. The validator returns every problem it finds, so one log line explains the failure.
Prose is not always the right output. Quotes, steps and a validated JSON object with an answer, citations and a confidence let a product show sources and let code check the model. Streaming needs one extra care: hold back half-finished citation markers.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write quote_ok(quote, source) that returns True when the quote appears in the source after lower-casing both and collapsing runs of whitespace to a single space. Print the result for the quote "unlimited rides" and "free rides" against "The monthly pass costs 25 dollars and includes unlimited rides of up to 45 minutes.".
Write next_step_ok(reply) that parses a JSON string and returns "ok" if it has a non-empty list steps of strings, else "bad" (also for invalid JSON). Print the result for '{"steps": ["Open the app", "Tap Forgot password"]}', '{"steps": []}' and 'oops'.
Explain it without notes
When would you ask for a quote, steps or JSON instead of prose?
What are the layers of validating a JSON answer?
Why is a model's own confidence not enough?
What goes wrong when you stream an answer with citations, and how do you fix it?
Expected output
1: ok
2: citation out of range: 3
3: not valid JSON
4: answer is missing or empty; an answer without citations must be the refusal; confidence must be one of high, medium, low, none
5: ok