Accept an answer only if its quote is in the source
The model is a scripted fake with four replies. The check is real code that you can use with a real model.
A model predicts likely text, not true facts, so it can produce fluent, confident answers that are false. Knowing the causes tells you the defences: ground answers in sources, ask for evidence, allow I do not know, and verify what matters.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write quote_ok(quote, source) that returns True when quote is non-empty and appears in source ignoring case and extra spaces. Test it with "Opens at 9 AM" against "The office opens at 9 am.", and an empty quote.
Write summarise_outcomes(statuses) that counts how many times each status appears, returning a dictionary in first-seen order. Use ["grounded", "rejected", "grounded", "honest"].
Explain it without notes
Why do language models hallucinate?
Name four situations that raise the chance of hallucination.
How does asking for a quote and checking it with code reduce hallucination?
Why must your prompt allow the model to say it does not know?
Expected output
Q: When does the Pune office open?
[grounded] 9 am
Q: Does the Pune office have a gym?
[rejected] I could not verify that, so I will not answer.
Q: Who manages the Pune office?
[honest] I could not find that in the document.
Q: When does the Pune office close?
[rejected] I could not verify that, so I will not answer.
grounded=1 honest=1 rejected=2