Check citations in the model's reply
Six scripted replies stand in for a model. The checker needs no model to run.
A grounded answer names the numbered source it came from, your code checks that the citation is real and supports the claim, and when the documents do not contain the answer the system says so instead of guessing.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 6 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write citations(text) that returns the list of integers found as [n] in a text, in order. Print it for "A [1] and B [2], again [1].".
Write valid_citations(text, n_sources) that returns True only if the text has at least one citation and every cited number is between 1 and n_sources. Print it for three texts.
Explain it without notes
Why do citations improve trust, and what must your code do with them?
Why is a fixed refusal sentence better than letting the model phrase its own?
Why can a similarity threshold not fully solve the refusal problem?
How would you test a RAG system's refusal behaviour?
Expected output
reply 1: ok Unused bikes can be returned within 30 days [1].
reply 2: BAD (number 60 is not in the cited source) Bikes can be returned within 60 days [1].
reply 3: BAD (cites [3] but there are only 2 sources) Express shipping costs 12 dollars [3].
reply 4: BAD (no citation) Returns are free for everyone.
reply 5: ok (refusal) Not in the documents.
reply 6: BAD (number 1 is not in the cited source; number 2 is not in the cited source) Express shipping takes 1 to 2 days [1].