A refusal threshold, and its limits
Uses the fake embedder from Topic 8.3. Scores are rounded to four decimals.
A grounded answer names the numbered source it came from, your code checks that the citation is real and supports the claim, and when the documents do not contain the answer the system says so instead of guessing.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 6 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write citations(text) that returns the list of integers found as [n] in a text, in order. Print it for "A [1] and B [2], again [1].".
Write valid_citations(text, n_sources) that returns True only if the text has at least one citation and every cited number is between 1 and n_sources. Print it for three texts.
Explain it without notes
Why do citations improve trust, and what must your code do with them?
Why is a fixed refusal sentence better than letting the model phrase its own?
Why can a similarity threshold not fully solve the refusal problem?
How would you test a RAG system's refusal behaviour?
Expected output
question best document at 0.30 at 0.80
get my money back 1.0000 returns answer answer
how long is the battery warranty 0.8575 warranty answer answer
how much does it cost 0.7071 shipping answer refuse
how much is a lock 0.7071 shipping answer refuse
do you sell locks 0.0000 returns refuse refuse
what is the weather today 0.0000 returns refuse refuse