The smallest RAG: one question it answers, one it gets wrong
Look at the "shared words" line of the second question: the right passage (D3) loses to D1 by one small word.
A complete RAG pipeline fits in forty lines: score passages by shared words, put the best one in a prompt, let the model answer and cite. It works, until a question uses different words from the document.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write score(question, text, stop) that counts shared different words, ignoring any word in the set stop. Print the scores of the refunds and password passages for the login question with stop = {"in", "get", "i", "my", "how", "do"}.
Write retrieve_top(question, docs, k) returning the ids of the k highest-scoring passages, best first, ties broken by position in the list. Print the top 2 for both questions.
Explain it without notes
Walk through the four steps of the smallest RAG pipeline.
Why did "I forgot my login, how do I get back in?" retrieve the refunds passage?
Why is this failure called silent, and why is that dangerous?
The model produced a wrong answer here. Is the model at fault? How would you prove it either way?
Expected output
Question: What does the monthly pass cost?
shared words: D1=0 D2=0 D3=1 D4=0 D8=3
retrieved: D8
answer: The monthly pass costs 25 dollars and includes unlimited rides of up to 45 minutes. [D8]
Question: I forgot my login, how do I get back in?
shared words: D1=2 D2=0 D3=1 D4=0 D8=0
retrieved: D1
answer: A ride cancelled at least 2 hours before its start time is refunded in full. [D1]