Three questions: from memory, then with sources
The memory is a scripted stand-in for what a real model might say with no sources. The notes after the arrows name the gap each answer falls into.
A model's knowledge stops on its training date, never included your private data, cannot be updated quickly and comes without sources. Retrieval closes each of these gaps at question time.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write gap(question, memory, docs) that returns "stale" if memory has an answer and the documents have a different one, "missing" if memory has no answer, and "same" otherwise. Represent both as dictionaries from question to answer.
Write answer_or_refuse(hit) that takes a (doc_id, text) tuple or None and returns the text followed by the id in square brackets, or the refusal sentence.
Explain it without notes
Name four reasons a language model needs retrieval to answer questions about a company's policies.
Why is a stale answer more dangerous than an "I don't know"?
Why is editing a document a better way to update knowledge than retraining a model?
What kinds of problem does retrieval not solve?
Expected output
Question: What does the monthly pass cost?
from memory: 20 dollars. <- out of date: the price changed after training
with sources: The monthly pass costs 25 dollars and includes unlimited rides of up to 45 minutes. [D8]
Question: Is a cancelled ride refunded?
from memory: Yes, within 30 days. <- made up: the policy is private, the model never saw it
with sources: A ride cancelled at least 2 hours before its start time is refunded in full. [D1]
Question: Do you rent scooters?
from memory: Yes, scooters are available. <- made up: it sounds likely, so the model says it
with sources: I can't find that in the handbook.