A stale cached answer, and invalidation by document
Each answer remembers which document it used. On an edit the entries that used it are dropped and nothing else is.
Most questions are repeats, so each stage of the pipeline (embedding, retrieval, the final answer) can remember its result. The hard part is the key: it must contain everything that changes the result, so a changed document, model or prompt never serves an old answer.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write cache_key(stage, model, index_version, question) that normalises the question (lower-case words joined by single spaces) and returns "embed|<model>|<q>" for stage embed and "<stage>|<model>|index<version>|<q>" otherwise. Show that two spellings of one question share a key and that a new index version changes the retrieval key but not the embedding key.
Write invalidate(cache, deps, doc_id) that removes from cache every key listed in deps[doc_id], removes the deps entry, and returns how many keys were removed (0 when the document is unknown).
Explain it without notes
What should the cache key of a RAG answer contain, and why?
Compare version keys with dependency tracking for invalidation.
Why is a semantic cache risky?
Which stages of a RAG pipeline are worth caching, and in what order do you look them up?
Expected output
invalidate on edit: no
The monthly pass costs 25 dollars. (fresh)
Shops open at 7 am. (fresh)
The monthly pass costs 25 dollars. (cached)
Shops open at 7 am. (cached)
invalidate on edit: yes
The monthly pass costs 25 dollars. (fresh)
Shops open at 7 am. (fresh)
dropped cache entry: pass price?
The monthly pass costs 30 dollars. (fresh)
Shops open at 7 am. (cached)