Cache keys: which change breaks which cache
The caches only remember that a key was seen. The expensive work is simulated by the WORK counters.
Most questions are repeats, so each stage of the pipeline (embedding, retrieval, the final answer) can remember its result. The hard part is the key: it must contain everything that changes the result, so a changed document, model or prompt never serves an old answer.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write cache_key(stage, model, index_version, question) that normalises the question (lower-case words joined by single spaces) and returns "embed|<model>|<q>" for stage embed and "<stage>|<model>|index<version>|<q>" otherwise. Show that two spellings of one question share a key and that a new index version changes the retrieval key but not the embedding key.
Write invalidate(cache, deps, doc_id) that removes from cache every key listed in deps[doc_id], removes the deps entry, and returns how many keys were removed (0 when the document is unknown).
Explain it without notes
What should the cache key of a RAG answer contain, and why?
Compare version keys with dependency tracking for invalidation.
Why is a semantic cache risky?
Which stages of a RAG pipeline are worth caching, and in what order do you look them up?
Expected output
answer key: toy-v1|index7|p3|what does the monthly pass cost
1 first ask: answer=miss retrieve=miss embed=miss
2 same, other case: answer=hit
3 prompt changed: answer=miss retrieve=hit
4 index changed: answer=miss retrieve=miss embed=hit
5 model changed: answer=miss retrieve=miss embed=miss
real work done: embed=2 retrieve=3 answer=4