A semantic cache at two thresholds
A toy embedder (sets of words) stands in for a real embedding model. The threshold logic is the real thing.
A semantic cache returns a saved answer when a new question means almost the same as an old one, found by comparing embeddings with a similarity threshold. It saves more calls than exact caching, but a threshold set too low returns the wrong answer to a question that only looks similar.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 6 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write similarity(a, b) for two sets of words returning shared / sqrt(len(a) * len(b)), 0.0 for an empty set. Print it to 3 decimals for {"reset", "password"} and {"change", "password"}.
Write lookup(question_words, cache, threshold) where cache is a list of (word_set, answer) and the function returns the answer of the best entry at or above the threshold, else None. Use the similarity function. Show a hit and a miss.
Explain it without notes
What problem does a semantic cache solve that an exact cache doesn't?
What is the role of the threshold, and what happens at the two extremes?
Why is a false semantic hit worse than a miss?
How would you choose a threshold for a real product?
Expected output
threshold 0.90:
'Can I reset the password?' -> HIT 'reset password' (1.000) right
'How to change my password' -> miss (best 0.500)
'cancel subscription today' -> miss (best 0.667)
'Is the office open today?' -> miss (best 0.333)
hits 1, wrong hits 0
threshold 0.60:
'Can I reset the password?' -> HIT 'reset password' (1.000) right
'How to change my password' -> miss (best 0.500)
'cancel subscription today' -> HIT 'cancel order today' (0.667) WRONG
'Is the office open today?' -> miss (best 0.333)
hits 2, wrong hits 1