HyDE with a toy embedder
A toy: six hand-made word groups (money, time, account, damage, safety, battery), summed and normalised. A real model has hundreds of learned directions. The guesses are scripted. Real scores are in the lab.
Expansion adds words or text to the query so it can match passages written in other words: synonyms and acronyms, or a short guessed answer that is embedded instead of the question. It raises recall, and it can also pull in noise.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 6 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write expand(words, table) that returns the words followed by the table's extra words for each of them, without repeats and keeping order. Print the result for ["refund", "cancel"] with table = {"refund": ["refunded", "credit"], "cancel": ["cancelled"]}.
Write score(weights, text) that sums the weights of the words of weights (a dict) found in the text. Print the score of the late-fee sentence for {"late": 1.0, "fee": 0.5, "charged": 0.5}.
Explain it without notes
What problem does query expansion solve, and what is the risk?
Explain HyDE in your own words. Why can a guess retrieve better than the question?
Give a situation where HyDE makes things worse.
How would you stop expansion from lowering precision?
Expected output
question: What happens if I'm slow getting back?
search: D2 1.000, D9 1.000, D6 0.962
guess: Returning the bike late costs extra, a fee for every 15 minutes after the booked end time.
search: D6 1.000, D1 0.983, D2 0.962
question: Can I take my dog along?
search: no signal
guess: Pets are not allowed on the bike and a cleaning fee of 20 dollars applies.
search: D8 0.970, D12 0.514, D4 0.500