The harness: versions 1, 3 and 6 on the same questions
Toy components, eight questions: read the shape of the comparison, not the numbers.
AskDocs version 6: keyword and vector search fused with RRF, a re-rank stage behind the same retrieve interface, a near-copy filter and an evidence bar on the re-ranker's score. A small harness compares it with versions 1 and 3.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write refuses(scores, bar) that returns True when no score in the list reaches the bar. Print it for [0.4, 0.3] and [0.4, 0.9] with bar 0.5, using the words refuse and answer.
Write version_table(rows) that takes a list of (name, hit1, total) and prints one line per version in the form name hit@1 x/y. Print it for [("v1", 4, 8), ("v6", 8, 8)].
Explain it without notes
Describe the flow of a question through AskDocs v6.
Why is the evidence bar applied to the re-ranker's score and not to the fused RRF score?
What did the lost-items question show about the evidence bar?
Why keep the retrieve interface unchanged?
Expected output
v1 keyword hit@1 4/8 hit@3 7/8 answers 2 of 2 unanswerable questions
v3 vector hit@1 2/8 hit@3 5/8 answers 2 of 2 unanswerable questions
v6 hybrid+rerank hit@1 8/8 hit@3 8/8 answers 0 of 2 unanswerable questions