The baseline harness: version 0 against version 1
Ten test questions, each with the passage that should be found. Both retrievers apply their own evidence bar (2 shared words, BM25 of 2.0) and return at most 3 ids.
AskDocs version 1 swaps Phase 0's word counting for an inverted index with BM25, behind the same retrieve(question, k) interface, and a small harness measures the change.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Add a function explain(question, doc_id) to a small BM25 setup that returns the list of (term, part) pairs for one passage, using the formula from Topic 1.4 with k1 = 1.2, b = 0.75. Test on two passages: "late fee" and "monthly pass" with the question "late".
Write refuse_if_weak(hits, min_score) that returns None when the best score is under min_score (or there are no hits) and the hits otherwise. Print the results for scores [2.43, 0.64] and [1.09] with a bar of 2.0.
Explain it without notes
Why can AskDocs swap its retriever without touching answer?
How does the Index avoid scoring passages that cannot match?
Why did the hit rate at 3 fall from 0.90 (Topic 1.6) to 0.70 here?
Why is MIN_SCORE = 2.0 only a guess?
Expected output
retriever at 1 at 3 not in top 3
v0 overlap 0.50 0.60 D3, D1, D9, D12
v1 BM25 0.60 0.70 D1, D9, D12