BM25 over the twelve handbook passages
Same formula and constants as the search lab's keyword ranker: k1 = 1.2, b = 0.75, idf = ln(1 + (N - df + 0.5) / (df + 0.5)).
BM25 repairs TF-IDF: repeated words add less and less to the score (saturation, controlled by k1), and long passages are scaled down against the average length (controlled by b). It is the default keyword ranker of search engines.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write idf(n, df) with BM25's form and print it for n = 12 and df = 1, 8 and 12, to 3 decimals.
Write factor(tf, k1=1.2) for a passage of average length (len = avg_len, so the length part is 1) and print it for tf = 1, 2, 5.
Explain it without notes
Write the BM25 score for one term and explain each part.
What does k1 control, and what is the maximum value of the saturating factor?
What does b control, and what do b = 0 and b = 1 mean?
Why does the 1 + inside BM25's idf matter?
Why can't you compare BM25 scores between two different queries?
Expected output
N=12 average length=19.417 words
the df=8 idf=0.425
monthly df=1 idf=2.159
pass df=1 idf=2.159
cost df=1 idf=2.159
top 3: D8=5.231, D10=2.430, D3=0.642
D8 has 15 words
the tf=1 part=0.468
monthly tf=1 part=2.381
pass tf=1 part=2.381