nDCG with graded relevance
Gains: D11 = 3, D4 = 2, everything else 0. The discount is log2(position + 1), written as log(x) / log(2) so all three languages agree to the last digits shown.
Recall and hit rate ignore order. MRR rewards the right passage for being near the top, and nDCG does the same for several passages with different importance. Both are recomputed here from the formula.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write mrr(first_ranks) taking a list of ranks (0 for a miss) and print it for [1, 3, 0, 2] with 3 decimals.
Write ndcg(gains, k) for a list of gains in returned order, using log2(i + 1) and an ideal built by sorting the same gains. Print it for [1, 0, 2] and k = 3.
Explain it without notes
Compute reciprocal rank for first relevant positions 1, 2 and 5, and explain why MRR is the mean of these.
Write the formula for DCG@k and nDCG@k, and compute nDCG for gains 0, 3 at positions 1 and 2 when the ideal is 3, 0.
Why can two retrieval systems have the same hit rate and different MRR?
When would you choose nDCG over MRR?
Expected output
good order gains 3, 2, 0, 0, 0 DCG 4.2619 ideal 4.2619 nDCG 1.0000
swapped gains 2, 0, 3, 0, 0 DCG 3.5000 ideal 4.2619 nDCG 0.8212
pushed down gains 0, 0, 0, 2, 3 DCG 2.0219 ideal 4.2619 nDCG 0.4744