Command Palette

Search for a command to run...

Hectal
Phase 9: Evaluating RAG
RAG and Vector Search · Exercise 2 of 5 PreviousNext
Intermediate

Mean reciprocal rank

Write first_rank(ranked, relevant) that returns the position (starting at 1) of the first id in ranked that is in relevant, or 0 if none is.

Write mrr(runs) where runs is a list of (ranked, relevant) pairs. It returns the mean of 1 / first_rank, counting a miss as 0.

What Check tests

  1. first_rank counts from 1
  2. A miss is 0
  3. The first relevant one counts, not later ones
  4. mrr averages reciprocal ranks
  5. A miss lowers the mean

Stuck? One hint at a time

The solution unlocks after your first Check. Try first: that is where the learning happens.

main.py
Sign in to run your code.

Output

Press Run to see what your program prints.