RAG and Vector Search · Exercise 5 of 5 Previous
Intermediate
Measure hit rate and MRR
Write evaluate(rankings, golds). rankings is a list of ranked id lists (one per question) and golds is the list of right ids in the same order.
Return a dict with three keys: "hit1" (questions with the right id first), "hit3" (right id within the first three) and "mrr" (the average of 1 / place of the right id, counting 0 when the id is missing).
What Check tests
- hit1 counts questions with the right id first
- hit3 counts the right id in the first three
- MRR averages 1 / place
- A missing right id counts as 0, not as an error
- The example ranking set
Stuck? One hint at a time
The solution unlocks after your first Check. Try first: that is where the learning happens.
main.py
Sign in to run your code.
Output
Press Run to see what your program prints.