RAG and Vector Search · Exercise 1 of 5Next
Intermediate
Recall, precision and hit rate
Write recall_at_k(ranked, relevant, k), precision_at_k(ranked, relevant, k) and hit_at_k(ranked, relevant, k). ranked is a list of passage ids, best first; relevant is a set of ids that answer the question.
Recall is the relevant passages found in the top k divided by all relevant passages. Precision is the same count divided by k. Hit is 1 when at least one relevant passage is in the top k, else 0.
What Check tests
- Recall counts relevant passages found
- Recall is partial when one is missing
- Precision divides by k
- Precision divides by k even when the list is shorter
- Hit is 1 or 0
Stuck? One hint at a time
The solution unlocks after your first Check. Try first: that is where the learning happens.
main.py
Sign in to run your code.
Output
Press Run to see what your program prints.