RAG and Vector Search · Exercise 5 of 5 Previous
Intermediate
Measure a retriever's hit rate
tests is a list of (question, expected_id) pairs; retriever(question, k) returns a ranked list of ids.
Write hit_rate(retriever, tests, k) returning the share of tests whose expected_id is in the retriever's top k results, as a float between 0 and 1. With no tests, return 0.0.
Write misses(retriever, tests, k) returning the questions (in order) whose expected id is not in the top k.
What Check tests
- Hit rate at 1
- Hit rate at 2 counts the second place
- No tests gives 0.0
- Misses are listed in order
- Works with a retriever that ignores k
Stuck? One hint at a time
The solution unlocks after your first Check. Try first: that is where the learning happens.
main.py
Sign in to run your code.
Output
Press Run to see what your program prints.