Retrieve, then re-rank: choosing the first stage's size
TOY pair scorer and toy vectors, as in Topic 6.4. The first-stage ranks of the three questions (1st, 5th, 6th) come from running this hybrid. A real pipeline's numbers will differ: measure yours.
Run a fast, wide first stage to collect candidates, then a slow, sharp second stage on only those. Two sizes matter: how many candidates you hand over, and how many you keep. Measure both.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write recall_at(rankings, golds, n) that returns how many questions have the gold id within the first n of its ranking. Print it for n = 1 and n = 3 given rankings = [["a", "b", "c"], ["b", "a", "c"], ["c", "b", "a"]] and golds = ["a", "a", "a"].
Write budget_ms(search_ms, pair_ms, candidates, parallel) returning the stage-one time plus the re-ranking time when parallel pairs are scored at once (round the number of rounds up). Print it for 5, 8, 30 with parallel 1, and with parallel 10.
Explain it without notes
Explain the two stages of retrieve-then-re-rank and what each is optimised for.
How do you choose the first stage's size?
What can go wrong if the re-ranker is a remote service?
A re-ranker did not improve results. List two possible reasons.
Expected output
Q: What is the age limit for helmets?
stage 1, hybrid, 8 candidates: D5 D1 D3 D6 D4 D9 D8 D10
stage 2, pair scores: D4=0.87 D5=0.00 D1=0.00 D3=0.00
final 3: D4 D5 D1
first stage pair calls gold first time per question
3 3 1 of 3 24 ms
5 5 2 of 3 40 ms
6 6 3 of 3 48 ms
8 8 3 of 3 64 ms
12 12 3 of 3 96 ms