What a pair score costs
The 8 ms and 5 ms are example numbers, not measurements. Measure your own model and hardware.
An embedding squeezes each passage into one vector before it ever sees the question. A cross-encoder reads the question and one passage together and scores the pair: much sharper, far too slow for the whole index, so it re-orders only a shortlist.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write rerank_ms(n, pair_ms) that returns the milliseconds to score n pairs sequentially. Print it for n = 30 and n = 1000 with pair_ms = 8.
Write order_by_pair(question, candidates, scorer) that returns the candidates sorted by scorer(question, candidate), best first, keeping the original order for ties. Print the result for candidates ["a", "bb", "cc"] with a scorer that returns the length of the candidate.
Explain it without notes
How does a cross-encoder differ from a bi-encoder, and why is it more accurate?
Why is a cross-encoder too slow to run over the whole index?
What happens to a relevant passage that the first stage did not return?
Why shouldn't you trust the toy scorer's results as evidence about real re-rankers?
Expected output
passages scored time at 8 ms per pair
10 80 ms (0.1 s)
50 400 ms (0.4 s)
100 800 ms (0.8 s)
1000 8000 ms (8.0 s)
100000 800000 ms (800.0 s)
vector search, then 30 pairs: 245 ms
re-ranking the whole index of 100000 would take 3265 times as long