Reciprocal Rank Fusion by Hand · RAG and Vector Search

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Reciprocal rank fusion (RRF): A way to merge ranked lists: each passage scores 1 / (k + rank) in every list that contains it, and the scores are added.
Reciprocal: One divided by a number. The reciprocal of 61 is 1/61.
k (the RRF constant): The number added to the rank before dividing. 60 in the search lab.
Fused score: A passage's total RRF score. Only the order of fused scores matters.
Tie: Two passages with exactly equal fused scores. The lab and the examples keep the passages' original order.
Missing from a list: A passage a ranker did not return. It adds nothing to the sum for that list.

Step by step

01Step 1: write the two lists

From the lab, copy the vector order (all 12 passages, best first) and the keyword order (only the 5 passages that share a word with the question). Each list is just a sequence of ids.

lists.pywhole filepython

VECTOR = ["D3", "D10", "D12", "D6", "D1", "D5", "D8", "D9", "D4", "D7", "D11", "D2"]
KEYWORD = ["D1", "D10", "D3", "D5", "D12"]

02Step 2: give points for each place

Walk down each list with enumerate(ranking, start=1) and add 1 / (k + rank) to the passage's total. Start counting at 1, so the best passage has rank 1.

rrf.pywhole filepython

def rrf(rankings, k=60):
    fused = {}
    for ranking in rankings:
        for rank, name in enumerate(ranking, start=1):
            fused[name] = fused.get(name, 0.0) + 1.0 / (k + rank)
    return sorted(fused.items(), key=lambda pair: (-pair[1], NAMES.index(pair[0])))

03Step 3: sort by the total

Highest total first. Break exact ties by the passage's position in the handbook, which is what the lab does. Passages that only one list returned still appear, with half the points.

04Step 4: compare with the lab

Run the example, then open the lab below with Rank by set to Hybrid. The order of the first five rows and the Fused column (to 4 decimals) must match. D3 and D10 both show 0.0323: the example prints six decimals to show that D3 is ahead.

Step 4: compare with the labdiagram

Rendering diagram…

05Step 5: change the constant

The second example prints how much each rank is worth for four values of k, and shows a case where the winner flips between k = 1 and k = 10. Choose k by testing on your own questions; 60 is a sensible start.

06Step 6: know the limits

RRF knows nothing about score gaps and its numbers cannot be thresholded. Keep the raw cosine and BM25 next to the fused score in your trace so that you can still see how strong each match was.

Try it yourself

1
Change the lists
In the first example, move "D3" to the end of KEYWORD (rank 5, pushing the others up). Predict whether D3 stays first, then run.
2
Try other k
Call rrf([VECTOR, KEYWORD], k=1) and k=1000 and print the order. Does D1 ever overtake D10? Try k=0, k=1 and k=2, and check each answer by hand with the formula.
3
Lab: break the tie
In the lab, find two passages whose Fused column looks equal. Which one comes first, and why?

Code & diagrams

Recompute the lab's Fused column New tab

The ranks are copied from the real embedding model in the search lab below, so this table is what the lab shows (the lab rounds the Fused column to 4 decimals).

Sign in to run this example in your browser.

Expected output

passage  vector  keyword  fused
D3            1        3  0.032266
D10           2        2  0.032258
D1            5        1  0.031778
D12           3        5  0.031258
D5            6        4  0.030777
D6            4        -  0.015625
D8            7        -  0.014925
D9            8        -  0.014706
D4            9        -  0.014493
D7           10        -  0.014286
D11          11        -  0.014085
D2           12        -  0.013889
D3 beats D10 by 0.000008

What the constant k does New tab

Sign in to run this example in your browser.

Expected output

k      rank 1   rank 2   rank 10  rank 50  rank 1 / rank 10
1      0.50000  0.33333  0.09091  0.01961  5.50
10     0.09091  0.08333  0.05000  0.01667  1.82
60     0.01639  0.01613  0.01429  0.00909  1.15
1000   0.00100  0.00100  0.00099  0.00095  1.01
fused score of A (ranks 1 and 20) and B (ranks 5 and 5):
  k=1   A 0.54762  B 0.33333  winner: A
  k=10  A 0.12424  B 0.13333  winner: B
  k=60  A 0.02889  B 0.03077  winner: B

Reproduce the Fused columnreal embeddings

Try: Press Search with Rank by set to Hybrid. For each row read the vector rank (#) and the keyword rank (or "no word") and compute 1/(60+vector rank) + 1/(60+keyword rank) on paper. Compare with the Fused column. Then change the question to "What does the monthly pass cost?" and predict which passages have no keyword rank before you press Search.

QuestionPassages (12)

1

2

3

4

5

6

7

8

9

10

11

12

Rank byTop k (retrieved)

Embedding modelall-MiniLM-L6-v2384 numbers per text. Runs in your browser: the first Search downloads about 23 MB once, and your text never leaves this page.

Goals

The password-reset passage ranks first

Press Search. Then change one thing (a word in the question, a passage, the ranker) and compare the scores.

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Set the constant to zero

In the Python tab of the first example, change def rrf(rankings, k=60): to def rrf(rankings, k=0): and run.

terminal

$ Run

── what you'll see ──

passage vector keyword fused

D3 1 3 1.333333

D1 5 1 1.200000

D10 2 2 1.000000

D12 3 5 0.533333

D5 6 4 0.416667

D6 4 - 0.250000

D8 7 - 0.142857

D9 8 - 0.125000

D4 9 - 0.111111

D7 10 - 0.100000

D11 11 - 0.090909

D2 12 - 0.083333

D3 beats D10 by 0.333333

Myth vs fact

Myth

RRF needs both rankers to return the same scale of scores.

Fact

RRF never looks at the scores, only at the order. That is its whole advantage.

Myth

A higher fused score means a better answer.

Fact

Fused scores only order the passages. The maximum possible value depends on how many lists you fused. A fused 0.032 is not "good" or "bad".

Myth

A passage missing from one list is penalised.

Fact

It just gets no points from that list. A passage found by only one ranker can still win if its rank there is first and the others are weak.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
Adding or removing one passage shifts the ranks of passages below it by at most one place in each list, so fused scores change by a tiny amount. Compare that with Topic 6.2, where one outlier rescaled everything.
▸
RRF fuses any number of lists: BM25, two different embedding models, a recency list, a popularity list. Each extra list costs another search. Weighted RRF multiplies a list's points by a weight when one ranker deserves more trust (Topic 6.9).
▸
Fusion needs enough candidates from each list. If each ranker returns only its top 3, a passage ranked 4th by one and 1st by the other is invisible in the first list. Fetch more candidates per list (often 20 to 100) than you finally keep, then fuse and cut.
▸
Some vector databases offer RRF hybrid search built in (Phase 5). The formula is the one on this page: check the constant and tie rules your database uses before comparing its numbers with the lab's.

Remember this

1
Why this exists. Topic 6.2 showed that mixing BM25 and cosine scores needs normalising, a weight and constant re-tuning. Reciprocal rank fusion (RRF) avoids all of that by throwing the scores away and keeping only the ranks (1 for the best passage in a list, 2 for the next, and so on).
2
The formula. For every passage, add up one term per list that contains it: 1 / (k + rank). The constant k is 60 in the search lab and a common default; it comes from the 2009 paper that introduced the method (Cormack, Clarke and Büttcher, SIGIR 2009), where 60 was fixed in a small pilot study and never changed afterwards. It is a convention that works well, not a law. A passage ranked first in one list gets 1/61 = 0.01639. Ranked first in both: 2/61 = 0.03279. A passage missing from a list simply gets nothing from that list. The passage with the highest total is first. Ties keep the passages' original order.
3
Every number, recomputed. The example below takes the exact ranks the search lab shows for "I forgot my login, how do I get back in?". D3 (the password passage) is #1 for vectors and #3 for keywords: 1/61 + 1/63 = 0.032266. D10 (the flat-tyre passage) is #2 and #2: 1/62 + 1/62 = 0.032258. D3 wins by only 0.000008. D1 is #5 and #1: 1/65 + 1/61 = 0.031778. D6 appears only in the vector list at #4: 1/64 = 0.015625.
4
Why agreement beats a single high rank. D6's only claim is one fourth place, worth 0.015625. A passage that is #10 in both lists scores 2/70 = 0.028571, almost twice as much. A passage that is first in only one list (0.01639) scores less than one that is tenth in both. That is the idea of hybrid search in one line: two moderate votes beat one strong vote.
5
**The constant k.** It decides how steeply the points fall with rank. Small k makes the top places count a lot: with k = 1, rank 1 is worth 5.5 times rank 10. With k = 60 it is worth only 1.15 times rank 10, so the top ranks count almost the same and *agreement across lists* matters more. In the second example, passage A (first in one list, 20th in the other) beats passage B (5th in both) at k = 1, but loses at k = 10 and k = 60. The lab uses 60.
6
What RRF gives up. It ignores how *far ahead* a first place is: a passage ahead by a mile and one ahead by a hair both get 1/61. It also produces scores that have no absolute meaning: 0.0323 does not mean "good", so you cannot use a fused score as an evidence threshold (use the raw cosine or a re-ranker's score, Topic 6.8). And the lab shows the Fused column rounded to 4 decimals, where D3 and D10 both read 0.0323; the order is still decided by the full numbers. Topic 7.4 uses RRF again to merge the results of several phrasings of one question.

Explain it without notes

01

State the RRF formula and compute the fused score of a passage ranked 2nd in one list and 7th in another, with k = 60.

02

Why does RRF not need normalised scores?

03

Explain what the constant k does, using two values.

04

What can you not do with a fused score?

Practice

01

Write rrf_score(ranks, k=60) that takes a passage's list of ranks (one per list that contains it) and returns its fused score. Print it for [1, 3] and [10, 10] with 6 decimals.

02

Using rrf_score, print the fused score of a passage first in one list only, then of a passage tenth in both lists. Which is higher?

Trade-offs

↔
RRF: no scale alignment, one constant, robust to outliers, but blind to score gaps and not thresholdable.
↔
Small k: the top of each list dominates. Large k: agreement across lists dominates.
↔
Fusing more lists: more recall, more searches to run and monitor.

Done when you can

Done when you can compute a fused score by hand from two ranks.
Done when you can reproduce the lab's Fused column from its ranks.
Done when you can explain what k = 1 and k = 60 do differently.
Done when you can explain why a fused score is not an evidence threshold.

Command Palette

Words you'll meet

Step by step

01Step 1: write the two lists

02Step 2: give points for each place

03Step 3: sort by the total

04Step 4: compare with the lab

05Step 5: change the constant

06Step 6: know the limits

Try it yourself

Change the lists

Try other k

Lab: break the tie

Code & diagrams

Break it on purpose

Set the constant to zero

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can