PHASE 6Intermediate ~21 min· topic 9 of 9

Topic 6.9

Playbook: Hybrid and Re-rank Recipes

In one line

Copy-ready recipes: a weighted RRF function, an MMR function, a two-stage retriever that survives a failing re-ranker, a hosted re-ranker call, and the checklist to run before you ship.

Think of it like this

A mechanic's toolbox has a few well-made tools that fit most jobs. You do not redesign a spanner for each bike. These recipes are the spanners of hybrid search: small, tested and meant to be copied.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Weighted RRF: RRF where each ranked list's points are multiplied by a weight.
Fallback: A safe default used when a stage fails, such as the first-stage order.
Timeout: The longest you wait for a call before giving up.
Rate limit: A provider's cap on how many requests you may send in a period.
Configuration: Settings kept outside the code, such as weights, lambda and sizes, so they can change without a release.

Step by step

01Step 1: fuse with weights

weighted_rrf(rankings, weights, k) adds weight / (k + rank) per list and sorts by score, then by name. With [1, 1] it equals RRF; with [1, 2] the keyword list counts double.

weighted_rrf.pywhole filepython

for ranking, weight in zip(rankings, weights):
    for rank, name in enumerate(ranking, start=1):
        fused[name] = fused.get(name, 0.0) + weight / (k + rank)

02Step 2: pick diverse results

mmr(query, items, lam, k) repeats: score each unchosen item, take the best, add it. The first of equal scores wins, so results are repeatable.

03Step 3: wrap the re-ranker

two_stage calls the first stage, then the re-ranker inside try. If it raises, print one line and return the first-stage candidates. The caller never sees an exception from stage two.

Step 3: wrap the re-rankerdiagram

Rendering diagram…

04Step 4: plug in the real call

The hosted-model code below has the same signature as the fake re-ranker, so it replaces it without changing two_stage. It sets a 2 second timeout so a slow answer becomes a fallback.

05Step 5: keep the table in your tests

Run the Topic 6.7 harness in CI, so a change to a stage shows up as a changed row.

06Step 6: ship with the checklist

Go through the checklist below. Each line prevents one failure that teams meet in production.

Try it yourself

1
Change the weights
Call weighted_rrf with weights [2, 1] and [1, 3]. At what weight does D4 pass D1?
2
Break the re-ranker on purpose
Make good_reranker raise an error for the question "q". Check that the output still returns candidates and prints one explanatory line.
3
Tune lambda
Run mmr with lam 1.0, 0.8, 0.6 and 0.4 on the three items. At which value does the second pick change?

Code & diagrams

Three recipes: weighted RRF, MMR, a two-stage retriever with fallback New tab

The re-rankers here are fakes (a table of scores and a function that raises). The recipe code itself is what you copy.

Expected output

plain RRF:    D1=0.03227, D3=0.03227, D2=0.01613, D4=0.01613
keyword x2:   D3=0.04866, D1=0.04814, D4=0.03226, D2=0.01613
mmr lambda 1.0: copy 2, copy 1
mmr lambda 0.5: copy 2, other
good re-ranker:   D4, D2
   re-ranker failed (timed out after 200 ms), keeping the first-stage order
broken re-ranker: D5, D2

Where a hosted re-ranker plugs in (Cohere Rerank over HTTP)python

Shown, not run. Checked against Cohere's published rerank reference. The key comes from an environment variable. In Java the course's examples avoid libraries, but real projects use Jackson to read the JSON, as here. Model names and prices change: check the current docs.

import os

import httpx

COHERE_API_KEY = os.environ["COHERE_API_KEY"]
RERANK_MODEL = "rerank-v4.0-pro"  # a model name from the provider's docs: check the current list


def cohere_reranker(question, candidates):
    """Plugs into two_stage(): candidates are (name, text) pairs, the result is [(name, score)]."""
    reply = httpx.post(
        "https://api.cohere.com/v2/rerank",
        headers={"Authorization": "Bearer " + COHERE_API_KEY},
        json={"model": RERANK_MODEL, "query": question, "documents": [text for _, text in candidates], "top_n": len(candidates)},
        timeout=2.0,  # a slow re-ranker must not stall the answer: two_stage() falls back when this raises
    )
    reply.raise_for_status()
    return [(candidates[hit["index"]][0], hit["relevance_score"]) for hit in reply.json()["results"]]

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Sort the re-ranker's scores the wrong way

In the Python tab, in two_stage, change key=lambda pair: -pair[1] to key=lambda pair: pair[1] and run.

terminal

$ Run

── what you'll see ──

plain RRF: D1=0.03227, D3=0.03227, D2=0.01613, D4=0.01613

keyword x2: D3=0.04866, D1=0.04814, D4=0.03226, D2=0.01613

mmr lambda 1.0: copy 2, copy 1

mmr lambda 0.5: copy 2, other

good re-ranker: D5, D9

re-ranker failed (timed out after 200 ms), keeping the first-stage order

broken re-ranker: D5, D2

Myth vs fact

Myth

A fallback hides problems.

Fact

Only if you do not count it. Log every fallback, count them and alert on a rising rate: the user gets an answer and you still hear about the failure.

Myth

Copy-paste recipes need no testing.

Fact

Each recipe has a known winner on a tiny case. Keep that case as a test so a later edit cannot silently reverse the order.

Myth

Weights are chosen once.

Fact

They depend on your questions and corpus. Re-run the Topic 6.7 table when either changes.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
Fetch deeper than you keep: 50 to 100 candidates per list, fuse, cut to first_size, re-rank, keep final_k. A passage rank 12 in one list and 3 in the other is lost if each list returns only 10.
▸
Cache re-rank results keyed by (question, passage version). Repeated questions then cost nothing, and a changed passage naturally gets a new key (Phase 11).
▸
Put stage timings in your traces: first-stage ms, re-rank ms, fallback used or not. Tail latency on the re-ranker is the usual cause of slow answers.
▸
Hosted re-rankers receive your passages. Check the provider's data terms against your corpus before sending anything private (Phase 10).

Remember this

1
Recipe: weighted RRF. When to use it: you fuse two or more ranked lists and one is more trustworthy for your questions. What it guards against: a weak ranker voting as loudly as a strong one. The function multiplies each list's 1 / (k + rank) by a weight. With weights 1 and 1 it is plain RRF; in the example, plain RRF ties D1 and D3 (both 0.03227, so the name decides), and doubling the keyword list's weight puts D3 first (0.04866 against 0.04814). How to adapt: choose weights on a test set (Topic 6.7), keep them in configuration, and keep k at 60 until you have a reason.
2
Recipe: MMR. When to use it: the candidates contain copies or near-copies and you will send only a few to the prompt. What it guards against: repeats filling the budget (Topic 6.6). The function takes the question vector, (name, vector) items, lam and k. In the example with two near-identical vectors and one different, lam = 1.0 returns both copies and lam = 0.5 returns one copy and the different passage. How to adapt: compute redundancy with the same similarity you retrieve with; start at lam = 0.7.
3
Recipe: two-stage retriever with a fallback. When to use it: always, when stage two is a model or a remote service. What it guards against: a slow or failing re-ranker taking the answer down. The function takes the first stage, the re-ranker, first_size and final_k; if the re-ranker raises, it logs one line and returns the first-stage order. In the example the good re-ranker puts D4 and D2 first and the failing one falls back to D5, D2. How to adapt: add a timeout, a counter of fallbacks and an alert when they become frequent.
4
Recipe: a hosted re-ranker call. When to use it: you do not want to run a model yourself. What it guards against: nothing by itself, so the call lives behind the same re-ranker interface and the same fallback. The code below was written against the provider's published reference for the Cohere Rerank API (POST /v2/rerank, a model, query, documents and top_n; results carry index and relevance_score). How to adapt: batch up to the provider's recommended document count, set a timeout from your latency budget, check the provider's current model names and prices.
5
Recipe: choose stages by evidence. When to use it: before you add a stage. Run the Topic 6.7 table. Keep BM25 only if vectors add nothing on your questions. Keep vectors only if there are no exact terms. Use hybrid when both kinds appear. Add a re-ranker when hit@3 is high and hit@1 is low and your latency budget allows. Add MMR or de-duplication when you see repeats in real traces.
6
Before you ship. Check the list at the end of this topic: lists fetched deeper than the final cut, a stable tie rule, a re-ranker timeout and fallback, a test table in CI, an evidence bar on a meaningful score, and the trace logged for every answer.

Explain it without notes

When would you use weighted RRF instead of plain RRF?

Why should the two-stage retriever catch exceptions from the re-ranker?

How does the hosted re-ranker call plug into the retriever?

What goes on your "before you ship" list for hybrid retrieval?

Practice

Write weighted_rrf for two lists with weights (1, 2) and print the top two ids with 5 decimals for lists ["a", "b"] and ["b", "c"].

Write safe_call(fn, fallback) that returns fn() and, if it raises any exception, prints fallback used and returns fallback. Print the results for a function that works and one that raises.

Trade-offs

↔
Fallback to first-stage order: availability over the best ordering.
↔
Deeper candidate lists: better recall, more work at fusion and re-rank time.
↔
Configuration over constants: easy tuning, easy to misconfigure without tests.

Done when you can

Before you ship: each ranker fetches more candidates than you finally keep.
Before you ship: ties break by a defined rule, so results repeat.
Before you ship: the re-ranker has a timeout, a fallback and a counter of fallbacks.
Before you ship: the measurement table (hit@1, hit@3, MRR) runs in CI against real questions.
Before you ship: the evidence bar uses a score with a stable meaning, set on answerable and unanswerable questions.
Before you ship: you know what data each hosted service receives and you are allowed to send it.

Command Palette

Words you'll meet

Step by step

01Step 1: fuse with weights

02Step 2: pick diverse results

03Step 3: wrap the re-ranker

04Step 4: plug in the real call

05Step 5: keep the table in your tests

06Step 6: ship with the checklist

Try it yourself

Change the weights

Break the re-ranker on purpose

Tune lambda

Code & diagrams

Break it on purpose

Sort the re-ranker's scores the wrong way

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can