Command Palette

Search for a command to run...

Hectal
Phase 6Intermediate7 of 14 in RAG and Vector Search

Hybrid Search and Re-ranking

Run keyword and vector search together, fuse them with reciprocal rank fusion, re-rank a shortlist, keep results diverse, and measure whether each stage pays.

Phase 1 gave you keyword search and Phases 2 and 3 gave you vector search. Each one wins questions the other loses: keywords find exact codes, numbers and rare names, vectors find paraphrases. This phase puts them together. You will see both kinds of question side by side, learn why their scores cannot be added directly, and build reciprocal rank fusion (RRF) by hand until you can reproduce the Fused column of the search lab with your own code.

Then comes a second stage. A cross-encoder reads the question and one passage together and gives a much sharper relevance score than a single vector can, but it is too slow to run on everything, so it only re-orders a shortlist. You will build that retrieve-then-re-rank pipeline, choose the size of each stage, keep results diverse with maximal marginal relevance, and learn to ask the question every added stage must answer: does it pay off on *your* questions?

The phase ends by upgrading AskDocs to hybrid retrieval with a re-rank stage behind the same retrieve interface, compared with the earlier versions by a small test harness, and a playbook of recipes to copy. A cross-encoder cannot run inside these examples, so a clearly labelled toy scorer stands in for it. The search labs use the real embedding model, so the scores they show are real.

0/9 · 0%
9 topics ~4 h 18 code blocks & diagrams
Start with the first topic
1
6.1

Why One Ranker Is Not Enough

Keyword search finds exact words, codes and numbers. Vector search finds the same meaning in different words. Each one confidently misses what the other finds, so serious search runs both.

29 min 2 code practice

2
6.2

Mixing Scores (and Why It Is Hard)

The obvious way to combine BM25 and cosine is to add the scores. They live on different scales, so you must normalise first, and even then the weights you tuned stop working when the data changes.

31 min 2 code practice

3
6.3

Reciprocal Rank Fusion by Hand

Reciprocal rank fusion merges ranked lists using only each passage's place in each list: it adds 1 / (60 + rank) per list. No scales to align, one constant, and you can recompute the search lab's Fused column by hand.

24 min 3 code practice

4
6.4

Re-ranking with Cross-Encoders

An embedding squeezes each passage into one vector before it ever sees the question. A cross-encoder reads the question and one passage together and scores the pair: much sharper, far too slow for the whole index, so it re-orders only a shortlist.

28 min 2 code practice

5
6.5

The Retrieve-Then-Rerank Pipeline

Run a fast, wide first stage to collect candidates, then a slow, sharp second stage on only those. Two sizes matter: how many candidates you hand over, and how many you keep. Measure both.

27 min 1 code practice

6
6.6

Diverse Results

When the same fact is stored in several places, the top results are copies of one passage and the context window fills with repeats. Maximal marginal relevance trades a little relevance for variety, and a de-duplication step removes true copies.

24 min 3 code practice

7
6.7

Does It Pay Off?

Every extra stage costs code, time and money. Measure each stage's gain on your own questions with hit rate and reciprocal rank, and keep the simplest pipeline that is good enough.

27 min 1 code practice

8
6.8

Build: Hybrid Retrieval for AskDocs

AskDocs version 6: keyword and vector search fused with RRF, a re-rank stage behind the same retrieve interface, a near-copy filter and an evidence bar on the re-ranker's score. A small harness compares it with versions 1 and 3.

38 min 2 code practice

9
6.9

Playbook: Hybrid and Re-rank Recipes

Copy-ready recipes: a weighted RRF function, an MMR function, a two-stage retriever that survives a failing re-ranker, a hosted re-ranker call, and the checklist to run before you ship.

21 min 2 code practice

Practise this phase

Hands-on exercises: write the code, run it, and press Check to have it tested automatically.

1

Fuse ranked lists with reciprocal rank fusion

Write rrf(rankings, k=60). rankings is a list of ranked lists of ids (best first). Every list adds 1 / (k + rank) to each id it contains, where the best id has rank 1.

Intermediate
2

Normalise two score lists and mix them

Write min_max(scores): rescale a list to 0..1 with (score - lowest) / (highest - lowest). If all scores are equal, return a list of 0.0 of the same length.

Intermediate
3

Re-rank a shortlist and survive a failing re-ranker

Write rerank_safe(question, candidates, scorer, final_k). candidates is a list of ids in first-stage order, and scorer(question, candidate) returns a number (higher is better).

Intermediate
4

Pick diverse results with MMR

Write mmr(query, vectors, lam, k). vectors is a dict from name to vector (a list of numbers). cosine(a, b) is given in the starter.

Advanced
5

Measure hit rate and MRR

Write evaluate(rankings, golds). rankings is a list of ranked id lists (one per question) and golds is the list of right ids in the same order.

Intermediate