Topic 3.9
Playbook: Picking and Tuning an Index
In one line
Recipes to copy into any vector search project: test whether you need an index at all, start from documented parameters, benchmark against exact search, shrink and re-score, choose a filter strategy, and keep recall from silently dropping.
Think of it like this
A mechanic does not tune an engine by feel. There is a checklist: is it even broken, start from the manufacturer's settings, change one thing, measure, write it down. These recipes are that checklist for vector search, and they do not depend on AskDocs or on any one database.
Words you'll meet
New words in this topic, in plain English. Come back here whenever one feels fuzzy.
- Playbook
- A set of tested recipes you can copy and adapt.
- Latency budget
- How long the whole answer may take, shared between embedding, search, the model call and the network.
- Throughput
- How many questions per second the system must handle.
- Recall floor
- The lowest recall you accept. A build fails if the measured recall is below it.
- Harness
- Code that runs candidates through the same test and prints comparable numbers.
- Starting parameters
- Documented values to begin from before measuring and tuning.
Step by step
01Step 1: write down the numbers of your collection
Count vectors, dimensions, questions per second and your latency budget. Every other recipe starts from these four numbers.
02Step 2: run the "do I need an index" check
The check estimates one exact query's cost and compares it with half the latency budget and with the cores. In the example, 12 chunks and 50,000 chunks pass; 200,000 chunks at 100 questions per second use 190 percent of four cores, so the answer is an index.
03Step 3: choose starting parameters
Use the documented rules for the index type, so you begin near a sensible point instead of at a guess. Record them.
04Step 4: run the harness on candidates
Give the harness every candidate as a function, plus the exact answers for your held-out questions. It prints recall and cost per candidate and picks the cheapest that meets the target.
def evaluate(name, search, truths):
hits = cheap = full = 0
for q, truth in zip(queries, truths):
ids, c, f = search(q) # any search returning (ids, cheap work, full work)
hits += len(set(ids) & set(truth))
cheap, full = cheap + c, full + f
recall = hits / (K * len(queries))
return recall, full05Step 5: turn one knob at a time
If no candidate meets the target, change one setting (a bigger shortlist, a larger nprobe or efSearch) and rerun. Never change two at once: you will not know which helped.
06Step 6: lock it in
Save the settings, the recall and the date. Add a CI check with a recall floor on a fixed question set. When you change the embedding model, the chunking or the data, rerun the harness before shipping.
Try it yourself
- 1
Your own numbers
In the first example add a scenario that matches your project: your chunk count, 768 dimensions, your expected questions per second. Which rule fires first, latency or throughput?
- 2
Raise the target
In the harness change
TARGETto 0.99. Which candidate is picked now, and what does it cost in full comparisons? Add a candidate of your own, such ascompressed(15, 8). - 3
Different data
Change the vector length from 16 to 64 in the harness (vectors and queries). Do the recalls of the compressed candidates change? Which would you pick, and would you trust a result from random vectors to decide for real embeddings?
Code & diagrams
The speed (1 billion multiply-adds per second per core), the 4 cores and the 50 percent rules are assumptions to reason with. Replace them with numbers measured on your hardware.
Expected output
handbook: 12 chunks: 0 ms per query, 0% of 4 cores at 1/s, 0 MB -> exact search is enough
help centre: 50,000 chunks: 19 ms per query, 2% of 4 cores at 5/s, 76 MB -> exact search is enough
busy help centre: 200,000: 76 ms per query, 190% of 4 cores at 100/s, 307 MB -> use an index: too many queries for the cores
big archive: 1,000,000: 384 ms per query, 192% of 4 cores at 20/s, 1536 MB -> use an index: too slow for one query and too many queries for the cores
huge: 10,000,000 x 1024: 10240 ms per query, 12800% of 4 cores at 50/s, 40960 MB -> use an index: too slow for one query and too many queries for the coresThe rules are the ones in pgvector's documentation (checked on 2026-10-11); they are starting points for any IVF or HNSW index, not guarantees.
Expected output
IVF for 10000 rows: start with lists=10, probes=3
IVF for 100000 rows: start with lists=100, probes=10
IVF for 1000000 rows: start with lists=1000, probes=31
IVF for 10000000 rows: start with lists=3162, probes=56
HNSW: start with m=16, ef_construction=64, ef_search=40
Then measure recall (Topic 3.6) and move one knob at a time.Candidates here are compressed searches on random vectors, but the harness accepts any function that returns ids and a work count: swap in IVF, HNSW or a database client.
Expected output
exact recall@3 1.000 cheap 0 full 1000 meets target
2-bit codes only recall@3 0.617 cheap 1000 full 0 below target
2-bit + re-score 10 recall@3 0.917 cheap 1000 full 10 meets target
4-bit + re-score 10 recall@3 1.000 cheap 1000 full 10 meets target
4-bit + re-score 5 recall@3 0.950 cheap 1000 full 5 meets target
cheapest that meets the target (fewest full comparisons): 4-bit + re-score 5Break it on purpose
Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.
Break #1
Pick the winner without a target
In the Python tab of the harness example, change the check recall >= TARGET and to nothing, so the last line chooses the candidate with the fewest full comparisons regardless of recall: the line becomes if best is None or full < best[1]:.
Myth vs fact
Myth
Every RAG system needs a vector index.
Fact
Up to tens of thousands of chunks (and often more) exact search is fast enough, simpler and never misses. Index when measurements say you must.
Myth
Copy the parameters from a blog post and ship.
Fact
A post's parameters were tuned for its data and its recall target. Use documented defaults as a starting point and measure on your own data.
Myth
Benchmark once, at launch.
Fact
Data, embedding models and traffic change. The benchmark belongs in your repository and your CI.
Pro corner
Extra depth for experienced readers. New to this? Skip it for now and come back later.
- ▸
Record benchmark results with the data size, the embedding model name and version, k, the question set's id and the index parameters. A number without that context cannot be compared next month.
- ▸
When tuning for latency, look at the tail (p95, p99), not the mean, and at the filtered queries, which usually behave worst.
- ▸
Treat index parameters as part of the schema: version them and store them with the index, so a rebuild reproduces the same behaviour and so you can see which setting was in force when quality changed.
Remember this
- 1
Recipe: Do I even need an index? When to use it: before you add any approximate index, and again whenever the collection or traffic grows. What it guards against: the most common mistake, a complex index for a collection exact search handles in milliseconds. How to adapt it: multiply vectors by dimensions for the cost of one query, divide by what one core does per millisecond on your machine (measure it; the example assumes a round 1 million multiply-adds per millisecond), keep half of your latency budget for everything else, and check throughput as queries per second times milliseconds against your cores. The first example runs five scenarios: the handbook and a 50,000-chunk help centre need no index; 200,000 chunks at 100 questions per second fail on throughput; a million chunks fails on both.
- 2
Recipe: Start from documented parameters. When to use it: when you do need an index. What it guards against: guessing, and defaults you never looked at. How to adapt it: for IVF, pgvector's documentation suggests
listsof rows / 1,000 up to a million rows and the square root of the rows above that, withprobesstarting near the square root oflists; for HNSW its defaults arem = 16,ef_construction = 64andhnsw.ef_search = 40; hnswlib recommendsMbetween 12 and 48 for most uses and saysefmust not be lower than k. These are starting points. The third example prints the IVF rule for four collection sizes. - 3
Recipe: A recall benchmark harness. When to use it: for every index choice and every change of data, model or settings. What it guards against: choosing by feel, and the silent drop in recall after a change. How to adapt it: keep the harness independent of the index. It takes any
search(q)that returns ids and counts of the work done, runs real held-out questions, scores recall@k against exact search, prints one line per candidate, and picks the cheapest candidate that meets the target. The second example compares exact search with four compressed variants and picks "4-bit + re-score 5" for a 0.90 target. Add timing and memory columns on your own machine. - 4
Recipe: Shrink, then re-score. When to use it: when vectors do not fit in memory, or scans are memory-bound. What it guards against: paying for 4-byte numbers when 1 byte, or 1 bit, ranks nearly as well. How to adapt it: search compressed vectors for a shortlist several times larger than k, re-score the shortlist with the full vectors, and choose the compression level and the shortlist size together with the harness. Keep the full-precision vectors somewhere you can fetch from (Topic 3.5).
- 5
Recipe: Choose the filter strategy by selectivity. When to use it: whenever searches carry a filter such as tenant, language or version. What it guards against: empty or short results for small tenants, and leaks across tenants. How to adapt it: scan the allowed set exactly when it is small, use the index with an over-fetch of about k / selectivity otherwise, and test with the strictest filter you serve. Assert on who owns the returned results, not only on how many came back (Topic 3.7).
- 6
Recipe: Keep recall from silently dropping. When to use it: always, once an index is in production. What it guards against: a new embedding model, a data change or a tuning edit that quietly lowers recall. How to adapt it: put the harness in CI with a fixed question set and a recall floor at your production settings, and rerun it when you rebuild. The practice below writes the check. The next phases add the other half: evaluation of the answers themselves (Phase 9).
Explain it without notes
Walk through the test for whether a project needs a vector index.
Where would you start with IVF and HNSW parameters, and what do you do next?
What does the benchmark harness need, and why must it work with any search function?
What belongs in CI to protect recall, and what does it not protect against?
Practice
Write check_recall_floor(recall, floor) that returns "ok: recall 0.970 meets the floor 0.950" when recall is at least the floor and "FAIL: recall 0.930 is below the floor 0.950" otherwise (3 decimals). Print the result for 0.93 and 0.97 with a floor of 0.95.
Write needs_index(n, dim, budget_ms) that estimates the exact query time as n * dim // 1000000 milliseconds and returns "yes" when twice that is above the budget, otherwise "no". Print the answer for 50,000, 200,000 and 1,000,000 vectors of 384 numbers with a 200 ms budget.
Trade-offs
- ↔
Exact search: nothing to tune and always right, with cost growing with the collection.
- ↔
An approximate index: speed at scale, with parameters to tune, recall to monitor and a rebuild cost.
- ↔
Compression with re-scoring: much less memory, with a second stage and the full vectors to store.
- ↔
A strict CI recall floor: early warning, and some friction when you change anything.
Done when you can
Before you ship, you have computed cost, latency and memory for exact search and written down why you do or do not need an index.
Before you ship, index parameters started from documented values and were then measured on your own embeddings and real questions.
Before you ship, recall@k against exact search is known at the production settings, with the worst questions inspected.
Before you ship, filters were tested with the strictest tenant, and results were checked for ownership.
Before you ship, a recall floor runs in CI, and the benchmark reruns when data, chunking or the embedding model change.