Topic 3.8
Build: A Vector Index for AskDocs
In one line
AskDocs version 3: an Embedder turns passages and questions into vectors, an exact index and an IVF index sit behind the same add and search interface, retrieval keeps only hits above a similarity bar, and recall against the exact index is measured. The login question is finally answered correctly.
Think of it like this
A building with only stairs gets a lift. The doors and rooms stay where they were, so nobody has to relearn the building. The stairs stay too: they are the fallback, and you can use them to check that the lift really stops at the floor it claims. A new index is a lift: same interface, faster for large collections, and checked against the stairs (exact search).
Words you'll meet
New words in this topic, in plain English. Come back here whenever one feels fuzzy.
- Embedder
- Anything with embed(text) that returns a vector. Real models and test stand-ins share this shape.
- Index interface
- The two methods every index offers: add(id, vector) and search(vector, k). Exact and IVF indexes both have them.
- Flat index
- An index that stores the vectors and compares with all of them: exact.
- MIN_SIM
- The lowest similarity a hit may have to count as evidence.
- Stand-in (canned) embedder
- A fake embedder that returns prepared vectors, so the rest of the system can be built and tested.
- Recall check
- Comparing an approximate index with the exact one on many queries.
Step by step
01Step 1: documents and vectors
The twelve handbook passages become Doc values with id, title and text. Each has a toy four-number vector (money, time, account, safety), the same as in Topic 3.1. The embedder knows the vectors of the passages and of four test questions, and gives an all-zero vector for any other text.
02Step 2: the exact index
FlatIndex keeps (id, vector) pairs. search scores all of them with cosine (returning 0 for a zero vector instead of dividing by zero) and returns the best k with a stable sort, so ties keep insertion order.
03Step 3: the IVF index behind the same interface
IvfIndex extends the flat one. build() picks 3 starting centroids, runs 3 rounds of k-means with cosine similarity and fills the lists. search opens the nprobe closest lists. Callers only know add and search.
04Step 4: the similarity bar
retrieve drops hits below MIN_SIM. answer refuses in code when nothing is left, so refusals stay free, instant and testable, as in version 0.
05Step 5: run the four questions
Each question prints the exact hits, the IVF hits and the answer. The two indexes agree on all four. The login question now finds the password passage, and the scooters question is refused.
06Step 6: measure recall against exact search
Query with each passage's own vector, take the exact top 3 as truth and compare. nprobe = 1 misses a few neighbours (recall@3 about 0.92), nprobe = 2 finds them all. The harness is the point: Topic 3.9 turns it into a reusable function.
Try it yourself
- 1
Move the bar
Set
MIN_SIMto 0.98. Which questions are still answered? Then set it to 0.5. What can now get through that should not? Find a value that keeps the three answerable questions and refuses the scooters one, and note how wide the range of good values is. - 2
Change the clusters
Change
IvfIndex(3, 1)toIvfIndex(6, 1)and read the recall lines. Then try 2 clusters. Which is better with only twelve vectors, and why does it matter so little here? - 3
Add a question
Add the question
"Is a helmet included?"toQUESTIONSwith a toy vector of your choice (for example[0.0, 0.0, 0.1, 0.9]). Which passage does each index return? Choose a vector that is close to two passages and see whether the indexes agree.
Code & diagrams
The vectors are a toy with four named directions, and the embedder returns prepared values for known texts. Real numbers from a real model are in the search labs of this phase and of Phase 2.
Expected output
Q: Can I get my money back if I cancel?
exact: D1 (0.991), D8 (0.974)
ivf: D1 (0.991), D8 (0.974)
answer: A ride cancelled at least 2 hours before its start time is refunded in full. [D1]
Q: What does the monthly pass cost?
exact: D8 (0.991), D1 (0.963)
ivf: D8 (0.991), D1 (0.963)
answer: The monthly pass costs 25 dollars and includes unlimited rides of up to 45 minutes. [D8]
Q: I forgot my login, how do I get back in?
exact: D3 (0.988)
ivf: D3 (0.988)
answer: To reset your password, open the app, tap Forgot password and follow the link we email you. [D3]
Q: Do you rent scooters?
exact: none
ivf: none
answer: I can't find that in the handbook.
recall@3 of the IVF index against the exact index, using each passage as the query:
nprobe 1: 0.917
nprobe 2: 1.000
nprobe 3: 1.000Break it on purpose
Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.
Break #1
Remove the evidence bar
In the Python tab, set MIN_SIM = 0.0.
Break #2
Use too many clusters
In the Python tab, change IvfIndex(3, 1) to IvfIndex(6, 1).
Myth vs fact
Myth
Adding a vector index fixes retrieval quality.
Fact
The index makes search fast. Quality comes from the embedding model, the chunks and the ranking (Phases 2, 4, 6 and 7). A fast index over bad chunks returns bad results faster.
Myth
A bigger similarity bar is always safer.
Fact
A bar that is too high refuses answerable questions. Both mistakes cost something, so the bar is measured on both kinds of question.
Myth
Keep the exact index only for tests.
Fact
It is also the right production choice for small collections and for small filtered sets, and the fallback while an approximate index rebuilds.
Pro corner
Extra depth for experienced readers. New to this? Skip it for now and come back later.
- ▸
Keep ids stable across indexes. The doc id in the index is the same id used in citations, tests and deletions. If an index uses its own internal numbers, map them at the edge and never leak them.
- ▸
Return the trace from retrieval (ids and scores, as the example prints them). The similarity of hit 1 against hit 2 is a useful confidence signal, and Phase 9's evaluation needs these scores.
- ▸
Give the index an "agree with exact" test that runs in CI on a fixed set of questions, with a recall floor. When someone changes nprobe, the embedding model or the vectors' normalisation, the build fails instead of the answers quietly getting worse.
Remember this
- 1
Why the product needs this now. Version 0 (Topic 0.7) found passages by shared words, and failed on "I forgot my login, how do I get back in?". Phase 2 gave AskDocs vectors that capture meaning, and this phase gives it a way to search them. Version 3 retrieves by vector similarity behind the same
retrieve(question, k)the product has had since Phase 0. - 2
What the step adds. An
Embedderwithembed(text), here a stand-in that returns canned toy vectors (Phase 2's build made the real one; the stand-in plays the part a fake embedder plays in tests). AFlatIndexwithadd(id, vector)andsearch(vector, k)that compares with every vector (Topic 3.1). AnIvfIndexwith the same two methods plusbuild(), which clusters the vectors and searches only thenprobenearest clusters (Topic 3.3). And a recall check of one against the other. - 3
How it plugs in.
retrieve(index, question, k)embeds the question, asks the index forkhits, and keeps only those at or aboveMIN_SIM, a similarity bar. This replaces version 0's shared-word bar (MIN_SCORE = 2). For a real embedding model the bar is a number you measure (Topic 8.4): in the lab of Topic 3.2 an unrelated scooters question still scored about 0.38. Here, with toy vectors, 0.90 is just a value that separates the handbook questions from the unanswerable one.answer(index, question)is unchanged in shape: refuse in code when nothing passes the bar, otherwise answer from the best passage and cite its id. - 4
The run. Four questions go through the IVF index and the exact index side by side. "Can I get my money back if I cancel?" returns Refunds first (0.99) and Monthly pass second (0.97) from both. "What does the monthly pass cost?" returns Monthly pass. "I forgot my login, how do I get back in?" returns Password reset (0.99), the passage word overlap could not find. "Do you rent scooters?" has no known words, so its toy vector is all zeros, nothing passes the bar, and the answer is the refusal without any model call.
- 5
The recall check. Using each of the twelve passages as a query, the example compares the IVF index with the exact one: recall@3 is 0.92 at
nprobe = 1and 1.00 atnprobe = 2. Be honest about what this shows. With twelve vectors the right index is exact search, and IVF is here to show the harness and the interface, not because it is needed. The same check, with a real collection and real questions (Topic 3.6), is how you would decide on it later. - 6
What is still missing. Real embeddings and a real threshold (Phase 8 measures it), documents turned into good chunks (Phase 4), storage that survives a restart, supports deletes and filters (Topic 3.7 and Phase 5), keyword and vector search combined (Phase 6), and an evaluation set that proves the answers improved (Phase 9). Version 3 is the first AskDocs that can answer a paraphrased question, and it is still a toy.
Explain it without notes
Why do FlatIndex and IvfIndex share the same two methods, and what does that buy?
In this build, why does the login question work now when version 0 failed it?
Why does the scooters question get the refusal, and what would change if MIN_SIM were 0?
What does the recall check measure, and why is it weak with only twelve vectors?
Practice
Write recall_against_exact(exact, other, k) where both arguments are lists of ranked id lists (one per query). Return the average, over queries, of the fraction of the exact top k that other also has in its top k. Print recall@1, @2 and @3 for exact = [["D1", "D8", "D6"], ["D3", "D2", "D9"]] and other = [["D1", "D8", "D5"], ["D3", "D2", "D9"]].
Write evidence(hits, min_sim) where hits is a list of (doc_id, similarity) pairs. Return the hits at or above min_sim, or the refusal text "I can't find that in the handbook." when none qualify. Print the result for three calls: two good hits, one weak hit (0.61), and no hits, with min_sim = 0.90.
Trade-offs
- ↔
Exact index in the build: simplest and always right, fine at this size, and it will need to be swapped at larger sizes.
- ↔
A high similarity bar: fewer wrong answers and more refusals of answerable questions. A low bar is the reverse.
- ↔
Keeping both indexes: extra memory and code, in return for an oracle that proves the approximate one.
Done when you can
Done when you can run AskDocs v3 in all three languages and explain each line of its output.
Done when you can say what add and search take and return, and why both indexes share them.
Done when you can explain the role of MIN_SIM and what a zero vector does.
Done when you can run the recall check and read its result honestly.