Topic 12.9
Build: Parent-Child Retrieval for AskDocs
In one line
AskDocs gains small-to-big retrieval behind the same retrieve interface. Three new handbook guides are cut into sentence children, the best child decides the parent, and a small test shows whole-section search missing questions it now gets right.
Think of it like this
Adding a table of contents and page numbers to a book that already exists. The pages are unchanged. You can just find things faster and read the right page.
Words you'll meet
New words in this topic, in plain English. Come back here whenever one feels fuzzy.
- Chunk
- A piece of a document that is searched. Here, one sentence with an id like D52.4.
- Doc id
- The id of the parent document a chunk belongs to.
- Golden set
- Questions with known right sources, used to compare versions.
- Jaccard
- Shared words divided by all different words in both texts.
- Threshold
- The lowest score that counts as evidence.
- Trace
- What the system did for one request: ids and scores.
Step by step
01Step 1: add the new documents
D50, D51 and D52 are new handbook guides. The earlier twelve passages are unchanged. Their ids start at D50 so no earlier id is reused.
02Step 2: cut guides into chunks
make_chunks splits each guide on sentences and gives each chunk an id built from the guide id and its position.
03Step 3: search chunks with a threshold
Score every chunk against the question, drop anything under MIN_SCORE, sort best first and keep k.
04Step 4: answer from the parent
The best hit names its parent. The prompt would use the parent's text; the stand-in replies with the matching sentence and cites the parent.
05Step 5: compare with whole-section search
The example runs the same four questions both ways. Whole sections right: 1 of 4. Small to big right: 4 of 4. Refusals: 2 of 2 both ways.
06Step 6: write down the next steps
Real retriever scores (Phases 3 and 6), a size budget for parents (Topic 12.10), access labels (Topic 10.3), and a bigger golden set (Topic 9.2). Each already has a topic that teaches it.
Try it yourself
- 1
Raise the bar
Set
MIN_SCOREto 0.4. Which answerable questions are now refused? Was the threshold too high? - 2
Add a question
Add
("Can I plan a route with hills?", "D51")to the answerable list. Does it pass in both modes? - 3
Change k
Change
retrieve(question, 2)to 1 inanswer. What disappears from thefoundline?
Code & diagrams
Self-contained: it re-declares the three new guides and the helpers it needs. The scoring is a toy stand-in for the real retriever.
Expected output
whole sections: right source 1/4 | refused 2/2
small to big: right source 4/4 | refused 2/2
Q: Do I stop at red lights in a group?
found: D52.4 (0.429)
read: Groups of up to 6 riders can share one booking. Everyone in the group needs their own helmet. The group leader pays at the shop before the ride. Keep the group together and stop at every red light.
reply: Keep the group together and stop at every red light. [D52]Break it on purpose
Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.
Break #1
Return the chunk id as the source
In the Python tab, change parent_id = hits[0][1].doc_id to parent_id = hits[0][1].id. Run.
Myth vs fact
Myth
Better retrieval needs a new retriever interface.
Fact
Here only the thing being searched changed. The interface stayed the same, so everything else kept working.
Myth
If the right section is retrieved, the answer is right.
Fact
Retrieval is half. Phase 8 checks the answer against the sources.
Myth
A toy test proves a method works.
Fact
It shows the mechanics. Prove the benefit on your own questions with real scores (Phase 9).
Pro corner
Extra depth for experienced readers. New to this? Skip it for now and come back later.
- ▸
Keep the whole-section retriever in the code as a baseline and keep comparing. A pattern that stops helping after a model change should be removable.
- ▸
Write the evaluation before the feature. Here the four questions were chosen before the numbers were known, which is the habit to keep.
- ▸
Return the child id and score in the trace as well as the parent id. Debugging needs to see which sentence matched.
Remember this
- 1
Why the product needs this now. Phase 4 added documents of different sizes. The handbook passages are one sentence or two, but real guides are sections of several sentences. Searching a whole section blurs it, and a precise question picks the wrong section. This phase adds three guides (D50 riding safety, D51 trip planning, D52 group rides) and the child index.
- 2
What the step adds.
make_chunkscuts each guide into sentence-sizedChunks with ids likeD52.4, each knowing itsdoc_id.retrieve(question, k)now searches chunks and returns scored hits, best first, andanswerreads the hit's parent. The interface ofretrieveis the one from Phase 0: a question and a k in, scored hits out. - 3
How it plugs in.
retrievestill returns hits with a score, andanswerstill builds a prompt and cites. The only new idea is that a hit is a chunk and its source id is the parent. The score here is Jaccard on non-stop words, a simple stand-in for the vector and BM25 scores of earlier phases, so the example runs in all three languages. - 4
Measure it. Four answerable questions and two unanswerable ones form a tiny golden set (Phase 9 builds the real harness). Whole-section search finds the right section for 1 of 4. Small-to-big finds 4 of 4, and both refuse the two unanswerable questions.
- 5
Read one trace. For "Do I stop at red lights in a group?" the best child is D52.4 (0.429). The second-best hit does not clear the 0.2 threshold, so one parent is read. The model-side stand-in replies with the matching sentence and cites the parent,
[D52]. - 6
What AskDocs can do now that it could not before. It can answer a question from one sentence and still cite the whole section that sentence belongs to, and it can show with numbers that this beats searching whole sections. What is still missing. The example scores with word overlap so it runs in three languages. In the real AskDocs you plug this step into the retriever you built in Phases 3 and 6 and use a threshold you measured (Topic 8.4), not the guessed 0.2. Parents have no size budget yet (the playbook, Topic 12.10, adds one). Access labels on parents (Topic 10.3) and a larger golden set (Topic 9.2) must be added before shipping.
Explain it without notes
What changed in AskDocs and what stayed the same?
Why did whole-section search fail on most questions?
How is a hit turned into a citation?
What would you add before shipping this?
Practice
Write hit_rate(find, golden) where golden is a list of (question, expected_id) pairs and find(question) returns an id or None. Return the share of questions where find returned the expected id. Print it with 2 decimals for a fake find.
Write margin(scores) that returns the best score minus the second best (or the best score alone if there is only one). Print it with 3 decimals for [0.429, 0.143, 0.095] and for [0.083, 0.082].
Trade-offs
- ↔
Sentence children: sharper matches and more chunks to store.
- ↔
Parent citations: readable, less exact than quoting the sentence.
- ↔
A fixed interface: easy swaps and some design upfront.
Done when you can
Done when you can run the example in three languages and read its trace.
Done when you can explain why the interface did not change.
Done when you can add a question to the golden set and compare both modes.
Done when you can list what is still missing and which phase adds it.