Topic 12.1
Small Chunks to Find, Big Chunks to Read
In one line
Search small pieces because their meaning is sharp, then hand the model the larger section each piece came from. Stable ids linking a child to its parent make this a few lines of code.
Think of it like this
Finding a recipe in a cookbook. You use the index to find the exact line, "fold in the egg whites", which takes you to a page. You do not cook from that one line. You read the whole recipe around it, because the line alone does not say how many eggs or what to do next. The index finds with small pieces. The reading uses the big piece.
Words you'll meet
New words in this topic, in plain English. Come back here whenever one feels fuzzy.
- Parent chunk
- A larger piece of a document (a section) that is stored and sent to the model but not searched directly.
- Child chunk
- A small piece cut out of a parent. Children are embedded and searched.
- Small-to-big retrieval
- Searching children, then sending their parents.
- Sentence-window retrieval
- A variant: send the matching sentence plus a few neighbouring sentences instead of a stored parent.
- Parent id
- A field on every child that says which parent it was cut from.
- Dilution
- A long text has a vector that blends many topics, so it matches any one of them less sharply.
- Document store
- A simple key-value store from an id to a text. Parents live here.
Step by step
01Step 1: see the trade-off
Take the safety guide (document D50, four sentences). Searching it whole gives one vector that blends helmets, lights, brakes and crossings. Searching one sentence gives a vector that is only about lights. The first is weaker for "when do I turn the lights on?", the second is thin if the answer needs the neighbouring sentence.
02Step 2: cut parents into children with ids
Split on sentences. Number the children inside each parent so the id says where it came from.
def split_children(parent_id, text):
sentences = [s.strip() + "." for s in text.split(".") if s.strip()]
return [(parent_id + "." + str(n), parent_id, s) for n, s in enumerate(sentences, start=1)]
# split_children("D50", D50_TEXT)[1] -> ("D50.2", "D50", "Switch on the bike lights from sunset to sunrise.")03Step 3: search only the children
Score every child, keep the top k. The example scores by shared words so it runs anywhere. In a real system this step is a vector or hybrid search over the child index.
04Step 4: climb to the parents and remove repeats
Walk the hits best first. Add each hit's parent id to a list unless it is already there. Two sentences from D50 give one parent. The example prints the word count of what would be sent, which is the number the context budget cares about.
05Step 5: read the example's output
For the lights question the two best children both come from D50, so one parent of 34 words is sent. For "Where do I return the bike?" the best child is in D51 and the second best, a weak match on small words, is in D50, so two parents (72 words) are sent. The second parent is noise. This is the reason for a minimum score and a budget.
06Step 6: protect the edges
Three checks keep small-to-big safe: a minimum score before any child counts, a word or token budget on the parents, and the same access filter on parents as on children. The playbook turns the first two into one function.
Try it yourself
- 1
Predict the dedupe
Change the second question to
"Is a helmet needed in a group?". Before running, write which children you expect and how many parents will be sent. Then run and compare. - 2
Raise k
Set
kto 4 in both calls. Which extra parents arrive, and do their hit scores justify them? Add a minimum score of 2 tosmall_to_bigand run again. - 3
Read the real scores
In the lab, press Search for the lights question and note the Cosine of the small chunk and of the whole guide (about 0.68 and 0.50). Then try the two other questions in the task. When does the small chunk lose?
Code & diagrams
Word overlap stands in for a real search so the example is exact. Children carry their parent's id, repeated parents are removed, and the last line of each question is the cost of what would be sent.
Expected output
Q: When should I turn on the bike lights?
hit D50.2 score 4
hit D50.4 score 2
parents sent: D50 (34 words)
Q: Where do I return the bike?
hit D51.4 score 3
hit D50.2 score 2
parents sent: D51, D50 (72 words)Try: Press Search. The one-sentence chunk about lights ranks above the whole safety guide, which contains it. Then ask "What are the safety rules for riding?": now the whole guide ranks first (about 0.48 against 0.45), because that broad question is about all of it. Try "Is a helmet required?": the group guide, which says everyone needs their own helmet, comes first (about 0.35), and the safety guide is second. Why does the guide about groups beat the guide about helmets?
Goals
- The small chunk ranks first
- The whole safety guide is still in the top two
Press Search. Then change one thing (a word in the question, a passage, the ranker) and compare the scores.
Break it on purpose
Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.
Break #1
Forget to remove repeats
In the Python tab, replace the if parent_id not in parent_ids: check and its append line in small_to_big with a plain parent_ids.append(parent_id). Run.
Myth vs fact
Myth
Bigger chunks are always better because they give the model more context.
Fact
Bigger chunks blur the vector and match precise questions worse. Parent-child gets the context without paying that price at search time.
Myth
Small-to-big means each child is stored twice.
Fact
Children are in the vector index. Parents are stored once in a plain store. The parent text is not embedded at all.
Myth
You can always send the whole parent.
Fact
Parents compete for the same context budget as everything else. When one does not fit, send the child or a window around it.
Pro corner
Extra depth for experienced readers. New to this? Skip it for now and come back later.
- ▸
Store the parent id and the position of each child (its number inside the parent) as metadata. Position lets you build a sentence window later (children n-1, n, n+1) without re-ingesting anything.
- ▸
Rank parents by the maximum child score. Summing rewards a long parent with many mediocre children. If you need both signals, log them separately and compare on your golden set (Topic 9.2) before choosing.
- ▸
When a parent is re-written, delete its old children and insert the new ones in one step (Topic 4.7). A stale child that points at a parent that no longer contains its text produces answers that cite a passage which does not say what the answer claims.
- ▸
Access control belongs on the parent id too. If a user may see child D50.2 they may not see its whole parent D50: check both, or build the rule so a parent inherits the strictest label of its children (Topic 10.3).
Remember this
- 1
Why this exists. Chunk size pulls two ways (Topic 4.4). A small chunk has a sharp meaning, so its vector matches a precise question well, but alone it may be too thin to answer from ("it costs 20": what does?). A big chunk has the context, but its vector is a blur of everything inside it, so it matches precise questions worse. Parent-child retrieval stops choosing: index small, read big.
- 2
How it works. Split each document into parents (a section, a few paragraphs) and split each parent into children (a sentence or a short paragraph). Only the children are embedded and searched. Every child stores the id of its parent. After a search you look up the parents of the best children, remove repeats, and put the parents in the prompt. This is also called small-to-big retrieval. A close cousin, sentence-window retrieval, sends the matching sentence plus a few neighbours instead of a stored parent.
- 3
The ids do the work. A child id such as
D50.2holds its parent idD50. In a real store keep the parent id as its own field, not something you cut out of a string, and keep the parents once in a plain document store keyed by id. The vector index then holds only children, and a parent costs one key lookup, not a search. If ids are not stable across re-ingestion (Topic 4.7), parents and children drift apart and you send the wrong section. - 4
Several children, one parent. Questions often match two sentences from the same section. Without care the section is sent twice. Remove repeats while keeping the order of first appearance, so the parent of the best child comes first. Rank parents by their best child's score; adding the scores instead rewards long sections with many weak matches.
- 5
Pick sizes from the questions. Make children about the size of what one question asks about, and parents about the size of what you need to understand the answer. Parents should be a section, not a whole document: a parent that is a whole manual brings back the blur and the cost you were avoiding. Always cap the total with the context budget (Topic 8.2). When a parent does not fit, fall back to the child alone or to a window of neighbours. The playbook has this recipe.
- 6
What it does not fix. A bigger parent can add text that distracts the model from the sentence that matters, so measure answers, not just retrieval. The parent must carry the same access labels as the child (Topic 10.3), or small-to-big becomes a way around permissions. And cite carefully: cite the parent's id for "read this section", and quote the child that matched, so the reader sees why it was chosen.
Explain it without notes
What problem does parent-child retrieval solve, and why can't a single chunk size solve it?
Describe the data you store and the steps at query time.
Two of the top three children share a parent. What do you do, and why?
Name two risks of small-to-big retrieval and how you reduce each.
Practice
Write parent_of(chunk_id) that returns the part of a chunk id before the first dot, and unique_parents(chunk_ids) that returns the parent ids without repeats, in the order they first appear. Print unique_parents for D50.2, D51.1, D50.4.
A parent section has 34 words and the matching chunk has 9. Write context_words(parent_ids, sizes) that adds up the words sent when each parent is sent once. Print the total for ["D50", "D50"] sent naively (a list with a repeat) and for the same list after removing repeats, with sizes = {"D50": 34}.
Trade-offs
- ↔
Small children: sharper matches and more vectors to store and search. Large children: fewer vectors, blurrier matches.
- ↔
Large parents: more context for the answer, more tokens and more distraction. Small parents: cheaper, may still miss the surrounding fact.
- ↔
Stored parents: exact and fast, but one more thing to keep in sync. Sentence windows: nothing extra to store, but a fixed shape that ignores section boundaries.
Done when you can
Done when you can explain why index-small, read-big beats either chunk size alone.
Done when you can write the split, the search, the parent lookup and the repeat removal from memory.
Done when you can say what to do when a parent does not fit the budget.
Done when you can name the access-control and freshness risks of storing parents separately.