Topic 12.2
Summaries and Hierarchies
In one line
A short summary of each document or section can be its own searchable layer. Searching summaries first picks the right document, then searching inside it finds the right sentence, and a whole-document question can be answered from the summary itself.
Think of it like this
A library. You do not read every book spine to find a recipe. You go to the cooking shelf first (a coarse choice), then look through the books on it (a fine one). A library works because there are layers: a floor, a shelf, a book, a page.
Words you'll meet
New words in this topic, in plain English. Come back here whenever one feels fuzzy.
- Summary
- A short text that says what a longer text is about.
- Coarse-to-fine
- Search a rough layer first to choose where to look, then search the detailed layer inside that choice.
- Hierarchical index
- Several layers of the same content at different sizes: sentences, sections, documents.
- Overview question
- A question about a whole document or topic, with no single sentence as its answer.
- Routing
- Deciding which index a question goes to.
- RAPTOR
- A published method (Sarthi et al., 2024) that builds a tree of summaries over chunks and retrieves from any level.
Step by step
01Step 1: a question with no matching sentence
"What is the trip guide about?" shares almost no words with any single sentence of the guide. Searching sentences either returns something arbitrary or nothing. The answer lives in the guide as a whole.
02Step 2: write a summary for each document
One line per document, written once at ingestion. Here they are written by hand; in a real system a model writes them, and you review a sample.
SUMMARIES = {
"D50": "Safety rules for riding: helmet, lights at night, braking on wet roads, crossings.",
"D51": "Planning a trip: routes, bike lanes, hills, returning the bike before closing.",
}03Step 3: choose the document by its summary
Score each summary against the question and take the best. The example prints all the scores (D50=3 D51=0), so you can see that the choice was clear, not a coin flip.
04Step 4: search inside the chosen document
Only the chosen document's sentences are compared with the question. The wrong document's look-alike sentences never get the chance to win.
05Step 5: route overview questions to the summary
The word "about" sends the second question to the summary level, so it is answered from the summary of D51. Without that rule it would return an arbitrary sentence of the guide.
06Step 6: keep a fallback
If the coarse step is unsure (a low or tied top score), search the sentences of every document instead. The summary layer should make good questions cheaper and broad questions possible, not make every other question fail.
Try it yourself
- 1
Add a third document
Add a document D52 with a summary and four sentences (the group rides guide works). Ask a question about group bookings. Does the coarse step pick D52?
- 2
Break the router
Ask
"Tell me what the trip guide covers". It contains no router word, so what comes back? Add"covers"to the router's word list. - 3
Read the lab
In the lab, compare the Cosine of the trip summary with that of the sentence
Plan your route in the app before you unlock the bike.for the overview question (about 0.49 and 0.35). Why does the summary score higher?
Code & diagrams
The summaries are hand-written stand-ins for model-written ones. The coarse line shows every document's score; the overview question skips the sentences entirely.
Expected output
Q: Are lights needed at night?
coarse: D50=3 D51=0
answer from D50 (sentence): Switch on the bike lights from sunset to sunrise.
Q: What is the trip guide about?
coarse: D50=0 D51=2
answer from D51 (overview): Planning a trip: routes, bike lanes, hills, returning the bike before closing.Try: Press Search. The summary of the trip guide ranks above every single sentence for an overview question. Change the question to "When should I turn on the lights?": which level wins now? Then edit the trip summary so it no longer mentions trips, and watch it fall.
Goals
- The trip summary ranks first
Press Search. Then change one thing (a word in the question, a passage, the ranker) and compare the scores.
Break it on purpose
Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.
Break #1
Turn off the overview route
In the Python tab, replace if "about" in tokenize(question): with if False:. Run.
Myth vs fact
Myth
Summaries replace the detailed index.
Fact
Summaries lose exact details such as numbers, codes and names. Keep the sentence index and use summaries as an added layer.
Myth
A model-written summary is always accurate.
Fact
A summary can omit or invent. Review a sample, regenerate when the document changes, and never cite a summary as if it were the source.
Myth
More layers are always better.
Fact
Each layer costs a model call per item at ingestion and a rebuild on every change. Add a layer when a measured question type needs it.
Pro corner
Extra depth for experienced readers. New to this? Skip it for now and come back later.
- ▸
Summaries are an ingestion cost you can compute: documents times the price of one summary call (Topic 11.5). For 100,000 documents at an example price per call that is a real bill, and a full rebuild repeats it, so build summaries incrementally from content hashes.
- ▸
Generate several short summaries for long documents (one per section) rather than one for the whole. A single summary of a long document averages it to nothing, and a question about one section will not match.
- ▸
Log which level answered each question. A rising share of questions answered from summaries can mean users ask broad questions you do not have good pages for.
Remember this
- 1
Why this exists. Two kinds of question defeat sentence-sized search. "What is the trip guide about?" has no matching sentence: the answer is spread over the whole document. And in a large collection, comparing a question with millions of sentences can pick a look-alike sentence from the wrong document. A summary is a short text that captures what a document or section is about. Searched on its own, it answers the first problem and narrows the second.
- 2
Two levels of search. Build a coarse index of summaries (one per document, or per section) and a fine index of the sentences. At query time first search summaries to choose a document, then search only that document's sentences. This is coarse-to-fine or hierarchical retrieval. The coarse step removes most of the false matches before the fine step starts.
- 3
Routing the question. An overview question ("what is it about?", "summarise the group rules") should be answered from the summary level. A detail question ("when do I need lights?") should go to the sentence level. The example uses a simple rule, the word "about". In practice a small classifier or a model decides (Topic 7.6). Getting the route wrong is the usual bug, so test both kinds of question.
- 4
Who writes the summaries. Usually a language model, once, at ingestion. That makes summaries an ingestion cost (a model call per document or section) and a freshness risk: when a document changes, its summary must be rebuilt (Topic 4.7). The example's summaries are written by hand as a stand-in for that model. A summary can also state something the document does not, so for important content check summaries the way you check answers (Topic 8.7).
- 5
Trees of summaries. Summarise sentences into a section summary, sections into a document summary, documents into a collection summary, and you have a tree. RAPTOR (Sarthi et al., 2024) is a published method of this kind: it repeatedly groups chunks, summarises each group and indexes every level so a question can match at the level that fits it. You will rarely need more than two levels. Add a third only when a measured question type needs it.
- 6
Mind the vocabulary gap. A summary is written in different words than the text, which is good for overview questions and bad for exact ones: a part number or error code in a sentence may not appear in its summary. Keep the sentence index in the search too, and let the router or a fusion of both levels decide. Summaries add recall for broad questions. They never replace the detailed index.
Explain it without notes
Why can't sentence-level search answer "what is this document about?"
Describe coarse-to-fine retrieval and the problem it removes in a big collection.
What new costs and risks do summaries introduce?
How would you decide which level a question should be sent to?
Practice
Write route(question) that returns "overview" when the question contains the word about, summary or overview, and "detail" otherwise. Print the route for three questions of your choice.
Write best_doc(question, summaries) where summaries maps a document id to its summary. Return the id whose summary shares the most different words with the question (the first wins a tie), or None when no summary shares a word. Print the result for one question that matches and one that does not.
Trade-offs
- ↔
Summaries: enable broad questions and narrow the search, at the price of a model call per item and a rebuild on change.
- ↔
One summary per document: cheap, but loses section detail. One per section: more calls, better matches.
- ↔
Strict routing: fast and predictable, brittle on unusual phrasings. Searching both levels and fusing (Topic 6.3): robust, costs two searches.
Done when you can
Done when you can explain coarse-to-fine search and its two benefits.
Done when you can say which question types need the summary level and how you would route them.
Done when you can list the ingestion cost and the freshness rule for summaries.
Done when you can say why the sentence index must stay.