Command Palette

Search for a command to run...

Hectal
Phase 7Intermediate8 of 14 in RAG and Vector Search

Query Understanding

The question is the weak link: rewrite it, expand it, split it, resolve "it" in a conversation, route it, and read filters out of it before retrieval runs.

Everything so far improved the index. This phase improves the other input: the question. Real questions are short, vague, misspelt, and full of "it" and "that". The best index in the world cannot find what the question never pointed at. You will fix questions before retrieval, with plain rules first and a model only where rules cannot do the job.

You will build the standard query tools by hand: cleaning and rewriting, expansion with synonyms and with a guessed answer (hypothetical document embeddings), several phrasings fused with reciprocal rank fusion, turning a follow-up into a standalone question, routing a question to the right place or to no retrieval at all, and reading metadata filters out of the question as validated structured output. Every tool costs an extra step, so each topic also shows how to measure whether the step earned its place.

The phase ends by adding a query layer to AskDocs, with each step switchable and scored, and a playbook with a checklist for deciding whether an extra model call is worth it.

0/9 · 0%
9 topics ~4 h 19 code blocks & diagrams
Start with the first topic
1
7.1

The Question Is the Weak Link

The index was built carefully, but the question is whatever the user typed: short, vague, misspelt, or pointing back at something said earlier. A retriever can only compare the text it is given, so a weak question beats a strong index.

23 min 1 code practice

2
7.2

Rewriting the Query

Rewriting turns the question the user typed into the question the retriever should see: first with plain rules (lower-case, drop greetings, fix known typos), then with a model for what rules cannot do. Keep the original, guard the model's reply, and count what the extra call costs.

30 min 3 code practice

3
7.3

Expanding the Query

Expansion adds words or text to the query so it can match passages written in other words: synonyms and acronyms, or a short guessed answer that is embedded instead of the question. It raises recall, and it can also pull in noise.

31 min 3 code practice

4
7.4

Many Queries for One Question

One question can be several questions, or one question in several phrasings. Search with each, then fuse the ranked lists with reciprocal rank fusion so passages that several searches agree on rise to the top.

26 min 2 code practice

5
7.5

Retrieval in a Conversation

In a chat, the latest message is often not a question on its own: "what about the second one?". Turn the history plus the follow-up into a standalone question before retrieval, and skip retrieval when the message does not need the documents.

31 min 4 code practice

6
7.6

Routing

Not every question belongs in the handbook index. Routing sends a question to the right place (documents, a database, or nowhere): clear rules first, then a small classifier, and a safe default when nothing is sure.

22 min 1 code practice

7
7.7

Filters from the Question

"The 2024 policy" is a fact about metadata, not a meaning to embed. Read it out of the question as structured output, validate every field against what the index really contains, and apply it as an exact filter before ranking.

20 min 1 code practice

8
7.8

Build: A Query Layer for AskDocs

AskDocs gets a query layer in front of retrieve: a standalone-question step, a rewrite step and a multi-query fusion step, each a switch, measured on seven questions. Two steps together reach 7 of 7, and the third step makes one answer worse.

24 min 1 code practice

9
7.9

Playbook: Query Recipes

Recipes to copy: a standalone-question prompt with a reply guard, a multi-query fuser, a router with a safe default, and a checklist with numbers for deciding whether an extra model call earns its place.

21 min 3 code practice

Practise this phase

Hands-on exercises: write the code, run it, and press Check to have it tested automatically.

1

Rewrite a question with rules

Write rewrite_query(question, drop, fix) that lower-cases the question, splits it into runs of letters and digits, removes every word in drop, replaces words found in the dict fix, and joins the rest with single spaces.

Beginner
2

Fuse ranked lists with RRF

Write fuse(rankings), where rankings is a list of lists of passage ids (best first). Give each id 1 / (60 + rank) for every list it is in (rank starts at 1), add the scores, and return the ids as a list, best first.

Intermediate
3

Guard a model's rewrite

A model was asked to rewrite a question. Write accept_rewrite(reply, original, max_words=12) that returns the reply (with surrounding whitespace removed) only if it is not empty, has no line break inside it, and has at most max_words words. Otherwise it returns original.

Intermediate
4

Validate a filter from a model

Write parse_filter(reply, allowed). reply is text from a model and allowed maps each allowed key to a list of allowed values.

Intermediate
5

Route a question, with a safe default

Write route(question, rules, default). rules is a list of (name, words, minimum) where words is a set. Count how many of a rule's words appear in the question (lower-cased words, each counted once).

Intermediate
6

Build the standalone-question prompt

history is a list of (role, text) pairs. Write standalone_prompt(history, message, max_turns=2).

Intermediate