PHASE 2Beginner ~31 min· topic 6 of 10

Topic 2.6

Queries vs Documents

In one line

A question and the passage that answers it look very different: one is short and vague, the other long and full of detail. Some models want different prefixes for each, and every model reads only a limited number of tokens and silently ignores the rest.

Think of it like this

A library catalogue card and the book. You search with a few words ('bike lock rules') and the card lists what the book is about. The card is short on purpose. A search model has the same two sides: a short *query* and a long *document*, and they are not the same kind of text.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Query: The text of the user's question, embedded at search time.
Document / passage: A piece of the knowledge base, embedded ahead of time. In RAG, one chunk.
Asymmetric search: Searching where the query and the documents are different kinds of text, such as a short question against long passages.
Prefix: A short marker such as 'query: ' added before a text, for models trained to expect one.
Token: A piece of text (often a word or part of a word) that a model reads. Limits are counted in tokens.
Truncation: Cutting an input to the model's limit. Most embedding models do it silently.

Step by step

01Step 1: count what the model will read

Before embedding, count the tokens of every text and compare with the model's limit. In the example, the passage has 32 words, and the model reads 24. The lab shows a token count next to every passage.

02Step 2: find what falls off the end

The 25th word is 'the', and what follows is the sentence 'the rider is charged a 150 dollar fee'. The model never sees it. The example prints the last word read ('jobs') and the first word ignored ('the').

limit.pywhole filepython

def embed(text, limit=None):
    words = tokenize(text)
    if limit is not None:
        words = words[:limit]      # a real model cuts tokens, and says nothing
    ...

03Step 3: compare scores

The question 'what fee is charged if my bike is stolen' scores 0.371 against the whole passage and 0.000 against what the model saw. The information is in the document but not in its vector.

Step 3: compare scoresdiagram

Rendering diagram…

04Step 4: prepare queries and passages the right way

The second part of the example prints a plain version and a version with query: and passage: . Which one is right depends on the model. Put the choice in one function so it cannot differ between indexing and searching.

05Step 5: run the lab

Search the four starting passages: 'Short' (just the sentence) scores about 0.64, 'Long first' about 0.29, 'Long last' about 0.24 and the monthly pass about 0.13. Open Vectors and read the token counts: both long passages are about 600, over the limit of 512. The first goal passes and the second fails.

06Step 6: fix the long passage

In 'Long last', delete filler until two copies of the paragraph remain (about 216 tokens) and search again. The score rises from about 0.24 to about 0.38 and the goal passes, because the sentence is now inside what the model reads. That is chunking, in one experiment.

Try it yourself

1
Move the limit
In the example, set MAX_WORDS = 40. The model now reads the whole passage. What are the two cosines? Then set it to 8. What happens?
2
Read the lab's token counts
Open the Vectors tab in the lab after searching. How many tokens does each passage have? Delete one copy of the filler at a time from 'Long last' and find the first length where its score jumps.
3
Compare the three fixes
In the long passage with the sentence last, try (1) moving the sentence to the start, (2) deleting most of the filler, (3) replacing the filler with one short sentence. Which gives the highest score? Which best matches what a chunker (Phase 4) would do?
4
Write the prefix function
Add a function prepare(text, kind) to the example that returns "query: " + text for kind "query" and "passage: " + text for "passage". Use it for both printed lines.

Code & diagrams

What a model with a limit actually reads New tab

A toy: it counts words, where a real model counts tokens. The limit of 24 is made up so the example stays short. The lab uses the real 512.

Expected output

the passage has 32 words, the model reads 24
last word read: jobs, first word ignored: the
question vs whole passage:     0.371
question vs what the model saw: 0.000
no prefixes:
  can I cancel?
  A ride cancelled at least 2 hours before ...
query/passage prefixes:
  query: can I cancel?
  passage: A ride cancelled at least 2 hours before ...

The same sentence, at the start or past the endreal embeddings

Try: Two long passages (about 600 tokens each, the model reads 512) hold the same text. 'Long first' has the fee sentence at the start, 'Long last' has it at the end. Press Search and compare their scores for the stolen-bike question, then open Vectors to read the token counts. Then solve the goal: edit 'Long last' so it scores at least 0.27, for example by deleting filler until the fee sentence is inside what the model reads.

QuestionPassages (4)

Long first. A bike stolen while unlocked costs the rider a 150 dollar fee. Lumen Bikes opened its first shop in a small town by the river. Our team services every bike each week, checks the brakes, and cleans the chain. The shops are decorated with photos from rides that our riders sent in. We serve coffee on Saturdays, and the shop dog likes to greet everyone at the door. New riders often ask where the best cycling routes are, and our staff love to share maps. The company began with ten bikes and now looks after hundreds in several neighbourhoods. Lumen Bikes opened its first shop in a small town by the river. Our team services every bike each week, checks the brakes, and cleans the chain. The shops are decorated with photos from rides that our riders sent in. We serve coffee on Saturdays, and the shop dog likes to greet everyone at the door. New riders often ask where the best cycling routes are, and our staff love to share maps. The company began with ten bikes and now looks after hundreds in several neighbourhoods. Lumen Bikes opened its first shop in a small town by the river. Our team services every bike each week, checks the brakes, and cleans the chain. The shops are decorated with photos from rides that our riders sent in. We serve coffee on Saturdays, and the shop dog likes to greet everyone at the door. New riders often ask where the best cycling routes are, and our staff love to share maps. The company began with ten bikes and now looks after hundreds in several neighbourhoods. Lumen Bikes opened its first shop in a small town by the river. Our team services every bike each week, checks the brakes, and cleans the chain. The shops are decorated with photos from rides that our riders sent in. We serve coffee on Saturdays, and the shop dog likes to greet everyone at the door. New riders often ask where the best cycling routes are, and our staff love to share maps. The company began with ten bikes and now looks after hundreds in several neighbourhoods. Lumen Bikes opened its first shop in a small town by the river. Our team services every bike each week, checks the brakes, and cleans the chain. The shops are decorated with photos from rides that our riders sent in. We serve coffee on Saturdays, and the shop dog likes to greet everyone at the door. New riders often ask where the best cycling routes are, and our staff love to share maps. The company began with ten bikes and now looks after hundreds in several neighbourhoods. Lumen Bikes opened its first shop in a small town by the river. Our team services every bike each week, checks the brakes, and cleans the chain. The shops are decorated with photos from rides that our riders sent in. We serve coffee on Saturdays, and the shop dog likes to greet everyone at the door. New riders often ask where the best cycling routes are, and our staff love to share maps. The company began with ten bikes and now looks after hundreds in several neighbourhoods.

Long last. Lumen Bikes opened its first shop in a small town by the river. Our team services every bike each week, checks the brakes, and cleans the chain. The shops are decorated with photos from rides that our riders sent in. We serve coffee on Saturdays, and the shop dog likes to greet everyone at the door. New riders often ask where the best cycling routes are, and our staff love to share maps. The company began with ten bikes and now looks after hundreds in several neighbourhoods. Lumen Bikes opened its first shop in a small town by the river. Our team services every bike each week, checks the brakes, and cleans the chain. The shops are decorated with photos from rides that our riders sent in. We serve coffee on Saturdays, and the shop dog likes to greet everyone at the door. New riders often ask where the best cycling routes are, and our staff love to share maps. The company began with ten bikes and now looks after hundreds in several neighbourhoods. Lumen Bikes opened its first shop in a small town by the river. Our team services every bike each week, checks the brakes, and cleans the chain. The shops are decorated with photos from rides that our riders sent in. We serve coffee on Saturdays, and the shop dog likes to greet everyone at the door. New riders often ask where the best cycling routes are, and our staff love to share maps. The company began with ten bikes and now looks after hundreds in several neighbourhoods. Lumen Bikes opened its first shop in a small town by the river. Our team services every bike each week, checks the brakes, and cleans the chain. The shops are decorated with photos from rides that our riders sent in. We serve coffee on Saturdays, and the shop dog likes to greet everyone at the door. New riders often ask where the best cycling routes are, and our staff love to share maps. The company began with ten bikes and now looks after hundreds in several neighbourhoods. Lumen Bikes opened its first shop in a small town by the river. Our team services every bike each week, checks the brakes, and cleans the chain. The shops are decorated with photos from rides that our riders sent in. We serve coffee on Saturdays, and the shop dog likes to greet everyone at the door. New riders often ask where the best cycling routes are, and our staff love to share maps. The company began with ten bikes and now looks after hundreds in several neighbourhoods. Lumen Bikes opened its first shop in a small town by the river. Our team services every bike each week, checks the brakes, and cleans the chain. The shops are decorated with photos from rides that our riders sent in. We serve coffee on Saturdays, and the shop dog likes to greet everyone at the door. New riders often ask where the best cycling routes are, and our staff love to share maps. The company began with ten bikes and now looks after hundreds in several neighbourhoods. A bike stolen while unlocked costs the rider a 150 dollar fee.

Rank byTop k (retrieved)

Embedding modelall-MiniLM-L6-v2384 numbers per text. Runs in your browser: the first Search downloads about 23 MB once, and your text never leaves this page.

Goals

Long first scores at least 0.27
Long last scores at least 0.27

Press Search. Then change one thing (a word in the question, a passage, the ranker) and compare the scores.

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Set the limit far too small

In the Python tab, change MAX_WORDS = 24 # to MAX_WORDS = 8 # (a much smaller limit). Run.

terminal

$ Run

── what you'll see ──

the passage has 32 words, the model reads 8

last word read: brakes, first word ignored: and

question vs whole passage: 0.371

question vs what the model saw: 0.000

Myth vs fact

Myth

If a text is too long, the model gives an error.

Fact

Most embedding models silently ignore everything past the limit. You get a vector, and it is quietly wrong for the cut-off part.

Myth

A longer passage always has a better vector.

Fact

Past a point a long passage gets a blurred vector that is a little about everything. A focused chunk beats a huge page.

Myth

Prefixes are a style choice.

Fact

For models trained with them, leaving them out lowers quality. For models not trained with them, adding them can also hurt. Follow the model card.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
Count tokens with the model's own tokenizer, not by splitting on spaces. Words, numbers and symbols split into several tokens each, so 400 words can exceed 512 tokens. In the lab, the long passages have about 610 tokens for about 540 words. The limit also depends on the library: the model card of all-MiniLM-L6-v2 says sentence-transformers truncates at 256 word pieces by default, while the browser setup in this lab reads 512, so check the limit in the tool you actually run.
▸
Make truncation loud in your own code: count the tokens before embedding and log or raise an error when a chunk is over the limit. A silent cut is a bug you only discover through wrong answers.
▸
Store which preparation (prefixes) was used with the index. If someone later changes it for queries but not for stored passages, quality drops and nothing fails (Topic 2.10 turns this into a check).

Remember this

1
Why this exists. Topic 2.4 explained that an embedding model learns from pairs. In real data the pairs are usually *asymmetric*: a short question paired with a long passage. If the model treats both sides the same way, it can do badly on a short, vague question. This topic covers the three practical consequences: prefixes, length and the input limit.
2
Query and passage modes. Some models are trained with a marker on each side, so you must add it: for example, the E5 family of models was trained with query: before a question and passage: before a passage, and their model cards say to use them. Other models, including the lab's, use no prefix at all, and some libraries have separate methods for queries and documents (encode_query and encode_document in recent sentence-transformers). Read the model card for each model. Using a prefix when the model does not expect one, or leaving it out when it does, quietly lowers quality. The example prints both ways of preparing a text.
3
Vague queries. A two-word query has little to match. In the lab of Topic 2.7, 'my money' is closest to the monthly-pass passage (about 0.19) and not to the refunds passage (about 0.09). People ask short questions, so real systems rewrite or expand them before searching (Phase 7).
4
The input limit. Every embedding model reads at most a fixed number of tokens (word pieces, LLM Engineering Topic 1.1). The lab's model reads 512. Anything after that is *silently ignored*: no error, no warning, just a vector computed from the first 512 tokens. In the toy example the model reads 24 words of a 32-word passage and never sees 'charged' or 'fee', so its vector has no money direction and the score against 'what fee is charged' drops from 0.371 to 0.000.
5
Seeing it for real. The lab below has two long passages made of the same pieces: six copies of a filler paragraph (about 600 tokens in all) and the fee sentence 'A bike stolen while unlocked costs the rider a 150 dollar fee.' In 'Long first' the sentence comes first, in 'Long last' it comes last. For 'What happens if my bike is stolen?' the first scores about 0.29 and the second about 0.24, and their vectors are almost the same (a similarity of about 0.98), because both are mostly filler. In the second, the fee sentence fell past the limit and the model never read it. The short passage with only the sentence scores about 0.64. The passages hold the same information and are ranked differently only because of where it sits.
6
What to do about it. Cut documents into chunks that fit well inside the limit, put the most important text first, and keep one topic per chunk (Phase 4). A chunk that is much shorter than the limit gets a sharper vector: in the lab, cutting the filler down to two copies (about 216 tokens), so the fee sentence is read, lifts the score of the 'last' passage to about 0.38, and about 0.42 when the sentence is first.

Explain it without notes

What happens when you embed a text longer than the model's input limit, and how do you guard against it?

Why do some embedding models need different prefixes for queries and passages?

Why can a short, vague query retrieve poorly even with a good model?

What is the difference between a model's limit measured in tokens and in words, and why does it matter?

Practice

Write fits(text, limit_words) that returns True when the text has at most limit_words words (use text.split()), and ignored(text, limit_words) that returns the words past the limit as a string. Print both for the 10-word text 'one two three four five six seven eight nine ten' with a limit of 6.

Write prepare(text, kind, query_prefix, passage_prefix) and print how it prepares the question 'Do you rent bikes?' and the passage 'We rent bikes.' with prefixes "query: " and "passage: ".

Trade-offs

↔
Long chunks: more context per vector, but blurred meaning and a higher risk of crossing the input limit.
↔
Short chunks: sharper vectors, but more of them and less context per chunk (Phase 4 balances this).
↔
Query rewriting: better matches for vague questions, at the cost of an extra step and an extra place to go wrong.

Done when you can

Done when you can explain what a model does with text past its input limit.
Done when you can count tokens with the right tokenizer and know your model's limit.
Done when you know whether your model wants query and passage prefixes.
Done when you can show in the lab that the same sentence scores differently at the start and past the limit.

Command Palette

Words you'll meet

Step by step

01Step 1: count what the model will read

02Step 2: find what falls off the end

03Step 3: compare scores

04Step 4: prepare queries and passages the right way

05Step 5: run the lab

06Step 6: fix the long passage

Try it yourself

Move the limit

Read the lab's token counts

Compare the three fixes

Write the prefix function

Code & diagrams

Break it on purpose

Set the limit far too small

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can