What a model with a limit actually reads
A toy: it counts words, where a real model counts tokens. The limit of 24 is made up so the example stays short. The lab uses the real 512.
A question and the passage that answers it look very different: one is short and vague, the other long and full of detail. Some models want different prefixes for each, and every model reads only a limited number of tokens and silently ignores the rest.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write fits(text, limit_words) that returns True when the text has at most limit_words words (use text.split()), and ignored(text, limit_words) that returns the words past the limit as a string. Print both for the 10-word text 'one two three four five six seven eight nine ten' with a limit of 6.
Write prepare(text, kind, query_prefix, passage_prefix) and print how it prepares the question 'Do you rent bikes?' and the passage 'We rent bikes.' with prefixes "query: " and "passage: ".
Explain it without notes
What happens when you embed a text longer than the model's input limit, and how do you guard against it?
Why do some embedding models need different prefixes for queries and passages?
Why can a short, vague query retrieve poorly even with a good model?
What is the difference between a model's limit measured in tokens and in words, and why does it matter?
Expected output
the passage has 32 words, the model reads 24
last word read: jobs, first word ignored: the
question vs whole passage: 0.371
question vs what the model saw: 0.000
no prefixes:
can I cancel?
A ride cancelled at least 2 hours before ...
query/passage prefixes:
query: can I cancel?
passage: A ride cancelled at least 2 hours before ...