Command Palette

Search for a command to run...

Hectal
Back to the lesson: Topic 8.2 — The Context Budget
RAG and Vector Search · Example 2 of 2

Best first, or best at both ends

The skimming reader is a toy that looks only at the first and last passage. Real models are not that extreme, but position can change what they use.

Everything in the prompt competes for one fixed window, so sources get a token budget. You count tokens with a stated rough rule, pack passages best-first, decide what to do with one that does not fit, and order the rest. More context is not better.

Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.

Ready for more? This phase has 5 hands-on exercises with automatic checks.

Practice questions

Write the code in the editor, run it, then open the model answer to compare.

01

Write pack(hits, budget, overhead) where hits is a list of (doc_id, text) pairs in rank order. Keep a passage only when its cost (estimate plus overhead) still fits; skip passages that do not fit and carry on. Return the kept ids. Print the result for the example hits with a budget of 80.

02

Write at_the_edges(ranked) that puts rank 1 first and rank 2 last, rank 3 second and rank 4 second to last, and so on. Print the order for ranks ["a", "b", "c", "d", "e", "f"].

Explain it without notes

01

How do you estimate tokens without a tokenizer, and when is that not good enough?

02

Describe the 'stop' and 'skip' packing policies and the trade-off between them.

03

Why is sending more passages not always better?

04

What can the order of passages change?

Best first, or best at both ends
Sign in to run this example in your browser.

Expected output

best first         D8 D3 D1 D6 D4
                   the skimming reader sees: D8 and D4
best at the edges  D8 D1 D4 D6 D3
                   the skimming reader sees: D8 and D3