Paste or retrieve? The arithmetic
The price and window size are examples, not any provider's. Change them and watch the verdicts move.
When the whole corpus fits in the model's context window, you can paste it instead of searching. That is simpler, but costs input tokens on every question. Work out the numbers, and consider using both.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write cost_to_paste(tokens, price_per_mtok, questions) returning the dollars spent on input tokens when the whole corpus is sent with each question. Print it with 2 decimals for 150000 tokens, a price of 1.00 per million tokens (an example price) and 1000 questions.
RAG costs 0.003 dollars per question and pasting costs 0.150 dollars per question. Building the RAG index costs 20.00 dollars once. Write break_even(paste, rag, index_cost) returning the smallest whole number of questions after which RAG is cheaper overall, and print it. Use math.ceil.
Explain it without notes
When might pasting the whole corpus beat RAG?
How do you compare their costs?
What non-cost factors can decide the choice?
How can you use both?
Expected output
handbook 1500 tokens | paste 1.50 | rag 3.00 | paste it
policy library 150000 tokens | paste 150.00 | rag 3.00 | RAG (cheaper)
all company docs 5000000 tokens | paste n/a | rag 3.00 | RAG (does not fit)