Paste everything or retrieve: the sums
The window size and the price are example values, not any provider's real numbers. Put in your own before deciding anything.
There are three ways to give a model knowledge it lacks: paste all of it into the prompt, retrieve only the relevant parts (RAG), or fine-tune the model. Each fits a different situation, and the choice is mostly arithmetic.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write fits(pages, window, tokens_per_page=500, overhead=200) returning True when pasting the whole corpus fits the window. Test it for 60 and 2,000 pages with a 128,000-token window.
Write cost_per_question(tokens, price_per_mtok) and print, to five decimals, the cost of a 30,200-token prompt and a 1,700-token prompt at an example price of 0.50 dollars per million tokens.
Explain it without notes
When is pasting the whole document into the prompt the right engineering choice?
Give three costs of a very long prompt.
Why is fine-tuning a poor way to teach a model your company's facts?
A team has 500 pages of documentation that change weekly and a model with a 200,000-token window. What do you recommend and why?
Expected output
RAG sends 1700 tokens per question: 8.50 dollars per 10000 questions, whatever the corpus size.
one policy (4 pages): pasting everything sends 2200 tokens, fits, 11.00 dollars
the handbook (60 pages): pasting everything sends 30200 tokens, fits, 151.00 dollars
a help centre (2000 pages): pasting everything sends 1000200 tokens, does NOT fit, 5001.00 dollars
all company documents (100000 pages): pasting everything sends 50000200 tokens, does NOT fit, 250001.00 dollars