Cost of one question, then of a month
All prices are example values in named variables. They are not any provider's prices. The sum is the lesson, not the figures.
RAG has a one-off cost (embedding the documents), a small monthly cost (storing the vectors) and a per-question cost (embedding, re-ranking and generation). Working out the cost of one question by hand shows which lever saves the most.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write generation_cost(prompt_tokens, answer_tokens, in_price, out_price) for prices per million tokens and print the cost of a 1,750-token prompt and 200-token answer at 0.50 and 1.50 dollars per million, to six decimals.
Write monthly_saving(cost_per_question, hit_rate, questions) returning what a cache saves in a month, and print it to two decimals for 0.0016754, 30 percent and 100,000 questions.
Explain it without notes
List the costs of a RAG service and say which are one-off, monthly and per question.
Why is generation usually the largest part of the cost per question?
How do you choose between sending fewer passages, skipping the re-ranker and adding an answer cache?
Why can the savings of several cost levers not simply be added?
Expected output
one-off: embed 6000000 tokens = 0.12 dollars
storage: 30720000 bytes of vectors = 0.0077 dollars per month
per question (k=5): embed question 0.0000004 | re-rank 0.0005000 | generate (input) 0.0008750 | generate (output) 0.0003000 | total 0.0016754
base, 100000 questions a month: 167.54 dollars
send 3 passages, not 5 137.54 dollars, saves 30.00
skip the re-ranker 117.54 dollars, saves 50.00
answer cache, 30% hits 117.28 dollars, saves 50.26
all three together 61.28 dollars, saves 106.26
biggest single lever: answer cache, 30% hits