A stage-by-stage triage of four failing cases
Real systems fill these fields from the trace of each failing question. The order of the checks is the point: a passage must be indexed before it can be retrieved.
A method for the most common RAG interview question: reproduce the failure, find the stage that lost the evidence, fix that stage, and add a test. Four realistic cases are triaged by code.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write place(ranking, expected) that returns the 1-based place of expected in ranking, or 0 when it is missing. Print the place of 'D3' in ['D1', 'D10', 'D3'] and in ['D1'].
Given the triage results ['ranked too low', 'ranked too low', 'cut by the budget', 'not indexed'], print each stage with its count, most frequent first and ties in alphabetical order.
Explain it without notes
Describe the four-step method for 'the answers are wrong'.
Name the five stages where evidence can be lost and what each points to.
What single experiment splits retrieval problems from generation problems?
Why add the question to the golden set after fixing it?
Expected output
brand-new policy page: not indexed
fix: check the loader, its filters and the last ingestion run
login question: ranked too low
fix: add a re-ranker or raise k
damage report: cut by the budget
fix: shorten the chunks or raise the context budget
lost items: ignored by the model
fix: fix the prompt, then check the order of the sources