Topic 13.1
The RAG Interview Map
In one line
RAG interviews probe five areas: fundamentals, design, debugging, trade-offs and production. This topic maps typical questions to the topic of this course that answers each, and gives a method for any answer.
Think of it like this
A driving test. The examiner does not ask you to recite the highway code. They ask you to park, to pull out at a junction and to explain what you would do if a tyre burst. A RAG interview works the same way: a few kinds of task, each testing whether you can use what you know, not whether you can list it.
Words you'll meet
New words in this topic, in plain English. Come back here whenever one feels fuzzy.
- Fundamentals question
- A question that checks you understand a concept: BM25, cosine, recall, an embedding.
- Design question
- A request to sketch a whole system for a stated product, with numbers.
- Debugging question
- A described failure ('answers are wrong') where you must show a method.
- Trade-off question
- A choice between options where you must name the costs of each.
- Clarifying question
- A question you ask the interviewer before answering, to fix the scope and numbers.
- Follow-up
- The interviewer's next, harder question, built on your last answer.
Step by step
01Step 1: sort the question
When a question arrives, decide which of the five areas it belongs to. It changes what a good answer looks like: a fundamentals answer is short and exact, a design answer is long and structured.
02Step 2: ask one clarifying question
Pick the question whose answer changes your design: how many documents, how fresh, who may see what, what an error costs. Asking shows judgement. Asking five shows stalling.
03Step 3: answer short first
Say the conclusion in one or two sentences, then support it. 'I would use hybrid search with reciprocal rank fusion, because the questions mix exact identifiers and paraphrases.' If you run out of time, the interviewer already has your answer.
04Step 4: use one number
One worked number does more than three adjectives. '200,000 chunks of 384 dimensions is about 300 megabytes of raw vectors' proves you can reason about scale. Topic 13.2 practises this.
05Step 5: name your own weakness
Say where your answer breaks: 'a fixed threshold works until the data changes, so I would recalibrate on the golden set'. Interviewers push on weak spots anyway. Naming it first takes the initiative.
06Step 6: close with the measurement
End with how you would know. Almost always that is the golden set and the metrics of Phase 9: hit rate, MRR, refusal rate, citation validity. 'I would measure it' without saying what is empty.
Try it yourself
- 1
Find your gaps
For each of the twelve questions, answer out loud in two sentences with one example before reading the pointed-to topic. Mark the ones you stumbled on. Read only those topics.
- 2
Add three questions
Add three questions you were asked or fear being asked, each with the topic that answers it. If you cannot find a topic, you have found a gap in your knowledge or in this course.
- 3
Time the short answer
Pick one fundamentals question and answer it in 30 seconds, then in two minutes. The 30-second version is what you say first in an interview.
Code & diagrams
Every pointer is a topic of this course. Add your own weak questions to the list and drill the areas with most rows.
Expected output
fundamentals (3)
Topic 1.4: Why does BM25 beat raw word counts?
Topic 2.2: Cosine, dot product or distance: does it matter?
Topic 2.4: What does an embedding model actually learn?
design (3)
Topic 4.4: How would you choose a chunk size?
Topic 3.4: Walk me through a vector index such as HNSW.
Topic 10.4: How do you keep one customer's data from another's?
debugging (2)
Topic 9.6: The answers are wrong. Where do you start?
Topic 9.7: Last week's answers were better. What changed?
trade-offs (2)
Topic 6.7: Hybrid search or vectors only?
Topic 12.8: RAG or a long context window?
production (2)
Topic 11.2: How fresh can answers be, and how do you prove it?
Topic 11.5: What does a question cost, and what is the biggest lever?
total: 12 questions in 5 areasBreak it on purpose
Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.
Break #1
Remove a row and lose an area
In the Python tab, delete both 'production' rows from QUESTIONS and run.
Myth vs fact
Myth
RAG interviews are about memorising library names.
Fact
They test reasoning about stages, numbers and failure modes. Library names are the easiest part to look up.
Myth
A longer answer is a better answer.
Fact
A short correct answer first, with depth on request, shows control. A long answer that buries the conclusion shows the opposite.
Myth
If I have not built a big system I cannot answer design questions.
Fact
You can reason from the numbers. This course's AskDocs gives you a small system whose every choice you can defend.
Interview problem
The problem
Explain RAG and defend your choices in two minutes
Imagine the interviewer says: 'Our support team keeps answering the same questions from a 500-page handbook. Explain how you would use RAG for that, and why not just fine-tune a model.' You have two minutes.
You're given
- State what RAG is in one sentence a manager understands.
- Say why not fine-tuning, with one reason about facts and one about change.
- Name the main failure and how you would find it.
The interviewer follows up
When would you not use RAG here?
How do you stop it inventing an answer?
What is the first thing you would measure?
Pro corner
Extra depth for experienced readers. New to this? Skip it for now and come back later.
- ▸
Senior interviews weight the questions you did not expect: 'what would make you turn this feature off?'. Prepare the answer with your own monitoring: refusal rate, thumbs down, leak tests (Topics 9.8 and 11.6).
- ▸
Interviewers often escalate a fundamentals answer into design: you explain HNSW, they ask what happens when you filter by tenant (Topic 3.7). Know the boundary where each idea breaks.
- ▸
Keep one failure story ready: a real or built case where retrieval looked fine and the answer was wrong, the stage you found, the fix and the test you added. One honest story beats ten abstract claims.
Remember this
- 1
Why a map helps. RAG touches search, machine learning, databases, security and cost, so the number of possible questions looks endless. It is not. Almost every question is a variation on five areas, and in each area a small set of ideas does most of the work. If you know the area, you know where to look in this course.
- 2
Fundamentals test whether the words mean something to you: why BM25 beats raw word counts (Topic 1.4), what cosine measures and why unit-length vectors matter (Topics 2.2 and 2.3), what an embedding model learns and what it cannot (Topic 2.4), what recall means for an approximate index (Topic 3.2). The test is a clear two-sentence answer plus one example.
- 3
Design questions give a product and ask for a system: question answering over company documents (Topic 13.2) or a customer-support assistant (Topic 13.3). They test structure. A strong answer clarifies requirements, does the arithmetic out loud, draws the two paths (ingest and query), names the evaluation, and mentions security without being asked.
- 4
Debugging questions describe a failure and ask what you would do (Topic 13.4). They test method: reproduce, find the stage that lost the evidence, fix that stage, add a test. The common weak answer is 'try a better model'. The strong answer says which stage and how you would prove it.
- 5
Trade-offs questions ask you to choose and defend: chunk size, k, hybrid or not, re-rank or not, which store, RAG or a long context (Topic 13.5). There is rarely a right answer. The test is whether you name what you give up and what you would measure to decide.
- 6
Production questions ask about the life of the system: freshness (Topic 11.2), cost (Topic 11.5), observability (Topic 11.6), incidents (Topic 11.9), changing the embedding model (Topic 11.8), deletion and privacy (Topics 10.5 and 10.6). They test whether you have thought past the demo.
- 7
A method for any answer. (1) Restate the problem and ask one clarifying question. (2) Give the short answer first. (3) Give one concrete number or example from a system you built. (4) Name the failure mode of your own answer. (5) Say how you would measure it. The runnable example prints the map as a study list: use it to find the areas where you cannot yet do step 3.
Explain it without notes
Name the five areas of RAG interview questions and what each one tests.
What is the method for structuring any answer, and why does it start with a clarifying question?
Why is 'try a better model' a weak answer to 'the answers are wrong'?
How do you prepare for questions you cannot predict?
Practice
Write by_area(questions, area) that returns the topic numbers of the rows in area, in order. Test it on a list of three rows and print the result for 'debugging'.
Write missing_areas(questions, practised) that returns the sorted list of areas present in questions but not in the set practised. Print it for a practised set of two areas.
Trade-offs
- ↔
Preparing by area: efficient and transferable, but you must still practise saying answers out loud.
- ↔
Preparing by question list: good for rehearsal, brittle when the question changes shape.
- ↔
Deep on a few ideas versus broad on all: depth impresses on follow-ups, breadth avoids blank spots. Cover the map first, then deepen your two strongest areas.
Done when you can
Done when you can name the five areas and give an example question for each.
Done when you can follow the five-step answer method without notes.
Done when every area has one number and one failure story you can tell.
Done when you know which topic of this course to open for any question on the map.