Command Palette

Search for a command to run...

Hectal
Phase 13Advanced14 of 14 in RAG and Vector Search

Interviews and Capstone

Turn twelve phases into interview answers and one shipped product: design questions, debugging method, trade-offs, and AskDocs assembled end to end.

This last phase has two jobs. The first is to help you talk about RAG: the questions interviewers ask, the way to structure a design answer, a method for the dreaded 'the answers are wrong' question, and how to argue a trade-off instead of reciting a preference. Every answer points back to the topic that teaches the detail, so you can fill gaps.

The second job is to finish the product. Topic 13.6 assembles AskDocs from every earlier step: keyword and toy-vector search fused with reciprocal rank fusion, an evidence threshold, a numbered-source prompt, a grounding check, a citation check, a refusal, an access filter and an evaluation report whose launch gate fails first and then passes. It runs in all three languages in your browser.

The last topic is the launch checklist: what must be true before real users arrive, and what to watch in the first week. Nothing here is new theory. It is the course used the way you will use it at work.

0/7 · 0%
7 topics ~3 h 12 code blocks & diagrams
Start with the first topic
1
13.1

The RAG Interview Map

RAG interviews probe five areas: fundamentals, design, debugging, trade-offs and production. This topic maps typical questions to the topic of this course that answers each, and gives a method for any answer.

21 min 1 code practice

2
13.2

Design: Question Answering over Company Documents

A worked system-design answer for 'build question answering over our company documents': requirements, back-of-envelope numbers, ingestion, retrieval, answers, evaluation, security and scale, with the arithmetic done by hand.

23 min 1 code practice

3
13.3

Design: A Customer-Support Assistant

A worked design for a multi-tenant support assistant: fresh tickets and articles, per-customer separation, hand-off to a human, and cost limits, with the hand-off and budget rules written and run.

20 min 1 code practice

4
13.4

"The Answers Are Wrong": Debugging Interviews

A method for the most common RAG interview question: reproduce the failure, find the stage that lost the evidence, fix that stage, and add a test. Four realistic cases are triaged by code.

23 min 2 code practice

5
13.5

Trade-off Questions

How to argue a choice instead of reciting a preference: chunk size, k, hybrid or not, re-rank or not, which store, RAG or long context. Each is a table of costs and a measurement.

21 min 2 code practice

6
13.6

Build: Ship AskDocs

AskDocs assembled: an access filter, keyword and vector search fused with reciprocal rank fusion, an evidence threshold, a numbered-source prompt, a grounding check, a citation check and a refusal, followed by an evaluation report that fails its launch gate first and passes it after the fix.

41 min 3 code practice

7
13.7

Playbook: The RAG Launch Checklist

The checklist to run before real users arrive, written as a gate that reports blockers, and the alerts to watch every day of the first week.

20 min 2 code practice

Practise this phase

Hands-on exercises: write the code, run it, and press Check to have it tested automatically.

1

Estimate memory and cost for a design

Write vector_mb(chunks, dimensions) that returns the raw size in megabytes (1 MB = 1,000,000 bytes) of 32-bit float vectors.

Intermediate
2

Write the hand-off rules

Write decide(best, refusals_in_row, asked_for_person, spent, limit) returning one of: "person", "budget", "hand off", "rephrase" or "answer".

Intermediate
3

Apply the access filter before ranking

Write visible(docs, role) returning the docs this role may see. Each doc is a dict with keys id, text and access. Docs with access "public" are visible to everyone and "staff" docs only to the role "staff".

Advanced
4

Build the launch gate

Write gate(checks) where checks is a list of tuples (label, value, op, limit) and op is ">=" or "<=". Return the list of labels whose check fails, in order.

Advanced
5

Daily alerts for the first week

Write alerts(day, baseline_cost) where day is a dict with keys questions, refused, thumbs_down, p95 and dollars, and baseline_cost is the cost per question of day one.

Advanced