Command Palette

Search for a command to run...

Hectal
Phase 8Intermediate9 of 15 in LLM Engineering

Retrieval Basics (RAG)

Give a model the right pages of your own data: keyword search, vector search, chunking, a working pipeline, numbered citations and honest refusals.

A model knows only what it read during training. It has never seen your price list, your policies or yesterday's incident report, and it will still answer questions about them in a confident voice. Retrieval-augmented generation (RAG) fixes this in the simplest possible way: before the model answers, your program looks up the few pages that matter and puts them in the prompt. The model then answers from those pages instead of from memory.

This phase builds retrieval by hand on a tiny shop: eight short documents from a made-up bike shop called Nimbus Bikes. You will write an inverted index and TF-IDF scoring, a fake embedder (a small deterministic stand-in that makes the maths visible, not a real learned model), cosine similarity with top-k and tie-breakers, a chunker, a full ingest-retrieve-prompt-answer pipeline, a citation checker and a rule that refuses to answer without evidence. Every example runs in Python, Java and JavaScript and prints the same numbers, small enough to check with a pencil.

Real embedding models and vector databases appear as short, clearly marked sections (Topics 8.3 and 8.7), and two real-model labs show a model citing a source and refusing when the answer is not there. The coming RAG course goes much deeper on production retrieval: hybrid search, re-ranking, evaluation of retrieval and scale.

0/8 · 0%
8 topics ~3 h 19 code blocks & diagrams
Start with the first topic
1
8.1

Why Models Need Your Data

A model knows only its training text, which ended months ago and never included your private documents. RAG finds the relevant pages at question time and puts them in the prompt, which is cheaper, fresher and easier to check than retraining.

20 min 1 code practice

2
8.2

Keyword Search First

An inverted index lists, for every word, the documents that contain it. TF-IDF scores a match higher when the word is frequent in the document and rare across all documents. This simple search is a strong baseline that real systems still use.

23 min 2 code practice

3
8.3

Vector Search by Hand

An embedding turns text into a list of numbers so that texts with similar meaning get similar lists. Cosine similarity measures how close two lists point, and top-k returns the best few. Here the embedder is a small fake so every number can be checked by hand.

26 min 3 code practice

4
8.4

Chunking Documents

Real documents are long, but a search result and a prompt need short, focused pieces. Chunking cuts documents into pieces of a chosen size, with some overlap or along their structure, and attaches metadata so each piece can be found, cited and updated.

23 min 2 code practice

5
8.5

A Working RAG Pipeline

A RAG pipeline is four small functions in a row: ingest the documents once, then for each question retrieve the best chunks, build a prompt with numbered sources, and let the model answer. Here the whole thing runs by hand in three languages.

23 min 1 code practice

6
8.6

Citing Sources and Refusing Without Evidence

A grounded answer names the numbered source it came from, your code checks that the citation is real and supports the claim, and when the documents do not contain the answer the system says so instead of guessing.

27 min 4 code practice

7
8.7

Vector Databases in Brief

A vector database stores embeddings and finds the nearest ones quickly. Exact search compares the query with every vector, which is fine for thousands of chunks; approximate indexes trade a little accuracy for speed at millions. Often a list in memory is enough.

23 min 2 code practice

8
8.8

Playbook: Minimal RAG Service

Five recipes to copy: a small service that chunks, indexes, retrieves, cites and refuses; a retrieval test set that measures hit rate and picks the refusal threshold from data; an incremental re-index; a prompt template; and the before-you-ship checklist.

33 min 4 code practice

Practise this phase

Hands-on exercises: write the code, run it, and press Check to have it tested automatically.

1

Build an inverted index

Write build_index(docs): docs is a dictionary from document name to text. Return a dictionary from each word to the sorted list of names of the documents that contain it (a document counts once per word, however often it repeats it).

Beginner
2

Rank documents with TF-IDF

Write rank(query, docs) returning the names of the documents with a score above 0, best first. A document's score is the sum, over the query words, of (how many times the word is in the document) times ln(N / df), where N is the number of documents and df is how many documents contain the word. Words no document contains add nothing.

Intermediate
3

Cosine similarity and top-k

Write cosine(a, b): the dot product divided by the product of the two lengths. If either vector has length 0, return 0.0.

Intermediate
4

Chunk words with overlap

Write chunk_words(words, size, overlap) that returns a list of strings, each the joined words (with single spaces) of one window of at most size words. Each window starts size - overlap words after the previous one, and the last window ends at the last word.

Intermediate
5

Parse and check citations

Write citations(text) returning the list of integers found as [n] in the text, in order (repeats included). Brackets that do not hold only digits, like [x], are ignored.

Intermediate
6

Answer with guards

Write answer(question, retrieve, model, min_score=0.3). retrieve(question) returns a list of (score, text) pairs, best first. Keep only the texts whose score is at least min_score.

Advanced