Command Palette

Search for a command to run...

Hectal
Back to the lesson: Topic 8.2 — Keyword Search First
LLM Engineering · Example 1 of 2

An inverted index and three searches

Document numbers start at 1 so they match the numbered sources used later in the phase.

An inverted index lists, for every word, the documents that contain it. TF-IDF scores a match higher when the word is frequent in the document and rare across all documents. This simple search is a strong baseline that real systems still use.

Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.

Ready for more? This phase has 6 hands-on exercises with automatic checks.

Practice questions

Write the code in the editor, run it, then open the model answer to compare.

01

Write idf(n_docs, df) returning ln(n_docs / df), and print it to 4 decimals for df = 1, 2, 4 and 8 with 8 documents.

02

Write tf_idf(word, words, n_docs, df) for one word in one document (tf times idf). Use 8 documents; the word appears twice in the document and in only 1 document overall. Print to 4 decimals.

Explain it without notes

01

What is an inverted index and why is it faster than scanning every document?

02

Explain tf and idf, and why their product is a useful score.

03

Name two limits of plain keyword search and one fix for each.

04

Why do you need an explicit tie-breaker when sorting by score?

An inverted index and three searches
Sign in to run this example in your browser.

Expected output

8 documents, 82 different words
days: returns, shipping, repairs
warranty: warranty
bike: repairs, helmets
bikes: returns, payment
the: returns, repairs, hours, battery
search 'free helmet' -> helmets
search 'bike' -> repairs, helmets
search 'get my money back' -> nothing