Command Palette

Search for a command to run...

Hectal
Back to the lesson: Topic 4.9 — Build: The Ingestion Pipeline
RAG and Vector Search · Example 1 of 1

AskDocs v1: load, clean, chunk with context, embed, upsert, run again

The embedder counts ten word families, a toy that stands in for a real model (Phase 2). The three Markdown files are made-up copies of handbook entries D1 and D6 and the new rental guide D14.

AskDocs version 1: documents become chunks with heading paths and metadata, are embedded and upserted by content id, and a second run changes only what changed. Retrieval and the refusal still work on the stored chunks.

Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.

Ready for more? This phase has 6 hands-on exercises with automatic checks.

Practice questions

Write the code in the editor, run it, then open the model answer to compare.

01

Write summary(added, deleted, store_size, embeds) returning the one-line run summary in the exact format added 4, deleted 0, store 4, embedding calls 4. Print it for those numbers.

02

Write stale_ids(store_ids, wanted_ids) returning the sorted ids in the store that are no longer wanted. Print them for {"x", "y", "z"} and {"y"}.

Explain it without notes

01

Walk through what ingest does for one file, from raw text to a stored vector.

02

Why is the second run supposed to do nothing, and how do the numbers show it?

03

How does the pipeline plug into the earlier retrieve and answer functions without changing them?

04

Name three things this version does not do yet and which phase adds each.

AskDocs v1: load, clean, chunk with context, embed, upsert, run again
Sign in to run this example in your browser.

Expected output

run 1: added 4, deleted 0, store 4, embedding calls 4
run 2: added 0, deleted 0, store 4, embedding calls 4
run 3: added 1, deleted 1, store 4, embedding calls 5  (late.md edited)
Q: What is charged for a late return?
   late.md > Late returns | Bikes returned late are charged 3 dollars for every 15 minutes after the booked end time.
Q: Do you rent scooters?
   I can't find that in the handbook.