An idempotent sync
The store is a dictionary. A real store also keeps the text and the vector, and a real sync would call the embedding model where the example counts embeddings.
Documents change, and rebuilding everything on each change is slow and costly. Stable chunk ids built from content let a sync add new chunks, skip unchanged ones and delete the ones that no longer exist, so running it twice does nothing the second time.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 6 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write plan(stored_ids, wanted_ids) returning two sorted lists, the ids to add and the ids to delete. Print them for stored {"a", "b"} and wanted {"b", "c"}.
Write unchanged_share(old_chunks, new_chunks) that returns the fraction (as a string with 2 decimals) of new chunks whose exact text already existed in the old list. Print it for old ["x", "y", "z"] and new ["w", "x", "y", "z"].
Explain it without notes
Why are content-based chunk ids better than positional ones for re-ingestion?
What does it mean for a sync to be idempotent, and why is it useful?
What belongs in the content hash and what does not?
How do you handle documents that are deleted at the source?
Expected output
first run added 3, updated 0, deleted 0, unchanged 0 | store 3, embeddings so far 3
same again added 0, updated 0, deleted 0, unchanged 3 | store 3, embeddings so far 3
helmet text edited added 1, updated 0, deleted 1, unchanged 2 | store 3, embeddings so far 4
section renamed added 0, updated 1, deleted 0, unchanged 2 | store 3, embeddings so far 4
document deleted added 0, updated 0, deleted 3, unchanged 0 | store 0, embeddings so far 4