Positional ids against content ids
The hash is FNV-1a written out by hand so that the three languages agree. Use SHA-256 in a real system: 32 bits are not enough to avoid collisions in a large store.
Documents change, and rebuilding everything on each change is slow and costly. Stable chunk ids built from content let a sync add new chunks, skip unchanged ones and delete the ones that no longer exist, so running it twice does nothing the second time.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 6 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write plan(stored_ids, wanted_ids) returning two sorted lists, the ids to add and the ids to delete. Print them for stored {"a", "b"} and wanted {"b", "c"}.
Write unchanged_share(old_chunks, new_chunks) that returns the fraction (as a string with 2 decimals) of new chunks whose exact text already existed in the old list. Print it for old ["x", "y", "z"] and new ["w", "x", "y", "z"].
Explain it without notes
Why are content-based chunk ids better than positional ones for re-ingestion?
What does it mean for a sync to be idempotent, and why is it useful?
What belongs in the content hash and what does not?
How do you handle documents that are deleted at the source?
Expected output
positional ids: d14:0, d14:1, d14:2, d14:3
chunks to embed again: 4 of 4
content ids: d14:73523ab5, d14:b00d60cf, d14:a53b8284, d14:2a4a4820
chunks to embed again: 1 of 4