Command Palette

Search for a command to run...

Hectal
Back to the lesson: Topic 4.4 — Chunking Strategies
RAG and Vector Search · Example 2 of 2

Three strategies on the rental guide

The rental guide is a new handbook document, D14: a Markdown file with three sections. D1 to D12 are unchanged.

A chunk is the unit that gets embedded, retrieved and shown to the model. Fixed-size, sentence, recursive and structure-aware chunking trade simplicity against keeping ideas whole, and size and overlap should be measured, not guessed.

Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.

Ready for more? This phase has 6 hands-on exercises with automatic checks.

Practice questions

Write the code in the editor, run it, then open the model answer to compare.

01

Write pack(sentences, limit) that joins sentences with single spaces into chunks no longer than limit characters (a sentence longer than the limit stays alone). Print the chunks for the sentences ["A.", "BB.", "CCC."] with limit 7.

02

Write starts_mid_sentence(chunks) that counts chunks whose first character is not an upper-case letter. Print the count for ["Hello there.", "ello there.", "world"].

Explain it without notes

01

Compare fixed-size, sentence, recursive and structure-aware chunking.

02

What does overlap do, what does it cost and what does it not fix?

03

Why can very large chunks hurt retrieval even when the answer is inside them?

04

How would you decide on a chunk size for a new corpus?

Three strategies on the rental guide
Sign in to run this example in your browser.

Expected output

fixed     4 chunks, longest 130, 3 start mid-sentence, 0 know their section
sentences 6 chunks, longest 121, 0 start mid-sentence, 0 know their section
structure 6 chunks, longest 121, 0 start mid-sentence, 6 know their section