An analyser pipeline, built from parts
Real engines use the same three stages with ready-made, tested parts. The stemmer here is the crude one from Topic 1.1.
You rarely write an inverted index yourself. Lucene-based engines (Elasticsearch, OpenSearch), PostgreSQL full-text search and SQLite FTS5 all implement the ideas of this phase, with analysers, BM25 and fields built in. Know what each gives you, and when your database is enough.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write pipeline(*steps) that returns a function applying each step to a text in order. Build clean from str.lower, str.strip and a step that replaces - with a space, and print clean(" Built-In Chain ").
Write same_terms(a, b, analyze) that is true when analyze(a) == analyze(b), with analyze lower-casing, splitting into letters and digits and dropping one final s from words longer than 3 letters. Print it for ("Riders", "rider") and ("Riders", "ride").
Explain it without notes
Name the three stages of an analyser and give an example of each.
When is PostgreSQL's built-in full-text search enough, and when would you choose a dedicated engine?
Why do two engines give different BM25 scores for the same query and data?
Expected output
html removed: Riders under 16 must wear one.
standard: riders under 16 must wear one
english: rider under 16 must wear one
english / english: rider must wear
english / standard: must wear