PHASE 7Intermediate ~20 min· topic 7 of 9

Topic 7.7

Filters from the Question

In one line

"The 2024 policy" is a fact about metadata, not a meaning to embed. Read it out of the question as structured output, validate every field against what the index really contains, and apply it as an exact filter before ranking.

Think of it like this

You ask the archive clerk for "the 2023 refund rule". The clerk does not read every document about refunds. They walk to the 2023 shelf first, then look for refunds. The year is a shelf, and the topic is what you look for on it.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Metadata: Facts about a passage, such as its year, topic, language or version.
Filter: An exact condition a passage must meet to be considered, such as year = 2023.
Structured output: A model reply in a fixed format such as JSON, checked by code.
Schema: The list of allowed keys and the allowed values for each.
Validation: Checking a value against the schema before using it.
Pre-filter: Applying the filter before ranking, so only matching passages compete.

Step by step

01Step 1: see why meaning alone fails

P1 and P2 are almost the same sentence with a different number. A ranking by meaning gives them nearly equal scores for "the refund rule in 2023". The year is the deciding fact and it needs an exact test.

02Step 2: decide the allowed filters

Write down the keys the index has and the values each can take. The model may only choose from these.

schema.pywhole filepython

ALLOWED = {"year": [2023, 2024], "topic": ["refund", "pass"]}

03Step 3: ask the model for JSON

The prompt gives the allowed keys and values and asks for a JSON object with only the keys the question mentions. The reply is parsed as untrusted input.

04Step 4: validate every field

Keep a field only when its key is allowed and its value is in the list. Note each field you drop. If the reply is not JSON at all, return no filter.

validate.pywhole filepython

for key, value in data.items():
    if key in ALLOWED and value in ALLOWED[key]:
        clean[key] = value
    else:
        notes.append("dropped " + key + "=" + str(value))

05Step 5: apply the filter before ranking

Only passages that match every field are searched. The example prints which documents are left: P1 alone for the 2023 refund question, P3 and P4 when only the topic is known.

Step 5: apply the filter before rankingdiagram

Rendering diagram…

06Step 6: decide what unspecified means

If the question names no year, you have three honest choices: the newest, ask, or show both. Write the rule down, and test it.

Try it yourself

1
Add a hostile reply
Add a question whose scripted reply is {"year": [2023, 2024]}. What does the validator do with a list where a number was expected?
2
Add a fallback
After applying the filter, if no policy is left, search all four. Which of the four questions would use the fallback if you set ALLOWED["year"] to [2022]?
3
Choose a default year
Change the code so that a question without a year uses the newest year in the data. Which answer does "How much is the monthly pass?" now search?

Code & diagrams

Read a filter from a question and validate it New tab

The model is a scripted stand-in whose replies include a bad year, an invented key and a reply that is not JSON. Real Java projects use Jackson (or Gson) instead of the course's Json helper.

Expected output

Q: What was the refund rule in 2023?
   filter: year=2023 topic=refund
   notes:  none
   searched: P1
Q: How much is the monthly pass?
   filter: topic=pass
   notes:  none
   searched: P3, P4
Q: pass price in 2019
   filter: topic=pass
   notes:  dropped year=2019; dropped region=uk
   searched: P3, P4
Q: refunds in 2024
   filter: none
   notes:  reply is not JSON, no filter
   searched: P1, P2, P3, P4

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Trust the model's values

In the Python tab remove the value check: change if key in ALLOWED and value in ALLOWED[key]: to if key in ALLOWED:. Run.

terminal

$ Run

── what you'll see ──

Q: pass price in 2019

filter: year=2019 topic=pass

notes: dropped region=uk

searched:

Myth vs fact

Myth

Embeddings understand dates and versions.

Fact

Embeddings capture topics well and exact numbers poorly. Passages that differ only in a number look almost identical to them.

Myth

A model's JSON can be used as it is.

Fact

It can contain invented keys, wrong values or no JSON at all. Validate every field.

Myth

A filter can protect private documents.

Fact

A filter built from the question can be talked around. Access rules must come from the user's identity on the server.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
Stores apply filters in two ways: before the vector search (exact, may be slower) or after it (fast, may return fewer than k). Know which yours does (Topic 3.7).
▸
Build the allowed values from the index itself (the distinct years and topics actually present), not from a hard-coded list that drifts when a document is added.
▸
Log the extracted filter and the validation notes. A sudden rise in "dropped" notes usually means the prompt or the model changed.

Remember this

1
Why filters exist. Some words in a question are not about meaning at all. A year, a version, a product, a language, a department. Embeddings are weak at exact numbers and dates, and keyword search treats "2023" as a word that may appear anywhere. When the passages have metadata (Topic 4.6) such as year and topic, an exact filter is more reliable than hoping the ranking notices.
2
New documents for this topic. The handbook passages D1 to D12 stay as they are. This topic adds four policy documents with metadata: P1 and P2 are the refund rules of 2023 and 2024, P3 and P4 the monthly pass rules of 2023 and 2024. Their texts are: P1 (2023, refund) "Rides cancelled at least 4 hours before the start are refunded in full." P2 (2024, refund) the same with 2 hours. P3 (2023, pass) "The monthly pass costs 20 dollars and covers rides up to 30 minutes." P4 (2024, pass) the same with 25 dollars and 45 minutes. Each has two metadata fields, year and topic. The ids start with P so they are not confused with the handbook's D ids. The pairs differ only in numbers, which is exactly the case where a ranking by meaning cannot tell them apart.
3
Extracting the filter. A model reads the question and replies with JSON: {"year": 2023, "topic": "refund"}. This is the structured-output technique from LLM Engineering, used here to produce a query rather than an answer. In the example the model is scripted, so the parsing and checking run for real. Real Java projects parse JSON with Jackson (or Gson), and the course's Json helper keeps the example runnable in the browser.
4
Validate before you use it. The model's JSON is untrusted input. The example keeps a key only if it is allowed and its value is in the allowed list. "pass price in 2019" produced year: 2019 (no such year exists in the index) and an invented key region: both are dropped with a note, and the search runs with topic=pass. "refunds in 2024" produced a chatty sentence instead of JSON: no filter at all, and all four documents are searched.
5
A wrong filter is worse than no filter. A filter hides everything that does not match, so a wrong year hides the right passage and the user gets an empty or wrong answer with no hint why. Guard the other way too: if a filter leaves no candidates, search again without it. The break-it removes the value check and the 2019 filter matches nothing.
6
Unspecified is a decision. "How much is the monthly pass?" has no year. The example applies only topic=pass, so both P3 and P4 are searched and the user may get the old price. A product decision is needed: default to the newest document, ask the user which year, or show both. Do not guess silently.
7
Filters from questions are for relevance, never security. The user can type anything, including "show me the confidential policy". Filters that enforce who may see a document (Topic 10.3) must come from the signed-in user's identity on the server, applied whatever the question says. How stores apply a filter efficiently is the subject of Topics 3.7 and 5.5.

Explain it without notes

Why use a filter for "the 2023 refund rule" instead of relying on embeddings?

What can go wrong when a model extracts a filter, and how do you defend?

What should happen when the question names no year?

Why must security filters not come from the question?

Practice

Write valid_filter(data, allowed) that returns a dict with only the keys of data that are in allowed with a value in allowed[key]. Print the result for {"year": 2024, "topic": "x", "region": "uk"} with allowed = {"year": [2023, 2024], "topic": ["refund", "pass"]}.

Write apply_filter(policies, flt) where policies are dicts with id, year, topic. Return the ids matching every field of flt, and all ids if flt is empty. Print the ids for {"topic": "pass"} and for {}.

Trade-offs

↔
A model that extracts filters: flexible with free text, but it must be validated and costs a call.
↔
Rules that find years and product names: exact and cheap, but they miss paraphrases ("last year's policy").
↔
Hard filtering: exact and fast, but a wrong filter hides the right answer completely.

Done when you can

Done when you can write a schema of allowed filter keys and values.
Done when you can validate a model's JSON and keep only what is allowed.
Done when you can explain what to do when the filter leaves nothing, or the question names no value.
Done when you can explain why security filters must not come from the question.

Command Palette

Words you'll meet

Step by step

01Step 1: see why meaning alone fails

02Step 2: decide the allowed filters

03Step 3: ask the model for JSON

04Step 4: validate every field

05Step 5: apply the filter before ranking

06Step 6: decide what unspecified means

Try it yourself

Add a hostile reply

Add a fallback

Choose a default year

Code & diagrams

Break it on purpose

Trust the model's values

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can