Topic 10.3
Access Control at Retrieval Time
In one line
If some documents are for some people only, the check must happen inside retrieval, before the model sees anything. Every chunk carries who may read it, the search filters on that first, and a test asks as every kind of user.
Think of it like this
A school library has a shelf for teachers only. The librarian does not fetch every book and then say "please do not read the teachers' ones". The pupil is simply never handed them. If a hidden book reaches the pupil's desk even once, the damage is done, however politely they are asked to ignore it.
Words you'll meet
New words in this topic, in plain English. Come back here whenever one feels fuzzy.
- Access label
- Metadata on a chunk saying who may read it, such as public, staff or a team name.
- ACL (access control list)
- A list of who may read an item. Here, the labels on a chunk compared with the user's groups.
- Pre-filter
- Remove chunks the user may not see before ranking, so the top k are all allowed.
- Post-filter
- Rank first and remove forbidden chunks afterwards. It can leave too few results or none.
- Fail closed
- When something is missing or broken, allow nothing. The opposite, fail open, allows everything.
- Canary
- A planted test chunk with a unique phrase, used to detect whether it ever shows up where it should not.
- Leak
- A chunk or an answer that includes information the user is not allowed to see.
Step by step
01Step 1: give every chunk a label at ingestion
The label comes from where the document lives (the staff wiki, the public site) and is set by code. Chunks inherit the label of their document. If the label is missing, ingestion stops with an error instead of defaulting to public.
02Step 2: pass the identity into retrieve
retrieve(question, groups, k) takes the user's groups from the server-side session. The question text has no say in it.
03Step 3: see the no-filter leak
For "How do I get the staff discount on the monthly pass?" the best two passages are staff-only (C4 with 7 shared words, C5 with 4). Without a filter a public user is shown both.
04Step 4: see why filtering after the search is not enough
After the search the two staff chunks are removed and nothing is left. The user is told the handbook has no answer, even though the public pass price (C3) exists. The filter must be part of the search.
05Step 5: pre-filter and fail closed
The filter always runs. A user with no groups gets an empty candidate list, and the product answers with the refusal from Topic 8.4.
06Step 6: write the leak test
Loop over users and questions, count returned labels the user does not hold, and fail on anything above zero. The buggy filter passes for Alice and Bob and leaks for the guest, which is why the guest must be in the test.
Try it yourself
- 1
Predict the leak count
Before running the second example, work out what the buggy filter returns for the guest on each of the three questions (look at the chunk scores). Then run it and compare.
- 2
Add a manager level
Add a chunk
("C6", "manager", ...)that matches the pass question, and a usercarolwith groups{"public", "staff"}. Does she see it? Addmanagerto her groups and rerun. - 3
Raise k
In the first example change
2to4. Does filtering after the search now return something? What does that tell you about post-filtering and k? - 4
Forget the label
Give a chunk the label
"". Which users can see it? Change the code so a chunk without a label is visible to nobody.
Code & diagrams
Real stores take the filter as part of the query (a WHERE clause or a filter expression), so this is the shape of what they do.
Expected output
Q: How do I get the staff discount on the monthly pass? (user groups: public)
no filter -> C4 (7), C5 (4)
filter after search -> nothing
filter before search -> C3 (3)The guest has no groups on purpose. Unusual users are where filters break.
Expected output
buggy filter alice: 0 leaks, bob: 0 leaks, guest: 5 leaks
fail-closed filter alice: 0 leaks, bob: 0 leaks, guest: 0 leaksPostgreSQL with pgvector, as in Topic 5.2: <=> is cosine distance, and the array operator && is true when the two arrays share at least one element, so an empty groups array matches nothing (and a missing label, NULL, matches nothing too). The groups array comes from the server-side session, never from the request. Not run here: the browser has no database.
import os
import psycopg # pip install "psycopg[binary]"
SQL = """
SELECT id, doc_id, 1 - (embedding <=> %s::vector) AS score
FROM chunks
WHERE allowed_groups && %s::text[] -- overlap: at least one group in common, an empty list matches nothing
ORDER BY embedding <=> %s::vector
LIMIT %s
"""
def search(query_vector, user_groups, k):
text = "[" + ",".join(str(x) for x in query_vector) + "]"
with psycopg.connect(os.environ["DATABASE_URL"]) as conn:
# user_groups comes from the server-side session, never from the request body
return conn.execute(SQL, (text, user_groups, text, k)).fetchall()
Break it on purpose
Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.
Break #1
A staff note labelled public
In the Python tab of the first example, change the label of C5 from "staff" to "public", as a careless import might, and run.
Myth vs fact
Myth
The system prompt can say "never reveal staff documents".
Fact
A prompt is a request, not a lock. The model should never receive text the user may not read. Do the check in code, before the prompt is built.
Myth
Filtering the answer afterwards is enough.
Fact
By then the model has used the forbidden text, and its answer can paraphrase or summarise it in ways no output filter recognises. Filter the inputs.
Myth
Labels are set by whoever writes the document.
Fact
Labels come from ingestion code, from where the document lives and who owns it. A label inside the document's own text is untrusted like anything else in it.
Pro corner
Extra depth for experienced readers. New to this? Skip it for now and come back later.
- ▸
Prefer pre-filtering inside the store. With approximate indexes, a filter that removes most vectors can lower recall: the index walks neighbours and finds few that qualify. Test recall for your most restrictive group (Topic 3.7 covers this) and consider a separate index for very small, very sensitive groups.
- ▸
Model sensitive groups as data, not as hard-coded strings: store
allowed_groupson each chunk as a list and pass the user's groups as an array, as in the SQL above. Per-document sharing ("shared with these ten people") then works with the same code. - ▸
Log the labels of the chunks that were retrieved for each request, together with the user's id and groups. When someone reports a leak, you can answer who saw what. Treat the log itself as sensitive (it contains question text) and apply the same access rules to it.
- ▸
Search-time checks are not the only layer. Cache keys must include the user's groups (Topic 10.4), and the final answer should cite only chunks that were sent. A citation to a chunk the user may not see is a leak by itself.
Remember this
- 1
Why it is needed. A search index mixes everything: public handbook pages, staff pricing, a manager's notes. A question about the staff discount will retrieve the staff-only passage because it matches best, and a model that has been given that passage will happily use it. A prompt is not a lock. "Do not reveal staff-only information" fails the first time the model is persuaded, confused, or just helpful. The only safe design is that the model never receives text the user may not read.
- 2
Every chunk carries a label. Topic 4.6 put access labels in the metadata:
public,staff, or a team name. They are written by ingestion code, not by the document's text, and they travel with each chunk through chunking, embedding and storage. A chunk without a label is a bug: fail closed, meaning an unlabelled chunk is visible to nobody until someone labels it. - 3
Filter before the search, not after. There are two designs. Post-filtering takes the top k results and removes what the user may not see. It looks fine and quietly fails: in the example, the best two passages for the staff question are both staff-only, so after the filter the public user gets nothing, and the public pass-price passage that could have helped was never in the top 2. Pre-filtering removes forbidden chunks from the candidates first, then ranks. The user gets the best chunk they are allowed to see. Vector stores support this as a filter inside the query (Topics 3.7 and 5.5).
- 4
Fail closed on missing identity. The classic bug is a filter written as "if the user has groups, restrict to them". A user with no groups, such as a guest or a broken profile, skips the filter and sees everything. The rule must be the reverse: the filter is always applied, and no groups means no results. The second example shows the leak count for each version.
- 5
Where the user's groups come from. From your login system, on the server, per request. Never from the request body, a cookie the client can edit, or the model. The retrieval function receives the identity as an argument from trusted code, and the user's question cannot change it. Group membership also changes (people leave teams), so check at query time, not only when the document was indexed.
- 6
Test it by asking as everyone. The control is only real if a test proves it. The example runs every question as every user and counts chunks returned whose label the user does not hold. The target is exactly zero, on every run, in CI. Add a canary: one chunk with a unique made-up phrase at each label, and assert that the phrase never appears in any answer for a user without that label. Logs and traces need the same labels (Topic 10.4).
Explain it without notes
Why is it not enough to tell the model in the system prompt not to reveal restricted documents?
Explain pre-filtering and post-filtering. What can go wrong with post-filtering?
What does "fail closed" mean for access control in retrieval, and what bug does it prevent?
How would you test, automatically, that a user never sees a chunk they are not allowed to see?
Practice
Write visible(chunks, groups) that returns the ids of chunks whose label is in groups, and returns an empty list when groups is empty. Test it with an empty set, {"public"} and {"public", "staff"}.
Write leaks(hits, groups) that returns the ids of hits whose label the user does not hold. Test it with the staff chunk and a public user.
Trade-offs
- ↔
Pre-filtering inside the store: correct results and one query, but a very restrictive filter can lower the recall of an approximate index. Test it, and consider a separate index for small, sensitive groups.
- ↔
Strict fail-closed labels: nothing leaks by accident, and new documents are invisible until labelled. That is the price worth paying.
- ↔
One shared index with labels versus one index per group: labels are simpler and cheaper, separate indexes give a stronger wall. Topic 10.4 discusses the choice for tenants.
Done when you can
Done when you can explain why a prompt is not an access control.
Done when you can show with an example how post-filtering loses results.
Done when you can write a retrieve function that fails closed for a user with no groups.
Done when you can write a leak test with a canary and run it in CI.