Topic 10.4
Keeping Tenants Apart
In one line
When one system serves many customers (tenants), no customer may ever see another's data. Isolation must hold in the index, and also in caches, logs, queues and every other place data is remembered.
Think of it like this
An apartment building. Every flat has its own front door and key, but the building also shares a lift, a post room and a bin store. Burglars rarely pick the flat's lock; they use the shared parts. A multi-customer RAG service has the same shape: the index is the flat, and the cache, the log and the queue are the shared lift.
Words you'll meet
New words in this topic, in plain English. Come back here whenever one feels fuzzy.
- Tenant
- One customer organisation using the service. Its data must stay invisible to other tenants.
- Multi-tenant
- One deployment of the system serving many tenants.
- Tenant filter
- A condition on every query that restricts results to one tenant's rows.
- Row-level security
- A database feature that adds a condition to every query automatically, based on who is asking.
- Cache key
- The text a cache uses to decide two requests are the same. If it omits something that matters, answers cross over.
- Cross-tenant leak
- Any case where one tenant receives data that belongs to another.
- Blast radius
- How much is exposed when one thing fails. Separate indexes shrink it.
Step by step
01Step 1: decide shared index or one index per tenant
Ask how sensitive the data is, what the contract says, and how many tenants there will be. A common answer: shared index with a mandatory tenant filter for most customers, a dedicated index for the few that require it.
02Step 2: derive the tenant on the server
The tenant id comes from the authenticated session. It is passed to retrieve as an argument by trusted code. The question text and the request body never choose it.
03Step 3: put the filter in one place
retrieve(tenant, question) is the only way to search. In the example it indexes into that tenant's list; in a database it is a mandatory WHERE tenant_id = ... or row-level security.
04Step 4: watch a cache leak
Acme asks how early to cancel and gets its 2-hour rule, which is cached under the question only. Zippy asks the same and gets a hit that returns Acme's rule. Retrieval was never called for Zippy.
05Step 5: put the tenant into the key
The key becomes tenant|question. Now Zippy misses, searches its own index and gets the 24-hour rule. The third call, a differently capitalised repeat from Zippy, is a legitimate hit.
06Step 6: audit every other store
List everything that stores data between requests and write down, for each, what keeps tenants apart. The table is short and every gap in it is a future incident.
Try it yourself
- 1
Add a third tenant
Add
"brisk"with a 48-hour rule and one more call. Predict the lines for both key styles before running. - 2
Remove the normalising
Change both key functions to use
questionas written (notokenize). Does the third call still hit? What do you lose, and what do you gain in safety? - 3
Key by groups too
Add a
groupsargument to the key functions (a sorted, joined string) and think of a case where two users of one tenant must not share a cached answer.
Code & diagrams
Retrieval is isolated in both runs. Only the cache key differs, and only one run leaks.
Expected output
cache key = question only
acme miss A ride cancelled at least 2 hours before its start time is refunded in full.
zippy hit A ride cancelled at least 2 hours before its start time is refunded in full.
zippy hit A ride cancelled at least 2 hours before its start time is refunded in full.
cache key = tenant + question
acme miss A ride cancelled at least 2 hours before its start time is refunded in full.
zippy miss A ride cancelled at least 24 hours before its start time is refunded in full.
zippy hit A ride cancelled at least 24 hours before its start time is refunded in full.Break it on purpose
Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.
Break #1
Forget the tenant in the "safe" key
In the Python tab, change key_with_tenant to return " ".join(tokenize(question)), as if someone cleaned up a "redundant" part of the key. Run.
Myth vs fact
Myth
If retrieval filters by tenant, we are isolated.
Fact
Retrieval is one of many places that remember data. Caches, logs, queues, temporary files and evaluation sets can each cross tenants.
Myth
A separate index per tenant removes all risk.
Fact
It shrinks the blast radius of a bad query, but a shared cache, log or worker still crosses tenants. Isolation has to hold end to end.
Myth
The user could not guess another tenant's id, so we are fine.
Fact
Never rely on secrecy of ids. Derive the tenant from the session on the server and check it on every request.
Pro corner
Extra depth for experienced readers. New to this? Skip it for now and come back later.
- ▸
In PostgreSQL, row-level security policies can enforce
tenant_id = current_setting('app.tenant_id', true), so a forgottenWHEREstill returns nothing: withtrueas the second argument a missing setting gives NULL, the comparison is never true, and no rows match. Table owners bypass policies unless the table hasFORCE ROW LEVEL SECURITY, and superusers and roles withBYPASSRLSalways bypass them, so test the policy as the application's own role. Topic 10.9 shows the full recipe. - ▸
Per-tenant encryption keys for stored text are possible and make deleting a tenant (destroy the key) strong, but vectors in the index are not protected by encrypting only the text; decide what the index stores and who can read it (Topics 10.5 and 10.7).
- ▸
Shared embedding caches are safe only for text that is not private. A cache keyed on the text alone returns the same vector to everyone, which is fine for a public phrase and a side channel for private text (whether a given sentence was seen before). When in doubt, key by tenant.
- ▸
Watch for noisy neighbours as well as leaks: one tenant's bulk ingestion can starve others. Per-tenant rate limits and queues are an isolation feature too (Topic 11.7).
Remember this
- 1
Why it is needed. A tenant is one customer organisation using your service (here, a bike company that has bought AskDocs for its own handbook). Selling to many tenants from one deployment is cheap, and one leak between them can end the business. Topic 10.3 separated users inside one organisation; this topic separates organisations from each other, where the stakes and the contractual duties are higher.
- 2
Two ways to hold the data. (1) One shared index with a tenant filter: every chunk has a
tenant_id, every query addsWHERE tenant_id = :tenant. It is cheap and simple, and its safety is exactly as good as the discipline that every query includes the filter. (2) One index (collection, schema or database) per tenant: a query cannot reach another tenant's data because it is aimed at a different store. It costs more to run and to migrate, and it makes the failure harder to cause. Large or regulated customers often get their own index; small ones share. - 3
Make the filter impossible to forget. In the shared design, do not rely on every developer remembering
tenant_id. Put the search behind one function that takes a tenant argument and refuses to run without it, derive the tenant from the login session on the server, and where the database allows it use row-level security so the database itself adds the condition. A query that cannot be written without a tenant cannot be written wrongly. - 4
Caches are where tenants leak. A cache is a saved copy of an earlier answer, so a repeated question is answered without searching again. The example caches answers by key. With a key made only of the question, the first tenant's answer (a 2-hour refund window) is stored; when the second tenant asks the same thing the cache returns it as a hit, and the second tenant is told a rule that belongs to another company. The retrieval was perfectly isolated and the leak still happened. The cure is a key that contains the tenant (and, from Topic 10.3, the user's groups when answers differ by group). Topic 11.3 covers cache design in general.
- 5
Everything that remembers needs the tenant. The same rule applies to embedding caches (safe only if keyed by text and model, and not shared if the text is private), query logs, traces, evaluation sets, background job queues, file stores, temporary files and analytics. For each, ask: if two tenants send the same input, can one see the other's output? If a log line contains question text, which tenant's log is it in, and who can read that log?
- 6
Test with two tenants and the same question. The test that finds these bugs is cheap: create two tenants with deliberately different answers to the same question, ask both, in both orders, with and without a warm cache, and check each gets its own answer. Add canary phrases per tenant and search every output (answers, logs, cache dumps) for another tenant's canary. Run it in CI.
Explain it without notes
What is a tenant, and why is cross-tenant leakage more serious than leakage between users of one organisation?
Compare a shared index with a tenant filter to an index per tenant.
Explain how a cache can leak between tenants even when retrieval is perfectly isolated.
List five places besides the index that need tenant isolation.
Practice
Write cache_key(tenant, question) that returns the tenant, a bar, and the question's lower-case words joined by single spaces. Print the key for two tenants asking the same question, and show that differently capitalised repeats from one tenant share a key.
Write no_cross_tenant(answers) where answers maps tenant to answer text and a canary dict maps tenant to a unique phrase. Return the tenants whose answer contains another tenant's canary.
Trade-offs
- ↔
Shared index with a filter: lowest cost, relies on discipline and row-level security. Index per tenant: stronger wall, more to run and migrate.
- ↔
Caching across tenants: higher hit rates and a leak risk. Caching per tenant: lower hit rates and a clear rule.
- ↔
Per-tenant keys and storage: stronger deletion and compliance stories, more key management work.
Done when you can
Done when you can explain the tenant filter and the index-per-tenant designs and when to choose each.
Done when you can show how a cache key without the tenant leaks, and fix it.
Done when you can list every store in your system and say what keeps tenants apart in each.
Done when you can write a two-tenant canary test.