A leak, its fix and per-tenant quotas
Acme must never see Globex's discount code. Prints the bug, the fix and a quota burst.
When many customers share one LLM service, each needs their own quota, their own data and their own failure domain. The core rules: identify the tenant first, put the tenant into every query, cache key and limit, and test that tenant A can never see tenant B.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 6 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write cache_key(tenant, model, question) that returns one string that differs for each of the three inputs and normalises the question to lower case with single spaces. Show that two tenants asking the same question get different keys.
Write TenantLimiter(limit) with allow(tenant) that counts per tenant and refuses after the limit. Show that one tenant's burst does not affect another.
Explain it without notes
What is a noisy neighbour and how do you contain it in an LLM service?
Where can cross-tenant data leaks happen in an LLM application?
How should the tenant id be established and passed around?
How do you test tenant isolation?
Expected output
shared: acme asks about 'discount' -> discount code is SAVE20
scoped: acme asks about 'discount' -> no match
scoped: globex asks about 'discount' -> discount code is SAVE20
acme:ok acme:ok acme:ok acme:429 acme:429 globex:ok