Quotas and Multi-Tenancy

In one line

Quotas cap produce bytes/sec, fetch bytes/sec and request CPU time per user or client ID; brokers throttle offenders by delaying responses instead of rejecting them. Combined with tenant-aware topic strategies, they keep noisy neighbours from degrading everyone.

0/5 · 0%

Think of it like this

A shared office internet connection with per-team bandwidth limits. One team downloading huge files slows down only itself, not everyone's video calls.

Key ideas

01
Quota types: producer_byte_rate, consumer_byte_rate (per broker), request_percentage (share of network and I/O thread time), and controller mutation rate (topic creation). Set on users, client IDs, or both, with defaults for everyone.
02
Enforcement: when a client exceeds its quota, the broker computes a throttle time and delays the response (and mutes the channel), so the client slows down naturally. Clients expose produce-throttle-time-avg/fetch-throttle-time-avg.
03
Quotas are per broker: a 50 MB/s producer quota on 6 brokers allows up to 300 MB/s cluster-wide if traffic is spread evenly across partitions and leaders.
04
Tenant strategies: topic per tenant (strong isolation, simple ACLs, but thousands of topics and partitions), shared topics keyed by tenant (efficient, but a huge tenant creates hot partitions and ACLs can't separate data), or tenant groups (big tenants dedicated, small ones shared). Very large or regulated tenants often get dedicated clusters.
05
Cost allocation: measure bytes in/out and storage per tenant principal for chargeback.

Code & diagrams

quotas.shbash

# Default for every user, stricter for a known noisy tenant
kafka-configs.sh --bootstrap-server $B --alter --entity-type users --entity-default \
  --add-config 'producer_byte_rate=20971520,consumer_byte_rate=41943040,request_percentage=50'
kafka-configs.sh --bootstrap-server $B --alter --entity-type users --entity-name tenant-a \
  --add-config 'producer_byte_rate=104857600,consumer_byte_rate=209715200'
kafka-configs.sh --bootstrap-server $B --describe --entity-type users --entity-name tenant-a
Quota configs for user-principal 'tenant-a' are producer_byte_rate=104857600.0, consumer_byte_rate=209715200.0

Interview problem

The problem

Noisy tenant: 500 MB/s vs 5 MB/s

Tenant A sends 500 MB/s; tenant B sends 5 MB/s. Tenant A is degrading latency for everyone. Design a quota strategy.

Explain it without notes

How does Kafka enforce quotas without rejecting requests?

Practice

Set a 1 MB/s producer quota on a lab user and run a perf test with that user.

Trade-offs

↔
Quotas protect shared clusters but can cause lag for legitimate bursts; set them with headroom and review regularly.

Done when you can

I can configure quotas and choose a multi-tenancy model.

Command Palette