Topic 5.10
Concurrent Rate Limiter
In one line
Applying everything in this phase to the limiter: per-key state, atomic claims, and the shared-map granularity decision.
Think of it like this
A water tap that refills a bucket at a fixed rate. Everyone shares the same tap logic, but each person (or API key) has their own bucket, so one thirsty person can't drain someone else's water.
Key ideas
- 01
One limiter object per key; shared ConcurrentHashMap<String, TokenBucket> as the registry.
- 02
Per-bucket synchronization (not global) keeps unrelated keys independent.
- 03
computeIfAbsent to create buckets atomically; the claim itself is the synchronized method.
- 04
Sliding-window alternative: store deques of timestamps (log) or counters with expiry — memory vs accuracy.
- 05
The discussion escalates to scale: single JVM ok; N instances need Redis (Phase 12).
Java / Spring map
- →
The full TokenBucket + RateLimiter is in phase-4 → rate-limiter-lld.
Explain without notes
Why lock per bucket and not the registry map? Show the contention difference.
Practice
Add a per-key max burst and a global total cap that share the same code path.
Trade-offs
- ↔
Synchronized bucket = simple, serialized per key; CAS per claim = lock-free but fiddlier on multi-step refill.
Run it in production
You've designed it. Now build, operate, and break the same idea hands-on in the DevOps courses:
Completion checklist
I can deploy the single-instance limiter and name precisely what breaks at instance #2.