System 12.2 — Rate Limiter (HLD)
In one line
The LLD limiter from Phase 4, now distributed: sticky to one instance won't work, so the counters move to Redis.
Think of it like this
The single bouncer idea from Phase 4, but for a whole nightclub CHAIN with branches worldwide. Every branch (server) must agree on the same headcount for the same VIP, so the counting now has to happen somewhere shared (Redis), not in each bouncer's own head.
Key ideas
- 01
Requirements: limit per user/IP/API-key per period (e.g. 100 req/min per key, 1000/day).
- 02
Enforcement point: middleware/filter BEFORE the API handler → reject fast (HTTP 429 + Retry-After).
- 03
Where: edge gateway (centralized, protects everything) vs per-service (latency-adjacent, duplicated policy).
- 04
Counting: Redis INCR + EXPIRE per key — atomic multi-instance; Lua for sliding-window-counter precision.
- 05
Sliding window counter: track window start + counter; weight previous window's remnant — smooth, memory-friendly (1–2 keys per user).
- 06
Failure handling: Redis DOWN → fail open (allow) vs fail closed (429 everyone)? Fail open at edge with local static budget, or have replicas.
- 07
Distributed correctness: read-modify-write races — single Redis command / Lua script makes check-and-set atomic.
- 08
Config: rules per endpoint dimension, async sync of config to gateways; allowlists (VIP users) exempt.
Java / Spring map
- →
Bucket4j + Redisson distributed; or a Lua script INCR/EXPIRE — the code from phase-4/phase-5 upgraded to Redis.
Code & diagrams
The atomic fixed-window counter — the script that makes the distributed limiter correct.
-- KEYS[1] = "rl:{key}:{window}"
-- ARGV[1] = limit, ARGV[2] = ttl seconds
local current = redis.call('INCR', KEYS[1])
if current == 1 then
redis.call('EXPIRE', KEYS[1], ARGV[2])
end
if current > tonumber(ARGV[1]) then
return 0 -- blocked
end
return 1 -- allowed
-- One atomic script = check-and-set with zero races across instances.Explain without notes
Walk the race if this were GET + SET instead of INCR/Lua — two instances both allow at the same millisecond.
Practice
Add the sliding-window-counter variant and argue memory vs accuracy for a 100k-rps API.
Trade-offs
- ↔
Centralized limiter = one ingress bottleneck; distributed rules = policy skew between nodes.
Run it in production
You've designed it. Now build, operate, and break the same idea hands-on in the DevOps courses:
Completion checklist
I can present distributed rate limiting with Redis EXPIRE/Lua and the fail-open/fail-closed decision.