Topic 8.5
Distributed WebSockets with Redis
In one line
When users are connected to different WebSocket servers, a message for user B arriving at server 1 must reach server 2. Redis provides the routing layer: Pub/Sub for fast fan-out, a presence map for connection ownership, and Streams or a database for messages that must survive disconnects.
Think of it like this
A large office with several reception desks. A delivery for Priya arrives at desk 1, but Priya is waiting at desk 2. The desks share an intercom (Pub/Sub) and a board showing who's waiting where (presence). If Priya has gone home, the parcel goes to her mailbox (durable storage) instead.
Key ideas
- 01
Connection ownership: each WebSocket server records
presence:{user} → server-id(a hash or string with a TTL refreshed by heartbeats) when a user connects, and removes it on disconnect. - 02
Routing options: (1) broadcast every message on one channel and let each server drop irrelevant ones (simple, doesn't scale); (2) per-server channels (
ws:server-2) using the presence map (efficient); (3) per-user or per-room channels subscribed by whichever server holds the user (flexible; with sharded Pub/Sub in Cluster). - 03
Missed events: Pub/Sub loses messages for disconnected users or servers. For chat or notifications, write the message to a durable store first (a per-conversation Redis Stream or the database), then publish a lightweight "new message" signal. Clients reconnect with their last-seen ID and fetch the gap.
- 04
Scaling WebSocket servers: they hold many long-lived connections (tens of thousands each); Redis handles routing, not the connections themselves. Use sharded Pub/Sub or per-server channels so each server only receives traffic for its own users.
- 05
Frameworks: Socket.IO's Redis adapter, Spring's STOMP broker relay (with an external broker), and many chat backends implement exactly these patterns.
Code & diagrams
# On connect (and every 20 s as a heartbeat)
HSET presence:user:42 server ws2 connId c-9f1
EXPIRE presence:user:42 60
# Route a message
HGET presence:user:42 server # "ws2"
PUBLISH ws:ws2 '{"to":42,"msgId":"1727520000123-0"}'
# On disconnect
DEL presence:user:42Interview problem
The problem
Chat across 50 WebSocket servers
Client A is connected to server 1 and client B to server 2. Server 1 must notify client B. Scale this to 50 servers and 2M connections, and handle disconnects without losing messages.
You're given
- 50 WS servers
- 2M concurrent connections
- Messages must not be lost
- Users on several devices
The interviewer follows up
Why not broadcast every message to every server?
When it breaks
Presence entries without TTL
What you see
When a server crashes, its users stay "online" on a dead server forever; messages are published to a channel nobody listens to.
Fix & prevent
Heartbeat-refreshed TTLs on presence; durable storage so published-but-undelivered messages can be fetched.
Explain it without notes
How do you avoid losing messages when WebSocket servers disconnect from Redis?
Practice
Build a two-server chat demo (two processes) that routes messages via per-server channels and recovers missed messages from a stream after restarting one server.
Trade-offs
- ↔
Per-server channels need a presence lookup per message; broadcast avoids it but doesn't scale.
Done when you can
I can design cross-server WebSocket delivery with presence, targeted Pub/Sub and a durable log.