PHASE 12Advanced ~7 min· topic 6 of 39Level 2

System 12.6 — Twitter

In one line

The classic read-heavy fan-out problem: write once, read everywhere, and the pull vs push decision for feeds.

0/39 · 0%

Think of it like this

A stadium announcer with 100 million fans. When they say something, it's WAY faster to have already pushed a copy of the announcement to every fan's pocket radio (push) than to make every fan tune in and ask 'what did they just say?' (pull) — except for the handful of announcers with 50 million followers, where pushing to everyone instantly would flood the system.

Key ideas

01
Scale: 100M DAU, 500M tweets/day → ~5.8k writes/s + 10x reads; timelines are 90% of the read load.
02
Components: API services, tweet service, feed/timeline service, user graph service, search (ES), media (object store+CDN).
03
The BIG decision — feed construction: PULL (read latest from followings on request: simple, but N queries per view) vs PUSH (fan-out tweet to followers' inbox at write time: fast reads, write amplification) vs HYBRID (push to active followers, pull for celebrities' fans).
04
Data: tweets (id, authorId, text, ts), follows (graph), timeline_inbox (denormalized), user profiles.
05
Fan-out math: a tweet by a 10M-follower celeb → 10M inbox writes → queue + batching + shard by userId.
06
Caching: timeline cache in Redis/whatever (LRU, hot set) → p99 < 100ms requirement.
07
Consistency: eventual for feeds; tweets appear slightly delayed on small followers — acceptable and said out loud.
08
Extras: likes/retweet counters (Redis INCR + async commit), trends (streaming aggregation), search queue → ES.

Java / Spring map

→
feed-service with a per-user inbox row; Kafka for fan-out jobs; Redis for counters; ES for search.

Code & diagrams

FeedDecision.mdmarkdown

The push/pull decision — the sentence that wins this interview.

PULL (read-time):
  timeline = SELECT * FROM tweets JOIN follow ON followee=… LIMIT 10
  + Simple, no write amplification
  − N sub-queries per view, slow for heavy users

PUSH (write-time fan-out):
  on tweet → insert into inbox_user_{followerId} for each follower
  + reads are ONE row lookup — p99 < 100ms
  − a 10M-follower account causes 10M inbox writes

HYBRID (the production answer):
  fan-out push for normal users
  pull-merge the 'celebrity' feeds on read
  → bounded write amplification + bounded read cost

Explain without notes

Walk a celebrity tweet through the hybrid: who gets pushed, who pulls, how big is the remaining write burst?

Practice

Size fan-out: 500M tweets/day, avg 150 followers → inbox writes/day and the queue's role.

Trade-offs

↔
Push = read speed for everyone + write amplification; hybrid trades a 'celebrity list' for sanity.

Run it in production

You've designed it. Now build, operate, and break the same idea hands-on in the DevOps courses:

Stateful · Redis & caching patterns

Cache-aside with a stampede lock, TTLs, rate limiting, and sessions in Redis.

Stateful · Kafka fundamentals

Run Kafka, create partitioned topics, and see per-key ordering on disk.

Completion checklist

I can present the pull/push/hybrid decision with numbers and defend it.

Back to phase

Command Palette