Topic S.4
From 1 User to 1 Million: The Scaling Story
In one line
The single most useful story in system design: start with one server, then add each building block only when a real problem appears. Every later phase is a zoom-in on one step of this journey.
Think of it like this
A food stall that becomes a restaurant chain. First one cook does everything. Then you hire a separate cashier (split roles), more cooks (horizontal scaling), a host who seats people evenly (load balancer), a prep station with ready ingredients (cache), a central warehouse (database with replicas), and a delivery service for takeaway orders (queue + workers). You only add each piece when the queue out of the door demands it.
Key ideas
- 01
STEP 1, ONE SERVER: app and database on one machine. Fine for a prototype. Problem: one machine's CPU and memory are the limit, and if it dies, everything is down (a SINGLE POINT OF FAILURE).
- 02
STEP 2, SEPARATE THE DATABASE: app servers and database on different machines, so each can be sized independently. STEP 3, MORE APP SERVERS + A LOAD BALANCER: run several identical app servers and put a load balancer in front to spread requests (HORIZONTAL SCALING). This needs STATELESS servers: no user data kept in a server's memory (Phase 9).
- 03
STEP 4, CACHE: put hot data (product pages, sessions) in Redis so most reads never touch the database (Phase 10). STEP 5, CDN: serve images, videos, and static files from servers near users (Phase 9). STEP 6, DATABASE REPLICAS: copies of the database handle reads, and one primary handles writes (Phase 8).
- 04
STEP 7, ASYNC WORK WITH QUEUES: slow tasks (emails, image resizing, analytics) go onto a queue and are handled by background workers, so users don't wait (Phase 10). STEP 8, SHARDING: when one database can't hold all writes or data, split it across several by a key such as user ID (Phase 8). STEP 9, MULTIPLE DATA CENTRES/REGIONS: survive a whole data centre failing and serve global users faster (Phase 14).
- 05
The key lesson: every step fixes one bottleneck and ADDS complexity (more parts to run, new failure modes, consistency questions). Good designers add them in response to measured needs, not all at once. That judgement is what interviews test.
Code & diagrams
Where a typical app ends up; each box is a phase of this course.
Explain without notes
Why must app servers be stateless before you can add a load balancer?
Why not build the full architecture from day one?
Practice
Your single-server blog gets featured on a popular site and slows to a crawl. Which step(s) do you take first and why?
Trade-offs
- ↔
Every scaling step trades simplicity for capacity or resilience. Scale in response to measured bottlenecks, and prefer the cheapest fix (cache, index, bigger machine) before the most complex (sharding, multi-region).
Run it in production
You've designed it. Now build, operate, and break the same idea hands-on in the DevOps courses:
Completion checklist
I can tell the scaling story in order and say which problem each step solves
I understand why statelessness enables horizontal scaling