Capacity & Resilience
When load or failure exceeds what the system was built for: surges, cascades, lost availability zones, and a database running out of room at 2 a.m.
These shifts are about systems under stress. The instinctive actions (scale everything, restart everything, fail over everything) are often exactly what turns a partial problem into a total one. The skill is choosing the smallest, most reversible lever that protects the most important user journeys.
- SHIFT 2.1SEV23 decisions
An influencer posted a link
ShopLite p99 3.8s and rising — traffic at 7× normal, unannounced; autoscaling at max tasks
- SHIFT 2.2SEV13 decisions
One slow dependency, everything down
SEV1 — shoplite, search, and account pages all timing out; only payments was 'slightly slow' ten minutes ago
- SHIFT 2.3SEV13 decisions
An availability zone goes dark
SEV1 — ap-south-1a network degradation; 50% of ShopLite requests failing; RDS primary is in 1a
- SHIFT 2.4SEV23 decisions
The database is 95% full at 2 a.m.
PostgresStorageExhaustionPredicted — orders-db free storage 4.1 GB, projected full in ~3 hours