Backpressure, Load Shedding & Admission Control

In one line

When demand exceeds capacity, a system must slow producers down, reject some work gracefully, or collapse. Bounded queues, backpressure, load shedding by priority, and adaptive concurrency limits keep it standing.

0/8 · 0%

Think of it like this

A popular restaurant on a Saturday. A good host stops seating people when the kitchen is full (backpressure), gives a waiting-list time, and serves reservations first (priority). A bad one seats everyone, the kitchen drowns, and nobody gets food: that's what an overloaded service with unbounded queues does.

Key ideas

01
UNBOUNDED QUEUES are the root cause of many outages: requests pile up in thread pools, connection pools, and message buffers; latency grows until clients time out; clients retry, adding more load. Work is done for requests nobody is waiting for any more. Every queue should be BOUNDED with a policy for 'full'.
02
BACKPRESSURE: signal upstream to slow down instead of buffering forever: TCP flow control, reactive streams (request(n) in Reactor/RxJava), Kafka consumers only pulling what they can process, 429 Too Many Requests with Retry-After. It pushes the wait to where it's cheapest.
03
LOAD SHEDDING: when overloaded, reject work early and cheaply (before doing expensive processing) and PRIORITISE: keep checkout and login, shed recommendations and analytics. Drop requests whose DEADLINE has already passed (propagate deadlines through calls, as gRPC does).
04
ADMISSION CONTROL and ADAPTIVE CONCURRENCY: instead of static limits, measure latency and adjust how many requests run concurrently (Netflix's concurrency-limits library, Envoy's adaptive concurrency filter), which follows Little's Law (Stateful Systems course, capacity). Combined with circuit breakers (Phase 11) and rate limiting (Phase 12), this is how large systems degrade gracefully instead of collapsing.

Java / Spring map

→
new ThreadPoolExecutor(n, n, 0, SECONDS, new ArrayBlockingQueue<>(100), new ThreadPoolExecutor.AbortPolicy()) gives a bounded queue that rejects when full instead of growing forever; Resilience4j's Bulkhead caps concurrent calls per dependency.

Code & diagrams

graceful vs collapsing under overloaddiagram

Rendering diagram…

Explain without notes

Why do unbounded queues make overload worse instead of absorbing it?

Practice

During a flash sale, the recommendation service is slow and checkout latency spikes. Design protections so checkout keeps working.

Trade-offs

↔
Rejecting work early disappoints some users but protects the majority; accepting everything feels generous until the whole system fails for everyone.

Run it in production

You've designed it. Now build, operate, and break the same idea hands-on in the DevOps courses:

SRE · Cascading failure

Timeouts, retries, and overload decisions during an incident.

Stateful · Bottlenecks & capacity planning

USE/RED, Little's Law, the utilisation knee, and profilers.

Stateful · Load testing with k6

Open vs closed models and SLO thresholds that gate CI.

Completion checklist

Every queue and pool in my design is bounded
I can explain backpressure, load shedding, and priority-based degradation

Back to phase

Command Palette