The herd: retry now, back off, back off with jitter
A simulated server and 100 clients on a simulated clock. The capacity, meltdown line and delays are made-up numbers chosen to show the shapes.
Many model errors (429, 503, timeouts) go away if you try again later, but a naive retry loop can turn a small hiccup into an outage. This topic builds the full retry from first principles: why immediate retries pile up, exponential backoff, random jitter, Retry-After, retrying only what is retryable, and idempotency.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 6 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write is_retryable(status) returning True for 0 (timeout), 408, 429 and 500 to 599, else False. Print it for 0, 200, 400, 429, 503.
Write full_jitter_ms(attempt, base_ms, cap_ms, rand) where rand is a number from 0 up to but not including 1. Return int(rand * (ceiling + 1)) with ceiling = min(cap_ms, base_ms * 2 ** (attempt - 1)). Print the result for attempts 1 to 4 with base 500, cap 8000 and rand = 0.5.
Explain it without notes
Explain the thundering herd and why retrying immediately makes an overload worse.
Why is exponential backoff without jitter not enough, and what does full jitter change?
Which errors should you retry and which not? What is the role of Retry-After?
What is an idempotency key and when do you need one?
Compare what tenacity, p-retry and resilience4j give you with what you built by hand.
Expected output
retry in the next slot
first slots (slot:arrivals): 0:100 1:100 2:100 3:100 4:100 5:100
busiest slot 100, succeeded 0, gave up 100
exponential backoff, no jitter
first slots (slot:arrivals): 0:100 2:100 6:100 14:100 30:100 62:100
busiest slot 100, succeeded 0, gave up 100
exponential backoff with full jitter
first slots (slot:arrivals): 0:100 1:100 2:44 3:28 4:27 5:6 6:3 8:4
busiest slot 100, succeeded 100, gave up 0