Topic 1.4
Timeouts and Retries
In one line
Give every request a time limit, retry only the failures that can succeed next time, wait longer between each try with a little randomness, stop after a few attempts, and never repeat a request that has already had a side effect unless it carries an idempotency key.
Think of it like this
Phoning a friend who does not pick up. You do not redial every second for an hour. You wait, try again a little later, then later still, and after a few tries you stop and send a message instead. And if you were calling to ask them to order a pizza, you first make sure the first call did not already do it before you ask again. Retries are the same manners, written in code: patience that grows, a limit, and care with things that cannot be undone.
Words you'll meet
New words in this topic, in plain English. Come back here whenever one feels fuzzy.
- Timeout
- A time limit after which a request is cancelled and treated as failed.
- AbortSignal
- An object you pass to
fetch. When it fires, the request is cancelled. - TimeoutError / AbortError
- The names of the errors for "the time limit passed" and "someone aborted it". Check
e.nameto tell them apart. - Retryable
- A failure that can succeed on a second try, such as a timeout or a 503.
- Exponential backoff
- Waiting longer after each failed try, usually doubling the wait each time.
- Jitter
- A random amount added to or used in the wait so many clients do not retry at the same instant.
- Idempotent
- Safe to do twice with the same effect as doing it once. Reading a page is. Placing an order is not, unless made so.
- Idempotency key
- A unique id sent with a request so the server can recognise a repeat and not do the work again.
Step by step
01Put a time limit on the request
The first live example sends the same request four times to a slow "model" whose first token takes 400 ms. Read the four lines: a 2-second limit works; a 100 ms limit fails with a TimeoutError; a Stop press fails with an AbortError; and a combined signal reports which of the two ended it, in signal.reason.
try {
const res = await fetch("/api/chat", { method: "POST", body, signal: AbortSignal.timeout(15_000) });
// …
} catch (e) {
if (e.name === "TimeoutError") showError("Mitra is taking too long. Please try again.");
else if (e.name === "AbortError") { /* the visitor pressed Stop: show nothing */ }
else showError("Cannot reach Mitra. Check your connection.");
}02Retry only what can succeed
The second example wraps the request in withRetry. It is built from three small decisions: is this failure in the retryable set, how long to wait (the larger of Retry-After and the jittered backoff), and have we hit the limit. Because the sleep function is passed in, the example pretends to wait, and the output is instant and repeatable.
Read the four scenarios. Two 503s then success: waits of 250 ms and 900 ms. A 429 with retry-after: 3: it waits the full 3000 ms. A 401: gives up at once, no wait. A 503 forever: three waits and then it stops with the last status.
03Where the wait goes in the user's eyes
While retrying, the screen should not look frozen and should not hide the problem for long. A short line such as "Trying again (attempt 2)…" tells the visitor it is working. After the last failure, show the error and a Try again button; a manual retry resets the count, so the visitor decides whether to spend more time on it. Mitra v1 (Topic 1.6) does exactly this.
04When a retry would do the work twice
The third example places an order through a slow server. The server records the order immediately, then takes 200 ms to reply. The client gives up at 100 ms and retries. Without a key the server creates ORD-1045 and then ORD-1046: the customer is charged twice. With an Idempotency-Key header the second request returns the first order, and only ORD-1045 exists.
The first attempt timed out in both runs, yet the work was done. A timeout never means "it did not happen".
05Cap everything
Three numbers keep retries honest: the number of tries, the cap on a single wait, and the total time you are willing to spend. A visitor-facing request that has not worked after about 15 to 30 seconds in total should become an error with a button. Background work can be more patient.
Try it yourself
- 1
Change the numbers
In the second example change
retries = 3to1and run. Then change the random list inseeded()to[0.99].With one retry the 503-forever case stops after two attempts. With a random value near 1 the waits are near the maximum of each step (495, 990 and 1980 ms instead of 250, 900 and 600): the jitter is the difference between waiting nearly nothing and waiting the whole backoff.
- 2
Retry a network failure
At the end of the second example, add
await withRetry(() => fetch("/api/nowhere", { method: "POST" }), { sleep: pretendSleep, random: seeded() });and delete the fourscenariolines above it.Console: three lines "attempt N failed: Failed to fetch → waiting …", then
Uncaught TypeError: Failed to fetch. Network failures are retried up to the limit and then thrown to the caller, whosecatchmust show the offline message. - 3
Reuse the key for a second click
In the third example call
placeOrder("order-attempt-7f3a")twice in a row with the same key in the keyed server.The second call returns the same order. That is correct if it is a retry, and wrong if the customer really clicked twice on purpose to buy two. This is why a key belongs to one *intent* (one click), not to one product.
Code & diagrams
The server here is a stand-in that is slow on purpose (in a real app, think of a busy model). The message texts come from the browser and can differ between browsers, so decide by `e.name`, not by the message.
Preview
Console
Nothing logged yet.Expected console output
patient (2 s limit) → answered 200
impatient (100 ms) → TimeoutError: signal timed out
user pressed Stop → AbortError: signal is aborted without reason
Stop or 2 s limit → AbortError: signal is aborted without reason
why it ended: AbortErrorThe waits are pretended (the sleep does nothing) and the random numbers are a fixed list, so you can read the policy without waiting. In your app, `sleep` is `(ms) => new Promise((r) => setTimeout(r, ms))` and `random` is `Math.random`. The cap and the base here are examples, not recommendations: tune them from your own measurements.
Preview
Console
Nothing logged yet.Expected console output
--- two 503s, then success
attempt 1 got 503 → waiting 250 ms
attempt 2 got 503 → waiting 900 ms
attempt 3 succeeded
final status: 200
--- 429 with retry-after: 3
attempt 1 got 429 (retry-after 3000 ms) → waiting 3000 ms
attempt 2 succeeded
final status: 200
--- 401: retrying cannot help
attempt 1 gave up on status 401
final status: 401
--- 503 forever: the retry limit stops it
attempt 1 got 503 → waiting 250 ms
attempt 2 got 503 → waiting 900 ms
attempt 3 got 503 → waiting 600 ms
attempt 4 gave up on status 503
final status: 503Not a real shop: an order is just an array here. The key (`order-attempt-7f3a`) is created once for the visitor's click and reused by every retry of that click. A new click makes a new key. Real servers keep keys for a limited time.
Preview
Console
Nothing logged yet.Expected console output
attempt 1 → TimeoutError
attempt 2 → TimeoutError
without a key, orders on the server: [ 'ORD-1045', 'ORD-1046' ]
attempt 1 → TimeoutError
with an idempotency key, orders on the server: [ 'ORD-1045' ]Break it on purpose
Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.
Break #1
Retry everything, including a bad key
In the second example change if (!RETRYABLE.has(res.status) || attempt >= retries) { to if (attempt >= retries) {.
Break #2
A time limit shorter than the model needs
In the first example change AbortSignal.timeout(2000) on the patient line to AbortSignal.timeout(100).
Myth vs fact
Myth
A timed-out request did not happen.
Fact
The server may have finished the work after you stopped listening. For anything with a side effect, retry only with an idempotency key.
Myth
More retries make the app more reliable.
Fact
Past two or three, retries mostly add waiting and load. A struggling server gets worse when every client keeps knocking.
Myth
Doubling the wait is enough; jitter is optional.
Fact
Without jitter, clients that failed together retry together, in waves. Randomness spreads them out.
Myth
fetch times out by itself.
Fact
The browser eventually gives up on a dead connection after a long time that you do not control. Set your own limit with AbortSignal.timeout.
Pro corner
Extra depth for experienced readers. New to this? Skip it for now and come back later.
- ▸
Retrying inside the browser multiplies load when many tabs and visitors retry at once. Put the main retry on the server (your route calling the provider), where you can also share the knowledge that the provider is struggling, and keep the browser's retries few and visible.
- ▸
Distinguish *connect* problems from *slow* ones. A request that never connects can be retried quickly; one that connected and then went silent may be mid-computation, and its work may be billed. Both end as a timeout in the browser, which is another reason to log the time and stage on the server.
- ▸
Do not retry silently forever. Each automatic retry should be visible to the visitor, at least as a short status line, and counted in your logs. A feature that "works" only because it quietly retries five times is a feature that is already failing.
- ▸
A page that was in the background may have its timers delayed. A visitor who switches tabs and returns can see a timeout fire the moment they come back. Treat a timeout as information, not as proof of a broken service.
Remember this
- 1
The problem this solves: A model call can hang for a minute or fail for a moment. With no time limit, the visitor stares at a spinner forever. With no retry, a one-second hiccup becomes a visible error. With careless retries, you hit a struggling server harder, burn your quota, or place the same order twice. This topic gives you the three rules that avoid all three: a time limit, a smart retry, and a safety check for requests that change something.
- 2
**Timeouts with
AbortSignal.timeout(ms).**fetchhas no time limit of its own: it waits as long as the connection stays open. Passsignal: AbortSignal.timeout(10_000)and the browser cancels the request after that many milliseconds, rejecting the promise with aDOMExceptionnamedTimeoutError. A visitor pressing Stop produces a different one,AbortError. Telling these apart matters, because a timeout deserves "Mitra is taking too long" and a stop deserves silence.AbortSignal.any([a, b])combines several signals so the request ends on whichever fires first. Current browsers support both; for an old one, asetTimeoutthat callscontroller.abort()does the same job. - 3
Choosing the limit. Too short, and you cancel requests that would have worked (the first live example does exactly that by accident). Too long, and the visitor waits for nothing. For a non-streamed answer the whole answer must be generated before the first byte arrives, so allow for the longest answer you will accept. Measure real response times (a high percentile such as the 95th) and add margin. Streaming (Phase 2) changes the question: you limit the wait for the first token and, separately, the silence between tokens (Topic 3.5).
- 4
Which failures to retry. (Recall from Topic 1.3: 4xx means the request was wrong, 5xx means the server failed, and 429 carries a wait time.) Retry what can succeed on a second try: a timeout, a dropped connection,
408,429(after waiting), and500,502,503,504. Do not retry400,401,403,404or413: the same request will fail the same way, and retrying only wastes time and quota. Keep the list as an explicit set in one place so it is easy to review. - 5
Exponential backoff. After a failure, wait before trying again, and wait longer each time: for example 0.5 s, then 1 s, then 2 s, doubling up to a cap. The wait is
base × 2^attempt. The reason is practical: a server that is overloaded needs breathing space, and if every client retries at once it is knocked over again. If the response hasRetry-After, wait at least that long. - 6
Jitter. If a thousand visitors all failed at the same moment, a plain backoff makes them all retry at the same moments again, in waves. Adding jitter means using a random wait between 0 and the backoff value ("full jitter"), so the retries spread out. The live example uses a seeded stand-in for the random number so its output is repeatable; in your app use
Math.random(). - 7
Limits, and side effects. Always stop: two or three retries is typical for a visitor-facing request. After that, show the error with a Try again button, which hands control back to the person. And be careful with requests that *change something*: placing an order, sending an email, charging a card. If such a request times out, you do not know whether the server did the work. Retrying may do it twice. The fix is an idempotency key: a unique id you send with the request (
Idempotency-Keyheader is the common name), which the server remembers, so a repeat with the same key returns the first result instead of repeating the work. Asking a model a question has no side effect except cost, so it is safe to retry; the tools you give it in Phase 8 may not be.
Explain it without notes
How do you put a time limit on a fetch, and how do you tell a timeout from a visitor pressing Stop?
Which failures should be retried and which not, and why?
What are exponential backoff and jitter, and what problem does each solve?
Why is retrying a timed-out POST that places an order dangerous, and how do you make it safe?
Practice
Write fetchWithTimeout(url, options, ms) that adds a timeout, and combines it with an optional options.signal so a Stop button still works. It should reject with an error whose name is TimeoutError or AbortError accordingly.
Change backoff to "equal jitter": half of the backoff is fixed and half is random. Why might some teams prefer it?
Trade-offs
- ↔
Automatic retries hide short failures from the visitor but delay the error message when the failure is real. Two quick retries then a visible error with a button is a good balance for a chat.
- ↔
Idempotency keys make retries safe but need server storage and an expiry policy. For read-only requests, such as asking a model a question, they are not needed.
Done when you can
I can add a timeout to a fetch and tell a timeout from a stop.
I can list which failures to retry and which to give up on at once.
I can write exponential backoff with jitter, and honour
Retry-After.I can explain why a timed-out order must not be retried blindly, and how a key fixes it.
I can set a retry limit and show a Try again button after it.