Topic 3.4
Retry and Regenerate
In one line
Retry repeats a request that failed. Regenerate asks again for a better answer to a request that worked. They send different messages, treat the old answer differently, and fail in different ways, and both can hurt a struggling server if they repeat too fast.
Think of it like this
At a restaurant you can say "the kitchen lost my order, please bring it again" (a retry), or "this dish is fine but I'd like another try at it" (a regenerate). The waiter does something different in each case: in the first nothing was served, in the second there is a plate on the table. A chat screen needs both buttons, and each must do what its name promises.
Words you'll meet
New words in this topic, in plain English. Come back here whenever one feels fuzzy.
- Retry
- Sending the same request again because the first attempt produced no usable answer.
- Regenerate
- Asking the model for a new answer to a question that was already answered.
- Retryable
- A failure that may succeed if repeated: no connection, 408, 429, 5xx. A 400 or 401 is not.
- Exponential backoff
- Waiting longer after each failed try, usually doubling the wait each time.
- Jitter
- A random amount added to a wait, so many clients do not all retry at the same moment.
- Retry storm
- A flood of retries from many clients that keeps a struggling server down.
- Idempotency key
- An id for one user action. The server remembers it, so a repeat of the same action returns the first result instead of running again.
- Answer version
- One of several answers to the same question, kept so the user can switch between them.
Step by step
01Retry sends the same thing, regenerate sends less
The first example logs on the server which roles each request carries. The retry after a 503 carries [user], the same as the failed one. The regenerate also carries [user]: the old answer is not included. The last line shows the mistake of leaving the old answer in, which would send [user, assistant].
02Keep every answer
The example collects each answer in a versions list, so the screen can show "2 of 2" instead of throwing the first answer away. Both answers are different because the stand-in model was scripted to word the second one differently, as a real model would.
03Which failures to retry
The second example starts with one line of logic: retryable(status). A 400 or 401 is not retried because the same request fails the same way. A 429 or 503 is. It is a one-line function, and it saves you from both the "retry forever on a bad request" bug and the "never retry anything" bug.
const retryable = (status) => status === 408 || status === 429 || status >= 500;
// no response at all (fetch threw a TypeError): also worth retrying04Storm versus backoff
The server in the second example is down: every request gets 503. A loop with no pause sends more than twenty requests in 0.4 seconds. A loop with exponential backoff and jitter sends four or fewer in the same time, and stops after its limit. That difference, multiplied by every user, decides whether an outage recovers.
05One click, one request
The third example counts how many times the paid model really runs. An unguarded Regenerate double-click runs it twice. A guard (if (busy) return, unlocked in finally) runs it once. An idempotency key makes a repeated request after a lost response return the first answer, so the model runs once even when the browser retries.
Try it yourself
- 1
Retry a 400
In the second example, change the server to answer
status: 400and add aconsole.log("requests:", hits)after each strategy.Both strategies now stop after one request, because
retryable(400)isfalse. Retrying a bad request can never help. - 2
Make the jitter fixed
Change every entry of the
jitterarray to1and run. The result is the same: four or fewer requests.That is fine for one client. The reason for jitter is the many: with identical schedules, every client that failed at the same moment retries at the same moments again, in waves.
- 3
Reuse a key for a different action
In the third example's part 3, use the same key for a second, different request body and read the answer.
The server replays the first answer even though the body is different. A key must identify one action; reusing it hides real work. Real servers also compare the body and reject a key reused with different content.
Code & diagrams
Both requests that carry [user] are identical to the failed one. Nothing in the retry or the regenerate adds the old answer.
Preview
Console
Nothing logged yet.Expected console output
(server) request carries roles: [user]
first try failed: HTTP 503
(server) request carries roles: [user]
retry worked: Returns are free within 7 days.
(server) request carries roles: [user]
regenerated: You have 7 days from delivery to return an unopened item.
versions kept: 2 | same text twice? false
wrong regenerate would send roles: user, assistant (the model would see its own answer as settled)The numbers are comparisons, not timings, so the result is the same on every machine. Real code replaces the fixed jitter list with Math.random().
Preview
Console
Nothing logged yet.Expected console output
retry 400? false | 401? false | 429? true | 503? true
no pause | more than 20 requests in 0.4 s: true | 4 or fewer: false
backoff | more than 20 requests in 0.4 s: false | 4 or fewer: trueThe idempotency check is code on YOUR server in this example (the Map). A real server would keep keys in a database or cache and expire them. Do not count on your model provider for this: the protection that matters (one paid run per user action) belongs in your own route.
Preview
Console
Nothing logged yet.Expected console output
unguarded double click -> model ran 2 times, answers: Answer #1, Answer #2
second click ignored
guarded -> Answer #1
guarded double click -> model ran 1 time
with a key -> Answer #1 then Answer #1 | replayed: true | model ran 1 timeBreak it on purpose
Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.
Break #1
Regenerate with the old answer left in
In the first example, replace the regenerate line with versions.push(await askMitra([...history, { role: "assistant", content: versions[0] }]));.
Myth vs fact
Myth
Retry and regenerate are the same button.
Fact
A retry repeats a request that failed. A regenerate asks again after success. They send different messages and treat the old answer differently.
Myth
Retrying more often makes recovery faster.
Fact
Retrying without a pause makes an outage worse. Backoff with jitter and a limit is what lets a service recover.
Myth
Every error is worth a retry.
Fact
400, 401, 403, 404 and 422 fail the same way every time. Only failures that can fix themselves (no connection, 408, 429, 5xx) are retryable.
Myth
Disabling the button prevents duplicate requests.
Fact
It prevents most. A request that is repeated by the network layer, by a library or by a second tab is not stopped by a disabled button. Guards on the server (keys) are the second line.
Pro corner
Extra depth for experienced readers. New to this? Skip it for now and come back later.
- ▸
"Full jitter" is a named strategy from AWS's article on exponential backoff and jitter: wait a random time between 0 and
min(cap, base * 2^attempt). It spreads clients out better than adding a small random amount on top of a fixed delay. The example uses a fixed list only so the output is repeatable. - ▸
Do not retry while the browser is offline.
navigator.onLineis only a hint (falseis reliable,trueonly means some network is attached), but theonlineevent is a cheap signal to retry once when the connection comes back, instead of burning attempts into a dead network. - ▸
A circuit breaker goes one step further than backoff: after a number of failures it stops sending requests for a while and fails fast with a clear message, then tries one request to see if the service is back. It protects both the server and the user from a long series of timeouts.
- ▸
Do not retry silently when a request costs money and the first attempt may have succeeded on the server (the response was lost on the way back). The provider may have billed for it. If you retry, accept that you may pay twice, or use your own idempotency layer to avoid it.
- ▸
SDKs retry for you. The AI SDK's
maxRetriesdefaults to 2 (so up to 3 attempts, with backoff between them), so a failing call can take noticeably longer than one request, and your own retry loop on top multiplies the attempts. Decide where retries live (Phase 5 shows the SDK side) and switch the others off.
Remember this
- 1
The problem this solves: Both buttons look like "send it again", so they often get the same code. Then a retry after an error sends the wrong messages, a regenerate leaves the old answer in the history, a double click bills twice, and an automatic retry loop hammers a server that is already down. This topic separates the two actions and makes repeating safe.
- 2
Retry means: the last request produced no usable answer (an error, a broken stream), so send the same request again. The messages are identical. The old, failed attempt is forgotten (or its partial text is kept on screen until the new one starts, Topic 3.3). Regenerate means: the request worked, but the user wants another answer. You send the conversation up to and including the user's question, but without the answer being replaced. If you leave the old answer in the list, the model sees it as settled. Providers differ in what a conversation that ends with an assistant message means: some reject it, some treat it as the start of the reply to continue. Either way it is not a fresh answer.
- 3
Replace or keep the old answer? If you replace, the screen is simple but the user loses an answer they might have preferred. If you keep, show "answer 1 of 2" with arrows and let them choose which one stays in the history. That is a small version of branching (Topic 4.6). Whichever you choose, the choice must be visible: never swap the text under the reader's eyes without a sign.
- 4
Not every failure deserves a retry. Retry things that can go away by themselves: no connection at all,
408(request timeout),429(rate limit, after the wait the server asked for, Topic 3.6) and5xx(the server or the model had a problem). Do not retry400(the request itself is wrong),401and403(not allowed),404or422: the same request fails the same way and you only add load. Retrying a deliberate stop is wrong too. - 5
Backoff and jitter. If a retry fails, wait before the next one, and wait longer each time: for example 0.5 s, then 1 s, then 2 s. That is exponential backoff. Add jitter, a random amount, so that a thousand users who failed at the same instant do not all come back at the same instant. Put a limit on it (three tries is typical). If the server sent
Retry-After, obey it instead of your own schedule. Without these three things, an outage turns into a retry storm: every client retries immediately, the extra load keeps the server down, and the outage lasts longer. - 6
Retries multiply across layers. A browser that makes up to 3 tries, calling your server that makes up to 3 tries at the provider, through an SDK that makes up to 3 tries, can send up to 27 requests for one click (3 x 3 x 3). Decide where retrying happens and turn it off elsewhere. Also guard the button: a double click on Regenerate must send one request, not two, and two billed answers. For actions with side effects (Phase 8), send an idempotency key, an id for the user's action that your server remembers, so a repeated request returns the first result instead of doing the work twice.
Explain it without notes
What is the difference between retry and regenerate, and what does each send?
Which HTTP failures should be retried, and which should not?
How do exponential backoff and jitter prevent a retry storm?
How do you make sure one click on Regenerate costs one model call?
Practice
Write askWithRetry(messages, maxRetries) that retries only retryable failures, waits 200 * 2 ** attempt ms between tries, and stops after maxRetries.
Add a "Regenerate" button to the first example's story: keep every answer in a versions array and show "Answer 2 of 2" with Previous and Next buttons. Which index should be sent to the model as history on the next question?
Trade-offs
- ↔
Replacing the old answer on regenerate is the simplest screen but loses an answer the user may have liked. Keeping versions costs a little state and UI and gives the user control.
- ↔
Automatic retries hide short failures from the user but can hide real outages and double the bill. A visible Retry button is slower for the user and more honest. Many products retry once silently for network errors and show a button after that.
Done when you can
I can say what a retry sends and what a regenerate sends.
I can name the failures worth retrying and the ones that are not.
I can write exponential backoff with jitter and a retry limit.
I can explain how retries in several layers multiply.
I can make a double click send one request, and describe what an idempotency key does.