PHASE 3Intermediate ~29 min· topic 5 of 7

Topic 3.5

Slow and Stuck Requests

In one line

A slow request will finish; a stuck one never will, and from the browser they look the same until you add a clock. Use messages that grow more honest as the wait grows, and a watchdog that gives up when a stream goes quiet.

Think of it like this

A lift that is slow and a lift that is stuck both leave you standing in front of closed doors. After ten seconds you assume it is slow. After two minutes you take the stairs. You need a rule for when to stop waiting, and the page needs to say what it is doing while you wait.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Time to first token: How long it takes from sending the request to the first piece of the answer arriving.
Stalled stream: A response that is still open but has stopped sending data.
Watchdog: A timer that is reset by activity and that raises an alarm (here, aborts the request) if it ever runs out.
Idle timeout: How long the stream may stay silent before you give up on it.
First-token timeout: How long you will wait for the first piece of the answer. It is usually longer than the idle timeout.
Total timeout: A limit on the whole request, however healthy. It also ends long answers.
Keep-alive: A tiny message the server sends only to show that the connection is still open.
role="status": An attribute that makes screen readers announce a changed message politely, without interrupting.

Step by step

01A wait that explains itself

The first example makes the model wait 0.9 seconds before the first token. Three timers are set when the request starts, one per message. The first token clears them all. If the answer arrived at once, the user would never see the second and third messages.

The messages are data (an array of { after, text }), not scattered if statements, so a designer can change the wording or the thresholds in one place.

waiting messageswhole filejs

const WAITING = [
  { after: 0, text: "Mitra is thinking…" },
  { after: 250, text: "Still working on it…" },
  { after: 600, text: "This is taking longer than usual. You can wait or press Stop." },
];
const timers = WAITING.map((step) => setTimeout(() => setMessage(step.text), step.after));
// first token arrives: timers.forEach(clearTimeout)

02A stream that goes quiet

In the second example the server sends "Orders ship in " and then waits forever, with the connection still open. The headers have arrived, so the request looks healthy. A watchdog restarts a 300 ms timer after every chunk. When it finally runs out, it aborts with a reason, and the read rejects with that reason.

Because the reason is your own Error("stalled"), the catch can tell a stall from a user's Stop: a Stop is an AbortError, a stall is the error you created.

A stream that goes quietdiagram

Rendering diagram…

03Two clocks

The third example runs the same watcher against four servers. A healthy one completes. A silent one never says anything, so the first-token deadline fires. One that sends only : ping comments looks alive but never produces a token, so the first-token deadline still fires. One that stalls after two tokens triggers the idle deadline.

The idle timer is armed only once a real token has arrived. Before that, silence is the first-token deadline's job.

04Pick the numbers from data

The numbers in the examples (a few hundred milliseconds) exist only so you can watch them run. In a real app, measure time to first token and the longest healthy gap between tokens at your busiest time, and set the deadlines well above them. A deadline that is too tight kills healthy answers, which the fourth tryIt shows.

Try it yourself

1
Make the answer fast
In the first example change firstDelayMs: 900 to 100 and run it.
Only the first message ever appears, and it is cleared almost at once. The delayed messages cost nothing when the answer is quick, which is why this design is safe to leave in.
2
Press Stop while it waits
In the first example, add setTimeout(() => document.getElementById("stop").click(), 500); at the end.
The request is aborted during the wait. The status becomes stopped (the status line is only drawn while waiting, so it vanishes), and the waiting messages disappear. The third message, already scheduled for 600 ms, is cleared by the finally block instead of firing on a stopped request, so it never prints.
3
Let the pings count
In the third example, change if (tokens > 0) armIdle(); to armIdle(); (so every chunk, even a ping, arms the idle timer) and change firstTokenMs: 400 to firstTokenMs: 1000. Run it again.
The pings line now reads gave up: stalled after 0 tokens. The pings stop after 400 ms, the idle timer fires 250 ms later, and it blames a stall in a stream that never produced a word. With the original condition the pings line reads no first token, which is the honest description. This is why the idle timer is armed only after the first token.
4
Set a deadline that is too tight
In the third example, change idleMs: 250 to idleMs: 20 and run it.
Even the healthy stream is aborted, because its tokens come 30 ms apart. A deadline must sit well above the largest normal gap.

Code & diagrams

A first token that takes a long time

The first token arrives after about a second here, so all three messages appear in turn. Make firstDelayMs smaller and watch the later messages disappear: a fast answer never shows them.

Preview

Press Run to see the page.

Console

Nothing logged yet.

Expected console output

waiting message: Mitra is thinking…
waiting message: Still working on it…
waiting message: This is taking longer than usual. You can wait or press Stop.
answer: Orders ship in 1 to 2 working days.

A watchdog for a stream that goes silent

The status was 200 and three tokens arrived, so nothing looked wrong. Without the watchdog the last line would never print. The (server) line shows that aborting also closed the connection on the server's side.

Preview

Press Run to see the page.

Console

Nothing logged yet.

Expected console output

headers arrived: HTTP 200 (the request looks healthy)
(server) the browser closed the connection
kept: "Orders ship in " | outcome: stalled

Two deadlines: no first token, and gone quiet

The pings arrive, so the connection is clearly open, and still the watcher gives up: only a real token counts as progress for the first-token deadline. (Here each value is one chunk, which the stand-in guarantees. A real parser would buffer lines as in Topic 2.4.)

Preview

Press Run to see the page.

Console

Nothing logged yet.

Expected console output

healthy  -> complete, 2 tokens
silent   -> gave up: no first token
pings    -> gave up: no first token
stalls   -> gave up: stalled after 2 tokens

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

A watchdog that is too strict

In the third example, change const limits = { firstTokenMs: 400, idleMs: 250 }; to { firstTokenMs: 400, idleMs: 20 }.

terminal

$ The console, the healthy line

── what you'll see ──

healthy -> gave up: stalled after 1 token

silent -> gave up: no first token

pings -> gave up: no first token

stalls -> gave up: stalled after 1 token

Myth vs fact

Myth

A fetch will time out by itself if the server hangs.

Fact

A stream can stay open and silent for a very long time. The browser has no idle timeout for an open response body you are reading. You must add one.

Myth

AbortSignal.timeout() is an idle timeout.

Fact

It limits the total time of the request, including a healthy long answer. An idle check needs a timer that restarts on each chunk.

Myth

A keep-alive ping means the model is working.

Fact

It only shows the connection is open. Progress means real tokens (or reasoning chunks) arriving.

Myth

A progress bar is the best answer to a slow wait.

Fact

You do not know how long a generation will take. An honest changing message is better than a bar that stalls.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
Put the watchdog in one helper that wraps the reader and yields chunks, so every streaming call in the app gets it for free. The reason it aborts with (stalled, no first token) can then be turned into a message by one function, shared with Stop and rate limits.
▸
Models that reason before they answer can be silent for a long time at the text level while streaming many small reasoning chunks. If you ignore those chunks the first-token deadline will fire on healthy requests. Count any model output as progress and show the reasoning state in the interface.
▸
Watch for timer throttling: browsers slow timers in background tabs (to about once a second, and less often after several minutes), and a sleeping laptop fires everything late on wake. If you see aborts clustered right after users return to a tab, reset your timers on the visibilitychange event.
▸
Log the stall and first-token timeouts by cause with the size of the request. A rise in "no first token" usually points at the provider or a cold start; a rise in "stalled" usually points at a proxy or network in the middle.

Remember this

1
The problem this solves: A normal fetch ends by itself, successfully or with an error, and a browser will give up on a connection that cannot be made. But a streaming response can stay open and silent for as long as the server (or any proxy in between) lets it. reader.read() then waits forever, the page shows "Mitra is thinking…" forever, and the Stop button is the only way out. This topic adds the clock that the browser does not.
2
Slow is normal; it has causes. Time to first token grows when the provider is busy and your request waits in a queue, when your server function starts from cold, when the prompt is long, or when the model spends tokens "thinking" before it writes anything. Mitra cannot fix any of these, but it can be honest about them. Replace a single "Thinking…" with messages that change as the wait grows: *Mitra is thinking…*, then *Still working on it…*, then *This is taking longer than usual. You can wait or press Stop.* Do not fake a progress bar: you do not know how long it will take, and a bar that stalls at 90 percent is worse than no bar.
3
Stuck is a different problem. A stream that has started and then goes quiet (the model hung, a proxy lost the upstream connection, a mobile network stalled) never reports an error. The fix is a watchdog: a timer that is restarted every time bytes arrive, and that, if it ever fires, aborts the request with controller.abort(new Error("stalled")). The read then rejects with your reason, and the code keeps the partial text and offers Try again (Topic 3.3).
4
Two deadlines, not one. Before the first token the model may legitimately need a long time; once text is flowing, a few seconds of silence is suspicious. So use a first-token deadline (long) and an idle deadline (shorter) that only starts after the first token. And keep a separate total limit if you want one: AbortSignal.timeout(ms) limits the whole request, which also ends a long but healthy answer, so it is a poor stand-in for an idle check. Choose the numbers from your own measurements: look at how long real first tokens take at your busiest hour, and set the deadline comfortably above that.
5
Keep-alives and reasoning change what "alive" means. A server may send : ping comment lines every few seconds so a proxy does not close an idle connection. They prove the connection is open, not that the model is making progress. Count only real data lines as progress for the first-token deadline. Models that "think" may stream reasoning chunks before any answer text: those are progress, and the interface should show them as such (Topic 5.4).
6
Be careful with timers. A background tab throttles timers, and a laptop that wakes from sleep fires every overdue timer at once, which can abort a healthy stream. Check document.visibilityState or restart the idle timer on the first chunk after a gap, and write messages that do not blame the user. Keep screen reader noise low: put the changing text in one role="status" element so it is announced politely, and change it only a few times.

Explain it without notes

Why can a streaming request hang forever in the browser, and what do you add to prevent it?

What is the difference between a first-token timeout and an idle timeout, and why use both?

Why is AbortSignal.timeout() not enough for detecting a stalled stream?

What should the screen say as a wait grows longer?

Practice

Change the first example so that the third message also shows a "Try again" button that aborts the request and starts a new one.

Add a total limit of 2 seconds to the second example using AbortSignal.any so that the request ends if either the watchdog or the total timer fires, and the outcome says which.

Trade-offs

↔
Short deadlines catch stuck streams quickly but risk cutting slow-but-healthy ones; long deadlines are safe for the model and bad for the user. Tune them from your own measurements and make them configuration, not constants buried in code.
↔
Detailed waiting messages reassure users but add moving text that can distract or be over-announced by screen readers. Three stages is usually enough.

Done when you can

I can show waiting messages that grow more honest and clear themselves when the first token arrives.
I can write a watchdog that restarts on each chunk and aborts with a reason.
I can explain why the first-token and idle deadlines are separate.
I can tell a stall from a user's Stop in the catch block.
I can explain why keep-alive pings do not count as progress.

Command Palette

Words you'll meet

Step by step

01A wait that explains itself

02A stream that goes quiet

03Two clocks

04Pick the numbers from data

Try it yourself

Make the answer fast

Press Stop while it waits

Let the pings count

Set a deadline that is too tight

Code & diagrams

Break it on purpose

A watchdog that is too strict

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can