PHASE 5Intermediate ~20 min· topic 5 of 8

Topic 5.5

Cancelling a Stream

In one line

When the user presses Stop or leaves the page, close the stream so the provider can stop writing and your program frees its resources. Hiding the text while you keep reading does not stop anything.

Think of it like this

A tap filling a bath. If you decide you have enough water, turning your head away does nothing: the water keeps running and spills. You have to turn the tap off. A Stop button that only hides the text leaves the tap running.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Cancel (abort): Telling a running operation to stop before it finishes.
AbortController: A JavaScript object that creates a signal you pass to fetch, and whose abort() cancels the request.
Context manager: Python's with block, which runs cleanup code when the block ends, even after a break or error.
Resource leak: Connections, memory or threads that were opened and never released.
finally: A block that runs at the end whether the code succeeded, failed or was stopped.
Partial reply: The text received before the stream ended early. It is real output and can be kept.

Step by step

01Step 1: the tap analogy in code

The example has a fake provider that counts how many words it "generated" and how many streams are still open. It has 20 words to write. We can then measure what each way of stopping costs.

02Step 2: read to the end

Scenario one reads everything: 20 words are generated and no stream stays open. The generator's finally block runs at the end and decrements the open count.

03Step 3: stop and close

Scenario two breaks out of the loop after 5 words and closes the stream. The provider only wrote 5 words, and the finally block (the cleanup) still runs because close() forces it.

Step 3: stop and closediagram

Rendering diagram…

04Step 4: stop and forget to close

Scenario three breaks out but never closes. The words were not generated beyond 5 in this fake because nobody asked for more, but the stream stays open: the count shows 1. With a real connection this is a socket held open and a provider that may keep writing until it fails to send.

05Step 5: wire it to the UI

A Stop button sets a flag (or calls abort()). The reading loop checks the flag after each piece, or the abort makes the pending read fail. Either way the loop leaves and the finally runs. The real-code tabs show this with AbortController, a with block and try-with-resources.

06Step 6: keep and label the partial reply

Store the text that was shown, with a flag such as stopped: true. The next request can include it as an assistant message so the conversation stays coherent.

Try it yourself

1
Change when to stop
Change the 5 to 10 in both stopping scenarios. Predict generated and saved before running. How many words does stopping save at 1, and at the last word?
2
Find the cleanup
Point at the line that makes open streams go down in each language. Why does it run in scenario one without anyone calling close()?
3
Check twice
Call close() a second time in the first scenario (after reading everything). Nothing should break: cleanup code must be safe to run twice, because a stream can be closed by two different paths.

Code & diagrams

Stop early, close the stream, count what was saved New tab

A fake provider counts the words it writes and the streams left open. The generator (Python, JavaScript) or the iterator (Java) has a cleanup that runs when it ends or is closed.

Expected output

read to the end : shown 20 words, generated 20, saved 0, open streams 0
Stop after 5    : shown 5 words, generated 5, saved 15, open streams 0
Stop, no close  : shown 5 words, generated 5, saved 15, open streams 1

Cancelling a real streampython

Not run here: they need a network. The Python SDK, browser fetch with an AbortController, and the JDK HttpClient.

# pip install openai
import os

from openai import OpenAI

client = OpenAI(base_url="https://api.groq.com/openai/v1", api_key=os.environ["GROQ_API_KEY"])
stop_requested = False  # set to True by your UI (a Stop button, a closed tab)

# "with" closes the HTTP response when the block ends, also when you break out early.
with client.chat.completions.create(
    model="qwen/qwen3.8-27b",
    messages=[{"role": "user", "content": "Write a long story about a lighthouse."}],
    stream=True,
) as stream:
    for chunk in stream:
        if chunk.choices and chunk.choices[0].delta.content:
            print(chunk.choices[0].delta.content, end="", flush=True)
        if stop_requested:
            break  # leaving the block ends the request, and the provider can stop writing

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Hide the text but keep reading

In the first language you use, replace the break in read with pass (JavaScript: remove the break; Java: remove the break). The Stop now only exists in the code, not in the loop.

terminal

$ Run

── what you'll see ──

read to the end : shown 20 words, generated 20, saved 0, open streams 0

Stop after 5 : shown 20 words, generated 20, saved 0, open streams 0

Stop, no close : shown 20 words, generated 20, saved 0, open streams 0

Myth vs fact

Myth

Stopping a stream means I pay nothing for the rest.

Fact

Tokens already generated are normally billed, including some in flight when the connection closed. You save the rest of the reply, not the part already written.

Myth

Closing the browser tab cancels the model call.

Fact

Only if your server notices and closes the provider request. Otherwise the server keeps reading to the end.

Myth

An AbortError means something went wrong.

Fact

It is the normal result of the user pressing Stop. Catch it, do not show an error, and run your cleanup.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
Add max_tokens as the backstop (and a timeout, Topic 2.7): cancellation depends on someone noticing, a token cap does not. Cap per request and use a time limit on the relay as well.
▸
Track the reason a stream ended (finished, user stopped, provider error, timeout) as a metric. A rising share of "user stopped" can mean replies are too long or too slow; a rising share of provider errors needs a different fix (Topic 6.8).
▸
When the user sends a new message while an answer is still streaming, cancel the old stream first. Otherwise two streams write into the same chat, and you pay for both.
▸
Test cancellation with the fake provider: assert that after Stop, generated is smaller than the full length and open_streams is 0. Both numbers are checks the tests in this course's exercises use.

Remember this

1
Why this exists. A long generation can take half a minute and cost real money. Users press Stop, close the tab, send a new message, or navigate away. If your code keeps reading, you keep an open connection, keep the provider writing tokens nobody sees, and may be billed for them. Cancellation is how a program says "I have enough".
2
What stopping actually does. Closing the connection tells the provider the reader is gone, and the provider normally stops generating soon after. Tokens already generated, including a few that were in flight, are normally billed. The saving is the rest of the reply: in the example, stopping after 5 of 20 words leaves 15 never generated. Provider behaviour differs in detail: read your provider's documentation and measure with usage numbers.
3
How, in each language. In JavaScript, an AbortController: pass its signal to fetch and call controller.abort(); the pending read() rejects with an AbortError, which is not a real error to report. In Python, use the SDK stream as a context manager (with ... as stream) or call stream.close(); with httpx, leaving the with httpx.stream(...) block closes the response. In Java, close the body InputStream (try-with-resources); to interrupt a blocked read, close it from another thread.
4
**Clean up in finally.** Cancelling is one of the ways a stream ends, next to finishing and failing. Put the cleanup (close the connection, re-enable the Send button, record the outcome) in a finally block or a try-with-resources, so it runs in all three cases. The example's fake provider counts open streams: the third scenario forgets to close and shows one stream left open, which at scale becomes a leak of connections.
5
Keep what the user saw. After Stop, the text on screen is a valid partial reply. Keep it in the conversation history so the next message has context, mark it as stopped (some UIs show "Stopped"), and do not treat it as a failure. If you save messages, store the partial text and the reason it ended.
6
Cancel on the server, too. If your own server relays the stream (Topic 5.4), cancellation must travel the whole chain: browser to your server to the provider. The browser's abort closes its connection, your server notices (a cancel hook or abort signal) and closes the provider request. A break anywhere in the chain leaves the provider writing for nobody.

Explain it without notes

What does cancelling a stream save, and what does it not save?

How do you cancel a streaming request in JavaScript, Python and Java?

Why does cleanup belong in a finally block or try-with-resources?

How must cancellation travel through a server that relays the stream?

Practice

Write read_until(stream, stop_word) that reads pieces from a generator, joins them, and stops (closing the generator) as soon as a piece equals stop_word. Return the text read before the stop word.

Write a generator numbered(limit, log) that yields 1, 2, 3 up to limit and appends "closed" to the list log in a finally block. Show that breaking out and calling close() appends "closed".

Trade-offs

↔
Cancelling saves money and capacity, but a half-finished answer may be worse than none for some tasks (such as tool calls), so decide per feature whether partial output is kept.
↔
Aggressive cancel-on-new-message keeps chats tidy, and loses a reply the user might still have wanted.
↔
Closing connections early costs a new handshake for the next request; connection pooling mostly hides that.

Done when you can

Done when you can cancel a streaming request in one language and handle the abort without an error message.
Done when you can show with a counter that stopping early generates fewer tokens.
Done when you can put cleanup in finally or try-with-resources.
Done when you can name every hop that must pass a cancel along.

Command Palette

Words you'll meet

Step by step

01Step 1: the tap analogy in code

02Step 2: read to the end

03Step 3: stop and close

04Step 4: stop and forget to close

05Step 5: wire it to the UI

06Step 6: keep and label the partial reply

Try it yourself

Change when to stop

Find the cleanup

Check twice

Code & diagrams

Break it on purpose

Hide the text but keep reading

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can