PHASE 7Intermediate ~27 min· topic 4 of 12

Topic 7.4

Context Managers and with

In one line

A with statement sets something up, runs your block, and then always cleans up, even if the block fails. Files, locks, database connections and temporary settings all use it, and any class with __enter__ and __exit__ can join in.

Think of it like this

Borrowing a library book with a stamped card. When you walk in, the librarian stamps the card (setup). You read the book. When you leave, the book goes back on the shelf whether you finished it, got bored, or ran out because of a fire alarm (cleanup). A context manager is that arrangement in code: something is set up when you enter a block, and put away when you leave it, no matter how you leave.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Context manager: An object that does setup when you enter a with block and cleanup when you leave it.
with statement: The syntax that uses a context manager: with thing as name: followed by an indented block.
__enter__: The special method called when the with block starts. Its return value goes to the name after as.
__exit__: The special method called when the block ends, however it ends. It does the cleanup.
Resource: Something limited that must be given back: an open file, a network connection, a lock.
Resource leak: Forgetting to give a resource back, so it stays used until the program ends.
Suppress: To stop an exception from travelling further. Done by returning True from __exit__.

Step by step

01The problem: cleanup that gets skipped

This code opens a file, writes to it and closes it. The third line fails on purpose (10 / 0 raises ZeroDivisionError), so f.close() never runs and the file stays open. A single error is enough to lose the "close" step. ("w" means open for writing, \n is a line break. Topic 7.5 explains both.)

main.pywhole filepython

f = open("log.txt", "w", encoding="utf-8")
f.write("started\n")
f.write(str(10 / 0))     # raises ZeroDivisionError
f.close()                # never reached

02The long way: try / finally

Recall from Topic 7.1: finally runs even when the block fails, so it fixes the problem. This works, but every use needs these extra lines, and it is easy to forget them.

main.pywhole filepython

f = open("log.txt", "w", encoding="utf-8")
try:
    f.write("started\n")
    f.write(str(10 / 0))
finally:
    f.close()

03The short way: with

with does the same thing without the try/finally. The file is closed when the block ends, by success or by error. You can check this with f.closed.

main.pywhole filepython

with open("log.txt", "w", encoding="utf-8") as f:
    f.write("started\n")
print(f.closed)   # True

The short way: withdiagram

Rendering diagram…

04Write your own context manager

__enter__ runs on the way in and its return value becomes the name after as. __exit__ runs on the way out and receives the exception details (all None if there was no error). Returning False means: do not hide the exception.

Read the class lines as a pattern for now (Topic 8.1 explains them). self means "this object", and __init__ runs once when the object is made. The point is the two methods __enter__ and __exit__.

main.pywhole filepython

class Section:
    def __init__(self, name):
        self.name = name

    def __enter__(self):
        print(f"[start {self.name}]")
        return self

    def __exit__(self, exc_type, exc_value, traceback):
        print(f"[end {self.name}]")
        return False

05Errors still pass through

If the block raises and __exit__ returns a false value, the exception continues after the cleanup. The cleanup ran first. You can inspect exc_type inside __exit__ to log what happened, but you almost never want to suppress an error silently.

main.pywhole filepython

try:
    with Section("download"):
        raise RuntimeError("network down")
except RuntimeError as e:
    print("caught:", e)

06Several managers, and the parenthesised form

Managers in one with are entered left to right and exited right to left. Python 3.10 lets you put the whole group in parentheses, one manager per line, so long lines stay readable. A trailing comma is allowed.

main.pywhole filepython

with (
    open("in.txt", encoding="utf-8") as source,
    open("out.txt", "w", encoding="utf-8") as target,
):
    for line in source:
        target.write(line.upper())

Try it yourself

1
Predict the order
In the second example, say the order of the printed lines before running. Then add a print as the first line inside the __exit__ method and predict again.
2
Suppress an error
Change return False to return True in Section.__exit__ and run the example again. The RuntimeError disappears, and the except never runs. This is exactly how silent bugs are born, so do this only to see it once.
3
Order of exit
Use two Section objects in one with (with Section("a"), Section("b"):) and watch the order of start and end lines. The last one entered is the first one left.

Code & diagrams

with closes the file for you New tab

The file is written and read back. Print f.closed to see that the block really closed it.

Expected output

inside the block, closed? False
after the block, closed? True
'first line\nsecond line\n'

A context manager you write yourself New tab

Cleanup runs before the exception reaches the except. Remove the raise to see the normal path.

Expected output

[start one]
working
[end one: ok]
[start two]
[end two: RuntimeError]
caught: network down

Two files, one with statement Python 3.10+ New tab

The parenthesised form needs Python 3.10 or later. Without the brackets it also works, on one long line.

Expected output

ALPHA
BETA

True True

Cleanup happens even when the block fails New tab

Expected output

error: something broke halfway
file closed? True
what was saved: partial result

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Use the file after the block

Read from the file after leaving the with block.

main.pywhole filepython

with open("a.txt", "w", encoding="utf-8") as f:
    f.write("hello")

f.write(" again")

terminal

$ python main.py

── what you'll see ──

Traceback (most recent call last):

File "main.py", line 4, in <module>

f.write(" again")

~~~~~~~^^^^^^^^^^

ValueError: I/O operation on closed file.

Break #2

Write to a file opened for reading

Open without a mode and then write.

main.pywhole filepython

with open("b.txt", "w", encoding="utf-8") as f:
    f.write("x")

with open("b.txt", encoding="utf-8") as f:
    f.write("y")

terminal

$ python main.py

── what you'll see ──

Traceback (most recent call last):

File "main.py", line 5, in <module>

f.write("y")

~~~~~~~^^^^^

io.UnsupportedOperation: not writable

Myth vs fact

Myth

with is only for files.

Fact

Any object with __enter__ and __exit__ works: locks, database connections, HTTP clients, temporary directories, timers, and switches like torch.no_grad().

Myth

The garbage collector closes files, so with is optional.

Fact

CPython often closes an unreferenced file immediately, but the language does not promise it, other implementations do not, and unflushed data can be lost on a crash. Use with to make the cleanup explicit and immediate.

Myth

A variable created in a with block disappears when the block ends.

Fact

Python blocks do not have their own scope. f and anything else assigned inside is still available afterwards. Only the resource it refers to has been cleaned up.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
Writing a file does not put data on disk immediately: it goes to a buffer (a small waiting area in memory, filled first and written out in bigger chunks). Closing flushes the buffer, which is another reason cleanup must be guaranteed. For crash-safe writes professionals write to a temporary file and rename it over the target.
▸
__exit__ receives the live traceback object. Holding on to it past the call can keep frames alive, so log what you need and let it go.
▸
contextlib.ExitStack (Topic 9.8) lets you enter a variable number of managers, such as opening every file in a list, and unwind them all correctly.
▸
Async code has the twin async with (__aenter__/__aexit__), used for network connections and database pools (Topic 12.8).
▸
Inside __exit__, never re-raise the exception you were handed; return a false value and Python continues the propagation with the original traceback. Raising a new exception there replaces the old one (the old becomes __context__, Topic 7.2). If cleanup itself can fail, as with closing a network connection, decide which error matters more and chain them on purpose.
▸
The crash-safe file-write recipe is: write to a temporary file in the same folder, call f.flush() and os.fsync(f.fileno()), then os.replace(temp, target). os.replace is atomic on the same file system, so readers see the old file or the new one, never half of a file. A plain with open(target, "w") truncates the target the moment it opens, so a crash mid-write leaves a damaged file.

Remember this

1
Why do we need this? Some resources are limited and must be returned. An open file uses one of a small number of file handles (the operating system's numbered tickets for open files). A database connection uses a slot on a server. A lock (a flag that makes other workers wait their turn) blocks them until it is released. If your code forgets to close a file because an error happened halfway, the resource leaks. You could write try/finally every time, but it is easy to forget or get wrong. with packages the pattern so it is hard to get wrong.
2
The most common use is with open("notes.txt", encoding="utf-8") as f:. (Topic 7.5 teaches files in full. Here you only need to know that open gives you a file object, a handle you can read from or write to.) with calls the object's __enter__ method, which returns the object, stored in f. It runs the indented block. Then it calls the __exit__ method, which closes the file. After the block f still exists but is closed, and using it raises ValueError: I/O operation on closed file.
3
Here is exactly what with does. It evaluates the expression and calls __enter__(). It binds the result to the name after as (if any). It runs the block. When the block ends, by any route (normally, with return, break, or an exception), it calls __exit__(exc_type, exc_value, traceback). On normal exit all three arguments are None. If the block raised, they describe the exception. If __exit__ returns a true value the exception is suppressed. Returning False or None (the usual choice) lets it continue upward.
4
Any object can be a context manager if its class defines __enter__ and __exit__. These are called special methods (written with double underscores) and are covered properly in Topic 8.1 and Phase 8; for now, treat the small class in the walkthrough as a pattern. Python's contextlib module (Topic 9.8) lets you write one as a simple generator function instead, with @contextmanager.
5
You can put several managers in one with: with open(a) as src, open(b, "w") as dst:. They are entered left to right and exited in reverse order (like unstacking plates). If entering the second fails, the first is still closed. Since Python 3.10 you can wrap them in parentheses and put one per line, which is the readable choice when the line is long.
6
Use with any time you open or acquire something. In AI work it appears everywhere. with open("data.jsonl") reads training data. with torch.no_grad(): switches off gradient tracking (extra bookkeeping used only while training a model) for a block. with httpx.Client() as client: reuses network connections. with lock: protects shared data. The shape is always the same: setup, your block, guaranteed cleanup.

Explain it without notes

What exactly happens when Python executes a with statement?

Why is with open(...) safer than calling open and close by hand?

What does it mean if __exit__ returns True? Is that a good idea?

In what order are several context managers in one with entered and exited?

Practice

Write a program that writes three lines to todo.txt with a with statement, then reads them back in a second with and prints the number of lines.

Write a class Counter whose __enter__ sets self.count = 0 and returns self, and whose __exit__ prints the word count: and the value of count. Use it in a with ... as c block that adds 1 to c.count three times.

Create src.txt with two lines of text (use a with to write it). Then copy it to dst.txt in upper case using one with statement that opens both files.

Trade-offs

↔
Class-based versus generator-based managers. A class with __enter__/__exit__ is explicit and can hold state. A @contextmanager generator (Topic 9.8) is shorter for simple cases. Pick the shorter one unless you need a reusable object.
↔
Suppressing versus propagating errors. Hiding an error inside __exit__ keeps callers simple but can hide real bugs. Prefer to let it pass and handle it where you can make a decision.

Done when you can

Done when you can explain what __enter__ and __exit__ do and when each runs.
Done when you can read and write a file using with open(..., encoding="utf-8").
Done when you can use two files in one with, including the parenthesised form.
Done when you can write a small class-based context manager.
Done when you can say why cleanup must be guaranteed.

Command Palette

Words you'll meet

Step by step

01The problem: cleanup that gets skipped

02The long way: try / finally

03The short way: with

04Write your own context manager

05Errors still pass through

06Several managers, and the parenthesised form

Try it yourself

Predict the order

Suppress an error

Order of exit

Code & diagrams

Break it on purpose

Use the file after the block

Write to a file opened for reading

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can