PHASE 11Intermediate ~28 min· topic 2 of 8

Topic 11.2

pytest: Writing and Running Tests

In one line

pytest finds functions named test_*, runs each one separately and reports the results. It rewrites your plain assert lines so a failure shows the actual values, which makes it the standard tool for Python tests.

Think of it like this

A teacher marking exams with an answer key. A bad marker writes only "wrong". A good marker writes "you wrote 6, the answer was 7, and 6 came from adding 1, 2 and 3". pytest is the good marker: when an assert fails, it shows both sides of the comparison and where each value came from.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

pytest: The most used Python test tool. It finds and runs tests and shows detailed failures.
Test discovery: How pytest finds tests by looking for files and functions with test in their names.
Test module: A .py file that contains tests, such as test_shop.py.
Collected: pytest says it "collected" N items: it found N tests to run.
Assert rewriting: pytest changes your assert lines at import time so failures show the values involved.
Node id: A test's full address, such as test_shop.py::test_total.
Exit code: A number a program returns to tell other tools whether it succeeded. 0 means success.
`pytest.raises`: A check that a block of code raises a particular exception.
`pytest.approx`: A way to compare numbers that are equal up to a tiny tolerance.

Step by step

01Install and write the first test file

Install pytest into your project's virtual environment (Topic 6.5). Then create two files next to each other: the code, and a test file whose name starts with test_.

Notice the test file just imports the code and uses plain assert. There is nothing to learn about pytest to write this.

shop.pywhole filepython

def total(prices):
    return sum(prices)


def apply_discount(amount, percent):
    if percent < 0 or percent > 100:
        raise ValueError("percent must be between 0 and 100")
    return amount - amount * percent / 100

terminal

$ python -m pip install pytest

── expected output ──

(pip downloads pytest and its small dependencies, then ends with a line that starts with "Successfully installed". Versions differ over time.)

02The test file

Three tests: a normal case, a numeric case, and an error case. The error case uses pytest.raises.

test_shop.pywhole filepython

import pytest
from shop import apply_discount, total


def test_total():
    assert total([30, 12]) == 42


def test_discount_ten_percent():
    assert apply_discount(200, 10) == 180


def test_bad_percent_raises():
    with pytest.raises(ValueError):
        apply_discount(100, 150)

03Run it

python -m pytest -v lists each test with its result. The header (platform, versions, plugins) differs between machines; the part you read is the list and the last line. Output below is trimmed: the header lines were cut.

terminal

$ python -m pytest -v

── expected output ──

collected 3 items

test_shop.py::test_total PASSED [ 33%]

test_shop.py::test_discount_ten_percent PASSED [ 66%]

test_shop.py::test_bad_percent_raises PASSED [100%]

============================== 3 passed in 0.01s ==============================

04Read a failure from the top down

When a test fails, pytest prints the test's source with a > in front of the line that failed, then lines starting with E that explain it. where 6 = total([1, 2, 3]) is the assert rewriting at work: it shows what each part of the expression was. Last comes the file and line number, then the one-line summary.

Read in this order: the E lines (what was different), the > line (which assert), then the summary (which test). The runnable example below produces this exact report in your browser.

05Expecting errors and comparing floats

pytest.raises(ValueError, match="zero") passes only if the block raises ValueError whose message matches the pattern. The as form gives you the exception to inspect: with pytest.raises(ValueError) as excinfo: then excinfo.value.

approx fixes the float problem from Topic 11.1: 0.1 + 0.2 == pytest.approx(0.3) is true. It also works on lists and dictionaries of numbers. You can set the tolerance with approx(3.0, abs=0.01).

06Run only what you need

-k filters by name, :: selects an exact test, -x stops at the first failure, and --lf repeats the last failures. A quick fix loop is: pytest -x --lf.

When no tests are found, pytest says "no tests ran" and exits with code 5. That is the sign that your file or function name does not match the rules.

terminal

$ python -m pytest -v -k discount

── expected output ──

collected 3 items / 2 deselected / 1 selected

test_shop.py::test_discount_ten_percent PASSED [100%]

======================= 1 passed, 2 deselected in 0.01s =======================

07Exit codes and what uses them

Your terminal, a pre-commit hook and a CI server all decide pass or fail by the exit code. 0 is green. 1 means tests failed. 2 means the run was interrupted, 3 is an internal error, 4 means pytest was called with bad options, and 5 means no tests were collected.

terminal

$ python -m pytest -q test_shop.py::test_total

echo "exit code: $?"

── expected output ──

. [100%]

1 passed in 0.32s

exit code: 0

Try it yourself

1
Make a failure readable
In the second example, change the last test's expected value to a wrong number such as 0.4. Read the E lines. Then wrap the numbers in pytest.approx and see the report change.
2
Rename a test and watch it vanish
Rename test_divide_normal to check_divide_normal and run the --collect-only step. It is no longer listed: pytest ignores names that do not start with test. Rename it back.
3
Predict the exit code
On your own computer, run pytest -q in an empty folder and check the exit code with echo $? (Linux/macOS) or echo $LASTEXITCODE (PowerShell). Guess first: it is 5.

Code & diagrams

Run pytest from a program and read a failure New tab

pytest is installed in the in-browser Python. The first lines (down to run_pytest) are plumbing that runs pytest inside this page and removes the timing text so the output stays the same each run: you do not need to study them. The test file is the text inside the triple quotes. Change the expected 7 to 6 and run again.

Expected output

..F                                                                      [100%]
=================================== FAILURES ===================================
____________________________ test_wrong_on_purpose _____________________________

    def test_wrong_on_purpose():
        prices = [1, 2, 3]
>       assert total(prices) == 7
E       assert 6 == 7
E        +  where 6 = total([1, 2, 3])

test_cart.py:16: AssertionError
=========================== short test summary info ============================
FAILED test_cart.py::test_wrong_on_purpose - assert 6 == 7
1 failed, 2 passed

raises, approx, -k and -x New tab

Four tests, one of them wrong on purpose (`0.1 + 0.2 == 0.3` without approx). The run_pytest helper is the same plumbing as in the first example. Notice -x stops the run at the first failure.

Expected output

--- --collect-only: list the tests ---
test_tools.py::test_divide_normal
test_tools.py::test_divide_by_zero_raises
test_tools.py::test_float_sum
test_tools.py::test_float_sum_exact_fails

4 tests collected
--- -k divide: only names that contain divide ---
..                                                                       [100%]
2 passed, 2 deselected
--- -x stops at the first failure ---
...F
=================================== FAILURES ===================================
__________________________ test_float_sum_exact_fails __________________________

    def test_float_sum_exact_fails():
>       assert 0.1 + 0.2 == 0.3
E       assert (0.1 + 0.2) == 0.3

test_tools.py:25: AssertionError
=========================== short test summary info ============================
FAILED test_tools.py::test_float_sum_exact_fails - assert (0.1 + 0.2) == 0.3
!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!
1 failed, 3 passed

Where pytest looks for testsdiagram

Rendering diagram…

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Expect an error that never comes

Use pytest.raises(ValueError) around a call that does not raise.

test_shop.pywhole filepython

import pytest
from shop import apply_discount


def test_discount_error():
    with pytest.raises(ValueError):
        apply_discount(100, 50)

terminal

$ python -m pytest -q

── what you'll see ──

E Failed: DID NOT RAISE <class 'ValueError'>

Myth vs fact

Myth

pytest tests need a special base class or special assert methods.

Fact

Plain functions and plain assert are enough. pytest rewrites assert to give detailed failures. (The older unittest style with TestCase and assertEqual still works under pytest, Topic 11.5.)

Myth

Passing tests that print nothing mean nothing ran.

Fact

pytest hides output of passing tests by default and prints dots. Use -v to see each test name, and -s to see prints.

Myth

pytest is part of Python.

Fact

It is a separate package. Python ships unittest and doctest (Topic 11.5); pytest is installed with pip or uv.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
Assert rewriting happens through an import hook, only for test modules, conftest.py files, plugins and modules registered with pytest.register_assert_rewrite. If your assertions live in a helper module outside the test files, that module is not rewritten and you lose the detailed report; register it before it is imported. Never run a test suite under python -O: it removes every assert, so all tests would pass without checking anything.
▸
pytest collects a Test* class only if it has no __init__ method (it warns and skips it otherwise), and it creates a new instance of the class for every test, so state set on self in one test never leaks into the next.
▸
python -m pytest and plain pytest differ only in sys.path: the module form puts the current directory first. (With the default prepend import mode, plain pytest also inserts the first folder without an __init__.py above each test file into sys.path, which is why a flat layout with tests beside the code often works by accident and then breaks in CI.) Real projects avoid the question with a package layout and an editable install (Topic 6.8), or with pythonpath = ["src"] under [tool.pytest.ini_options] in pyproject.toml.
▸
Configure pytest once in pyproject.toml: [tool.pytest.ini_options] with testpaths = ["tests"] and addopts = "-ra". Settings in that table apply to every run, locally and in CI.
▸
Tests run in the order they appear in the file, but never rely on it. Plugins such as pytest-randomly shuffle order on purpose to expose tests that secretly depend on each other, and pytest-xdist (-n auto) runs them in parallel processes.

Remember this

1
In Topic 11.1 you wrote a tiny runner. pytest is a full one, and it is a third-party package (written by other people, not shipped with Python). Install it with pip install pytest (or uv add --dev pytest, Topic 6.6). It is the tool almost every Python project uses, and it runs plain assert tests without any base class or special syntax.
2
pytest uses naming rules to find tests, called test discovery. It looks in files named test_*.py or *_test.py (the * stands for any text). Inside them it picks functions named test_*, and classes named Test* that have test_* methods (classes are Topic 8.1). Starting from the folder you run it in, it searches subfolders too.
3
Misspell the name (tests_total, check_total) and the function is silently ignored. That is the most common beginner surprise.
4
You run it with pytest or python -m pytest. The second form also adds the current folder to Python's import path (the list of folders Python searches when you write import, Topic 6.1), so from shop import total works without extra setup.
5
The output shows a dot for each pass and an F for each failure, then a section for every failure, then a summary line. The exit code (the number a finished program hands back) tells scripts and CI (a server that runs your checks on every change, Topic 11.8) what happened: 0 means all passed, 1 means some tests failed, and 5 means no tests were collected.
6
pytest does something clever to plain assert. When it imports your test file, it rewrites the assert statements so that they record the value of every part of the expression. That is why a failure of assert total(prices) == 7 shows assert 6 == 7 and where 6 = total([1, 2, 3]) without you writing any message. You get the detail for free, so most pytest asserts need no message at all.
7
To expect an error, use with pytest.raises(ValueError):. The test passes only if the block raises that exception, and fails if it raises nothing or something else. Add match="zero" to check the message too (it is a pattern in the style of Topic 3.6, searched in the text of the error). Recall from Topic 1.3 that 0.1 + 0.2 is not exactly 0.3. To compare floats, use pytest.approx: assert 0.1 + 0.2 == pytest.approx(0.3) passes, because approx allows a tiny difference (one part in a million by default).
8
While working, you rarely run everything. pytest -x stops at the first failure. pytest -k discount runs only tests whose name contains "discount" (the match is case-insensitive and also looks at file and class names, and -k accepts expressions such as "discount and not bad"). pytest test_shop.py::test_total runs one test by its node id.
9
A few more options: -v lists every test by name, -q makes output short, -s shows your print output (pytest hides it by default), and --lf re-runs only the tests that failed last time.
10
In AI projects, pytest is where you check the plain-Python parts: that a function turns a model's JSON reply into the right object, that a prompt builder fills in every field, that bad replies raise a clear error. Those tests run in milliseconds and never call a model.

Explain it without notes

How does pytest find tests, and what do you do if it reports "no tests ran"?

How does pytest show useful failure messages for plain assert statements?

How do you test that a function raises an exception, and how do you check the message?

Why not compare floats with == in a test, and what do you use instead?

What do -k, -x, -v, -s and --lf do?

Practice

Write is_palindrome(text) and a test file (as a string you run with the pytest helper from the examples) with at least three tests: a palindrome, a non-palindrome and the empty string.

Write a test that checks int("abc") raises ValueError and that the message contains the word "invalid". Use pytest.raises with match.

Write a check comparing 0.1 + 0.2 to 0.3 that passes using pytest.approx, and explain why the plain == version fails.

Trade-offs

↔
pytest versus unittest. pytest needs an install and uses a little magic (assert rewriting, fixtures), but tests are shorter and failures clearer. unittest is built in and explicit, and its style is common in older code.
↔
Detailed output versus quiet output. Verbose runs help while learning and debugging; in CI a short summary plus full failure detail is usually enough. Use -q or -v to match.
↔
One big test file versus many small ones. Small files named after the module they test (test_shop.py for shop.py) keep failures easy to find; very large files get slow to read.

Done when you can

Done when you can name the file, function and class naming rules pytest uses to find tests.
Done when you can run all tests, one file, one test by node id, and tests matching -k.
Done when you can read a pytest failure: the > line, the E lines and the summary.
Done when you can test an expected exception with pytest.raises and match.
Done when you can compare floats with pytest.approx and say why == fails.
Done when you know what exit codes 0, 1 and 5 mean.

Command Palette

Words you'll meet

Step by step

01Install and write the first test file

02The test file

03Run it

04Read a failure from the top down

05Expecting errors and comparing floats

06Run only what you need

07Exit codes and what uses them

Try it yourself

Make a failure readable

Rename a test and watch it vanish

Predict the exit code

Code & diagrams

Break it on purpose

Expect an error that never comes

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can