Topic 16.14
Writing Python with AI Assistants
In one line
AI coding assistants such as Copilot, Cursor, Claude Code and Codex write plausible Python quickly, and plausible is not the same as correct. Give them context, ask for tests, review what they produce, and verify with tests, type checks and Ruff before you trust it.
Think of it like this
A very fast intern who has read a lot but never ran the code you work with. They produce a draft in seconds, sound confident, and sometimes invent a detail. You would not ship the draft unchecked. You would read it, run it, and test it. The skill is not typing less. It is checking well.
Words you'll meet
New words in this topic, in plain English. Come back here whenever one feels fuzzy.
- AI assistant
- A tool that suggests or writes code from your instructions and files.
- Context
- The files, messages and instructions the assistant can see.
- Hallucination
- A confident answer that is wrong, such as a function that does not exist.
- Edge case
- An unusual input, such as empty, one item or very large.
- Linter
- A tool that reads code and warns about likely mistakes. Ruff is one.
- Type checker
- A tool that checks type hints without running the code. mypy and Pyright are two.
- Code review
- Reading someone else's changes to find problems before they are merged.
Step by step
01A plausible bug
There is no real assistant in this page, so the "AI-written" code below is written by us on purpose. That way the bug is the same every time you run it. The first example is code an assistant might write for "median of a list". It looks right and passes the odd-length example it was probably shown. Two of the tests the example runs expose the problems: it returns 3 for the even-length list instead of 2.5, and it sorted the caller's list in place (the last line prints the changed list). Both are hard to spot when reading. "In place" means the function changes the list it was given instead of working on a copy (Topic 4.2).
02A test that catches it
Tests turn "looks right" into evidence. The second example is a corrected version that raises a clear error for empty input, uses sorted to leave the caller's list alone, and averages the two middle values for even length. The tests include the cases that failed before, plus an input-unchanged check. Write the failing test first when you can, then ask the assistant to fix the code.
03A prompt that works
Compare "write a median function" with a prompt that states the rules. The second leaves less to guess.
Write median(values) for Python 3.13.
- values is a list of numbers. Do not modify it.
- Empty list: raise ValueError("median of an empty list").
- Even length: return the mean of the two middle values.
- Use only the standard library. Add pytest tests for odd, even, single, empty, and "input unchanged".04Automated review with the ast module
The third example uses Python's ast module (it reads code as a tree without running it) to flag three classic traps: a mutable default argument, a bare except and eval. Ruff does this and much more with ready-made rules. The point is the habit: a machine checks the things humans skim over.
05Invented APIs
When the break-it example runs, Python reports AttributeError: 'str' object has no attribute 'contains'. Strings have in and .find(), not .contains() (that is JavaScript and Java). A type checker finds this without running the code. When an assistant uses a function you do not recognise, look it up in the official documentation before trusting it.
06Tools on the command line
Not runnable here (the browser has no command line). These are the usual checks. Output depends on your project.
pytest # run the tests
ruff check . # lint
ruff format --check . # formatting
mypy . # types (or: pyright)Try it yourself
- 1
Add a failing test
In the second example add a test for the list
[1, 2, 2, 3](expect 2.0) and one for[-5, -1](expect -3.0). Do they pass? - 2
Extend the reviewer
Add a check to
reviewthat flags calls toprintinside a function namedload. Which AST node do you need?
Code & diagrams
Find the two problems before you read the output: one in the even-length case, one in what the function does to its input.
Expected output
PASS odd length: got 2, expected 2
FAIL even length: got 3, expected 2.5
PASS single value: got 7, expected 7
caller's list after the call: [1, 4, 5]Expected output
empty input -> median of an empty list
all tests passedThe code being reviewed is held in a string and parsed, never executed.
Expected output
line 2: mutable default argument in add_item()
line 8: eval() runs arbitrary code
line 9: bare except hides every errorBreak it on purpose
Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.
Break #1
Use a method that does not exist
Call a method an assistant invented.
text = "AI assistants sometimes invent methods"
print(text.contains("invent"))Myth vs fact
Myth
If the assistant wrote it and it runs, it is correct.
Fact
Running once proves little. Edge cases, mutation of inputs and missing error handling often hide behind a working demo.
Myth
Assistants make learning to code unnecessary.
Fact
You need to read, test and debug what they produce. That takes the skills in this course, and beginners who skip them cannot tell good output from bad.
Myth
AI-written tests prove AI-written code.
Fact
Tests from the same source can share the same misunderstanding. Review the tests and add cases you chose yourself.
Pro corner
Extra depth for experienced readers. New to this? Skip it for now and come back later.
- ▸
Treat repository-level instructions as code: many tools read a project instruction file describing build commands, style rules and conventions. Keeping that file accurate helps every assistant and every new teammate.
- ▸
Prefer small, reviewable changes with tests over big generated rewrites. A good check is whether you could explain each changed line in a code review.
- ▸
Dependency suggestions are a supply-chain risk. Verify that a suggested package exists, check its name carefully against the official one, pin versions, and review what it needs before installing.
- ▸
Agent-style assistants that read files, run commands and edit code are themselves exposed to prompt injection (Topic 16.7): a README, an issue, a web page or a dependency's docs can contain text aimed at the assistant. Keep them to least privilege (sandbox or container, no production credentials in the environment, an approval step for shell commands, network access and installs), review the commands and diffs they propose, and do not let them run on untrusted repositories with your real secrets available.
- ▸
Review generated code with a checklist: does it solve the stated requirement (not a nearby one), is every API real for your installed version, what happens on empty, huge and malformed input, does it mutate its arguments, are errors handled in the right place, are there new dependencies or network or file access, and do the tests fail when you break the code (try deleting a line to see). A test that cannot fail is not evidence.
Remember this
- 1
Give context. An assistant works from what it can see: the open files, the error message, your instructions. Say what the function must do, its inputs and outputs, the Python version, the libraries you use, the style rules of the project, and an example. Point it at the relevant files. A vague request like "fix this" gets a vague fix; "this function mutates its argument, here is a failing test, keep the signature" gets a useful one.
- 2
Ask for tests, and read them. Request tests together with the code, including edge cases (empty input, one item, duplicates, huge values). Then check that the tests actually test something: a test that repeats the implementation's logic proves nothing. The best flow is often tests first: write or review the tests yourself, then let the assistant make them pass.
- 3
Typical mistakes in AI-written Python. (1) Invented APIs: a method or argument that does not exist (the break-it shows
str.contains). (2) Outdated code: deprecated functions or old library versions, because training data lags. (3) Missing error handling: the happy path only. (4) Hidden bugs in edge cases: the median example sorts the caller's list in place and ignores even length. (5) **Mutable defaults, bareexcept,evaland other known traps. (6) Over-engineering:** extra classes and layers nobody asked for. - 4
Verify with tools, not feelings. Run the tests. Run a type checker (mypy or Pyright) to catch wrong types and invented attributes. Run Ruff (the linter and formatter from Phase 11) to catch unused imports, bare excepts and style problems. Try the code on an input you chose, not one the assistant chose. Read the diff (the list of lines the assistant added and removed) as if a stranger wrote it, because one did.
- 5
Security and privacy. Do not paste secrets, customer data or private code into tools your organisation has not approved. Treat generated code that handles input, files, subprocesses, SQL or credentials with extra suspicion (Phase 15 shows the safe patterns). Be careful with packages an assistant suggests: check that a package exists, is the real one, and is maintained, because names can be invented or mimic real ones.
- 6
Stay in charge of design. Assistants are good at boilerplate, explanations, test scaffolding, refactors with tests as a safety net and unfamiliar libraries. They are weaker at deciding what the system should do. Keep changes small, commit often so you can undo, and make sure you understand every line you keep: you are the one who will debug it at midnight. A full Hectal course on LLM engineering is planned and will cover using models for software work in depth.
Explain it without notes
How do you give an AI assistant enough context to produce useful Python?
Name typical mistakes in AI-written Python and a way to catch each.
Why is it risky to accept AI-written code without tests?
A teammate opens a 600-line pull request that was mostly written by an AI assistant. How do you review it?
Practice
Write find_bare_excepts(source) that returns the line numbers of bare except: clauses using ast.
Write slugify(title) and a function check_slugify() whose assertions cover normal, extra-space, empty and upper-case input.
Write first_word(text) that returns the first word, or an empty string for empty or blank text.
Trade-offs
- ↔
Speed versus understanding. Accepting generated code is fast and leaves you with code you may not understand. Slow down on anything you will maintain.
- ↔
Autocomplete versus agent mode. Small inline suggestions are easy to review. Larger autonomous changes save time and need stronger tests and smaller commits.
Done when you can
Done when you can write a prompt with context, constraints and examples.
Done when you can list at least five typical mistakes in AI-written Python.
Done when you verify generated code with tests, a type checker and Ruff.
Done when you can write tests that expose an edge-case bug.
Done when you know a full Hectal course on LLM engineering is planned.