PHASE 17Advanced ~30 min· topic 5 of 10

Topic 17.5

Interview Deep Dive: Python in Production

In one line

The questions that separate someone who has written scripts from someone who has run Python services: packaging and environments, async versus threads versus processes, finding memory and speed problems, testing strategy, typing, and API design.

Think of it like this

Opening a restaurant versus cooking dinner at home. At home you can improvise: one pot, no schedule, you taste as you go. A restaurant needs a written recipe (so any cook produces the same dish), a stocked and labelled pantry (so nothing is missing at 7 pm), a way to check quality (tasting before serving), and a plan for a rush of customers. Production Python is the restaurant: reproducible builds, controlled environments, tests, and a plan for load and failure.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Virtual environment: A private folder of installed packages for one project.
Lock file: A file listing the exact version of every dependency, so installs are repeatable.
Wheel: A pre-built Python package file that installs without compiling.
Profiler: A tool that measures where a program spends its time.
Dependency injection: Passing the things a function needs (a clock, a client) in as arguments instead of reaching for globals.
Idempotent: Safe to repeat: doing it twice has the same effect as doing it once.
Semaphore: A counter that limits how many tasks may do something at the same time.
Timeout: A maximum time to wait before giving up.

Step by step

01A repeatable project from an empty folder

The commands below create an isolated environment, install the project and its development tools, and lock the versions. They run in a terminal on your computer, not in the browser. With uv, uv sync does the environment and install in one step from pyproject.toml and uv.lock.

The interview point: the same two files (pyproject.toml, the lock file) must be enough for a stranger, or a CI machine, or a Docker build to get the same environment.

terminalwhole filebash

python -m venv .venv
source .venv/bin/activate        # on Windows: .venv\Scripts\activate
pip install -e ".[dev]"          # the project itself, plus its dev tools
pytest                           # run the tests
ruff check .                     # lint
mypy src                         # type check

02Choose the concurrency model from the bottleneck

Ask what the program waits for. Waiting for the network or disk: asyncio (many cheap tasks) or threads. Burning CPU in pure Python: processes or native code. Waiting for a database with a blocking driver: threads, or an async driver. Mixed: use asyncio.to_thread or run_in_executor to run blocking calls without freezing the event loop.

Whatever you pick, bound it. The first runnable example runs three fake model calls through an asyncio.Semaphore(2): the third call starts only when one finishes, which you can read from the log.

Choose the concurrency model from the bottleneckdiagram

Rendering diagram…

03Timeouts and cancellation

Every outbound call needs a timeout, because a call with no timeout can wait forever and hold a worker. asyncio.timeout(seconds) cancels the block when time runs out and raises TimeoutError. The example's last lines show it: the one-second sleep is cancelled after 0.05 s.

Cancellation is delivered as an exception at an await, so use try/finally for cleanup and never swallow asyncio.CancelledError.

04Make code testable by injecting the clock

A rate limiter that calls time.time() directly can only be tested by really waiting. The second runnable example passes the clock in as an argument. The tests use a FakeClock object whose time they change by hand, so a ten-second window is tested in microseconds with no sleep and no flakiness.

The runner at the bottom uses unittest objects directly and prints counts, so the output does not contain timing numbers. In your project you would run pytest, which also finds unittest tests.

05Memory and speed: measure, then change one thing

A typical interview scenario: 'a service's memory grows until it is killed'. Walk through it: reproduce under load, take tracemalloc snapshots at two times and compare to see which lines allocate, look for unbounded caches or lists that only grow, check for reference cycles holding large objects, and for slow data pipelines switch from building lists to generators. Change one thing, measure again, and keep the measurement.

profile_demo.pywhole filepython

import tracemalloc

tracemalloc.start()
data = [str(i) * 10 for i in range(100_000)]      # something suspicious
snapshot = tracemalloc.take_snapshot()
for stat in snapshot.statistics("lineno")[:3]:
    print(stat)                                    # file, line, size, count

Try it yourself

1
Find the bottleneck by reading the log
In the first example, make the third call (c) take 0.2 s instead of 0.02 s, rerun, and describe in one sentence why the log still shows start c before end a.
2
Add a test
In the second example, add a test that a limiter with limit=3 allows three calls and blocks the fourth, and another that checks calls are allowed again after the window passes. Run it and make sure both pass without any real waiting.

Code & diagrams

Bounded concurrency and a timeout with asyncio New tab

Change `Semaphore(2)` to `Semaphore(1)` and `Semaphore(3)` and watch the start/end order in the log change. Results always keep the order of the calls.

Expected output

results keep call order: ['A', 'B', 'C']
log: ['start a', 'start b', 'end b', 'start c', 'end c', 'end a']
timed out and cancelled the slow call

Injecting a clock so tests need no sleep New tab

One test fails on purpose: the first call is allowed, so `allow()` is `True`, not `False`. Fix the test to see `failures: 0`.

Expected output

ran: 3 | failures: 1 | errors: 0
failed: test_this_one_fails_on_purpose -> AssertionError: True != False

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Depend on a package you never declared

Import a package that is not installed in the current environment (here a deliberately misspelled name).

main.pywhole filepython

import yaml_typo

print("loaded")

terminal

$ python main.py

── what you'll see ──

Traceback (most recent call last):

File "main.py", line 1, in <module>

import yaml_typo

ModuleNotFoundError: No module named 'yaml_typo'

Myth vs fact

Myth

asyncio makes everything faster.

Fact

It makes waiting cheaper. CPU-bound code gets no faster, and one blocking call inside an async function stalls all other tasks.

Myth

Type hints slow Python down or are enforced at run time.

Fact

The interpreter ignores them. They are for checkers, editors and libraries such as Pydantic.

Myth

pip install without a lock file is fine for production.

Fact

It works until a dependency releases a new version and your next build changes. Production builds should install exact, locked versions.

Myth

More threads always means more throughput.

Fact

Beyond a point, extra threads add memory use and contention, and they overload the service you call. Bound concurrency deliberately.

Interview problem

The problem

Call a rate-limited API for 1,000 documents

You must send 1,000 documents, one request each, to a slow external API that allows at most 5 concurrent requests and sometimes returns a temporary error. Write the design and the core code idea. The job should finish in reasonable time, never exceed the limit, and keep the order of results.

You're given

At most 5 requests in flight.
Retry temporary errors, but not forever.
Results line up with the input order.
One bad document must not kill the job.

The interviewer follows up

How would you resume after the job is killed halfway?

What changes if the API charges per token?

When it breaks

A batch job calls a model API for 10,000 documents with unbounded asyncio.gather

What you see

The provider returns rate-limit errors (HTTP 429) for most calls, the job retries them all at once, and memory climbs because 10,000 tasks and responses are alive together.

Fix & prevent

Bound concurrency with a semaphore, process in batches, retry with exponential backoff and jitter, and stream results to disk instead of collecting everything in memory.

A web service grows in memory over days until the container is killed

What you see

Requests slow down, then the process is restarted by the orchestrator. The cause is an unbounded cache dictionary keyed by user input.

Fix & prevent

Use functools.lru_cache(maxsize=...) or a cache with a size limit and expiry. Compare tracemalloc snapshots from the running service to find the growing allocation.

Tests pass on a laptop but fail in CI

What you see

Different dependency versions, a missing system library, a test that depends on the current time or on the order of a set, or a file left behind by a previous run.

Fix & prevent

Lock dependencies, run tests in a clean environment (or the same container as production), inject clocks and random seeds, and use temporary directories for files.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
For services, prefer an ASGI framework (FastAPI is the common one) with async handlers for I/O-heavy code, and keep CPU-heavy work out of the event loop using a process pool or a separate worker service. See Phase 15 for the API topics.
▸
Python 3.11 added asyncio.TaskGroup, which gives structured concurrency: tasks started inside async with asyncio.TaskGroup() as tg: are all finished (or cancelled, if one fails) before the block exits, and failures arrive together as an ExceptionGroup caught with except*. It is safer than loose create_task calls whose errors nobody awaits. It cancels siblings on the first error, so use gather(..., return_exceptions=True) instead when one bad document must not stop the rest.
▸
python -X dev enables extra runtime checks, and -W error turns warnings into errors in CI. PYTHONHASHSEED pinned in tests removes ordering flakiness caused by hash randomisation.
▸
Observability for Python services means structured logs with a request ID, metrics (latency, error rate, queue depth) and traces. Add them before the first incident, not after. The DevOps and SRE courses on Hectal cover the monitoring side.

Remember this

1
Packaging and environments (Topics 6.5 to 6.7). A virtual environment (python -m venv .venv) is a private folder of installed packages for one project, so projects do not fight over versions. A project describes itself in pyproject.toml (name, version, Python version, dependencies, tool settings). A lock file (from uv, pip-tools or Poetry) records the exact version of every package, so a build on your laptop and one in a container install the same thing. Distribute with a wheel (a built, ready-to-install file) rather than relying on source builds. The interview answer: *pin what you ship, range what you publish*.
2
Async versus threads versus processes (Topics 12.1 to 12.7). The bottleneck is the thing the program spends most of its time on, such as waiting for the network or doing arithmetic. Choose by the bottleneck. **asyncio** runs many tasks on one thread by switching at each await; it suits thousands of waiting network calls (for example many model API requests) and costs the least per task, but one blocking call stalls everything. Threads suit blocking libraries and modest I/O concurrency, and share memory, so shared data needs locks. Processes use several CPU cores and avoid the GIL, at the cost of start-up time and copying data between them. Put a limit (a semaphore or pool size) on any of them, or you will overload the service you are calling.
3
Finding memory and speed problems (Topic 13.7). Measure first. cProfile (or a sampling profiler such as py-spy) shows where time goes; tracemalloc shows which lines allocate memory and can compare two snapshots to find a leak; timeit times small snippets reliably. The usual culprits are accidental O(n^2) loops, building huge lists instead of streaming with generators, caches that never evict (an lru_cache with no maxsize, or a global dict), and holding references to big objects in a long-lived structure.
4
Testing strategy (Topics 11.1 to 11.4). A unit test checks one small piece, an integration test checks several pieces together, and an end-to-end test runs the whole program. Many fast unit tests, fewer integration tests, very few end-to-end tests. Make code testable by passing in its dependencies (a clock, a database connection, a model client) instead of reaching for globals, so a test can pass a fake. Test behaviour and edge cases, not implementation details. For code that calls models or the network, fakes and recorded responses give deterministic tests, and a small number of real calls run separately.
5
Typing in production (Topics 10.1 to 10.3 and 15.4). Type hints cost nothing at run time and let a checker such as mypy or pyright catch whole categories of bugs before deployment: passing None where a value is needed, wrong argument types, forgotten cases. Use them at module boundaries first (public functions, data classes), start lenient and tighten gradually, and run the checker in CI (a server that runs your checks on every change). Pydantic turns type hints into validation of data that arrives from outside.
6
API design questions. Interviewers ask how you would design the Python or HTTP interface of a service: stable and explicit input and output types, validation at the edge, meaningful errors, timeouts on every outbound call, idempotency (the same request sent twice has the same effect, via an idempotency key; Topic 15.2), pagination for lists, versioning, and structured logging with a request ID. These ideas continue in the System Design course (/system-design), and the Docker course (/docker) covers packaging the result for deployment.

Explain it without notes

How do you make a Python project reproducible across a laptop, CI and a container?

When do you pick asyncio, threads or processes? Give one example of each.

A Python service's memory keeps growing. How do you investigate?

How do you test code that depends on the current time or a remote API?

What do you consider when designing the interface of a service that calls a slow model API on behalf of users?

Where do configuration and secrets (API keys, passwords) belong in a Python service, and where must they never go?

What is the difference between a mock and a fake, and what does the rule "patch where it is used" mean?

Practice

Write an async function fetch_all(names, limit) that fakes a 0.01-second call per name, runs at most limit at a time, and returns the results in input order.

Write a FakeClock and a Cache class whose entries expire after ttl seconds, using the injected clock. Show an entry expiring without sleeping.

Write a function retry(fn, attempts) that calls fn() up to attempts times, re-raising the last error if all fail.

Trade-offs

↔
Lock everything versus allow flexibility. Exact locks make builds repeatable but mean you must update on purpose (and handle security fixes). Ranges for libraries, locks for applications is the usual compromise.
↔
Mocks versus real services in tests. Fakes are fast and deterministic but can drift from reality. Keep a small set of contract or smoke tests against the real thing.

Done when you can

Done when you can describe a repeatable setup: pyproject.toml, a lock file and a clean environment.
Done when you can choose between asyncio, threads and processes and bound each.
Done when you can outline how to find a memory leak with measurements.
Done when you can make time and network dependencies injectable and write a fast test.
Done when you can name five points of API design: types, validation, timeouts, idempotency, errors.

Command Palette

Words you'll meet

Step by step

01A repeatable project from an empty folder

02Choose the concurrency model from the bottleneck

03Timeouts and cancellation

04Make code testable by injecting the clock

05Memory and speed: measure, then change one thing

Try it yourself

Find the bottleneck by reading the log

Add a test

Code & diagrams

Break it on purpose

Depend on a package you never declared

Myth vs fact

Interview problem

Call a rate-limited API for 1,000 documents

When it breaks

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can