Concurrency vs Parallelism: Threads, Processes, Async

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Concurrency

Several tasks are in progress at the same time, though maybe only one runs at any instant.

Parallelism

Several tasks really run at the same instant on different CPU cores.

I/O-bound

Spends most of its time waiting for the network, a disk or a database.

CPU-bound

Spends most of its time calculating, so the CPU is the bottleneck.

Thread

A flow of code inside a process. Threads in one process share the same memory.

Process

A running program with its own memory. Processes do not share variables.

Event loop

The asyncio loop that runs tasks and resumes each one when what it awaited is ready.

LLM API

A web service you send text to and get a language model's answer from. The call mostly waits for the reply.

Coroutine

A function made with async def. It can pause at an await and be resumed later.

Cooperative switching

A task keeps running until it chooses to wait (at an await).

Preemptive switching

The operating system pauses a thread at any moment to let another run.

1. 1

   Make the waits longer

   In the first example change the kettle's wait to 0.2 and the toaster's to 0.01. Predict which done line comes first in the "together" half, then run it. The order depends only on the waits, never on the order you wrote the calls.

2. 2

   Add a third job

   Add a third boil("oven", 0.07) to the gather call. Where does its done line appear? How many start lines come before the first done?

3. 3

   Classify your own programs

   In the second example add three jobs you know from your own life of coding (downloading a file, sorting a million numbers, asking a chatbot a question) and decide the three flags for each. If you cannot decide, ask: "if I gave this a faster internet connection, would it speed up? If I gave it a faster CPU?"

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

import asyncio
import time


async def boil(name, seconds):
    print(f"{name}: start")
    time.sleep(seconds)            # blocking sleep: nobody else can run meanwhile
    print(f"{name}: done")


async def main():
    await asyncio.gather(boil("kettle", 0.1), boil("toaster", 0.05))


asyncio.run(main())

1. 1

   A program is I/O-bound when most of its time is spent waiting for something outside the CPU: a network reply, a disk read, a database answer, an AI model generating text. It is CPU-bound when most of its time is spent calculating: resizing images, training a model in pure Python, hashing files. This one question decides almost everything in this phase, so ask it first about any slow program.

2. 2

   Concurrency helps I/O-bound programs. While one request waits for a reply, another can be sent. You do not need more CPU for that, because the CPU was idle anyway. Calling an LLM API takes about a second of waiting for a few milliseconds of your own work, so sending 100 calls together finishes in about the time of one call instead of 100.

3. 3

   Parallelism helps CPU-bound programs, but only if the work runs on several CPU cores at the same time. Python's threads cannot do this for pure-Python code (the GIL, Topic 12.2); separate processes can, because each process has its own interpreter and its own memory. Libraries like NumPy also run in parallel inside their C code. That is why vectorised maths (one command that works on a whole array of numbers at once, Topic 14.3) is often the best answer.

4. 4

   Python has three tools. Threads (threading, Topic 12.3) run several flows of code inside one process, sharing memory; the operating system switches between them at any moment. Processes (multiprocessing, Topic 12.5) are separate programs with separate memory; they use all cores but cost more and must send data by copying. asyncio (Topics 12.6 to 12.8) runs many tasks on one thread, and a task only gives way at an await, a point where it says "I am waiting, someone else can go".

5. 5

   The three tools differ in who decides when to switch. Threads use preemptive switching: the operating system can interrupt a thread anywhere, even in the middle of counter += 1, which is why threads need locks. asyncio uses cooperative switching: a task is never interrupted, it hands control back only at await, which is why asyncio needs far fewer locks but one task that never awaits freezes everything.

6. 6

   In the AI era most Python concurrency is I/O-bound: many model calls, many web pages, many database rows. That is why asyncio is the default for LLM applications, and why it is the longest part of this phase. CPU-bound work (embeddings, matrix maths) usually runs in NumPy, PyTorch or another native library that already uses many cores for you.

• ↔

  Threads: easy to start and work with blocking libraries, but they share memory, so you need locks, and they do not speed up pure-Python calculation.

• ↔

  Processes: real parallelism and strong isolation, but heavy to start, and every argument and result must be copied between them.

• ↔

  asyncio: very cheap tasks and few race conditions, but every library in the chain must be async-aware, and one blocking call stalls everything.

• Done when you can explain concurrency vs parallelism with the kitchen analogy.

• Done when you can tell whether a program is I/O-bound or CPU-bound with the faster-network or faster-CPU question.

• Done when you can name the three Python models and say which kind of work each one helps.

• Done when you can read the printed order of two overlapping coroutines and say why it came out that way.

• Done when you can explain why time.sleep inside a coroutine defeats gather.