Command Palette

Search for a command to run...

Hectal
PHASE 13Advanced ~30 min· topic 9 of 9

Topic 13.9

Why NumPy Is Fast: C Underneath

In one line

A Python list holds pointers to separate objects, while NumPy, array and memoryview use one contiguous block of raw C data that is shared through the buffer protocol. That layout, compiled loops and releasing the GIL in native code are why NumPy is fast.

0/9 · 0%

Think of it like this

A bag of marbles versus a labelled egg tray. In a Python list, every marble sits in its own little box somewhere in the warehouse, and the list is a notebook of box addresses. To add all the marbles, you must fetch every box. A NumPy array is an egg tray: all the marbles lie side by side in a single row, and a worker can run his hand along the tray without looking anything up. Contiguous data is fast to read, and C can loop over it directly.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Contiguous
Stored in one unbroken block of memory, item after item.
Buffer protocol
The standard way for objects to share their raw memory without copying.
`memoryview`
A window onto another object's memory. Slicing it does not copy.
`array.array`
A standard-library list-like object holding packed C numbers of one type.
dtype / strides
The item type of a NumPy array and how many bytes to step to reach the next item.
C API
The C functions that let compiled code create, inspect and call Python objects.
Wheel
A pre-built binary package (.whl) that pip installs without compiling.
SIMD
A CPU feature that does the same operation on several numbers in one instruction.

Step by step

01A list of objects versus a block of numbers

A Python list holds pointers; every item is a separate object somewhere else in memory. A packed array holds the numbers themselves. In the runnable example, sys.getsizeof(lst) alone is misleading (it counts only the pointers), so we add the size of each int object before comparing with the array.

A list of objects versus a block of numbersdiagram
Rendering diagram…

02The array module

array('i', ...) uses the C type code i (an int, 4 bytes on current desktop platforms). tobytes() shows the raw bytes: the number 1 is the four bytes 01 00 00 00 on a little-endian CPU (one that stores the smallest byte of a number first, as almost every modern computer does). The array checks every assignment: a float raises TypeError, and a number too big for the C type raises OverflowError.

main.pywhole filepython
from array import array

nums = array("i", [1, 2, 3, 4])
print(nums.itemsize, nums.tobytes())

03memoryview: slicing without copying

data[2:5] on a bytearray copies three bytes into a new object. memoryview(data)[2:5] makes a window onto the same memory. Changing data shows through the view, and writing through the view changes data. For a 100-megabyte buffer parsed in chunks, that is the difference between no extra memory and copying it chunk by chunk.

main.pywhole filepython
data = bytearray(b"abcdefgh")
copy = data[2:5]
view = memoryview(data)[2:5]
data[2] = ord("X")
print(bytes(copy), bytes(view))   # b'cde' b'Xde'

04Reinterpret the same bytes

memoryview.cast changes how the same memory is read: three C ints become twelve bytes. The memory is not touched or copied. NumPy's frombuffer does the same for arrays: the ndarray and the array share one block, so a change in one shows in the other.

05NumPy: one block, shape and strides

An ndarray is a small Python object that describes a block: a pointer to the data, a dtype, a shape and strides (how many bytes to step along each axis). A slice makes a new description over the same block, which is why writing through a slice changes the original (a view). This is the same trade as memoryview: cheap, but you must remember the data is shared.

06Where the speed comes from

a * b is one Python call. Inside, a compiled loop walks both blocks, multiplying with SIMD instructions where possible, and produces a new block, with no Python object created per element. Matrix multiplication (@) calls an optimised BLAS library. And while that C code runs, it may release the GIL, so other Python threads can run.

Where the speed comes fromdiagram
Rendering diagram…

07Wheels and compiled packages

pip install numpy normally downloads a wheel that already contains the compiled C code for your platform. Libraries such as pandas, SciPy, scikit-learn, Pydantic (its core is Rust), orjson and cryptography ship compiled wheels too. Installing in a minimal container or on a new Python version sometimes finds no wheel; then pip needs a compiler and system headers, which explains many confusing install errors (see the Docker course, /docker, for building images that install such packages).

Try it yourself

  1. 1

    Slice with and without copying

    In the first example, change memoryview(data)[2:5] to data[2:5] for view and re-run. The line that shows Xde now shows cde. Which version would you use to parse a 1 GB buffer?

  2. 2

    Change the item type

    Change the typecode "i" to "h" (2-byte ints) and to "d" (doubles) and look at itemsize and tobytes(). Try the OverflowError with a smaller number, such as 40000, for "h".

  3. 3

    Write through a NumPy view

    In the second example, change b[0] = -1.0 to b = a[1:3].copy() and then b[0] = -1.0. Is a still changed? A copy breaks the link.

Code & diagrams

array and memoryview: packed data and zero-copy windows New tab

`sys.getsizeof` numbers differ between 32-bit (this browser) and 64-bit desktops, so the example compares sizes instead of printing them.

Sign in to run this example in your browser.

Expected output

typecode: i item size: 4 bytes
raw bytes: b'\x01\x00\x00\x00\x02\x00\x00\x00\x03\x00\x00\x00\x04\x00\x00\x00'
buffer_info length: 4
list holds pointers to objects, array holds raw numbers
getsizeof(list) alone counts only the pointers, so add the int objects
array is smaller than list plus its ints: True
array storage = 1000 * itemsize: True
TypeError: 'float' object cannot be interpreted as an integer
OverflowError: Python int too large to convert to C long
copy: b'cde'
view: b'Xde'
data after writing through the view: bytearray(b'abXYefgh')
format: i itemsize: 4 shape: (3,)
as bytes: [1, 0, 0, 0, 2, 0, 0, 0, 3, 0, 0, 0]
NumPy shares memory through the buffer protocol New tab
Sign in to run this example in your browser.

Expected output

numpy sees: [1.0, 2.0, 3.0, 4.0]
shares memory with the array: True
change the array, numpy changes: [10.0, 2.0, 3.0, 4.0]
list items are separate objects: True
ndarray: one block, itemsize 8 bytes, strides (8,) shape (4,)
dtype: float64 contiguous: True
original after writing through the slice: [10.0, -1.0, 3.0, 4.0]
base is the original buffer: True
sum of 4 doubles in C: 16.0
Python object layout versus a packed blockbash

Arithmetic from the 64-bit sizes: sys.getsizeof(1.0) is 24 and a list slot is 8 bytes. The browser's 32-bit build reports smaller numbers.

# 64-bit CPython 3.12, one million floats (typical figures)
list of floats : 8 bytes pointer + 24 bytes float object = about 32 MB, scattered
numpy float64  : 8 bytes per item                        = 8 MB, contiguous
Parsing a binary log with zero-copy memoryviews New tab

Each record is a 2-byte length followed by that many bytes. Slicing the `memoryview` makes windows, not copies, and the open views block resizing until you release them.

Sign in to run this example in your browser.

Expected output

['start', 'user=ada', 'stop']
views point into the original buffer: True
a view sees the change: Start
BufferError: Existing exports of data: object cannot be re-sized
after releasing the views, resizing works: 24

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Resize a buffer while a view exists

Append to a bytearray that has a live memoryview.

main.pywhole filepython
data = bytearray(b"abcdef")
view = memoryview(data)
data.append(103)
terminal
$ python main.py
── what you'll see ──
Traceback (most recent call last):
File "main.py", line 3, in <module>
data.append(103)
~~~~~~~~~~~^^^^^
BufferError: Existing exports of data: object cannot be re-sized

Myth vs fact

Myth

NumPy is fast because it is written in C, and that is all.

Fact

Language matters, but so does layout: contiguous typed memory, compiled loops over it, SIMD and optimised libraries. A C loop over a list of Python objects would still be slow.

Myth

memoryview makes a copy of the data.

Fact

It is a window onto the original memory. Slicing it creates another window, not a copy.

Myth

A list of numbers and an array of numbers are the same, only the type differs.

Fact

A list stores pointers to separate objects. An array stores the values packed, which is smaller, cache-friendly and what compiled code needs.

Myth

Using NumPy means no Python loop anywhere.

Fact

NumPy removes the loop over elements. Your Python still loops over batches, files and epochs.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

  • ▸

    A NumPy array that is not contiguous (a strided view such as a[::2] or a transposed array) cannot always be handed to C code as one block. Some functions then make a contiguous copy. a.flags and np.ascontiguousarray are the checks to know when you debug memory or speed.

  • ▸

    The buffer protocol also carries readonly flags. np.frombuffer on bytes gives a read-only array, and a write raises ValueError. Zero-copy hand-offs between libraries (Arrow, pandas, PyTorch through torch.from_numpy, DLPack) all rely on these rules, and mutating shared data from two libraries at once is a common source of subtle bugs.

  • ▸

    The buffer protocol is defined in PEP 3118 and describes memory with a pointer, a length, an item format, a shape and strides, so multi-dimensional and non-contiguous layouts can be shared too. Since Python 3.12 (PEP 688) a class written in Python can implement __buffer__ and __release_buffer__, and collections.abc.Buffer is the type to use in annotations. memoryview.cast only converts between a byte format (B, b, c) and another format, not directly between two non-byte formats. pickle protocol 5 (PEP 574) uses the same idea to send large arrays out of band without copying them into the pickle stream.

  • ▸

    Native code can release the GIL only when it does not touch Python objects, which is why NumPy, hashlib, zlib and file I/O run in parallel across threads while a pure-Python loop does not. On the free-threaded build (Topic 12.2) extension modules must declare that they are safe to run without the GIL.

  • ▸

    The stable ABI (the fixed binary interface of the C API; abi3 wheels) lets one compiled wheel work across several Python versions, and tools such as cibuildwheel build wheels for many platforms in CI (see the CI/CD course at /cicd). Maintaining compiled wheels for every platform is the real cost of native extensions.

Remember this

  1. 1

    (Recall Topic 13.1: every Python value is a full object.) On a 64-bit CPython a float object takes 24 bytes (a reference count, a type pointer and the 8-byte value), plus an 8-byte pointer in the list, so a million floats cost about 32 MB and are scattered in memory. A NumPy float64 array of a million items takes exactly 8 MB: only the raw numbers, packed. The browser here is a 32-bit build and shows smaller numbers, so the examples compare sizes rather than print them. Less memory means more data fits in the CPU cache, which is a large part of the speed.

  2. 2

    The **array module** is the standard-library version of this idea: array('i', [1, 2, 3]) stores C ints packed side by side and checks the type on every write, so putting 3.5 in an 'i' array raises TypeError, and a value that does not fit raises OverflowError. It has few operations, but it shows the layout and is handy for binary data.

  3. 3

    The buffer protocol is how objects share memory without copying. An object such as bytes, bytearray, array or a NumPy array can expose its raw memory to others. memoryview(obj) gives a view over that memory: slicing a memoryview makes a new view onto the same bytes (no copy), while slicing bytes copies. Writing through a view changes the original. view.cast("B") or cast("H") reinterprets the same memory as a different item type, and np.frombuffer(buf) makes a NumPy array that shares the memory. This is how NumPy, pandas, image libraries, sockets and PyTorch hand data to each other with zero copies.

  4. 4

    NumPy is fast because of three things together: contiguous typed memory (the ndarray knows its dtype, shape and strides), loops written in compiled C or C++ that run over that memory without creating Python objects, and SIMD CPU instructions plus optimised libraries (BLAS, a long-tuned maths library, for matrix multiplication). One Python call such as a * b runs a loop over a million items inside C. Python only dispatches.

  5. 5

    Compiled code can run outside the interpreter's rules. C extension code can release the GIL (Topic 12.2) while it works on a buffer, so threads running NumPy, hashing, compression or I/O can use several cores at once. The extension API is the C API (Python.h): functions to create objects, call back into Python and manage reference counts. Tools such as Cython, Numba, PyBind11, PyO3 (Rust) and cffi are friendlier ways to write extensions.

  6. 6

    Compiled packages reach you as wheels: ready-built binary files (.whl) for your operating system, CPU and Python version, so pip install numpy needs no compiler. If no wheel exists for your platform, pip builds from source and needs a compiler, which is the usual reason an install fails. Phase 14 builds on all of this: Topic 14.1 — NumPy Arrays shows dtype, shape and views in depth, and Topic 14.3 — Vectorisation and Broadcasting uses the loops-in-C idea.

Explain it without notes

01

Give three reasons a NumPy operation is faster than the equivalent Python loop.

02

What is the buffer protocol, and what does memoryview do with it?

03

Why does a Python list of a million floats use so much more memory than a NumPy array?

04

How can native code make multi-threaded Python faster, if the GIL exists?

Practice

01

Create an array('h', [1, 2, 3]) and print its item size, the number of bytes in tobytes() and its sum.

02

Take a bytes object b"abcdef", build a memoryview, print the bytes of the slice [1:3], whether view.obj is the original object and the length of the view.

03

Show that memoryview(b"\x01\x00\x02\x00").cast("H") reads the same four bytes as two 16-bit numbers.

Trade-offs

  • ↔

    Zero-copy sharing versus safety. Views avoid copies and save memory, but two names can change the same bytes, and a view blocks resizing. Copy when you need independence.

  • ↔

    Compiled extensions versus portability. Native code gives speed, but you must build and ship wheels for each platform and Python version. Pure Python runs everywhere.

  • ↔

    Typed arrays versus flexible lists. An array holds one type and fixed-size items, which is what makes it fast; a list holds anything and is slower.

Done when you can

  • Done when you can explain the memory layout of a list of floats versus a NumPy array.

  • Done when you can use array and memoryview and say which operations copy and which do not.

  • Done when you can explain the buffer protocol and why libraries can share data with no copy.

  • Done when you can name three reasons NumPy is fast and where the GIL is released.

  • Done when you can say what a wheel is and why a missing wheel makes pip install try to compile.