Vectors and Matrices: The Maths AI Runs On

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Vector

A list of numbers, which you can think of as a point or an arrow in space.

Activation function

A simple function applied to each output of a layer. ReLU turns negatives into 0.

Gradient

The numbers that say which way, and how much, to nudge each weight to improve a model.

Embedding

A vector that represents the meaning of something, such as a word, sentence or image.

Dot product

Multiply matching positions of two vectors and add everything up: one number.

Norm

The length of a vector.

Unit vector

A vector of length 1.

Cosine similarity

How closely two vectors point in the same direction: from -1 to 1.

Matrix multiplication

Combine every row of one matrix with every column of another using dot products. Written A @ B.

Transpose

Swap rows and columns. Written A.T.

Weights and bias

The numbers a layer learns: W is multiplied with the inputs and b is added.

Top-k

The k items with the highest scores.

1. 1

   Find the closest to dog

   In the search example, set the query to the dog row (index 2) and print the top three. Does puppy come second, as you would expect?

2. 2

   Dot product of unit vectors

   Check that unit[0] @ unit[1] equals the cosine from the first line. Why do they match?

3. 3

   Break the shape rule

   In the layer example change W to shape (4, 2) and read the error. What must W's first dimension be equal to?

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

import numpy as np

X = np.ones((4, 3))
W = np.ones((3, 2))
print((X * W).shape)

1. 1

   Recall from Topic 14.1: a vector is a 1-D array (one row of numbers) and a matrix is a 2-D array (rows and columns). You can also picture a vector as an arrow from the starting corner of a map to a point. The numbers say how far to go in each direction.

2. 2

   The dot product of two vectors multiplies matching positions and adds the results: [3, 4] . [1, 2] = 3*1 + 4*2 = 11. The everyday version is a shopping bill. Put the prices in one vector and the quantities in another. Multiply each price by its quantity, add up the line totals, and you have the bill. In NumPy write a @ b (or np.dot(a, b)). Notice a * b is something else: element-wise multiplication that gives another vector (the line totals).

3. 3

   The norm (length) of a vector is np.linalg.norm(v). For [3.0, 4.0] it is 5.0: walk 3 steps east and 4 steps north and the straight-line distance from the start is 5 (the longest side of a 3-4-5 right triangle). Dividing a vector by its norm gives a unit vector (length 1) that keeps the direction. Normalising embeddings to unit length makes comparisons simpler.

4. 4

   Cosine similarity measures how closely two vectors point in the same direction, ignoring their length: a @ b / (norm(a) * norm(b)). It is 1 for the same direction, 0 for unrelated (at right angles) and -1 for opposite. For unit vectors it is just the dot product. Search engines for text, images and chatbot memory use it to find the items whose embeddings are nearest to a question's embedding.

5. 5

   A top-k search is then three lines: normalise the rows of the embedding matrix, compute unit @ query to get one score per item, and take np.argsort(-scores)[:k] for the best k. This is what a vector database does, with clever indexes to avoid scoring every row. In Topic 16.8 you will use embeddings from a real model with exactly this code.

6. 6

   Matrix multiplication A @ B combines rows of A with columns of B using dot products. The shapes must fit: (n, k) @ (k, m) gives (n, m). The inner sizes k must match, and they vanish. It is not element-wise, and A @ B differs from B @ A. The transpose A.T swaps rows and columns, and np.eye(n) is the identity matrix, which leaves a matrix unchanged under @.

7. 7

   A neural-network layer is Z = X @ W + b. X has shape (batch, inputs), one row per example. W has shape (inputs, outputs), the learned weights. b has shape (outputs,), the learned bias (a number added to each output), broadcast across the batch (Topic 14.3). Z has shape (batch, outputs). An activation function such as ReLU (np.maximum(Z, 0), which turns every negative number into 0) is then applied element-wise. A deep network is many of these layers stacked. Training means adjusting W and b so the outputs get closer to the right answers.

8. 8

   NumPy runs these on the CPU, in compiled code that uses optimised linear-algebra libraries. Frameworks such as PyTorch use the same ideas on GPUs (graphics chips that do huge amounts of arithmetic at once), with automatic gradients (the numbers that say which way to nudge each weight) for training. If you can read shapes, you can read their code.

• ↔

  Exact versus approximate search. Scoring every row is exact but slow for millions of items; indexes are fast but approximate.

• ↔

  Cosine versus dot product versus distance. Cosine ignores length; the dot product rewards length; Euclidean distance suits some embeddings. Use what the model's authors recommend.

• ↔

  Precision versus memory. float32 embeddings halve storage compared with float64, with negligible effect on ranking.

• Done when you can compute dot products and norms and explain them.

• Done when you can normalise vectors and compute cosine similarity by hand and with NumPy.

• Done when you can write a top-k search with @ and argsort.

• Done when you can state the shape rule for @ and tell @ from *.

• Done when you can describe X @ W + b and give the shape of every piece.