A neural-network layer is X @ W + b
Change the shape of W to (3, 5) and watch the output shape and bias both need to become 5 long.
A vector is a list of numbers that can describe anything from a point on a map to the meaning of a sentence. The dot product, norm, cosine similarity and matrix multiplication (@) are the four operations behind embeddings, search and every neural-network layer.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 6 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write normalise(v) that returns v divided by its length. Test it on [3.0, 4.0].
Given A of shape (2, 3) and B of shape (3, 4), say what A @ B shape is and confirm it with np.ones.
Compute the cosine similarity between [1.0, 0.0] and [1.0, 1.0].
Given three embeddings as the rows of emb = np.array([[1.0, 0.0], [1.0, 1.0], [0.0, 1.0]]), compute the whole 3-by-3 table of cosine similarities between every pair with one matrix product, rounded to 3 places. Hint: unit @ unit.T.
Explain it without notes
What is an embedding and why is cosine similarity a useful way to compare two of them?
State the shape rule for A @ B and the shape of the result. How is it different from A * B?
Explain Z = X @ W + b with shapes for a layer that maps 128 inputs to 64 outputs on a batch of 32.
How do you find the top 3 most similar items to a query with NumPy?
When are cosine similarity, dot product and Euclidean distance equivalent for ranking, and when do they disagree?
What can go wrong when you normalise vectors, and how do you guard against it?
Expected output
X (4, 3) W (3, 2) b (2,) -> Z (4, 2)
after ReLU, negatives become 0: True
first row Z: [ 0.024 -0.633]
first row A: [0.024 0. ]