PHASE 16Advanced ~27 min· topic 8 of 14

Topic 16.8

Embeddings and Similarity Search with NumPy

In one line

An embedding turns text into a vector so that texts with similar meaning end up close together. With NumPy you can score a query against every stored vector with one matrix multiplication and take the top results, which is the heart of semantic search.

Think of it like this

A map of a city. Every shop is a dot with two numbers (how far east, how far north). Shops that are close on the map are close in real life. An embedding is a map for text, with hundreds or thousands of directions instead of two: sentences that mean similar things land near each other, so "where is the nearest cafe?" can find "coffee shop opening hours" even though they share no words.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Embedding: A vector of numbers that represents a piece of text, so similar meanings are close.
Vector: A list of numbers, viewed as a point or an arrow in space.
Dimension: How many numbers are in the vector.
Cosine similarity: A score from -1 to 1 for how much two vectors point the same way.
Normalise: Scale a vector to length 1 without changing its direction.
Top-k: The k best-scoring items.
Vector database: A database built to store vectors and quickly find the nearest ones.

Step by step

01Text to numbers: a toy embedding

A toy is used here because a real embedding model is a big trained network that needs a service or a large download, and you could not see inside it anyway. The first example counts how often each of nine known words appears and divides by the vector's length. The result is a length-1 vector with nine dimensions. It is not a learned embedding; it only shows what you do with one.

main.pywhole filepython

def embed(text):
    words = re.findall(r"[a-z]+", text.lower())
    vec = np.array([words.count(w) for w in VOCAB], dtype=float)
    length = np.linalg.norm(vec)
    return vec / length if length else vec

02Cosine similarity step by step

Take two vectors a and b. Multiply matching entries and add (the dot product). Divide by the length of a times the length of b. If both are already length 1, you can skip the division. A zero vector has no direction, so guard against dividing by zero, as the example does.

03Score every document at once

Stack the document vectors as rows: a matrix of shape (documents, dimensions). One multiplication, matrix @ q, gives all scores. Compare this with a Python loop: same result, but the loop is far slower once the numbers grow.

Score every document at oncediagram

Rendering diagram…

04Top-k with stable ties

np.argsort(-scores, kind="stable") sorts from highest to lowest and keeps the original order for equal scores. Without a stable rule, two documents with the same score could swap places between runs or machines, which makes tests flaky.

05What the toy cannot do

Look at the last line of the first example's output: two phrases that mean nearly the same score 0.000, because they share no vocabulary. This is the gap real embedding models close. In real use you call an embedding API or a local model for every document once (and store the vectors) and for every query at search time.

real_embeddings.pywhole filepython

# OpenAI-compatible embeddings endpoint (not runnable here)
result = client.embeddings.create(model="your-embedding-model", input=["boil water", "simmer liquid"])
vectors = [item.embedding for item in result.data]   # list of lists of floats

06What a vector database adds

The third example wraps the matrix in a small class with ids and metadata, and normalises on the way in. A real vector store adds disk storage, filters, approximate indexes and scale. Understanding this small version tells you what to ask of any bigger one.

Try it yourself

1
Teach the toy a word
Add "simmer" and "liquid" to VOCAB in the first example. Does the synonym test change? Why is it still not "understanding"?
2
Change k
In the second example set k = 4 and run. Do the scores decrease from rank 1 to 4? Which documents score exactly zero?

Code & diagrams

Toy embeddings and cosine similarity New tab

The synonym test scores 0 because the toy only knows exact words. Real embeddings would not.

Expected output

0.000  python list and loop basics
0.866  cook pasta: boil water first
0.000  a vector model in python
synonym test: 0.0

A document matrix and top-k in one multiplication New tab

Expected output

matrix shape: (4, 9) (documents x dimensions)
row lengths: [1.0, 1.0, 1.0, 1.0]
top 1: doc 3 score 1.000 -> boil water and boil more water
top 2: doc 1 score 0.707 -> cook pasta: boil water first

A tiny vector store New tab

Expected output

0.981 {'id': 'a', 'source': 'guide.md'}
0.832 {'id': 'c', 'source': 'guide.md'}

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Compare vectors from different models

Take the dot product of two vectors that have different lengths.

main.pywhole filepython

import numpy as np

query = np.array([0.1, 0.9, 0.2])           # from one embedding model (3 numbers)
document = np.array([0.3, 0.1, 0.5, 0.7])   # from ANOTHER model (4 numbers)
print(np.dot(query, document))

terminal

$ python main.py

── what you'll see ──

Traceback (most recent call last):

File "main.py", line 5, in <module>

print(np.dot(query, document))

~~~~~~^^^^^^^^^^^^^^^^^

ValueError: shapes (3,) and (4,) not aligned: 3 (dim 0) != 4 (dim 0)

Myth vs fact

Myth

Embeddings understand text like a person.

Fact

They capture patterns of how words are used together in training text. They are useful for finding related text, and they can still miss negation, rare names and exact numbers.

Myth

Higher cosine similarity always means a correct answer.

Fact

It means similar by the model's measure. The most similar chunk may still not contain the answer, which is why RAG systems check and cite (Topic 16.9).

Myth

You need a vector database for any semantic search.

Fact

For thousands of vectors a NumPy matrix is enough. Move to a database when size, filtering, updates or concurrency demand it.

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
Exact search over N vectors costs O(N·d) per query. Approximate indexes (graph-based such as HNSW, or clustering-based) trade a little recall for much lower latency. Always measure recall on your own data.
▸
Normalise once at write time. Some embedding APIs return unit-length vectors already; check your provider. Mixing normalised and unnormalised vectors gives rankings that depend on vector length.
▸
Float32 is usually enough for embeddings and halves the memory of float64. With NumPy in the browser, small integer dtypes are 32-bit by default (Topic 14.1), but floats are 64-bit, as in these examples.
▸
For unit-length vectors the three common measures give the same ranking: cosine similarity equals the dot product, and the squared Euclidean distance equals 2 - 2 × cosine, so smaller distance means higher similarity. That is why many indexes accept any of them once you normalise. Cosine distance is usually defined as 1 - cosine similarity. For unnormalised vectors the measures disagree, and some embedding models are trained for a specific one, so follow the model's documentation.
▸
Some embedding models expect different treatment for queries and documents (for example a prefix or an input-type setting), and long inputs are truncated at the model's input limit. A silent truncation means the tail of a long chunk is never searchable, so check the limit and chunk below it.

Remember this

1
An embedding is a vector: a list of numbers (Topic 14.5). A trained embedding model reads text and returns a vector of fixed length, for example a few hundred to a few thousand numbers (the length depends on the model). You get it from an API or a local model, usually in one function call that takes strings and returns arrays.
2
To compare two vectors, use cosine similarity: the cosine of the angle between them, a number from -1 to 1. Close to 1 means pointing the same way (similar meaning), near 0 unrelated, negative opposite. It ignores length and cares only about direction. Recall from Topic 14.5: the dot product multiplies matching numbers and adds them up, and a vector's length is the square root of the sum of its squared numbers. The cosine formula is the dot product divided by the product of the two lengths.
3
Normalise vectors to length 1 once, when you store them. Then cosine similarity is just the dot product, and scoring a query against N stored vectors is one matrix multiplication, matrix @ query, with shapes (N, d) times (d,) giving N scores. NumPy does this in compiled code, so thousands of vectors are instant.
4
Top-k search takes the k highest scores: np.argsort(-scores)[:k]. For very large N you can use np.argpartition to avoid a full sort. Sorting ties needs a stable rule (kind="stable"), or the order can vary between runs and break your tests.
5
The example uses a toy embedding that counts a fixed list of words. It teaches the mechanics, but it has a big weakness you can see in its output: "boil water" and "simmer liquid" share no known words, so the score is 0. A real embedding model, trained on huge amounts of text, places them close together. That meaning-awareness is what you are paying for.
6
A vector database adds what raw NumPy lacks once the data grows: persistence on disk, approximate nearest-neighbour indexes that avoid scoring every vector, metadata filters ("only documents from this source"), updates and deletes, and concurrent access. For a few thousand documents, a NumPy matrix is often enough. A full Hectal course on RAG and agents is planned to cover vector stores properly.

Explain it without notes

What is an embedding and why is cosine similarity a good way to compare two of them?

Why normalise vectors, and how does that make search a single matrix multiplication?

What does a vector database add over a NumPy matrix?

When do cosine similarity, dot product and Euclidean distance rank results identically, and what happens when you change the embedding model?

Practice

Write cosine(a, b) that returns 0.0 when either vector is all zeros.

Write top_k(query, matrix, k) that returns the indices of the k best rows by dot product, with ties broken by lower index.

Write normalise_rows(matrix) that scales each row to length 1 and leaves all-zero rows as zeros.

Trade-offs

↔
Exact versus approximate search. Exact is simple and always right but slows with data size. Approximate indexes are fast at scale and may miss a true neighbour.
↔
Small versus large embedding dimension. Larger vectors can hold more nuance but cost more memory, storage and compute. Choose by measuring retrieval quality on your own questions.

Done when you can

Done when you can explain what an embedding is without using the word "magic".
Done when you can compute cosine similarity by hand with NumPy, guarding zero vectors.
Done when you can score a query against a matrix and return the top k with stable ties.
Done when you can say what a vector database adds.
Done when you know a full Hectal course on RAG and agents is planned for embeddings and retrieval at scale.

Command Palette

Words you'll meet

Step by step

01Text to numbers: a toy embedding

02Cosine similarity step by step

03Score every document at once

04Top-k with stable ties

05What the toy cannot do

06What a vector database adds

Try it yourself

Teach the toy a word

Change k

Code & diagrams

Break it on purpose

Compare vectors from different models

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can