PHASE 8Intermediate ~26 min· topic 3 of 8

Topic 8.3

Vector Search by Hand

In one line

An embedding turns text into a list of numbers so that texts with similar meaning get similar lists. Cosine similarity measures how close two lists point, and top-k returns the best few. Here the embedder is a small fake so every number can be checked by hand.

Think of it like this

A librarian who knows what you mean. You say "I want my money back" and she walks you to the returns desk, even though you never said "refund". She does not match letters; she matches ideas. A vector search does the same: it compares meanings, not spelling.

Words you'll meet

New words in this topic, in plain English. Come back here whenever one feels fuzzy.

Embedding: A list of numbers that represents a text, built so that similar meanings give similar lists.
Vector: An ordered list of numbers. An embedding is a vector.
Dimension: One position in the vector. Real models use hundreds or thousands; the fake here uses 9.
Normalise: Scale a vector to length 1 by dividing each number by the vector's length.
Cosine similarity: How closely two vectors point the same way: 1 is identical direction, 0 is nothing in common.
Top-k: The k highest-scoring results, best first.
Nearest neighbour: The stored vector most similar to a query vector.
Embedding model: A learned model that turns text into a vector. Documents and questions must use the same one.

Step by step

01Step 1: the problem keyword search cannot solve

Search the eight shop documents for "get my money back" with the Topic 8.2 index and you get nothing. The returns document is the right answer, but it says "refund".

02Step 2: put meaning on a map

Give each concept a dimension. The fake embedder counts how many words of a text fall into each concept, then normalises. "get my money back" has two refund words, so it becomes 1.0 on refund and 0 elsewhere, exactly like the sentence "full refund".

embed.pywhole filepython

# 9 dimensions: refund, shipping, warranty, repair, hours, helmet, payment, battery, cost
# "get my money back"           -> money, back      -> refund 2 -> [1, 0, 0, 0, 0, 0, 0, 0, 0]
# "how much does delivery cost" -> delivery, much, cost -> shipping 1, cost 2 -> normalised

Step 2: put meaning on a mapdiagram

Rendering diagram…

03Step 3: compare vectors with cosine

For two vectors multiply matching numbers, add them up, divide by the two lengths. The second example embeds all eight documents once, then compares each question against all of them.

04Step 4: sort, break ties, keep the top k

Round scores to four decimals, sort by score (best first) then by document number, keep the first k, and drop zeros. The output shows a real tie (shipping and helmets at 0.7071) and the last query, "tell me a joke", which contains no concept word at all.

05Step 5: swap in a real embedding model

With a real model the only change is embed(): it calls an embeddings endpoint and returns hundreds of floats. Everything after it (cosine, sort, top-k) stays the same. The one rule: documents and questions must be embedded with the same model.

Try it yourself

1
Predict the cosine
Before running the second example, predict the score of the payment document for "how much does it cost". The payment document has 4 payment-word hits and 1 cost-word hit, so its vector is 4/√17 and 1/√17. The query is all cost.
2
Add a synonym
Add reimburse to the refund concept (in every language you try). Search for "reimburse me". What changed in the vectors, and what does a real model do instead of a word list?
3
Make a tie vanish
Edit the helmets document so it says "Extra helmets cost 25 dollars, 30 for kids." Does the tie between shipping and helmets for "how much does it cost" survive? Why or why not?

Code & diagrams

A fake embedder: text to numbers New tab

A hand-made, deterministic stand-in so every number can be checked. Real embedding models are learned and far better.

Expected output

dimensions: refund, shipping, warranty, repair, hours, helmet, payment, battery, cost
get my money back
  [1.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000]
  length 1.0000
how much does delivery cost
  [0.0000, 0.4472, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.8944]
  length 1.0000
tell me a joke
  [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000]
  length 0.0000

Cosine similarity, top-k and a tie New tab

Scores are rounded to four decimals before sorting. Equal scores keep document order.

Expected output

query: get my money back
  1.0000  returns
query: how much does it cost
  0.7071  shipping
  0.7071  helmets
  0.2425  payment
query: what time do you open
  1.0000  hours
  0.3162  battery
query: tell me a joke
  no concept words, nothing to compare

The same step with a real embedding model (over HTTP)python

Not run here: it needs a network call and your own key. The request shape (POST /embeddings with model and input, and a data array of embeddings in the reply) is the OpenAI-style one many providers copy. The lab models on this site are chat models, so the labs cannot embed.

# pip install httpx
# Embeddings over plain HTTP. Many providers expose the OpenAI-style POST /embeddings endpoint.
# Groq's free tier (used by the labs) offers chat models only, so use a provider that sells embeddings.
import math
import os

import httpx

BASE_URL = os.environ.get("EMBEDDINGS_BASE_URL", "https://api.openai.com/v1")
MODEL = os.environ["EMBEDDING_MODEL"]  # an embedding model name from your provider's docs


def embed_many(texts):
    r = httpx.post(
        BASE_URL + "/embeddings",
        headers={"Authorization": "Bearer " + os.environ["EMBEDDINGS_API_KEY"]},
        json={"model": MODEL, "input": texts},
        timeout=30,
    )
    r.raise_for_status()
    data = sorted(r.json()["data"], key=lambda item: item["index"])
    return [item["embedding"] for item in data]  # one list of floats per input text


def cosine(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    return dot / (math.sqrt(sum(x * x for x in a)) * math.sqrt(sum(y * y for y in b)))


# Embed the documents once and store the vectors. Embed each question at query time, with the SAME model.

Break it on purpose

Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.

Break #1

Skip the normalising

In the second example, make embed return the raw counts (replace its last line with return vec) and make cosine return just dot.

terminal

$ Run

── what you'll see ──

query: get my money back

4.0000 returns

query: how much does it cost

4.0000 shipping

4.0000 helmets

2.0000 payment

query: what time do you open

5.0000 hours

1.0000 battery

query: tell me a joke

no concept words, nothing to compare

Myth vs fact

Myth

Embeddings understand the text like a person.

Fact

They place text on a map learned from patterns in data. Nearby usually means related, which is useful, but they can place a wrong document close and cannot tell you why.

Myth

A high cosine score means the document answers the question.

Fact

It means the two texts are about similar things. The battery document scores 0.3162 for an opening-hours question. Score thresholds, citations and refusals (Topic 8.6) guard against this.

Myth

Vector search replaces keyword search.

Fact

Each finds things the other misses: vectors catch paraphrases, keywords catch exact names and codes. Production systems often use both (hybrid search, covered in the coming RAG course).

Pro corner

Extra depth for experienced readers. New to this? Skip it for now and come back later.

▸
Embedding endpoints take many texts per request (the OpenAI-style one accepts an array of up to 2,048 inputs, each within the model's token limit), so embed documents in batches rather than one call per chunk, and match results by each item's index, as the real-model code above does.
▸
Documents and queries must use the same embedding model and the same preprocessing. Mixing models, or upgrading the model without re-embedding the documents, silently ruins results because the two maps no longer line up.
▸
Store the model name next to every stored vector and re-embed everything when you change it. Embedding cost is usually small compared with chat calls, but it is a real line item for large collections.
▸
For vectors of length 1, cosine similarity equals the dot product and gives the same ranking as Euclidean distance, so databases offer whichever is fastest. For vectors that are not normalised, the three can rank differently.
▸
Top-k with ties is a determinism question: sort by (score, stable id). Exact float ties are rare with real embeddings, but near-ties are common and a rounded score plus an id makes tests stable.

Remember this

1
Why this exists. Keyword search (Topic 8.2) fails whenever the question uses different words from the document: "get my money back" shares no word with "full refund". People paraphrase all the time, so retrieval needs to compare meaning. The trick is to turn text into numbers in such a way that similar meanings land close together.
2
An embedding is a list of numbers (a vector) that stands for a text. Think of coordinates on a map: two cafes on the same street have nearby coordinates. An embedding model places texts on a map with hundreds or thousands of directions (dimensions), so that "refund" and "money back" land near each other and "warranty" lands elsewhere (Topic 1.3 explained how such models are learned).
3
The embedder in this topic is a fake, on purpose. It has 9 dimensions, one per concept (refund, shipping, warranty, repair, hours, helmet, payment, battery, cost), and a short word list for each. To embed a text, count how many of its words belong to each concept, then scale the list to length 1 (normalise it: divide every number by the list's length, the square root of the sum of squares). It is deterministic and prints identical numbers in all three languages. Real embedding models are learned from data and far better: they handle any word, word order, context and many languages, with no hand-made word lists. This fake only makes the mechanics visible.
4
Cosine similarity compares directions. For two vectors, cosine = (sum of products of matching numbers) divided by (length of the first times length of the second). It is 1 when they point the same way, 0 when they share nothing, and (for these non-negative vectors) never below 0. Because our vectors already have length 1, cosine equals the plain dot product. Check one by hand: the query "how much does delivery cost" has one delivery word and two cost words (much, cost), so it is 0.4472 on shipping and 0.8944 on cost. The shipping document is 0.7071 on both, so cosine = 0.4472 × 0.7071 + 0.8944 × 0.7071 = 0.3162 + 0.6325 = 0.9487.
5
Top-k means keep the k best. Score every document, sort best first, take the first k (for example 3). Always add a tie-breaker: for the question "how much does it cost", the shipping and helmets documents score exactly 0.7071 each. Without a rule their order could change; the example sorts by (score descending, document number ascending), so shipping always comes before helmets. Also drop scores of 0: a document that shares nothing is not a result.
6
What vector search gets wrong. It always returns the nearest things, even when nothing is near, so you need a minimum score (Topic 8.6). Close in meaning is not the same as answering: "what time do you open" also scores the battery document (0.3162) because it says "4 hours" and the fake maps hours to the opening-hours concept. Real models make fewer mistakes of this kind, not none. And exact things (product codes, names, error numbers) are often found better by keywords, so many systems use both.

Explain it without notes

What is an embedding and why does it help find a document that shares no words with the question?

Compute the cosine similarity of [1, 0] and [0.6, 0.8], and explain what the number means.

Why must the documents and the question be embedded with the same model?

Why is a tie-breaker needed in top-k, and what would you choose?

Practice

Write normalise(vec) that scales a list of numbers to length 1 (return the list unchanged if its length is 0). Print the result for [3, 4] with 4 decimals.

Write cosine(a, b) and print it to 4 decimals for [1, 1, 0] and [1, 0, 0], and for [1, 0] and [0, 1].

Trade-offs

↔
Vector search finds paraphrases but is fuzzier and harder to explain than keywords.
↔
More dimensions capture more meaning but cost more memory and time per comparison.
↔
A strict minimum score avoids irrelevant results but may refuse questions that were answerable.

Done when you can

Done when you can explain what an embedding is without maths.
Done when you can normalise a vector and compute cosine similarity by hand.
Done when you can implement top-k with a stable tie-breaker.
Done when you can say why a fake embedder differs from a real one and why documents and queries need the same model.

Command Palette

Words you'll meet

Step by step

01Step 1: the problem keyword search cannot solve

02Step 2: put meaning on a map

03Step 3: compare vectors with cosine

04Step 4: sort, break ties, keep the top k

05Step 5: swap in a real embedding model

Try it yourself

Predict the cosine

Add a synonym

Make a tie vanish

Code & diagrams

Break it on purpose

Skip the normalising

Myth vs fact

Pro corner

Remember this

Explain it without notes

Practice

Trade-offs

Done when you can