A toy embedder: text to four numbers, ranked by cosine similarity
A toy: its four directions and its word table were written by hand. A real model has hundreds of unnamed directions learned from data. Real numbers are in the lab below.
An embedding model turns a text into a list of numbers so that texts with similar meaning get similar numbers. This topic runs a real one in your browser on the question that word matching got wrong, and explains how every example and lab in the course works.
Change the code and press Run (Ctrl+Enter). Predict what will happen first, then break it on purpose and read what changes. Your edits are saved and match the lesson page.
Ready for more? This phase has 5 hands-on exercises with automatic checks.
Practice questions
Write the code in the editor, run it, then open the model answer to compare.
Write cosine(a, b) for vectors that are not already length 1 (divide by both lengths). Print the similarity of [3, 0, 1, 0] with [1, 0, 0, 0] and with [0, 2, 0, 0], to three decimals.
Using the toy idea, write embed(text, meaning, size) and rank three short texts for the question "is it open on sunday" with the table {"open": 0, "sunday": 0, "hours": 0, "price": 1, "costs": 1}.
Explain it without notes
What is an embedding, and what property makes it useful for search?
Why do the runnable examples use a toy embedder while the labs use a real model?
In the lab, the best passage for the login question scores about 0.47. A colleague says "that's below 50 percent, so it's a bad match". What do you tell them?
The scooter question gets a best score of about 0.38 although nothing in the handbook is about scooters. Why, and what does that mean for building a product?
Expected output
Directions: account, money, time, safety
Question: I forgot my login, how do I get back in?
vector: [1.000, 0.000, 0.000, 0.000]
0.962 D3
0.000 D1
0.000 D2
0.000 D4
0.000 D8
Question: When do you close?
vector: [0.000, 0.000, 1.000, 0.000]
1.000 D2
0.447 D1
0.316 D8
0.275 D3
0.000 D4