Topic 5.7
Sizing and Cost
In one line
How much memory and disk a vector store needs follows from three numbers you control: how many vectors, how many numbers each has, and how many bytes a number takes. Add the index, the copies and some headroom, and you can estimate before you buy.
Think of it like this
Before buying a cupboard for your books, you count the books and measure how thick they are. A thousand paperbacks need a short shelf, a million need a warehouse, and picture books need more space than their number suggests. Sizing a vector store is counting books and measuring thickness.
Words you'll meet
New words in this topic, in plain English. Come back here whenever one feels fuzzy.
- Byte
- The unit of storage. A 4-byte number takes 4 bytes.
- RAM (memory)
- Fast working storage. Much faster than disk, and more expensive per gigabyte.
- Index overhead
- Extra space an index needs on top of the vectors.
- Replica
- A full copy of the data, for availability or more read capacity.
- Shard
- One part of the data, kept on one machine, so no machine holds everything.
- Headroom
- Spare capacity kept for peaks, rebuilds and other work.
- Half precision
- Storing each number in 2 bytes instead of 4, with fewer digits of accuracy.
Step by step
01Step 1: count the vectors
One vector per chunk. 10,000 documents of 5 pages with about 4 chunks a page is 200,000 chunks, 200,000 vectors. Count chunks after you have settled chunking (Topic 4.4), not documents.
02Step 2: pick the dimensions and the byte size
The embedding model decides the dimensions: the lab model in this course gives 384. Most stores keep 4 bytes per number; some offer 2 or 1.
03Step 3: multiply
n x (bytes x dims + 8). The example prints 7.7 MB for 5,000 chunks of 384 numbers and 1,544.0 MB for a million. The handbook chatbot fits in the memory of a phone; a million chunks fit in a small server.
04Step 4: add the index and the copies
Add the (assumed) index overhead, then multiply by the number of copies. Compare with the memory of the machine you can afford, and keep half free as headroom.
05Step 5: try the levers
Row 3 to row 4: the same million 1536-number vectors, stored in 2 bytes a number, need 3,080 MB of vectors instead of 6,152 MB. Row 2 versus row 3: a model with four times the dimensions needs about four times the memory.
06Step 6: check with a real sample
Load 10,000 real vectors into your real store, build the index, and read the table and index sizes from the database (in PostgreSQL, pg_total_relation_size). Multiply up. The measured value replaces every assumption in this topic.
Try it yourself
- 1
Size your own project
Add a scenario for your own numbers: documents, chunks per document, the dimensions of the model you chose. Does it fit in 16,000 MB?
- 2
Halve the dimensions
Change the 1536 row to 768 and read the new total. By what factor did it change, and why is it not exactly a half?
- 3
Raise the link assumption
Change
LINKS_PER_VECTORto 64 (a larger M). Which scenario's yes/no changes, if any? - 4
Add a price
Add
PRICE_PER_GB_MONTH = 0.0 # fill in from your providerand print the monthly memory cost per scenario. Leave it a variable: never trust a hard-coded price.
Code & diagrams
The 8 bytes per row come from pgvector's documentation. The 32 links of 4 bytes are an assumption for HNSW, stated in the code, so check them against your own store.
Expected output
scenario vectors MB index MB total MB fits in 16000 MB
5 thousand chunks, 384 7.7 0.6 8.4 yes
1 million chunks, 384 1544.0 128.0 1672.0 yes
1 million chunks, 1536 6152.0 128.0 6280.0 yes
same, 2 bytes a number 3080.0 128.0 3208.0 yes
100 million, 384, 3 copies 154400.0 12800.0 501600.0 noBreak it on purpose
Errors are the best teachers. Make each change, read the error, guess what went wrong, then reveal the answer.
Break #1
Forget the copies
In the Python tab, change total = (vectors_mb + index_mb) * copies to total = vectors_mb + index_mb. Run.
Myth vs fact
Myth
Vector storage is cheap because numbers are small.
Fact
Numbers are small, but there are many: a million 1536-number vectors is over 6 GB before the index.
Myth
Disk size is the only number that matters.
Fact
Search speed depends on what fits in memory. A store that spills to disk may be slow.
Myth
More dimensions always means better search.
Fact
Not necessarily. Size grows with dimensions; accuracy depends on the model. Compare models on your own questions (Phase 9).
Pro corner
Extra depth for experienced readers. New to this? Skip it for now and come back later.
- ▸
Memory is dominated by
n x d. If the estimate is too big, the cheapest lever is almost alwaysn: de-duplicate, drop boilerplate, merge tiny chunks. Dimension reduction and quantisation are the second lever, and they cost recall. - ▸
Index build memory can exceed index size. Build on a larger machine or with the store's documented memory setting, and test the rebuild path before you need it in an emergency.
- ▸
Capacity for queries per second is a separate calculation: measure the queries per second one copy sustains at your recall target, then add copies. Never convert between storage size and query capacity.
Remember this
- 1
Why this topic exists. Vector stores are usually fast because they keep most of what they need in memory (RAM). A store that does not fit in memory gets much slower, or fails. You can estimate the size on paper, long before you have the data, and that estimate decides which machine, which store and which design (smaller vectors, fewer chunks) you can afford.
- 2
The formula. One vector of
dnumbers at 4 bytes each takes4 * dbytes, plus a little row overhead. pgvector documents4 * dimensions + 8bytes for avector. So 1,000,000 vectors of 384 numbers take 1,000,000 x (4 x 384 + 8) = 1,544,000,000 bytes, about 1,544 MB (decimal megabytes). With 1536 numbers it is 6,152 MB, four times as large, because size grows in proportion to the dimensions. - 3
The index adds to it. An approximate index stores extra structure. For HNSW, each vector keeps a list of neighbour links. The example assumes about 32 links of 4 bytes per vector (about twice
M= 16 at the bottom layer), which is 128 bytes per vector or 128 MB per million. This is an assumption to make the table readable, not a measured number: the real overhead depends on the store, its version andM. Build an index on a sample of your data and read its size from the database. - 4
What shrinks it. Fewer dimensions (a smaller model, or a model whose vectors can be shortened), fewer bytes per number (half precision, or the 8-bit scalar quantisation from Topic 3.5), fewer vectors (bigger chunks, de-duplication from Topic 4.3, dropping what nobody asks about). The example's fourth row stores 2 bytes per number and halves the vector part. Smaller numbers cost a little accuracy, so measure recall (Topic 3.6) after the change.
- 5
Copies multiply it. Replicas for availability or read capacity each hold a full copy; shards split the data across machines so each holds a part. The last row of the example shows 100 million vectors with 3 copies: about 501,600 MB, far beyond one machine, which is the signal for a sharded design or a different plan.
- 6
What grows with what. Storage grows with documents (more chunks) and dimensions. Memory follows storage unless the store keeps vectors on disk. Query cost grows with the number of vectors visited per search (everything for exact search, a small share for an index) and with dimensions. Write cost grows with the index build. The number of queries per second decides how many copies you need, not how much data there is.
- 7
Headroom. Plan for peaks: building or rebuilding an index needs extra memory, compaction needs extra space (Topic 5.6), and the operating system and other queries need room. The decision example keeps half of the memory free as an assumption. Replace it with what you measure.
- 8
Money. A cost is memory and disk per month times the number of machines, plus embedding costs (you pay per token to embed documents and every query) and the hours of operating the store. Prices vary by provider and change, so keep them as variables and recompute:
PRICE = 0.00 # check your provider.
Explain it without notes
How much memory do 2 million vectors of 768 numbers take (vectors only, 4 bytes a number, 8 bytes overhead)?
Why does the index add to the size, and why do you measure that part?
Name three ways to shrink a vector store.
What decides how many copies you need?
Practice
Write vector_bytes(n, dims, bytes_per_number=4) using pgvector's formula and print the size in MB (decimal) of 250,000 vectors of 1024 numbers with 2 decimals.
Write max_vectors(ram_mb, dims, bytes_per_number=4, overhead_bytes=128) that returns how many vectors fit in half of ram_mb, rounding down. Print it for 8,000 MB and 384 numbers.
Trade-offs
- ↔
Everything in memory: fastest queries, highest cost per vector.
- ↔
Quantised or smaller vectors: much less memory, a small loss in accuracy.
- ↔
Replicas: more availability and read capacity, at a full copy of the memory each.
Done when you can
Done when you can compute the memory of n vectors with d dimensions.
Done when you can say which parts of an estimate are measured and which are assumed.
Done when you can name three levers that shrink a store.
Done when you can explain what sets the number of copies versus the number of shards.