Embeddings and Vector Databases: The Limits That Actually Bite

Embeddings and vector databases are the two halves of semantic search. The model turns text into a list of numbers. The database stores those numbers and finds the closest ones fast. Everything that goes wrong in production goes wrong at the seam: a 3072-dimension vector your index refuses to build, a 40 KB metadata cap you find at upsert time, a memory bill four times what you planned. Below are the real limits and prices from vendor docs, checked in August 2026.

What an embedding actually is

An embedding is a fixed-length array of floating point numbers representing text, an image or audio. OpenAI’s text-embedding-3-small returns 1536 numbers, text-embedding-3-large returns 3072, Cohere’s embed-v4.0 returns 1536 and Voyage’s voyage-4 returns 1024.

Each number is a 4-byte float32 by default. That fact drives most cost decisions later: a 1536-dimension vector is 6,144 bytes, and a million of them is 6.1 GB before you store any of the original text.

Two texts that mean similar things get vectors that sit close together. “How do I reset my password” and “I forgot my login” share almost no vocabulary and still land near each other.

The hard rule: one model, one space. Vectors from text-embedding-3-small and embed-v4.0 cannot be compared. Change model and you re-embed the corpus. There is no migration path.

Picking an embedding model: dimensions, context, price

Four vendors cover most production use. All figures come from vendor docs as of August 2026.

Model	Default dimensions	Max input	Price per 1M tokens
OpenAI text-embedding-3-small	1536	8192 tokens	$0.02
OpenAI text-embedding-3-large	3072	8192 tokens	$0.13
Cohere embed-v4.0	1536 (also 256, 512, 1024)	128k tokens	no per-token rate published, see Cohere pricing
Voyage voyage-4	1024 (also 256, 512, 2048)	32,000 tokens	$0.06
Voyage voyage-4-lite	1024	32,000 tokens	$0.02
Voyage voyage-4-large	1024	32,000 tokens	$0.12
Google gemini-embedding-2	3072 (128 to 3072 supported)	8192 tokens	$0.20

Voyage gives the first 200 million tokens of its v4 models free on every account. Cohere’s 128k context window is the outlier: it embeds a whole document without chunking, useful for classification but usually wrong for retrieval, since one vector over 128k tokens averages away the passage you wanted.

Google trains its Gemini embeddings with Matryoshka Representation Learning, and OpenAI’s v3 models behave the same way: the leading part of the vector is a usable embedding on its own. Pass a dimensions parameter and get a shorter vector with no retraining. OpenAI’s documentation gives the case: on MTEB, text-embedding-3-large shortened to 256 dimensions still beats full-length 1536-dimension ada-002, a 12x storage reduction. Google recommends 768, 1536 or 3072 output dimensions for gemini-embedding-2 and says the full 3072 can be truncated without losing quality. Truncate manually instead of through the API parameter and you must re-normalise to unit length, or cosine similarity stops meaning what you think.

Rather than trusting a vendor chart, check the model on the MTEB leaderboard and read the retrieval column. A model that wins on classification can lose on retrieval.

Why a vector database instead of a WHERE clause

Comparing a query vector against every stored vector is exact and needs no index. At a few thousand rows that is the right answer and you should install nothing. The cost is linear: every query touches every row.

A vector database replaces that scan with an approximate nearest neighbour index, trading a little recall for orders of magnitude in query speed, and adds the parts people forget: metadata filtering, incremental updates, deletes and backups.

Weaviate puts a number on the crossover. Its dynamic index stays flat and exact until a collection passes 10,000 objects, the default threshold, then builds the HNSW index in the background and swaps over.

HNSW, IVFFlat and what the parameters do

HNSW builds a layered graph where each node links to a bounded number of neighbours, and search walks it from the top layer down. In pgvector the defaults are m = 16 connections per layer and ef_construction = 64 for the build-time candidate list. At query time hnsw.ef_search defaults to 40. Raising it raises recall and latency together, and it is a session variable, so you tune per query without rebuilding.

IVFFlat partitions vectors into lists and searches only the closest ones. pgvector recommends rows / 1000 lists up to 1M rows and sqrt(rows) above that. The ivfflat.probes setting defaults to 1: one probe searches a single partition and misses anything on the wrong side of a boundary.

Build memory matters more than people expect. pgvector states indexes build significantly faster when the graph fits into maintenance_work_mem, and parallel build workers default to 2. On a 6 GB table with the stock 64 MB setting, an HNSW build thrashes for hours.

Similarity metrics and the operator you type

Cosine similarity compares angle and ignores magnitude, and it is the default for text. Inner product accounts for both and is faster on normalised vectors. Euclidean distance suits image and geospatial data.

pgvector exposes six operators: <-> for L2, <#> for negative inner product, <=> for cosine, <+> for L1, <~> for Hamming and <%> for Jaccard. The index must match the operator you query with. Create a cosine index, query with <->, and Postgres falls back to a sequential scan rather than raising an error.

The hard limits that will bite you

System	Max vector dimensions	Other limits worth knowing
pgvector vector type	16,000 stored, 2,000 indexable by HNSW or IVFFlat	Storage is 4 x dimensions + 8 bytes per vector
pgvector halfvec	16,000 stored, 4,000 indexable	Half precision, so 3072 dimensions become indexable
Pinecone	20,000 dense	40 KB filterable metadata per record, top_k max 10,000, 1,000 records or 2 MB per upsert, ID max 512 characters
Milvus	32,768	65,536 collections, 64 fields each, VARCHAR 65,535, topk 16,384, 64 MB per insert
Elasticsearch dense_vector	4,096	Element types float, bfloat16, byte and bit

The pgvector line catches teams out: text-embedding-3-large emits 3072 dimensions. The vector type stores it happily, then refuses to build an HNSW index, which caps at 2,000 there. Cast to halfvec, indexable to 4,000, or request 1536 dimensions from the API in the first place.

Sizing and compression: from 6 GB down to 200 MB

Work out the footprint before picking a plan. A float32 vector of d dimensions is 4d bytes, and pgvector’s row overhead makes it 4d + 8 bytes.

• 1M vectors at 768 dimensions: 3,080 bytes each, about 3.08 GB
• 1M vectors at 1536 dimensions: 6,152 bytes each, about 6.15 GB
• 1M vectors at 3072 dimensions: 12,296 bytes each, about 12.3 GB

The HNSW graph sits on top, and so do metadata and the original text. Qdrant’s free cloud tier gives 0.5 vCPU, 1 GB RAM and 4 GB disk, so 4 GB holds roughly 1.3 million 768-dimension vectors before overhead. Pinecone’s Starter plan allows 2 GB of storage, 1M read and 2M write units a month across 5 serverless indexes and 100 namespaces.

When those numbers get uncomfortable, you compress. Three vendors publish the same ladder.

• Scalar quantization, float32 to 8-bit integer. Qdrant and Weaviate both state 4x. Elasticsearch int8 gives the same 75 percent cut.
• 4-bit. Elasticsearch int4 reports 87 percent, or 8x. Weaviate’s 4-bit rotational quantization is roughly 8x.
• Binary quantization, one bit per dimension. Qdrant states up to 32x compression and up to 40x speedup. Weaviate’s is a straight 32x since 32 bits become 1. Elasticsearch bbq reports 96 percent.
• Product quantization. Qdrant reports up to 64x, but its distance maths is not SIMD-friendly and runs slower than scalar. Weaviate’s example compresses a 3072-byte vector to 128 bytes, roughly 24x.

Binary quantization sounds ruinous. With oversampling it usually is not. Qdrant’s own testing on text-embedding-ada-002 at 1536 dimensions reports 0.98 recall@100 with 4x oversampling, and 0.98 recall@50 on Cohere embed-english-v2.0 at 4096 dimensions with 2x oversampling, while warning that lower-dimensional models may need their own experiments. Voyage’s v4 models return int8, uint8, binary and ubinary straight from the API.

Metadata filters and hybrid search

Pure vector search returns things that are semantically close and completely wrong: the right answer for a different customer, a document the user may not see, a price list from 2023. Pinecone caps filterable metadata at 40 KB per record, so put tenant ID, document type, language and date there and keep body text out.

Hybrid search adds sparse keyword matching so exact tokens still land: product codes, error numbers, surnames. Qdrant runs both as prefetch sub-queries and fuses the lists with either Reciprocal Rank Fusion, where the constant k defaults to 2, or Distribution-Based Score Fusion, which normalises scores using 3-sigma extremes as endpoints. Weaviate offers BM25 and vector search in one query. If users search by SKU or part number, hybrid is not optional: dense vectors are bad at exact strings by design.

Chunking and the 67 percent number

Chunking sets your ceiling. Too large and the fact is diluted across a vector that mostly describes something else. Too small and you get a sentence with no context.

Anthropic published measured figures for a technique worth copying. Prepending 50 to 100 tokens of document-level context to each chunk before embedding, which they call contextual retrieval, cut the top-20 retrieval failure rate by 35 percent, from 5.7 to 3.7 percent. Adding contextual BM25 took it to 49 percent, or 2.9 percent failure. Adding reranking reached 67 percent, or 1.9 percent. Their assumptions were 800-token chunks and 8k-token documents, and with prompt caching the one-off preprocessing cost about $1.02 per million document tokens. That is the highest-leverage change most retrieval systems can make, and for the wider pipeline it sits inside, see our walkthrough of how RAG works in practice.

What it costs to embed a corpus

One million chunks of 400 tokens each is 400 million tokens of input.

• text-embedding-3-small at $0.02 per 1M tokens: $8.00
• text-embedding-3-large at $0.13 per 1M tokens: $52.00
• voyage-4-lite at $0.02 per 1M tokens, first 200M free: $4.00
• voyage-4-large at $0.12 per 1M tokens, first 200M free: $24.00

Embedding is cheap and one-off. Storage and queries are neither. At Pinecone’s Standard rate of $0.33 per GB per month, 6.15 GB of 1536-dimension vectors is about $2.03 a month, which counts against a $50 monthly usage minimum, with reads at $16 to $18 per million read units and writes at $4 to $4.50 per million. Builder is $20 flat for up to 10 GB. Qdrant Cloud’s paid tiers are usage-based rather than a published flat rate, so check the vendor’s current pricing on the Qdrant pricing page.

Re-embedding is the cost that hurts: the corpus again, plus the rebuild, plus a window where old and new vectors coexist. The trigger is almost always a model upgrade, so size the budget against your vendor’s release cadence rather than the calendar.

How to tell whether it works

Build 50 to 100 real questions with the chunk that should be returned for each, then measure recall@k. Anthropic’s benchmark used top-20 against a 5.7 percent baseline failure rate. Move one variable at a time: ef_search from 40 to 100, chunk size from 400 to 800 tokens, quantization on and off.

Four mistakes that show up in production

1. Mixing embedding models. One batch on ada-002, the next on 3-small. Incompatible spaces, and results degrade quietly rather than failing.
2. Storing 3072 dimensions in pgvector’s vector type. It stores fine, then will not index: HNSW caps at 2,000 there.
3. Skipping the metadata filter. Multi-tenant systems leak the moment a query lands closer to another customer’s document.
4. No sparse channel. Users search exact codes, and dense-only retrieval will not find them.

FAQ

Do I need a vector database at all?

Under roughly 10,000 items, no. Weaviate’s dynamic index stays flat and exact until 10,000 objects before switching to HNSW. Below that, brute force in Postgres is simpler.

Is pgvector enough for production?

For most retrieval workloads under a few million vectors, yes, provided you respect the 2,000-dimension HNSW limit on the vector type or use halfvec to 4,000, raise hnsw.ef_search above its default of 40, and give the build enough maintenance_work_mem.

Which dimension count should I choose?

1024 or 1536 covers most cases. OpenAI reports text-embedding-3-large truncated to 256 dimensions still beats full-length ada-002 at 1536, so higher is not automatically better, and doubling dimensions doubles the storage bill.

How much does it cost to embed a million documents?

At 400 tokens per chunk, 400M tokens costs $8.00 with text-embedding-3-small at $0.02 per 1M tokens, or $52.00 with text-embedding-3-large at $0.13. Voyage’s first 200M tokens are free on every account.

Will quantization ruin search quality?

Usually not, with oversampling. Qdrant reports 0.98 recall@100 on 1536-dimension OpenAI vectors under binary quantization with 4x oversampling, at up to 32x compression. Test on your own set first: Qdrant notes lower-dimensional models may behave differently.