A vector database stores vector embeddings — the numeric representations of text, images, or other content — and answers one question fast: which stored items are most similar to this one? That operation, similarity search, is the workhorse behind semantic search, recommendations, and the retrieval step in RAG systems.
Why ordinary indexes don’t work
A regular database index answers exact questions: rows where status = 'active', values between two dates. Similarity search is a different shape of problem — given a query vector with hundreds or thousands of dimensions, find its nearest neighbors by a distance measure like cosine similarity. Checking every stored vector gives the exact answer but scales linearly; at millions of vectors, per-query latency becomes unusable.
Vector databases solve this with approximate nearest neighbor (ANN) indexes — structures like HNSW (a graph you navigate greedily toward the neighborhood of the query) or IVF (vectors clustered into partitions so only the closest partitions get scanned). The trade is explicit: you give up the guarantee of finding the exact nearest neighbors in exchange for queries that return in milliseconds. Recall — the fraction of true neighbors actually found — is tunable, and tuning it up costs speed and memory. For most retrieval use cases, 95% recall at 10ms beats 100% at 2 seconds.
The other essential feature is metadata filtering: fetching the nearest neighbors among documents this user can see or published after last March. Combining a filter with an ANN index efficiently is genuinely hard, and implementations differ more here than in raw speed.
Dedicated product or Postgres extension?
The category includes dedicated systems (Pinecone, Weaviate, Milvus, Qdrant) and vector support added to databases you already run — most prominently pgvector, the Postgres extension that adds a vector column type plus HNSW and IVF indexes to ordinary tables.
The honest default for most teams is the extension. If your application data already lives in Postgres, pgvector means one database, real transactions, joins between vectors and the rest of your data, and no new system to operate. Dedicated vector databases earn their keep at larger scale — hundreds of millions of vectors, heavy write loads, or recall/latency requirements that need specialized tuning. Starting there because it’s the fashionable architecture is buying a distributed system you may never need.
One boundary worth drawing: a vector database is not an analytical database. It’s optimized for nearest-neighbor lookups, not for scanning and aggregating billions of rows the way an OLAP engine does. In an AI application’s stack the two coexist — vectors serve retrieval, while usage logs and traces land in Postgres or ClickHouse, where a BI tool like Metabase can query them to track latency, cost, and retrieval quality over time.
Related terms
Put it to work
- LLM analytics — Overview
- LLM observability — Integrations
- P95 LLM latency — Metric
