How Vector Databases Actually Work
/ Arvid Andersson
Type "cheap flights to Lisbon" into a search box built on keyword matching, and it looks for the words "cheap," "flights," and "Lisbon." Type "budget travel Portugal" and a keyword search finds nothing, even though both mean roughly the same thing. Vector search closes that gap by comparing meaning instead of matching words. A vector database is the infrastructure that makes that comparison fast at scale.
Turning text into numbers
An embedding model reads a piece of text and outputs a vector: a list of a few hundred to a few thousand numbers. The model is trained so that texts with similar meaning produce vectors that sit close together, and texts with different meaning produce vectors far apart. "Budget travel Portugal" and "cheap flights to Lisbon" land near each other. "Budget travel Portugal" and "recipe for banana bread" do not.
"Close together" has a precise meaning here: distance in that list-of-numbers space, usually measured with cosine similarity (the angle between two vectors) or Euclidean distance (straight-line distance between them). Once text becomes a vector, "find things that mean something similar" becomes "find the vectors nearest this one," a geometry problem a computer can solve directly.
Why you can't just check every vector
The direct way to find the nearest vectors is to compare your query against every vector in the collection and sort by distance. That works, and for a few thousand vectors it's fast enough that you don't need anything else. It stops working as the collection grows: a million vectors means a million comparisons per query, and a search API that scans everything on every request will not hold up under real traffic.
This is where an index comes in. Algorithms like HNSW (Hierarchical Navigable Small World) build a structure over the vectors ahead of time, so a query only has to check a small, well-chosen fraction of them instead of all of them. HNSW connects each vector to a handful of nearby neighbors, then layers in longer shortcut links between distant regions. A search starts on a shortcut and narrows in step by step, checking maybe a few hundred vectors instead of a million.
The trade-off is that this is now an approximate answer. The index might return the 8th-nearest vector instead of the exact nearest, in exchange for being orders of magnitude faster. That's usually the right trade: for "find documents about roughly this topic," near-enough is enough, and the speed is what makes vector search usable in production at all.
What the database adds on top of the index
An index alone answers one question: given a query vector, which stored vectors are closest? A vector database wraps that index with everything a real application needs beyond that single answer.
It stores more than the vector. Each entry keeps the vector alongside the original content (or a pointer to it) and metadata: which document it came from, a timestamp, a user ID, an access-control tag. A raw index has no concept of any of that.
It filters. A real query is rarely "find anything similar to this," it's "find things similar to this, from the last 30 days, that this user is allowed to see." The database combines the vector search with those filters, which is a harder problem than either alone: filter too early and you might discard the actual best matches before comparing them; filter too late and you waste work computing distances you'll throw away. Production vector databases handle that trade-off so the application doesn't have to.
It groups and preserves structure. A single source document is usually split into several chunks before embedding, each becoming its own vector, and the database keeps track of which chunks belong to which document so results can be reassembled and deduplicated. A bare collection of vectors floating in a mathematical space has no notion of "these five points are chunks of the same PDF."
A vector database is not a space of points. It's a database, storage, filtering, grouping, access control, built around an approximate-nearest-neighbor index as its core search primitive.
Where it fits
The most common use case is retrieval-augmented generation (RAG): before an LLM answers a question, the application embeds the question, searches a vector database for the most relevant chunks of your own documents, and includes those chunks in the prompt. The vector database is the retrieval half of that pipeline, it doesn't generate anything, it finds what's relevant.
Pure vector search has a known weak spot in that setup: it matches on meaning, so it can miss exact strings a user typed verbatim, like an SKU, an error code, or a person's name. Two things address that. Hybrid search runs a keyword query (usually BM25) alongside the vector query and merges the results, which recovers the exact matches similarity alone slides past. Reranking then takes that candidate set, often the top 50, and re-scores it with a slower, more accurate cross-encoder model to fix the ordering before the results are used. That's the "rerank" step in the diagram above. Both are covered properly in RAG Retrieval Architectures.
Beyond RAG, the same mechanism powers semantic search, recommendation (find items similar to what a user liked), deduplication (find near-identical records), and anomaly detection (find points with no close neighbors).
The options
The pieces described here (storage, an index, filtering, grouping) can be assembled in genuinely different ways, and that is why the category has not collapsed into one product. A vector index can be a feature of a database you already run, as with pgvector on PostgreSQL or Supabase Vector, which means no new infrastructure and one less system to operate. It can treat object storage as the source of truth and trade cold-query latency for cost, which is the approach Turbopuffer takes. It can run at the edge as a managed service, like Cloudflare Vectorize. Or the index can sit inside a general search engine that also ranks on keywords. That is where Vespa, Meilisearch and TopK live, and it is why hybrid search comes built in there rather than bolted on. Each of those is an answer to a different constraint.
A team with 50,000 chunks and an existing Postgres instance and a team serving latency-critical search over a billion vectors are not solving the same problem, and a field with one dominant option would serve one of them badly. The useful question is which constraint binds first for you: operational simplicity, scale, cost, latency, or where the data is allowed to live.
Infrabase tracks the current field of vector databases, from purpose-built services like Pinecone, Qdrant and Milvus through to in-process libraries like Chroma and LanceDB, which run inside your application with no server to operate. For a full comparison of hosting, performance, and pricing across the field, see Choosing a Vector Database for RAG.
Looking for European alternatives? Infrabase maintains a page of EU-headquartered, hand-verified providers, vector databases included.
Related reading
The theory here feeds directly into these.
Frequently asked questions
What is a vector database?
A vector database stores embeddings (numeric vectors that represent the meaning of text, images, or other data) alongside their original content and metadata, and answers nearest-neighbor queries against them quickly at scale. It combines an approximate-nearest-neighbor index with storage, metadata filtering, and grouping, the things a raw index doesn't provide on its own.
Is a vector database just a vector space?
No. A vector space is a mathematical structure, points with distances between them and nothing else. A vector database is a database: it stores the vector alongside the original content, metadata like source document or timestamp, and access-control tags, and it groups vectors that belong to the same record (a document is usually split into several chunks, each with its own vector). A raw index has no concept of any of that.
Why can't I just compare a query against every stored vector?
You can, and for a few thousand vectors it's fast enough. It stops scaling once the collection reaches millions of vectors, since a full comparison means a million distance calculations per query. Vector databases use an approximate-nearest-neighbor index, commonly HNSW, that checks a small, well-chosen fraction of the vectors instead of all of them, at the cost of an occasional near-miss instead of the exact nearest result.
How does HNSW work?
HNSW (Hierarchical Navigable Small World) connects each vector to a handful of nearby neighbors, then layers in longer shortcut links between distant regions of the space. A search starts on a shortcut link and narrows in step by step, checking maybe a few hundred vectors instead of a million, trading a small amount of accuracy for a large amount of speed.
What is hybrid search, and do I need it?
Hybrid search runs a keyword query (usually BM25) alongside the vector query and merges the results. It exists because pure vector search matches on meaning and can miss exact strings a user typed verbatim, like an SKU, an error code, or a person's name. If your corpus contains identifiers people search for literally, hybrid search recovers matches that similarity alone slides past. It is often paired with reranking, which re-scores the merged candidate set with a slower cross-encoder model to fix the ordering.
Why are there so many vector databases?
Because the same components (storage, an index, filtering, grouping) can be assembled for different constraints. Adding a vector index to a database you already run avoids operating a second system. A purpose-built service buys filtering during search and multi-tenancy at large scale. An in-process library removes the server entirely. Building on object storage lowers cost and accepts slower cold queries. Which of those matters depends on the size of the collection, the latency budget, and where the data is allowed to live.
What's the difference between a vector database and pgvector?
pgvector adds a vector index and similarity search functions to an existing PostgreSQL database rather than running as a separate system. It performs well up to a few million vectors and avoids adding infrastructure if your data already lives in Postgres. Purpose-built vector databases like Pinecone or Qdrant add more built-in features (multi-tenancy, hybrid search, larger-scale indexing) at the cost of another system to run.
Browse all vector databases on Infrabase.ai
Is your product missing?