What Is a Vector Database? How It Works and Which One to Choose

A vector database stores numerical representations of data and searches for items that are similar in meaning or characteristics. It helps answer questions such as “Which documents explain this problem?” or “Which products look like this image?” even when the query and the stored content use different words.

My default advice is to start with your existing database if its vector search meets your needs. A dedicated vector database makes sense when retrieval performance, filtering, scaling or operational requirements justify running another system.

For a quick shortlist:

• Already using PostgreSQL? Evaluate pgvector first.
• Need a dedicated vector service with substantial metadata filtering? Start with Qdrant.
• Want a managed service with little infrastructure work? Evaluate Pinecone.
• Want an integrated semantic and keyword retrieval platform? Consider Weaviate.
• Already rely on Elasticsearch for search? Evaluate its vector capabilities before adding another engine.
• Have a large distributed retrieval workload? Include Milvus in your benchmark.
• Building locally? Chroma and LanceDB are useful starting points, with different production paths.

These are architectural recommendations, not speed rankings. The right choice depends on your documents, filters, query traffic and team. The product capabilities below reference official documentation checked in October 2026.

What a vector database actually stores#

An embedding model converts text, images, audio or other inputs into a list of numbers called an embedding:

"How can I reset my password?"
                ↓
        Embedding model
                ↓
 [0.12, -0.48, 0.73, ..., 0.09]

Each value is a coordinate in a multidimensional space. A suitable model places related content near one another, so “I forgot my login credentials” can match an article titled “Reset your password.”

Real embeddings often contain hundreds or thousands of dimensions. Individual coordinates generally do not have simple human-readable meanings such as “password” or “security.” The representation works through the values together.

The embedding model creates the representation; the database stores and searches it. Some products integrate embedding generation, but these remain separate responsibilities. A database cannot compensate for a model that represents your domain poorly.

A typical record looks like this:

{
  "id": "article-42-chunk-3",
  "vector": [0.12, -0.48, 0.73],
  "text": "To reset your password, open account settings...",
  "metadata": {
    "tenant_id": "company-a",
    "language": "en",
    "source": "account-help",
    "document_version": 7
  }
}

The three-number vector is illustrative. Depending on the system, the original text may live in the same record or in another store referenced by its identifier.

Vector search vs SQL and keyword search#

These methods solve different retrieval problems, and an application can combine them.

A vector database does not replace transactions, joins or business rules. Account balances, invoice totals and exact inventory counts still need conventional database queries.

The product categories also overlap. PostgreSQL supports vector search through pgvector, and Elasticsearch combines lexical and vector retrieval. You do not necessarily need a separate database to add semantic search.

How ingestion and querying work#

A document-search system has two pipelines: one that prepares searchable data, and another that answers queries.

Ingestion: making documents searchable#

1. Load and parse the source. Extract useful text from documents, web pages or application records. Preserve headings, tables and source references where possible.
2. Split long documents into chunks. Keep related information together. A passage should be specific enough to retrieve accurately and complete enough to be useful.
3. Generate embeddings. Send each chunk through the embedding model.
4. Store vectors and metadata. Include stable IDs, document versions and the fields required for filtering.
5. Build or update the index. The database creates structures that help it locate nearby vectors efficiently.

Chunking creates a trade-off. Very small chunks may match a question but omit the surrounding explanation. Very large chunks can dilute the relevant passage with unrelated content. Evaluate chunk boundaries and sizes using real questions rather than selecting a universal number.

Querying: finding useful results#

1. Encode the query. Use the compatible query encoder and preprocessing expected by your embedding model.
2. Apply constraints. Scope retrieval to the correct tenant, access permissions, language or product category.
3. Retrieve candidates. Find the closest vectors under the chosen similarity measure.
4. Optionally combine keyword results. This helps preserve exact names, identifiers and technical terms.
5. Optionally rerank. A more expensive model evaluates the strongest candidates more closely.
6. Return results. Display the documents or provide the passages to an LLM.

Queries and documents must occupy a compatible embedding space. Equal dimensions alone do not make vectors from unrelated models comparable. Some models prescribe different query and document prefixes or encoders; follow their intended workflow. See the Elasticsearch kNN prerequisites for the basic compatibility requirements.

Where vector databases fit in RAG#

Retrieval-augmented generation, or RAG, gives a language model relevant source material before it writes an answer:

Documents → parse → chunk → embed → retrieval index
                                         ↑
Question → query embedding → search with access filters
                                         ↓
                      candidate passages → rerank
                                         ↓
                        LLM + sources → cited answer

The database retrieves candidates. The LLM writes the answer. Neither step guarantees that the source is correct or that the final answer is supported by it.

For example, a support assistant might retrieve an outdated refund policy that is semantically similar to the question. Version metadata and publication filters matter just as much as vector similarity.

RAG also does not require vector search in every case. A small knowledge base might work well with keyword search, and structured questions may be better answered with SQL. For the surrounding application framework, see my LangChain vs LlamaIndex guide.

How similarity is calculated#

Three common measures determine which vectors count as nearby:

Cosine similarity is calculated as:

similarity(a, b) = dot(a, b) / (length(a) × length(b))

Choose the metric recommended for your model. When vectors are normalized to unit length, cosine similarity, dot product and Euclidean distance produce equivalent rankings.

A similarity score is not a probability. A score of 0.85 does not mean an 85% chance that a passage answers the question. Thresholds must be calibrated against your model, content and queries, and may not transfer between systems.

How vector indexes make search faster#

Exact search compares the query with every eligible vector. It finds the true nearest neighbors under the selected metric, but the work grows with both vector count and dimensions.

Approximate nearest-neighbor search, or ANN, examines a smaller candidate set. It trades some completeness for lower latency or resource use. Exact search can still be the right choice for small datasets or narrowly filtered subsets. Elasticsearch documents both approaches.

Quantization is a compression technique that can be combined with other indexing approaches. It is not a mutually exclusive alternative to HNSW or IVF. Supported algorithms and configuration options differ by product; consult Faiss, Weaviate's indexing documentation and Milvus's architecture documentation.

One useful measure is Recall@k. If an approximate search returns nine of the ten neighbors found by exact search, its Recall@10 is 90%.

That measures agreement with exact vector retrieval, not whether the documents actually answer a human's question. Evaluate both ANN recall and human relevance. An exact search over poor embeddings can have perfect ANN recall and still produce useless results.

Why filtering and hybrid search matter#

Real searches usually include constraints:

Find relevant documents belonging to this customer, written in English, that the current user is allowed to access.

Searching globally and filtering only the top results afterward can leave too few matches. Engines may filter before search, during traversal, after candidate retrieval, or use an adaptive combination. Restrictive filters can change latency and recall substantially.

Qdrant provides payload indexes for efficient metadata filtering. With pgvector's approximate indexes, filtering can leave fewer results than requested; iterative scans can search further within configured limits.

Tenant and permission constraints must come from trusted application state and be enforced throughout retrieval. Do not rely on an LLM to decide which customer's records it may search. Apply the same scope to keyword retrieval, vector retrieval, reranking and caches.

Hybrid search combines semantic matching with lexical precision. It helps when queries contain error codes, product SKUs, version numbers, function names or uncommon names that dense embeddings may not distinguish reliably.

One approach is to retrieve candidates using both keyword and vector search, then merge rankings with reciprocal rank fusion. Reranking is a separate optional stage that evaluates the candidates more closely. See Elasticsearch's hybrid search documentation for a concrete implementation.

Comparing eight vector database options#

This comparison focuses on architectural fit. It does not rank products by speed or price because those depend on the workload, configuration and deployment.

PostgreSQL with pgvector#

pgvector adds vector similarity search to PostgreSQL. Its main attraction is architectural simplicity: business records and vectors can live together, reducing the need to synchronize two systems.

Choose it when relational queries and existing PostgreSQL operations are central. Evaluate query plans, selective filters and resource contention under realistic load. There is no universal vector-count threshold at which you must migrate away. Official documentation.

Qdrant#

Qdrant is a focused vector database with metadata filtering and hybrid retrieval capabilities. Its open-source and cloud offerings let teams choose between infrastructure ownership and managed operation.

I would shortlist it for a dedicated semantic-search service with substantial tenant or metadata constraints. Index the fields used for filtering and test the actual filter distributions. The extra service also needs an ingestion and synchronization strategy. Overview and hybrid queries.

Weaviate#

Weaviate combines vector retrieval with keyword search and embedding integrations. Its hybrid search supports configurable fusion and weighting, while its vector indexing options allow different resource trade-offs.

Choose it when that integrated workflow reduces application code. Budget time to understand collections, vectorization, indexing and retrieval configuration. Convenient defaults still need evaluation against your documents. Hybrid search and vector indexes.

Pinecone#

Pinecone is a fully managed vector database with an architecture that separates storage from query compute. Its main appeal is reducing database infrastructure ownership.

Choose it when managed operation matters more than low-level control. Evaluate costs with real ingestion, query traffic, metadata and tenancy patterns. Also test freshness requirements: a successfully acknowledged write and its visibility in search are separate concerns. Architecture overview.

Milvus#

Milvus is an open-source vector database designed for large-scale retrieval, with multiple indexing approaches and a distributed architecture that separates major responsibilities.

Include it when scale and workload requirements justify dedicated retrieval infrastructure. Distributed self-hosting requires planning for dependencies, capacity, monitoring and recovery. A small application should establish that it needs this complexity before adopting it. Milvus overview and architecture.

Elasticsearch#

Elasticsearch combines vector retrieval with a mature lexical-search workflow. It is a strong candidate for catalogs, document search and products where exact terms remain important.

If Elasticsearch is already part of your stack, evaluate its hybrid search before adding another engine. Operating and tuning a broad search platform still takes expertise, and available features depend on the chosen deployment and edition. Vector search and hybrid search.

Chroma#

Chroma provides local embedded, single-node and distributed deployment modes, with Chroma Cloud as a managed distributed offering. It is useful for getting a retrieval application working quickly.

It is inaccurate to dismiss Chroma as only a toy database. Equally, a successful local demo does not prove production capacity. Evaluate the intended deployment for concurrency, recovery and operational needs. Architecture documentation.

LanceDB#

LanceDB combines retrieval with columnar multimodal data workflows. Its OSS offering is an embedded retrieval library; its enterprise platform covers a broader AI data lifecycle.

Consider it when embeddings, source data and dataset workflows belong together, particularly for local or multimodal applications. Evaluate the capabilities of the specific offering rather than assuming the embedded library supplies every distributed-service feature. LanceDB documentation.

Where Faiss fits#

Faiss is a similarity-search and clustering library, with CPU and GPU capabilities, rather than a complete database service. It is useful for offline experiments, custom retrieval systems and direct control over indexing algorithms.

Your application must provide the surrounding service, metadata handling, access control and operational lifecycle. Comparing a bare library with a managed database without accounting for that work gives a misleading cost comparison. Faiss repository.

Which database to choose for your project#

Use this table to build a shortlist, then benchmark the candidates using your own data.

These choices are not exclusive. A team might begin with embedded retrieval, move to a managed service, or keep PostgreSQL as its source of truth while using a separate search index.

Avoid rules such as “one million vectors means you need a distributed database.” A large collection serving ten queries per minute can be easier to handle than a smaller one serving thousands of concurrent queries with restrictive filters.

Storage, cost and performance trade-offs#

For uncompressed float32 vectors, the raw storage calculation is:

raw vector bytes = vector count × dimensions × 4

For one million vectors:

These are arithmetic storage estimates, not vendor benchmarks or RAM requirements. Add index structures, source text, metadata, replicas, backups and temporary rebuild space. Compression and storage tiers change the actual footprint.

Also count chunks, not only documents. Ten thousand documents split into fifty chunks each produce five hundred thousand vectors. Multiple embeddings per chunk increase that further.

Total cost includes embedding generation, database infrastructure, storage, reads and writes, reranking, data transfer and engineering time. Re-embedding after a model change can be a substantial one-time expense. A free open-source license does not make operation free, and a managed service is not automatically more expensive after staff time is included.

For the broader application budget, see my RAG development cost guide and RAG cost calculator.

How to evaluate a database fairly#

Build a representative dataset with real documents, questions, filters and expected results. Include difficult cases: rare terms, misspellings, multilingual queries, narrow tenant filters, updated documents and questions with no relevant answer.

Compare products at similar recall targets and realistic resource budgets. An engine can look faster simply because it searches less thoroughly. Warm-cache measurements also do not establish cold-start performance.

Measure against your application's latency budget. Saving a few milliseconds in retrieval may have little visible impact if model inference dominates the response, while predictable tail latency may matter greatly for interactive search.

Common mistakes to avoid#

• Changing embedding models without migrating vectors. Track the model and preprocessing version. Re-embed into a separate index or vector field, evaluate it, then switch traffic.
• Treating similarity as certainty. Even unrelated queries have nearest neighbors. Test no-answer cases and support abstention.
• Ignoring exact terms. Evaluate hybrid search when names, codes and versions matter.
• Dropping document structure. Better parsing and chunking can improve results more than switching databases.
• Only testing unfiltered search. Include realistic permissions and tenant distributions in benchmarks.
• Forgetting updates and deletions. Use stable IDs and retryable ingestion so old chunks do not survive source changes.
• Assuming acknowledged writes are immediately searchable. Test the freshness guarantees of the chosen deployment.
• Using the retrieval index as the only source copy. Preserve authoritative content and enough metadata to rebuild the index.
• Choosing by a single vendor benchmark. Reproduce the workload that matters to your users.

Frequently asked questions#

Do I need a vector database for every AI application?#

No. An application that summarizes a supplied document or answers structured questions may not need similarity search. Use a vector index when retrieving related content solves a real problem.

Does a vector database understand meaning?#

It compares numerical representations produced by a model. Those representations can capture useful semantic relationships, but similarity is not reasoning or verification.

Can PostgreSQL handle production vector search?#

Yes, when its performance and operational characteristics meet the workload. Evaluate filters, concurrency, index size and contention with other queries rather than deciding from vector count alone.

Are dedicated vector databases always faster?#

No. Results depend on hardware, index configuration, recall targets, dimensions, filters and traffic. Dedicated systems may offer a better operational or scaling fit, but that must be established for the application.

Which database is best for RAG?#

There is no universal winner. Retrieval quality depends on source preparation, embeddings, filters, hybrid search and reranking as well as the database. Start with a short list that fits your existing infrastructure and evaluate the complete pipeline.

When should I avoid vector search?#

Use exact queries for identifiers, accounting, inventory and deterministic business rules. For a small corpus, keyword or exact vector search may already meet the requirements without a dedicated service.

A practical implementation path#

Keep authoritative documents and business records in their normal storage system. Build a retryable ingestion pipeline with stable document and chunk IDs, embedding versions, source versions and access metadata. Propagate deletions as carefully as new content.

Start with a baseline retrieval method, build a set of real evaluation questions, then add hybrid search or reranking when the results justify the added cost. Choose the database after measuring relevance, filtered recall, latency and operational requirements together.

If you need help implementing the full pipeline, I build custom RAG systems with ingestion, access rules, source citations and retrieval evaluation. Tell me about your documents and application.