What a Vector Database Actually Does

A relational database finds rows where userid = 42. A vector database finds rows whose meaning is closest to a query. You convert text, images, or structured data into a dense numerical vector (an embedding) using a model like text-embedding-3-small or nomic-embed-text, then store that vector alongside a payload. At query time, you embed the query the same way and retrieve the k nearest vectors by cosine similarity, dot product, or Euclidean distance. The critical insight: the quality of retrieval depends more on embedding consistency—using the same model for indexing and querying—than on which vector database you pick. Switching from Chroma to Pinecone won't fix a retrieval problem caused by mismatched embedding models.

Work through these in order: 1. Already on Postgres? Start with pgvector or Supabase Vector. You eliminate a network hop, stay in SQL, and keep one infrastructure dependency. 2. Need sub-100ms p99 at production scale? pgvector's HNSW is competitive up to a few million vectors. Beyond that, dedicated systems win. 3. Need complex metadata filtering? Qdrant's payload filtering evaluates conditions before the ANN search, not after. This matters when your filter is selective (e.g., tenantid = X AND category IN [...]). 4. Building a multi-tenant SaaS? Qdrant collections-per-tenant or Weaviate's multi-tenancy API both work. Namespaces in Pinecone serverless are the managed equivalent. 5. Prototyping and just need something to run? Chroma locally, Supabase Vector in the cloud.

Embedding model drift — updating the embedding model without re-indexing existing vectors corrupts retrieval. Version your embeddings. Treat a model swap as a migration, not a config change. - Chunking too coarse — retrieving 10,000-token chunks burns context window and dilutes relevance. Target 256–512 tokens per chunk with overlap, tested against your actual query distribution. - Ignoring payload filters — running ANN search and then filtering results in application code wastes compute and returns fewer results than requested. Use native payload filtering. - No recall evaluation — shipping RAG without measuring retrieval recall is flying blind. A simple evaluation: sample 50 queries with known good documents, measure what fraction appear in top-5 results.

If you're on Supabase already, enable pgvector and test RAG on your existing dataset before evaluating alternatives. - For filtering-heavy production workloads, run Qdrant locally with Docker and benchmark your actual query patterns before committing to a managed service. - Evaluate embedding models independently of the vector db—nomic-embed-text (open-weight, runs locally) vs text-embedding-3-small (OpenAI, cheap, high quality) will have more impact on retrieval quality than db choice. - Add a retrieval recall metric to your CI pipeline before the system grows.

AI Vector DB

AI Vector Databases 2026 TL;DR. Vector databases store high-dimensional embeddings and retrieve them by similarity—the core primitive behind semantic search, RAG, and agent memory. The right choice depends on whether you need a managed service that ships fast, a self-hosted option you control completely, or an extension bolted onto Postgres you already operate. This guide cuts through the noise a…

All guides

Related guides

AI Edge Computing
AI DBA
AI Fullstack
AI Backend
AI Frontend