{"canonical_url":"https://engineer.company/glossary/vector-search/","content_markdown":"# Vector search\n\n## What it is\n\nVector search finds the stored items whose vectors lie closest to the vector of\na query. It is also called vector similarity search. A vector here is a\nfixed-length list of numbers that an embedding model produces from a piece of\ntext, an image or a product record. The model is trained so that items with\nsimilar content come out with nearby vectors, which means a question and the\npassage that answers it can sit close together even when they share no words.\n\n\"Closest\" needs a measure. pgvector, the PostgreSQL extension this article uses\nfor its examples, supports six: L2 (Euclidean) distance, inner product, cosine\ndistance and L1 distance, plus Hamming and Jaccard distance for binary vectors.\nThe embedding model's own documentation says which measure its vectors are meant\nfor. For vectors normalised to length 1, cosine distance and inner product put\nthe neighbours in the same order, and pgvector recommends inner product there\nfor the best performance.\n\nThere are two ways to answer the question. Exact search compares the query with\nevery stored vector and returns the true nearest neighbours; pgvector does this\nby default, and its documentation describes it as perfect recall. Approximate\nsearch can use an index that visits only part of the data, trading some recall\nfor speed. How many true neighbours come back depends on the data, the index\nand its settings; even a majority is not guaranteed.\nThat is why pgvector warns that, unlike an ordinary index, an approximate index\nchanges what a query returns.\n\npgvector offers two approximate index types. HNSW builds a graph in several\nlayers, a method Malkov and Yashunin first posted in 2016: each layer holds a\nsubset of the layer below, a search starts in the small top layer and walks down\nto the full one, and their paper reports logarithmic scaling of the search cost.\nIVFFlat divides the vectors into lists and searches only the lists nearest the\nquery.\n\n## Why it matters\n\nFull-text search matches words. With word matching and stemming alone, a\ncustomer who writes \"the screen stays black\" does not find the manual section\nheaded \"display does not power on\", because the two share no word a stemmer can\nfold together. Configured synonyms can bridge some such gaps. Vector search compares meaning as\nthe embedding model captured it, so the complaint and the section can land close\ntogether. The same idea works beyond text: an image like this one, a support\nticket like one already solved, a product like the one in the basket.\n\nIt is also one way to do the retrieval in retrieval-augmented generation, where\na large language model answers from passages fetched for the question. The\nquestion is embedded with the same model as the passages, and the nearest\npassages go into the prompt. The answer can only be as good as what that\nretrieval brought back, so the search deserves the same care as the model.\n\nKeeping the vectors in the database you already run has a practical side, and\npgvector opens its documentation with that argument. The vectors get the same\ntransactions, point-in-time recovery and joins as every other column, and\nbecause pgvector writes to the write-ahead log, replication carries them too. A\nfilter on customer, date or permission is an ordinary `WHERE` clause on the same\nrow. A snapshot taken after a deletion commits no longer sees that document,\nbut a query using an older snapshot may still see it. A separate vector store is a second system to fill and keep in step\nwith the first.\n\nA team should also know what vector search does not do. It ranks by closeness,\nnot by truth. The nearest passage is the one most like the question; it may\nstill be outdated, or wrong, or from a document the reader may not see. Some\nquestions turn on an exact word, such as a name, a part number or an error code,\nand pgvector's documentation suggests using vector search together with\nPostgreSQL's full-text search. That combination is called hybrid search.\n\n## What it looks like in practice\n\npgvector, at version 0.8.7 in October 2026, supports PostgreSQL 13 and later.\nEnable the extension once in each database, then give the table a column of type\n`vector` with the number of dimensions the model produces:\n\n```sql\nCREATE EXTENSION vector;\n\nCREATE TABLE passages (\n    id        bigserial PRIMARY KEY,\n    source    text NOT NULL,\n    body      text NOT NULL,\n    embedding vector(3)\n);\n\nCREATE INDEX ON passages USING hnsw (embedding vector_cosine_ops);\n```\n\nThree dimensions keep the example readable; a real model produces far more, and\nthe column declares the number it produces. The application computes each\nembedding with the model and inserts it next to the text it came from. The index\nnames one distance: `vector_cosine_ops` serves queries ordered by cosine\ndistance, and a query ordered by L2 distance needs a second index built with\n`vector_l2_ops`. HNSW has no training step, so the index can be created on an\nempty table.\n\nThe query embeds the question with the same model and asks for the nearest\npassages:\n\n```sql\nSELECT id, source, body\nFROM   passages\nORDER  BY embedding \u003c=\u003e $1\nLIMIT  5;\n```\n\n`\u003c=\u003e` is cosine distance and `$1` is the query vector, passed as a parameter.\nTwo details let the planner use the index: the query has both `ORDER BY` and\n`LIMIT`, and the `ORDER BY` is the bare distance operator in ascending order. A\nsimilarity score for display, `1 - (embedding \u003c=\u003e $1)`, belongs in the select\nlist. The other operators are `\u003c-\u003e` for L2 distance, `\u003c+\u003e` for L1 and `\u003c#\u003e` for\ninner product. That last one returns the negative inner product, because\nPostgreSQL scans an index on an operator in ascending order only.\n\nThe HNSW index has two build parameters: `m`, how many neighbours a node links\nto on each layer (16 unless set), and `ef_construction`, the size of the\ncandidate list while building, 64 by default. At query time, `hnsw.ef_search`\nsets the size of the candidate list, 40 by default. Raising it finds more of the\ntrue nearest neighbours and makes each query slower, and `SET LOCAL` raises it until the end of the current transaction. To limit\nthe setting to one query, put just that query in the transaction. To learn what the index costs in\nrecall, run the same query in a transaction with\n`SET LOCAL enable_indexscan = off`, which gives the exact answer, and compare\nthe two lists.\n\nThe company's [work on custom AI agents](/ai-agents/) keeps an agent's memory of\npast cases in the client's own database rather than in a vendor's chat history.\nA table like `passages` is one shape such memory can take: when a new case\narrives, the past cases most like it are one query away, in the same database\nand under the same permissions as the rest of the data.\n\nHybrid search runs both searches and merges the two lists. pgvector's\ndocumentation names Reciprocal Rank Fusion, from a 2009 paper by Cormack, Clarke\nand Büttcher: every document scores 1 / (k + rank) for each list it appears in,\nand the scores are added. The authors fixed k at 60, and their pilot runs showed\nthat 60 was \"near-optimal, but that the choice was not critical\". Given a\n`tsvector` column `search_vec` on the same table, built as for full-text search,\nthe fused query reads:\n\n```sql\nWITH semantic AS (\n    SELECT id, row_number() OVER (ORDER BY dist) AS r\n    FROM  (SELECT id, embedding \u003c=\u003e $1 AS dist\n           FROM   passages\n           ORDER  BY dist\n           LIMIT  20) AS nearest\n),\nkeyword AS (\n    SELECT id, row_number() OVER (ORDER BY score DESC) AS r\n    FROM  (SELECT id, ts_rank_cd(search_vec, query) AS score\n           FROM   passages, plainto_tsquery('english', $2) AS query\n           WHERE  search_vec @@ query\n           ORDER  BY score DESC\n           LIMIT  20) AS matches\n)\nSELECT id,\n       coalesce(1.0 / (60 + s.r), 0)\n     + coalesce(1.0 / (60 + k.r), 0) AS rrf\nFROM   semantic AS s\nFULL   JOIN keyword AS k USING (id)\nORDER  BY rrf DESC\nLIMIT  5;\n```\n\nEach search keeps its own top 20 and numbers them from 1. A passage that both\nsearches place high collects two shares and rises to the top; one found by only\none search keeps one share. The fusion uses ranks rather than scores because a\ncosine distance and a text rank sit on scales that cannot be compared. pgvector\nnames a cross-encoder as the other way to combine the lists.\n\n## Where it goes wrong\n\n**Fewer rows than `LIMIT` asked for.** With an approximate index, the `WHERE`\nclause is applied after the index scan. pgvector gives the arithmetic: if a\ncondition matches 10% of rows, an HNSW index with the default `hnsw.ef_search`\nof 40 leaves about 4 rows on average. Since version 0.8.0, iterative index scans\nscan more of the index until enough rows pass the filter or scan limits are\nreached. Turn them on with\n`SET hnsw.iterative_scan = strict_order`, or `relaxed_order` for better recall\nwith results slightly out of order. For HNSW,\n`hnsw.max_scan_tuples`, 20,000 by default, is an approximate limit on additional\ntuples visited and does not affect the initial scan. A separate memory limit\ncan also stop further scanning. For a filter with only a few distinct\nvalues, a partial index per value also works.\n\n**A query misses the nearest-neighbour index.** `ORDER BY 1 - (embedding \u003c=\u003e $1) DESC`\nsorts by an expression, and a query without `LIMIT` does not request a bounded\nnearest-neighbour result. Neither meets pgvector's requirements for using that\nindex. How many rows the chosen plan reads also depends on filters and other\nplanner choices. An index built with `vector_l2_ops` does nothing for a query\nordered by `\u003c=\u003e`. Order by the bare operator, ascending, with a `LIMIT`; build\none index per distance you query; and check the plan with\n`EXPLAIN (ANALYZE, BUFFERS)`.\n\n**An IVFFlat index built too early.** IVFFlat runs k-means over the rows present\nwhen the index is built and divides the vectors into lists from the result. An\nindex created with too little data for its number of lists returns fewer rows\nthan the query asked for, and pgvector's advice is to drop it until the table\nhas more data. Load the data first, then build. pgvector suggests starting with\nrows / 1000 lists up to a million rows and the square root of the row count\nabove that, and probes, 1 by default, at the square root of the number of lists.\nHNSW has no such training step.\n\n**Rows that are never found.** NULL vectors are not indexed, and neither are\nzero vectors under cosine distance, which has no direction to compare. A row\nwhose embedding is still missing is invisible to an indexed search. Add\n`NOT NULL` once loading is reliable, and run a scheduled count of rows that\nstill lack an embedding.\n\n**Vectors from two models in one column.** Two embedding models, or two versions\nof one, place the same text in different spots, and their vectors cannot be\ncompared even when the dimension counts agree. Embed the question with the model\nthat embedded the passages, store the model in a column, and re-embed everything\nwhen the model changes. For a column that holds vectors of several sizes,\npgvector shows an expression index with a condition per model,\n`WHERE (model_id = 123)`.\n\n**Too many dimensions for the index.** The `vector` type stores up to 16,000\ndimensions, but HNSW and IVFFlat index at most 2,000. Index a `halfvec` cast for\nup to 4,000, use binary quantization for up to 64,000 and re-rank the shortlist\nby the full vectors, or index a subvector when the model supports it.\n\n**One customer's vectors slowing another's.** pgvector warns that an approximate\nindex shared between tenants lets one tenant's vectors affect recall and speed\nfor the others. Partition the table by customer with list partitioning, or give\neach customer its own table.\n\n**An index build that blocks the application.** HNSW builds much faster when the\ngraph fits in `maintenance_work_mem`, and pgvector prints a notice when it no\nlonger does; setting that memory so high that the server runs short is its own\noutage. On a live system, build with `CREATE INDEX CONCURRENTLY`, which does not\nblock writes.\n\n**Passages the reader may not see.** A search over all passages can hand a\nlanguage model text from a document the user has no right to read, and the model\ncan quote it. Put the permission condition in the same `WHERE` clause as the\nsearch, so such a row never leaves the database, and turn on iterative scans to\nlook for more permitted passages when filtering removes candidates, subject to\nthe scan limits.\n\n**A distance read as confidence.** A cosine distance of 0.2 says a passage is\nnearer than one at 0.4, not that it answers the question. A threshold tuned for\none model means nothing for the next. Order by distance, cap with `LIMIT`, and\njudge the search by a set of real questions whose right passages are known.\n\n## Further reading\n\n- [pgvector](https://github.com/pgvector/pgvector) — the extension's own\n  documentation: types, distance operators, HNSW and IVFFlat options, filtering,\n  iterative index scans, hybrid search and troubleshooting.\n- [Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs](https://arxiv.org/abs/1603.09320)\n  — Malkov and Yashunin's paper that introduced HNSW.\n- [Reciprocal Rank Fusion outperforms Condorcet and individual Rank Learning Methods](https://cormack.uwaterloo.ca/cormacksigir09-rrf.pdf)\n  — the SIGIR 2009 paper with the fusion formula and the pilot runs behind its\n  constant.\n- [Introduction to full text search](https://www.postgresql.org/docs/current/textsearch-intro.html)\n  — PostgreSQL's word-based search, the other half of a hybrid query.\n- [Controlling Text Search](https://www.postgresql.org/docs/current/textsearch-controls.html)\n  — `plainto_tsquery` and `ts_rank_cd`, as used in the keyword half above.\n- [Transaction Isolation](https://www.postgresql.org/docs/current/transaction-iso.html)\n  — when PostgreSQL snapshots see committed changes, including why an older\n  snapshot may still see a document after its deletion commits.\n\n\u003chttps://engineer.company/glossary/vector-search/\u003e\n","date_modified":"2026-11-09T21:50:25Z","description":"What vector search is: embeddings compared by distance, exact and approximate indexes in pgvector, hybrid search with full-text, and where it breaks.","formats":{"gemtext":"https://engineer.company/glossary/vector-search/index.gmi","gopher":"https://engineer.company/glossary/vector-search/gophermap","html":"https://engineer.company/glossary/vector-search/","json":"https://engineer.company/glossary/vector-search/index.json","markdown":"https://engineer.company/glossary/vector-search/index.md","pdf":"https://engineer.company/glossary/vector-search/index.pdf"},"kind":"page","language":"en","title":"What is vector search?","url":"https://engineer.company/glossary/vector-search/","version":1}