{"canonical_url":"https://engineer.company/glossary/database-index/","content_markdown":"# Database index\n\n## What it is\n\nA database index is a separate structure, stored alongside a table, that lets\nthe database go straight to the rows matching a condition instead of reading the\ntable from the first row to the last. In everyday talk, and in most of the\nPostgreSQL documentation, it is simply an index. The documentation compares it\nto the index at the back of a book: a reader who wants one topic looks it up\nthere and turns to the pages listed, rather than leafing through the whole\nvolume.\n\nTake a table of orders and the condition `WHERE customer_id = 42`. With no index\non `customer_id`, PostgreSQL has one way to answer: a sequential scan, which\nreads every row and keeps the ones that match. That costs the same whether the\ncustomer has two orders or two thousand, and it grows with the table. With an\nindex on the column, PostgreSQL walks a few levels of a search tree to the\nentries for 42 and fetches only those rows. Creating an index is a single\nstatement, `DROP INDEX` removes it again, and either can happen at any time.\n\nOnce an index exists, PostgreSQL keeps it in step with the table on every write,\nand the query planner decides, query by query, whether using it is cheaper than\na scan. Nothing in the SQL names the index, so one index can speed up a\n`SELECT`, an `UPDATE` or `DELETE` with a search condition, and a join.\n\nPostgreSQL ships several kinds, each built for a different shape of question:\n\n- **B-tree** is the default, and what `CREATE INDEX` builds unless told\n  otherwise. It answers equality and range comparisons (`\u003c`, `\u003c=`, `=`, `\u003e=`,\n  `\u003e`), `BETWEEN`, `IN`, `IS NULL`, and a `LIKE` whose pattern is anchored at\n  the start. It keeps its entries in order, so it can also return rows already\n  sorted.\n- **Hash** stores a 32-bit hash code of each value and answers equality only.\n- **GiST** and **SP-GiST** are frameworks for data such as geometry, where the\n  question is \"overlaps\", \"contains\" or \"nearest to\" rather than \"equals\".\n- **GIN** is an inverted index for values made of many parts, such as arrays,\n  and finds the rows whose value contains a given element.\n- **BRIN** keeps only a summary for each range of table blocks. It suits a\n  column whose values follow the physical order of the rows, as an insert\n  timestamp often does.\n\nOn top of its type, an index can be shaped to the question. A unique index\nrefuses duplicate values. A multicolumn index covers several columns, up to 32.\nA partial index has a `WHERE` clause and holds only the rows that satisfy it. An\nexpression index stores a computed value such as `lower(email)`. And an\n`INCLUDE` list carries extra columns in the index without searching on them, so\nthat a query can be answered from the index alone.\n\n## Why it matters\n\nThe difference an index makes grows with the table. On a small table a scan is\ncheap, and the planner will often, rightly, prefer one: reading a handful of\npages in order costs less than walking a tree to reach them. On tens of millions\nof rows, the gap between a scan and a lookup is the gap between a page that\nloads and one that times out. A query that was quick at launch can turn slow a\nyear later without a line of code changing, simply because the table it reads\nhas grown.\n\nIndexes are not free, and the bill arrives on writes. Every insert adds an entry\nto every index on the table. An update can avoid that only as a heap-only tuple\n(HOT) update, where the new row version stays on the same page and the index\nentries already there still lead to it. PostgreSQL allows that only when the\nupdate leaves alone every column the table's indexes use, BRIN indexes aside,\nsince they hold only a summary. Each index therefore narrows the cheap path,\nwhich is why the documentation advises dropping indexes that are seldom or never\nused. A table with a dozen indexes pays a dozen times on every insert.\n\nBuilding an index costs something too. A plain `CREATE INDEX` lets other\nsessions read the table but blocks its writes until the build finishes, and the\ndocumentation warns that on a very large table that can take many hours.\n\nFinally, an index is only as good as the planner's decision to use it, and the\nplanner works from statistics gathered by `ANALYZE`. The question to ask is not\n\"is there an index?\" but \"do the queries that matter use it?\", and `EXPLAIN` is\nhow to find out.\n\n## What it looks like in practice\n\nAn online shop's account page lists a customer's twenty most recent orders:\n\n```sql\nSELECT id, created_at, total\nFROM   orders\nWHERE  customer_id = 42\nORDER  BY created_at DESC\nLIMIT  20;\n```\n\nWith only a primary key on `id`, PostgreSQL has to read the whole of `orders`,\nkeep the customer's rows, sort them and throw away all but twenty. `EXPLAIN`\nshows it: a `Seq Scan` on `orders` with the condition as a `Filter`, and a sort\nabove it. `EXPLAIN ANALYZE` also runs the query and prints the real row counts\nand timings beside the estimates.\n\nThe index that fits this question puts the equality column first and the sort\ncolumn second:\n\n```sql\nCREATE INDEX CONCURRENTLY orders_customer_created_idx\nON orders (customer_id, created_at);\n```\n\nRun `EXPLAIN` again and the scan becomes an index scan using\n`orders_customer_created_idx`, with `customer_id = 42` now shown as an\n`Index Cond` rather than a filter. The sort goes too. A B-tree keeps the entries\nfor customer 42 ordered by `created_at`, PostgreSQL can read them backwards to\nget the newest first, and with a `LIMIT` it stops after twenty entries instead\nof touching the rest.\n\nThe order of the columns matters. The same index serves a query on `customer_id`\nalone, its leading column, but a report on every order placed yesterday,\nfiltering on `created_at` alone, gets far less from it: in a multicolumn B-tree,\nequality on the leading columns plus a range on the next one are what narrow the\npart of the index that is read. PostgreSQL 18 can skip-scan, repeating the\nsearch once per distinct leading value, but the planner usually picks that only\nwhen the leading column has few distinct values, and customer IDs have many. If\nthe report matters, it gets its own index on `created_at`. Turned round, as\n`(created_at, customer_id)`, the index would scatter customer 42's orders across\nthe whole of it.\n\nThe note on\n[optimising the database before you shard it](/notes/optimise-before-you-shard/)\nputs indexes first among the fixes it ranks by payoff, once\n`EXPLAIN (ANALYZE, BUFFERS)` has shown which scans a hot query runs. Its own\nexample is a partial index, `units_unsold`, holding only the units still for\nsale, so it shrinks as a sale goes on instead of growing. The note also counts\nthe price. On its table, 2,000 claims that set `reserved`, `reserved_at` and\n`order_id` were all HOT updates, while of 2,000 updates that set `sold`, a\ncolumn the partial index reads, none were. Its advice is to drop the indexes\nwhose `idx_scan` in `pg_stat_user_indexes` stays at zero.\n\nThe case study on\n[a better choice of key](/portfolio/quicker-answers-from-a-better-choice-of-key/)\nshows why the values going into a B-tree matter as much as the columns. Random\nUUIDs scatter inserts across the index, which can cause fragmentation. The\nplatform made time-ordered UUID v7 the default for entity primary keys, with a\ntimestamp in the leading bits so new identifiers carry time ordering. Reduced\nfragmentation and faster queries were design aims; the recorded result is the\nconfigured key convention, with no before-and-after workload benchmark\nestablished by the case.\n\n## Where it goes wrong\n\n**Adding an index stops the table's writes.** A plain `CREATE INDEX` on a busy\ntable blocks inserts, updates and deletes for the whole build, and a routine\ndeployment turns into an outage. Use `CREATE INDEX CONCURRENTLY`: it scans the\ntable twice and waits for transactions that could use the index, so it takes\nsignificantly longer, but writes carry on. It cannot run inside a transaction\nblock, so a migration tool that wraps every step in one has to be told to run\nthis statement on its own. Only one concurrent build can run on a table at a\ntime.\n\n**A concurrent build fails and leaves an invalid index behind.** A deadlock, or\na duplicate value while building a unique index, makes\n`CREATE INDEX CONCURRENTLY` fail, but the index it started stays in the\ncatalogue, marked invalid. Queries ignore it, yet every write still pays to\nmaintain it, and an invalid unique index can go on enforcing uniqueness. psql's\n`\\d` lists it with `INVALID` after its definition. Drop it and run the\nconcurrent build again, or rebuild it with `REINDEX INDEX CONCURRENTLY`.\n\n**The query hides the column inside a function.** An index on `email` does\nnothing for `WHERE lower(email) = 'ana@example.com'`, because the index holds\n`email`, not `lower(email)`. Index the expression the query uses:\n\n```sql\nCREATE INDEX users_lower_email_idx ON users (lower(email));\n```\n\nThe expression is computed on every insert and every update that is not HOT,\nwhich is why the documentation recommends expression indexes where retrieval\nspeed matters more than write speed.\n\n**Indexes pile up that nobody uses.** Each one added \"just in case\" costs every\nwrite and can rule out HOT updates. `pg_stat_user_indexes` counts the scans\nstarted on each index in `idx_scan` and records the time of the last one in\n`last_idx_scan`:\n\n```sql\nSELECT relname, indexrelname, idx_scan, last_idx_scan\nFROM   pg_stat_user_indexes\nWHERE  idx_scan = 0\nORDER  BY relname;\n```\n\nLook before dropping. An index behind a primary key or unique constraint does\nits job whether or not a query ever reads it, and a count of zero means little\non a server that has only just started taking traffic.\n\n**The planner ignores a good index.** After a bulk load the statistics lag\nbehind the data, the estimates are wrong, and a scan wins on paper. Run\n`ANALYZE`; the documentation calls judging index use without it \"a lost cause\".\nTest on realistic volumes too: on a hundred rows that fit in one disk page, no\nindex beats reading that page. To learn whether an index can be used at all, set\n`enable_seqscan = off` in a test session and run `EXPLAIN` again. If the plan\nstill scans, the condition does not match the index.\n\n## Further reading\n\n- [Indexes: Introduction](https://www.postgresql.org/docs/current/indexes-intro.html)\n  — what an index is for, how it is created and dropped, and the overhead it\n  adds to every write.\n- [Index Types](https://www.postgresql.org/docs/current/indexes-types.html) —\n  B-tree, Hash, GiST, SP-GiST, GIN and BRIN, and the operators each one can\n  answer.\n- [Multicolumn Indexes](https://www.postgresql.org/docs/current/indexes-multicolumn.html)\n  — how column order decides what a B-tree can narrow, and when skip scan\n  applies.\n- [CREATE INDEX](https://www.postgresql.org/docs/current/sql-createindex.html) —\n  the full syntax, including `CONCURRENTLY`, `INCLUDE`, partial indexes and\n  recovering from an invalid build.\n- [Using EXPLAIN](https://www.postgresql.org/docs/current/using-explain.html) —\n  reading a plan: scan types, estimated costs and the real figures from\n  `EXPLAIN ANALYZE`.\n- [Examining Index Usage](https://www.postgresql.org/docs/current/indexes-examine.html)\n  — why `ANALYZE` and real data come first, and how to test whether an index can\n  be used.\n\n\u003chttps://engineer.company/glossary/database-index/\u003e\n","date_modified":"2026-11-08T02:22:23Z","description":"How a database index turns a table scan into a lookup: the index types PostgreSQL offers, the price every write pays, and how to tell one is being used.","formats":{"gemtext":"https://engineer.company/glossary/database-index/index.gmi","gopher":"https://engineer.company/glossary/database-index/gophermap","html":"https://engineer.company/glossary/database-index/","json":"https://engineer.company/glossary/database-index/index.json","markdown":"https://engineer.company/glossary/database-index/index.md","pdf":"https://engineer.company/glossary/database-index/index.pdf"},"kind":"page","language":"en","title":"What is a database index?","url":"https://engineer.company/glossary/database-index/","version":1}