{"canonical_url":"https://engineer.company/glossary/scalability/","content_markdown":"# Scalability\n\n## What it is\n\nScalability is how a system's capacity grows as resources are added to it. A\nsystem scales well when twice the machines carry roughly twice the load at the\nsame response times, and badly when the extra hardware mostly waits on\nsomething it cannot help with. The word describes a relationship between load\nand cost, not a feature a system either has or lacks.\n\nThere are two directions to grow in. Scaling up, or vertical scaling, means a\nbigger machine: more cores, more memory, faster disks. It needs no change to\nthe software and stops at the largest machine worth buying. Scaling out, or\nhorizontal scaling, means more machines sharing the work. It has no such\nceiling, but only work that can be divided gains from it.\n\nHTTP was shaped with scaling out in mind. RFC 9110 notes that many\nimplementations rely on HTTP being stateless to reuse proxied connections or to\nload balance requests dynamically across multiple servers. A web tier that keeps\nnothing between requests can grow by adding copies of itself behind a load\nbalancer, and any copy can answer any request.\n\nLoad has more than one dimension, and a system can scale along one and stall\nalong another: requests per second, the volume of stored data, the number of\ntenants, and the number of writers contending for the same record. The last is\nthe hardest, because a single row can be changed by one transaction at a time,\nhowever many servers stand around it.\n\nThe arithmetic behind that is Amdahl's law. When part of the work has to run in\nsequence, parallel hardware speeds up only the rest, so the sequential part\nsets the limit: with 5% of the work serial, the whole job can never run more\nthan 20 times faster, even on a thousand machines. In a database the serial\npart is often a lock. In an application it is often one shared counter, queue\nor session store.\n\n## Why it matters\n\nScalability decides what growth costs. When a system scales well, a doubling of\ntraffic is a line on an invoice. When it scales badly, the bill grows faster\nthan the load, or more hardware stops helping altogether, and the fix becomes a\nredesign carried out while the traffic is arriving.\n\nHardware can relieve bottlenecks in processors, memory or disks, but inefficient\nwork and serialization can remain. A query that needlessly scans a whole table\ncan still waste work on a larger server. A bigger machine does not remove the\nserialization of updates to one shared row, and more application servers can add\nwriters to the queue. Most systems that look as if they need more machines need\nreading first.\n\nIn PostgreSQL the place to start reading is `pg_stat_statements`, an extension\nthat records each normalised statement with its call count and its execution\ntime. Sorted by total time rather than by mean, it shows where the server\nactually spends itself: a query that takes 2 ms and runs a million times an\nhour costs 2,000 seconds of that hour, while a two-second report run twice\ncosts four.\n\n```sql\nSELECT calls, round(total_exec_time) AS total_ms,\n       left(query, 60) AS query\n  FROM pg_stat_statements\n ORDER BY total_exec_time DESC\n LIMIT 10;\n```\n\nEach step towards more machines brings running costs of its own, which is why\nthe order matters. A tighter statement or a better index can improve performance\nwithout adding another server. A read replica is configuration. Partitioning\nchanges one table. Sharding, splitting the data across separate database\nservers, changes the surrounding operations: migrations, backups, restores and\nreports have to account for separate shards, joins across shards may move into\napplication code, and a query that does not know its shard key may have to ask\nall of them.\n\n## What it looks like in practice\n\nA PostgreSQL database behind a mid-sized application held user accounts,\ntenants, geographical data and daily electricity consumption. It ran under low\ntraffic, yet slowed as the data volumes grew. The brief was to make it\n[ten times faster without overhauling the architecture](/portfolio/accelerated-postgresql-performance-by-10x/),\nwith room for future data growth as part of the goal. The work was GiST indexes\nfor the spatial data, composite indexes on tenant and date range, the\nelectricity table partitioned by time, hash partitioning for the geographical\ndata, rewritten queries checked with `EXPLAIN ANALYZE`, and connection pooling.\nResponse times improved tenfold, the database's resource use fell by 40%, and\nthe system took growing data volumes without slowing down, with less need for\nhardware upgrades.\n\nAt a mid-sized company running ten PostgreSQL and SQL Server database servers,\nthe infrastructure was\n[scaled to carry a 40% rise in user traffic without service degradation](/portfolio/ran-10-postgresql-and-ms-sql-servers-on-ubuntu/).\nAlongside it, query response times improved by 30% through configuration\ntuning and index optimisation.\n\nThe order the work follows is set out in\n[a note on optimising before sharding](/notes/optimise-before-you-shard/):\nlook, tighten, take the reads off the primary, partition the tables around the\nhot spot, and shard only when one machine is full of good work. It names three\ncases where sharding is reasonable: the primary is saturated by statements\nthat are already lean, the data has outgrown its maintenance, or isolation is\nitself the requirement.\n\n## Where it goes wrong\n\n**More web servers in front of one database.** Each application server opens\nits own connections, and when many times more sessions are active than the\ndatabase has cores, the server spends its time switching between them. A\nconnection pool fixes that before any hardware does, and another web server\nmakes it worse.\n\n**One hot row.** When every request writes to the same record, such as a stock\ncounter or a balance, throughput is capped at roughly the writes one lock can\ncarry divided by how long the lock is held, on any number of machines.\nAnything slow inside that lock lowers the ceiling. Keep the transaction short,\nkeep external calls outside it, and when one item outgrows its machine, split\nits stock into smaller pieces that each have exactly one owner.\n\n**State that lives on one server.** Sessions held in one process's memory,\nuploads written to its local disk, or a scheduled job that assumes it is the\nonly copy all stop a second instance from being an equal one. Keep that state\nin a shared store, so that any instance can answer any request.\n\n**Replicas treated as copies of the primary.** A streaming replica runs a\nlittle behind, so a user who has just written something and then reads from the\nreplica may not see it; read recent writes from the primary. A long report on\na standby can also be cancelled when it holds up the changes being replayed.\nPostgreSQL documents these as query conflicts, together with the settings that\ntrade replica lag against them.\n\n**Partitions the queries never name.** Partitioning splits one table by a key,\nsuch as a month or a tenant, so a query that names the key reads only the\npartitions it needs. A query that leaves the key out reads every partition and\ngains nothing from the split. Partition by what the queries already filter on,\nand expect it to pay off only for large tables: the PostgreSQL manual's rule of\nthumb is a table bigger than the database server's physical memory.\n\n## Further reading\n\n- [RFC 9110: HTTP Semantics, section 3.3](https://www.rfc-editor.org/rfc/rfc9110#section-3.3)\n  — connections, clients and servers, and the stateless design that lets\n  implementations spread requests across multiple servers.\n- [PostgreSQL: Table Partitioning](https://www.postgresql.org/docs/current/ddl-partitioning.html)\n  — what partitioning is, when it pays, and how partition pruning skips the\n  partitions a query cannot need.\n- [PostgreSQL: pg_stat_statements](https://www.postgresql.org/docs/current/pgstatstatements.html)\n  — the extension that keeps call counts and timings for every normalised\n  statement.\n- [PostgreSQL: Hot Standby](https://www.postgresql.org/docs/current/hot-standby.html)\n  — read-only queries on a standby and the conflicts that can cancel them.\n- [RFC 9111: HTTP Caching](https://www.rfc-editor.org/rfc/rfc9111)\n  — how HTTP caches store and reuse responses, so repeat requests need not\n  reach the origin server.\n\n\u003chttps://engineer.company/glossary/scalability/\u003e\n","date_modified":"2026-11-09T21:50:25Z","description":"What scalability means, scaling up versus out, why a shared row or a serial step caps growth, and what to measure before adding a database server.","formats":{"gemtext":"https://engineer.company/glossary/scalability/index.gmi","gopher":"https://engineer.company/glossary/scalability/gophermap","html":"https://engineer.company/glossary/scalability/","json":"https://engineer.company/glossary/scalability/index.json","markdown":"https://engineer.company/glossary/scalability/index.md","pdf":"https://engineer.company/glossary/scalability/index.pdf"},"kind":"page","language":"en","title":"What is scalability?","url":"https://engineer.company/glossary/scalability/","version":1}