{"canonical_url":"https://engineer.company/glossary/rate-limiting/","content_markdown":"# Rate limiting\n\n## What it is\n\nRate limiting is a cap on how many requests one client may send to a service\nwithin a given stretch of time, enforced by the service or by something standing\nin front of it. Every limit answers three questions: who counts as the client,\nhow the requests are counted, and what happens to the request that goes over.\nThe HTTP status code built for the last of these, 429 Too Many Requests, was\ndefined in RFC 6585 in April 2012, and that RFC deliberately leaves the first\ntwo open. A server may count per resource, across the whole server or across a\ngroup of servers, and may recognise the client by its authentication credentials\nor by a stateful cookie.\n\nIn practice the key is one of a handful of things: the client's IP address, the\nlogged-in account, an API key, or one of those combined with the path, so that a\nlogin form and a product page get different limits. The answer to a request over\nthe limit comes in three kinds. The server can refuse it with a 429, preferably\nwith a `Retry-After` header that says how long to wait, given either as a number\nof seconds or as an HTTP date. It can hold the request back and process it a\nmoment later, which smooths a burst instead of rejecting it. Or, when the volume\nitself is the attack, it can drop the connection without answering. RFC 6585\nsuggests exactly that, because writing a polite 429 for every request in a flood\ncosts resources too.\n\nHow the counting works decides how a limit behaves at its edges. A fixed window\ncounts requests per calendar minute or hour and starts again from zero at the\nboundary. It is cheap and easy to explain, and it lets a client spend its whole\nallowance in the last second of one window and again in the first second of the\nnext: with a limit of 100 per minute, that is 200 requests in two seconds. A\nsliding window always looks back over the same span from the present moment, so\nthere is no boundary to exploit; HAProxy's stick tables keep counters of this\nkind, and one example in its manual counts the connection rate over a 30-second\nsliding window. The leaky bucket, which nginx uses, lets requests through at a\nfixed average rate and keeps a small queue for those that arrive faster,\nrefusing whatever would overflow it. A token bucket works the other way round:\ntokens accumulate at a steady rate up to a ceiling, each request spends one, and\na client that has been quiet for a while can spend its savings in a burst.\n\n## Why it matters\n\nA public endpoint has no say in who calls it or how often. A scraper copying a\ncatalogue, a script stuck in a retry loop, an attacker trying leaked passwords\nagainst the login form and a partner whose nightly job was written in a hurry\nall look the same to the server: more requests than it was sized for, from a few\nsources. Without a limit, the busiest caller decides how much capacity everybody\nelse gets. With one, a single client can only use its share, and the rest of the\nservice carries on at normal speed while that client waits.\n\nSome requests are far more expensive than others. A product page might be served\nfrom a cache in a millisecond, while a search, an export or anything that calls\na paid external API costs real CPU time or real money on every call. Those are\nthe paths an abuser targets, precisely because they cost the caller nothing. A\nsingle global limit set low enough to protect the export would strangle ordinary\nbrowsing, so useful rate limiting is set per path, with each limit chosen for\nwhat that path costs and how it is legitimately used.\n\nA limit is also part of an API's contract. Clients that know the rules can pace\nthemselves and never hit them, which is cheaper for both sides than discovering\nthe limit through refusals. HTTP has no standard header for it. Many APIs send\n`X-RateLimit-Limit`, `X-RateLimit-Remaining` and `X-RateLimit-Reset`, but the\nsame name carries different values from one implementation to the next, as an\nIETF draft that sets out to standardise the idea records. That draft defines two\nfields, `RateLimit-Policy` for the quota and its window and `RateLimit` for what\nremains; its eleventh version is dated 23 May 2026 and, like every\nInternet-Draft, is a work in progress rather than a standard.\n\n## What it looks like in practice\n\nIn nginx, a limit takes two directives: one in the `http` block that sets up the\ncounter, and one where it applies:\n\n```nginx\nlimit_req_zone $binary_remote_addr zone=perip:10m rate=1r/s;\nlimit_req_status 429;\n\nserver {\n    location /search/ {\n        limit_req zone=perip burst=5 nodelay;\n        proxy_pass http://app;\n    }\n}\n```\n\nThe key is the client's address in binary form, 4 bytes long for an IPv4 address\nand 16 for an IPv6 one, and the counters live in a 10-megabyte shared memory\nzone called `perip`. The documentation puts the state for one key at 128 bytes\non a 64-bit platform, so a megabyte holds about 8 thousand addresses and this\nzone about 80 thousand; when it fills up, nginx evicts the least recently used\none. `rate=1r/s` allows one request per second on average. `burst=5` lets up to\nfive requests above that rate wait their turn instead of being refused, and\n`nodelay` serves them at once instead of spacing them out at the configured\nrate. Anything beyond the burst is rejected. nginx would answer that rejection\nwith 503 by default, which is why `limit_req_status` sets 429. Rates below one\nper second are written per minute, so `30r/m` is one request every two seconds.\nBefore a new limit goes live, `limit_req_dry_run on` keeps counting as usual and\nmarks would-be refusals as `REJECTED_DRY_RUN` in the `$limit_req_status`\nvariable, without refusing anything.\n\nOn the receiving end, a client that hits the limit should read the answer rather\nthan retry at once. A 429 from an API that follows the IETF draft might look\nlike this:\n\n```http\nHTTP/1.1 429 Too Many Requests\nContent-Type: application/problem+json\nRetry-After: 20\nRateLimit-Policy: \"default\";q=100;w=60\nRateLimit: \"default\";r=0;t=20\n```\n\nThe policy allows 100 requests per 60-second window, none are left, and the\nquota comes back in 20 seconds. Where both are present, the draft says\n`Retry-After` wins. A well-behaved client sleeps for at least that long, adds a\nsmall random delay, and if there is no `Retry-After` at all, falls back to\nexponential backoff rather than a fixed interval.\n\nThe\n[22 rate limiters built to fail closed](/portfolio/built-fail-closed-rate-limiting-and-anti-abuse/)\nfor a public directory of companies and professionals, from 2025, follow the\nper-path rule: each was built for the path it protects, because a login attempt,\na search and a bulk export become abuse at very different rates. Their state\nlives in Redis, so a limit holds across every instance instead of being kept per\nprocess. The decisive choice was what happens when Redis is down: the limiters\nrefuse traffic rather than wave it through. The page is frank about the price,\nthat a Redis problem becomes one users can see, and about why it was accepted:\nthe alternative was a Redis problem turning into a billing one. Cloudflare\nTurnstile, an origin lock and 351 lines of idempotency middleware, so that a\nretried write is not applied twice, sit around them.\n\n## Where it goes wrong\n\n**Each server keeps its own count.** A limit held in process memory multiplies\nwith every copy of the application: four instances behind a load balancer let a\nclient through at four times the intended rate, and every restart resets the\ncounters. nginx's zone is shared memory on one machine, and keeping zones in\nstep across servers is part of its commercial subscription; HAProxy's peers\nprotocol copies stick-table entries between instances but, its manual notes,\nwithout adding them up. Keep the counters in a store every instance consults, as\nthe directory above does with Redis, or enforce the limit at the single point\nall traffic passes through.\n\n**Everyone arrives from the same address.** Behind a reverse proxy, a CDN or a\nload balancer, the application sees one client address for all users unless the\noriginal one is passed along, and a per-IP limit then throttles everybody at\nonce. A whole office or mobile network behind NAT has the same effect in a\nmilder form. Key the limit on the real client address taken from a header that\nonly your own proxy may set, and for logged-in traffic prefer the account or API\nkey over the address.\n\n**The limiter reports a server fault.** nginx's default answer to a rejected\nrequest is 503 Service Unavailable, which in HTTP means the server is overloaded\nor down for maintenance. Dashboards and alerts that treat 5xx responses as\nserver errors will then wake someone up because a single client was greedy, and\nthe client learns nothing about its own behaviour. Answer with 429, which puts\nthe cause on the client's side, and log the rejections at a level of their own.\n\n**Every throttled client comes back in the same second.** If a quota resets at a\nfixed moment, or every refused client is told the same `Retry-After`, they all\nreturn together; the IETF draft gives the example of a daily reset at 18:00:00,\nwhen all clients are likely to show up at 18:00:00. The burst that follows can\nbe worse than the one that was limited. Add jitter on both sides: vary the wait\nthe server announces, and let clients add a random delay of their own.\n\n**Nobody decided what happens when the store goes down.** A limiter that keeps\nits counts in Redis or a database must do something when the store is\nunreachable. Code that catches the connection error and lets the request through\nhas chosen to fail open, which turns the outage into an unlimited window for\nwhoever is watching; code that refuses has chosen to fail closed, which turns it\ninto an outage for everyone. Neither is wrong everywhere, but the choice belongs\nin a design decision per path, as the directory above made it, not in whatever a\nlibrary does by default.\n\n**Counting refusals leaks information.** If 401 and 403 responses count against\na quota, the IETF draft warns, a malicious client could probe an endpoint to\nlearn about another user's traffic. Decide which responses count, give failed\nauthentication a limit of its own, and keep quota details from clients that have\nnot authenticated.\n\n## Further reading\n\n- [RFC 6585: Additional HTTP Status Codes](https://www.rfc-editor.org/rfc/rfc6585.html#section-4)\n  — defines 429 Too Many Requests and states what the code leaves to the server:\n  who the user is and how requests are counted.\n- [RFC 9110: HTTP Semantics](https://www.rfc-editor.org/rfc/rfc9110.html#section-10.2.3)\n  — the `Retry-After` field and its two forms, and what 503 means when the\n  server sends it.\n- [RateLimit header fields for HTTP](https://datatracker.ietf.org/doc/draft-ietf-httpapi-ratelimit-headers/)\n  — the IETF HTTPAPI working group's draft of `RateLimit-Policy` and\n  `RateLimit`, with a security section on synchronised returns and information\n  leaks.\n- [Module ngx_http_limit_req_module](https://nginx.org/en/docs/http/ngx_http_limit_req_module.html)\n  — `limit_req`, `burst`, `nodelay`, `delay`, dry-run mode and the sizes of the\n  shared memory zone.\n- [HAProxy 3.4 Configuration Manual](https://docs.haproxy.org/3.4/configuration.html)\n  — stick tables and their rate counters, and the peers protocol that copies\n  them between instances.\n\n\u003chttps://engineer.company/glossary/rate-limiting/\u003e\n","date_modified":"2026-10-29T11:17:41Z","description":"How a service caps the requests one client may send, what it answers when the cap is hit, and the mistakes that let a limit leak or misfire.","formats":{"gemtext":"https://engineer.company/glossary/rate-limiting/index.gmi","gopher":"https://engineer.company/glossary/rate-limiting/gophermap","html":"https://engineer.company/glossary/rate-limiting/","json":"https://engineer.company/glossary/rate-limiting/index.json","markdown":"https://engineer.company/glossary/rate-limiting/index.md","pdf":"https://engineer.company/glossary/rate-limiting/index.pdf"},"kind":"page","language":"en","title":"What is rate limiting?","url":"https://engineer.company/glossary/rate-limiting/","version":1}