{"canonical_url":"https://engineer.company/glossary/exponential-backoff/","content_markdown":"# Exponential backoff\n\n## What it is\n\nExponential backoff is a rule for how long to wait before trying again. After\neach failed attempt, the wait is multiplied by a fixed factor, most often two,\nuntil it reaches a ceiling: one second, then two, four, eight and so on. The\nrule lives in the client, but it exists for the sake of whatever the client is\ncalling. A server, a network path or a database that fails under too much load\ndoes not recover sooner for being asked again at once. It recovers when its\ncallers ease off, and a wait that grows with every failure makes them ease off\nmore the longer the trouble lasts.\n\nThe oldest and most widely deployed example is in TCP. RFC 6298, the Standards\nTrack document from June 2011 that specifies how TCP computes its retransmission\ntimeout, sets that timeout to one second before the first round-trip time has\nbeen measured, and says that whenever the computed value falls below one second\nit should be rounded up. Each time the timer expires and a segment has to be\nsent again, the host must double the timeout. An upper bound is allowed, as long\nas it is at least 60 seconds. When a fresh round-trip measurement arrives, the\ntimeout is computed from the measurements again, which can bring it straight\nback down after a series of doublings. The algorithm goes back to Van Jacobson's\n1988 paper Congestion Avoidance and Control, and the RFC's security section\nstates what is at stake: the Internet relies on a correct retransmission timer\nto keep the network stable and to avoid congestion collapse.\n\nIn application code the same idea comes with a handful of settings: an initial\nwait, a multiplier, a maximum wait at which the growth stops, and a limit on the\nnumber of attempts or on the total time spent retrying. Written out, the wait\nafter the nth failure is the smaller of the cap and base × multiplier to the\npower of n, counting from zero. The AWS analysis discussed below calls this\ncapped exponential backoff. The cap matters more than it looks. Doubling a\none-second wait ten times gives 1,024 seconds, more than 17 minutes, which is\nlonger than most callers will wait for anything.\n\nThe last setting is jitter, a random element in each wait. The gRPC connection\nbackoff protocol, which describes how gRPC clients reconnect after a failed\nconnection, starts at one second, multiplies by 1.6, stops growing at 120\nseconds and applies a jitter of 0.2, so each wait is moved up or down at random\nby as much as 20%. It also gives every connection attempt at least 20 seconds to\ncomplete, and it resets the backoff to its initial value once the server has\naccepted a connection, which it takes to be the moment the SETTINGS frame\narrives. The protocol allows other implementations, for example on a mobile\nphone that should wake up as rarely as possible, but requires implementations to\nspread simultaneous retries apart and to avoid a substantially higher attempt\nrate.\n\n## Why it matters\n\nA retry is extra load, and it arrives at the worst moment: exactly when the\nthing being called is failing. If a service slows down because it is overloaded,\nand every caller responds by trying again at once, the load rises as capacity\nfalls. The Google Cloud Storage documentation lists retrying without backoff as\nan anti-pattern for that reason, since immediate or very short retries can lead\nto cascading failures, in which one failure triggers others. Backoff turns the\nretry from an amplifier into a brake: the longer the trouble lasts, the less\neach client adds to it.\n\nBackoff on its own is not enough, because clients that fail together also wait\ntogether. In Exponential Backoff And Jitter, published by AWS in March 2015,\nMarc Brooker simulated many clients using optimistic concurrency control to\nupdate the same row in a remote database. With every client competing in every\nround, the total work grew with the square of the number of clients. Capped\nexponential backoff helped only a little. The calls still came in clusters, with\nquiet gaps in between, so contention hardly fell. Adding jitter, by drawing each\nwait at random between zero and the capped exponential value, removed the gaps,\nleft an approximately constant rate of calls, and in the case with 100\ncontending clients cut the number of calls by more than half. Of the variants he\ncompared, backoff without jitter was, in Brooker's words, \"the clear loser\". He\nwas also explicit about the limit: none of the approaches changes the fact that\nthe work grows with the square of the contention. Jitter reduces the waste; it\ndoes not remove the conflict. In a May 2023 update, the post notes that most AWS\nSDKs now include exponential backoff and jitter in their standard and adaptive\nretry modes.\n\nBackoff decides when to try again, not whether it is safe to. The Cloud Storage\ndocumentation puts two questions first. One is whether the response points to a\ntransient problem: HTTP 408, 429 and 5xx status codes, socket timeouts and TCP\ndisconnects. The other is whether the request is idempotent. A list operation\ncan be repeated freely. Creating a Pub/Sub notification cannot, because each\nsuccessful call creates a new notification. Between the two sit conditionally\nidempotent operations, which become safe to retry when they carry a\nprecondition, such as an upload that succeeds only if the object's generation\nnumber still matches the one the client expects.\n\nSometimes the server knows better than the client's formula. The Retry-After\nheader field, defined in RFC 9110, tells the client how long it ought to wait\nbefore a follow-up request, either as a date or as a number of seconds, and with\na 503 Service Unavailable response it says how long the service expects to be\nunavailable. The 429 Too Many Requests status from RFC 6585, which signals rate\nlimiting, may carry the same field. A client with a backoff loop can use that\nvalue instead of its own computed wait, since it comes from the side that can\nsee its own load.\n\n## What it looks like in practice\n\nA retry loop with capped exponential backoff and full jitter fits in a few lines\nof Python:\n\n```python\nimport random\nimport time\n\nclass TransientError(Exception):\n    \"\"\"A timeout, a 408, 429 or 5xx response, or a dropped connection.\"\"\"\n\ndef call_with_backoff(op, attempts=5, base=1.0, cap=30.0):\n    for attempt in range(attempts):\n        try:\n            return op()\n        except TransientError:\n            if attempt == attempts - 1:\n                raise\n            ceiling = min(cap, base * 2**attempt)\n            time.sleep(random.uniform(0, ceiling))\n```\n\nWith these values, five attempts are separated by four waits whose ceilings are\n1, 2, 4 and 8 seconds. The loop therefore sleeps for at most 15 seconds before\nit gives up, and for about 7.5 seconds on average, since each wait is drawn\nevenly from zero to its ceiling. Only `TransientError` is retried; an\nauthorization failure or a malformed request raises at once. And the loop\nrepeats whatever `op` does, so `op` has to be safe to repeat: a read, or a write\nthat the server can recognise as a repeat.\n\nClient libraries ship their own versions of this loop, and their defaults\ndiffer, even for the same service. For Cloud Storage, the Python client starts\nat one second, doubles each wait up to 60 seconds and gives up after a default\ndeadline of 120 seconds. The C++ client also starts at one second and doubles,\nbut it waits up to five minutes between attempts and keeps retrying for up to 15\nminutes, more than seven times as long. It also retries operations that are\nnever idempotent, unless the program opts into a strict idempotency policy. The\ngcloud command line uses what Google calls truncated binary exponential backoff,\nwith up to 32 retries by default. Two programs that call the same API with the\nsame failure can therefore report it after two minutes or after a quarter of an\nhour, depending on the language they are written in.\n\nThe waiting does not have to block a thread. In background work, a job that\nfails can go back on a [queue](/glossary/#queue) with a later due time, and the\nbackoff becomes a property of the job rather than of a sleeping process. That is\nthe natural place for it whenever nobody is waiting on the answer.\n\nIn\n[email with failover and delivery logging](/portfolio/built-email-with-failover-and-delivery-logging/),\nfrom 2025, backoff sits on the asynchronous side of the system. Mail goes out\nthrough three providers, with SendGrid as the primary and SMTP2GO and Azure\nCommunication Services behind it, and failover between them is automatic. The\nsend path itself goes through a client with a 30-second timeout, and the\nasynchronous delivery path has retries and backoff, so that a short hiccup does\nnot cost a message. The page sums up the result: when one provider has a bad\nafternoon, mail takes a degraded path rather than suffering an outage. It also\nnames the price, which is three integrations to keep working instead of one.\n\n## Where it goes wrong\n\n**Every client waits the same amount.** Without jitter, clients that failed\ntogether retry together, and the server sees the same spike again after one,\ntwo, four and eight seconds. Brooker's simulation found that plain exponential\nbackoff barely reduced contention for this reason. Add jitter: in his\ncomparison, Full Jitter and Decorrelated Jitter both cut client work and server\nload substantially, and either is a sound default.\n\n**The wait has no ceiling, or the retries have no end.** An uncapped doubling\npasses 17 minutes after ten failures, and a loop without a limit keeps a request\nalive long after anyone is waiting for its answer. Google lists ignoring retry\nlimits as an anti-pattern: retrying indefinitely can exhaust the application's\nresources or keep sending requests to a service that will not recover without\nintervention. Set a maximum wait and a total deadline that suit the caller,\nshort where latency matters and longer for batch work.\n\n**Permanent errors are retried.** An authorization failure or an invalid request\nfails the same way on every attempt. Retrying it only delays the error, and\nwithout a limit the client can end up in an infinite loop, as the Cloud Storage\ndocumentation warns. Sort errors into transient and permanent before the loop,\nretry only the transient ones, such as 408, 429, 5xx and dropped connections,\nand log the rest with their cause.\n\n**A non-idempotent operation is retried blindly.** If a request timed out after\nthe server had already acted on it, a retry repeats the action, which Google's\nlist of anti-patterns describes as unintended overwrites or deletions of data.\nRetry only idempotent operations, make writes conditional where the API allows\nit, and otherwise make sure the receiving side can detect a duplicate.\n\n**Retries are stacked in several layers.** If the application makes three\nattempts and the client library under it makes three attempts for each of those\nattempts, one failure becomes nine attempts, which is Google's own example, and\nevery further layer that retries on its own multiplies the count again. The\ndocumentation adds that so many retries can lead to request throttling and\nhigher latency without improving the success rate. Retry in one layer, usually\nthe library, and turn retries off or down in the others.\n\n**The backoff is never reset, or nobody sees it.** A client that keeps its last,\nlong wait after a recovery sits out the maximum at the next small failure; gRPC\nresets to the initial backoff as soon as a connection is accepted, and TCP\nrecomputes its timer from fresh measurements. Reset after a success. And because\nretries hide trouble from users, they also hide it from the team. Count them in\n[monitoring](/glossary/#monitoring) and raise an [alert](/glossary/#alert) when\nthe retry rate climbs, because a dependency that needs three attempts per call\nis already failing.\n\n## Further reading\n\n- [RFC 6298: Computing TCP's Retransmission Timer](https://www.rfc-editor.org/rfc/rfc6298.html)\n  — the Standards Track rules for TCP's retransmission timeout, including the\n  doubling each time the timer expires.\n- [gRPC Connection Backoff Protocol](https://github.com/grpc/grpc/blob/master/doc/connection-backoff.md)\n  — the reconnection algorithm, its five parameters with their values, and when\n  the backoff is reset.\n- [Exponential Backoff And Jitter](https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/)\n  — Marc Brooker's 2015 simulation of contending clients, comparing no jitter\n  with Full, Equal and Decorrelated Jitter.\n- [Retry strategy](https://docs.cloud.google.com/storage/docs/retry-strategy) —\n  what Cloud Storage considers safe to retry, the defaults of each client\n  library and five retry anti-patterns.\n- [RFC 9110: HTTP Semantics](https://www.rfc-editor.org/rfc/rfc9110.html#section-10.2.3)\n  — the Retry-After field, with which a server tells a client how long to wait\n  before its next request.\n\n\u003chttps://engineer.company/glossary/exponential-backoff/\u003e\n","date_modified":"2026-11-08T05:31:18Z","description":"Why a client should wait longer after every failed attempt, how jitter stops retries from arriving together, and where retry loops do more harm than good.","formats":{"gemtext":"https://engineer.company/glossary/exponential-backoff/index.gmi","gopher":"https://engineer.company/glossary/exponential-backoff/gophermap","html":"https://engineer.company/glossary/exponential-backoff/","json":"https://engineer.company/glossary/exponential-backoff/index.json","markdown":"https://engineer.company/glossary/exponential-backoff/index.md","pdf":"https://engineer.company/glossary/exponential-backoff/index.pdf"},"kind":"page","language":"en","title":"What is exponential backoff?","url":"https://engineer.company/glossary/exponential-backoff/","version":1}