webhook

What it is

A webhook is an HTTP request that one service sends to another the moment something happens on its side: a payment goes through, an invoice is paid, a long task finishes. The receiving side registers a URL in advance, and the sender posts a description of each event to it. The Standard Webhooks specification calls webhooks “a common name for HTTP callbacks” and describes them as a channel through which one service announces an event to another. The term is sometimes written as two words, web hook, but Stripe, GitHub and the specification all write it as one.

The specification also describes a webhook as part of a service’s API, a kind of API running in reverse. Normally the client calls the service when it wants something. With a webhook, the service calls the client when it has news. That makes webhooks server-to-server traffic in which both sides run HTTP servers, a service may even expose webhooks as its only API. The sender is called the producer, the receiver the consumer, and the URL the receiver registers is its webhook endpoint.

Under the specification, the event travels as the body of an HTTP POST, preferably as JSON. The body has a type, written with full stops and grouped hierarchically, such as invoice.paid; a timestamp for when the event occurred, in ISO 8601; and the event itself under data. Everything else goes in headers: webhook-id identifies the event, webhook-timestamp records when this attempt was made, in Unix seconds, and webhook-signature proves who sent it. Stripe and GitHub use header names of their own, Stripe-Signature and X-Hub-Signature-256 among them, but the pieces are the same.

A payload can be full, carrying the whole changed object, or thin, carrying little more than an identifier and leaving the receiver to fetch the rest through the API. The specification weighs the two. A full payload saves the receiver extra API calls. A thin one means less data to generate and send, more flexibility and more room to change later, since a thin payload can always grow into a full one but not the other way round. It also makes data access auditable, because the receiver has to ask explicitly for what it reads. Either way the specification advises keeping payloads small, usually under 20 kB. Stripe offers a thin format and marks it as the recommended one.

Why it matters

Without webhooks, a system that needs to know about a change somewhere else has to poll: ask again and again, mostly to hear that nothing has happened, and still learn about the change only at the next poll. A webhook turns that around. The service that knows about the event calls once, when it happens. That is how payment providers, code hosting services and email providers report back to the systems built on them.

The reversal has costs, and most of them land on the receiving end. Its endpoint is a public URL that anyone on the internet can POST to, and nothing about an incoming request shows who sent it, so the specification makes verifying authenticity a requirement for any secure implementation. Delivery is retried until the receiver answers with a 2xx status, so the same event can arrive more than once; Stripe warns that endpoints might occasionally receive duplicates. Order is not promised either: Stripe states plainly that it does not guarantee events arrive in the order they were generated.

The producer, for its part, has to keep trying. The specification recommends a retry schedule spanning several days, with exponential backoff and some random jitter. Its example makes ten attempts, the last one 75 hours, 35 minutes and 5 seconds after the first. Stripe retries for up to three days in live mode. A receiver that comes back within that window catches up on its own; after it, Stripe lets you resend an event by hand for up to 15 days after it was created, or up to 30 days with its CLI. When delivery keeps failing for a long time, the specification recommends that the producer tell the consumer through another channel, such as email, and disable the endpoint.

What it looks like in practice

These are the headers of a delivery in the specification’s format. The identifier, timestamp and signature are the example values the specification itself uses; the Host value and the path are made up:

POST /webhooks/billing HTTP/1.1
Host: example.com
Content-Type: application/json
webhook-id: msg_2KWPBgLlAfxdpx2AI54pPJ85f4W
webhook-timestamp: 1674087231
webhook-signature: v1,K5oZfzN95Z9UVu1EsfQmfVNQhnkZ2pj9o9NDN/H/pI4=

And this is the body, a thin payload from the same specification, pretty-printed here. On the wire the specification recommends sending it minified:

{
  "type": "contact.created",
  "timestamp": "2022-11-03T20:26:10.344522Z",
  "data": {
    "id": "1f81eb52-5198-4599-803e-771906343485"
  }
}

The v1 prefix marks a symmetric signature: HMAC-SHA256 over the event identifier, the timestamp and the body joined by full stops, keyed with a secret unique to the endpoint. Because the identifier and timestamp are signed along with the body, neither can be altered without breaking the signature. The specification also allows asymmetric ed25519 signatures, marked v1a, and recommends them when different parties are responsible for securing the producer and the consumer.

GitHub signs its deliveries with HMAC-SHA256 too, puts the hex digest in X-Hub-Signature-256 with a sha256= prefix, and publishes a secret, a payload and a signature for testing a verifier. This is a verifier in Python, checked against that example:

import hashlib
import hmac


def verify(body: bytes, secret: str, header: str) -> bool:
    mac = hmac.new(secret.encode("utf-8"), body, hashlib.sha256)
    expected = "sha256=" + mac.hexdigest()
    return hmac.compare_digest(expected, header)


secret = "It's a Secret to Everybody"
header = (
    "sha256=757107ea0eb2509fc211221cce984b8a"
    "37570b6d7586c22c46f4379c8b043e17"
)
print(verify(b"Hello, World!", secret, header))  # True
print(verify(b"Hello, World! ", secret, header))  # False

One trailing space in the body is enough to fail. hmac.compare_digest takes the same time whether the strings differ early or late; GitHub’s documentation tells you never to compare signatures with a plain ==. A sound handler then works in a fixed order: read the raw bytes, verify the signature over exactly those bytes, check that the timestamp is recent, check whether this event identifier has been seen before, put the event on a queue and answer with a 2xx. The real work happens afterwards. Stripe gives the example of returning a 200 before marking a customer’s invoice as paid in the accounting system.

Two projects in this site’s portfolio deal with webhooks from the receiving end. In the Stripe and in‑app purchase entitlements, a subscription has to survive, among other things, a webhook that arrives twice and a webhook that arrives out of order. In the email system with failover and delivery logging, inbound webhooks report what each provider did with a message, and the project records both what was sent and what came back, in a send log and a delivery event table.

Where it goes wrong

The signature never matches. A common cause is that the receiver’s framework parses the JSON body and the code serialises it again before computing the HMAC. The specification calls this a very common failure mode: JSON serialisation differs between implementations, and a single stray space invalidates the signature. Stripe needs the raw body for the same reason and warns that any manipulation of it makes verification fail. GitHub adds that a proxy or load balancer in front of the endpoint must not modify the payload or the headers, and that the payload should be handled as UTF-8. Compute the signature over the bytes exactly as they arrived, before anything parses them.

The same event arrives twice. Retries and network trouble both produce duplicates. Under the specification, webhook-id stays the same across every retry of an event and serves as an idempotency key; its example is to keep the identifiers in Redis for five minutes. Stripe tells you to log the event IDs you have processed and skip ones you have seen. It also notes that one underlying occurrence can produce two distinct Event objects; you can recognise those by the object ID in data.object together with the event type. GitHub’s X-GitHub-Delivery header stays the same when a delivery is redelivered. Processing should be idempotent, so a second delivery changes nothing.

A captured delivery is played back. A valid signature stays valid, so an attacker who intercepts one delivery can send it again. That is why the timestamp is signed: the specification says to reject a webhook-timestamp outside an allowed tolerance. Stripe’s libraries default to five minutes, Stripe warns that a tolerance of 0 switches the check off entirely, and it recommends NTP to keep the server’s clock accurate. Both Stripe and the specification put a fresh timestamp on every retry, so a legitimate retry is never too old. Keep the seen identifiers for at least as long as the tolerance, and anything replayed inside the window is caught as a duplicate.

Events arrive out of order. Stripe’s example: creating a subscription might produce customer.subscription.created, invoice.created, invoice.paid and, if there is a charge, charge.created, and nothing guarantees they arrive in that order. The created field on Stripe snapshot events has one-second resolution, allowing several events to carry the same timestamp, and Stripe says not to use it to order events or detect duplicates. The robust approach treats an event as a notice that something changed and fetches the object’s current state from the API before acting on it.

The handler is too slow. GitHub expects a 2xx within 10 seconds; after that it terminates the connection and counts the delivery as failed. The specification recommends a request timeout of somewhere between 15 and 30 seconds. A handler that does its real work inline times out, the producer retries, and the work runs twice. Stripe and GitHub both recommend a queue. Stripe points to the spike at the start of the month, when all subscriptions renew.

Only a 2xx counts. Anything else is a failure, a 3xx included, to the specification and to Stripe alike, so an endpoint that moved behind a redirect fails every delivery until its registered URL is updated. The specification gives a few codes a meaning for the producer: a 410 Gone means disable the endpoint, a 429, 502 or 504 means slow down, and a retry-after header should shape the next attempt.

For the producer, the URL is hostile input. Customers can enter any URL they like, and the producer’s servers will call it. The specification calls webhook systems especially vulnerable to server-side request forgery (SSRF): a URL that points at an internal address makes the producer call into its own network. Its advice is to send every webhook request through a proxy that filters internal IP addresses, such as Stripe’s smokescreen, and to run the webhook workers in a private subnet that cannot reach internal services.

Further reading

• Standard Webhooks specification — version 1.0.0: payload structure, the signature scheme, headers, the retry schedule, status-code handling and SSRF.
• Receive Stripe events in your webhook endpoint — thin and snapshot payloads, retries, duplicate and out-of-order events, and replay protection.
• Best practices for using webhooks — GitHub’s list: few events, a webhook secret, HTTPS, an answer within 10 seconds, redelivery and X-GitHub-Delivery.
• Validating webhook deliveries — X-Hub-Signature-256, constant-time comparison and a test example to check a verifier against.