{"canonical_url":"https://engineer.company/glossary/queue/","content_markdown":"# Queue\n\n## What it is\n\nA queue is a holding place between two programs: one side puts messages or jobs\nin, the other takes them out when it is ready, usually oldest first. Most systems\nrun several queues, one per kind of work. The program that adds work is the\nproducer, the program that takes it is the consumer, and whatever stores the\nmessages in between is the broker, which can be a dedicated server or a table in\na database the system already has.\n\nThe arrangement lets the two sides stop waiting for each other. A web request\nthat needs an email sent adds a message and answers the user at once; a worker\npicks the message up moments later, and if a hundred arrive together, it drains\nthem at its own pace. A burst becomes a backlog, and a dependency becomes a\ndelay.\n\nTwo ideas describe what a queue promises about delivery. The first is\nacknowledgement: a consumer tells the broker when it has finished with a\nmessage, and only then is the message removed. In the AMQP 0-9-1 model, which\nRabbitMQ implements, a message whose consumer dies before acknowledging goes to\nanother consumer. The second is durability: a durable queue of persistent\nmessages survives a broker restart, and a queue held only in memory does not.\n\nThree neighbouring terms are worth keeping apart. A message queue carries data\nbetween services. A job queue carries units of work for background workers, each\nwith its arguments, a state and a count of attempts. A log, the model behind\nApache Kafka, looks like a queue from the outside but keeps events after they\nhave been read, for a retention period chosen per topic, so several readers can\nreplay the same history. Reading from a queue removes the message; reading from a\nlog only moves the reader's position.\n\nA queue need not be a separate product at all. PostgreSQL can play the part with\na table of jobs and a query that claims the next row while stepping over rows\nanother worker has already locked. The manual names queue-like tables as the\nreason the `SKIP LOCKED` clause exists:\n\n```sql\nWITH next AS (\n  SELECT id\n    FROM jobs\n   WHERE state = 'available'\n   ORDER BY id\n   LIMIT 1\n     FOR UPDATE SKIP LOCKED\n)\nUPDATE jobs\n   SET state = 'running',\n       attempt = attempt + 1\n  FROM next\n WHERE jobs.id = next.id\nRETURNING jobs.id, jobs.args;\n```\n\n## Why it matters\n\nA queue decides what a user waits for. Work placed on one happens after the\nresponse has gone, so a page that triggers an email, a PDF, a search reindex and\na recalculation can answer in the time it takes to write four rows, and the four\nslow tasks finish behind it. For a buyer, that is the difference between a\ncheckout that responds in a few hundred milliseconds and one that responds\nwhenever the mail server is ready.\n\nIt also decides what survives a failure. Work started in a background thread\nlives in that process's memory, and a deploy that restarts the process takes the\nwork with it, leaving no trace that it was ever due. Work on a durable queue is\na record instead: it can be retried, inspected and counted.\n\nIt separates two speeds, too. Producers move at the speed of incoming traffic;\nconsumers move at the speed of the slowest service they call. A queue between\nthem absorbs the difference, and its depth becomes one of the most truthful\nnumbers a system can report. A queue that has grown for an hour says the\nconsumers are losing ground, whatever the processor graph suggests.\n\nThe costs arrive with the benefits. Work now sits out of sight between being\nsent and being done. Delivery is almost always at least once, so a consumer has\nto tolerate seeing a message twice. Ordering is weaker than it looks. And a\ndedicated broker is one more server to run, secure, watch and upgrade.\n\n## What it looks like in practice\n\nSending mail, rebuilding a search index, generating documents and recalculating\nstandings were all moved off the request path on one platform. Work handed to a\ngoroutine vanishes when the process restarts in the middle of a deploy, so\n[the work went onto a River job queue kept in PostgreSQL](/portfolio/moved-slow-work-onto-a-river-job-queue/).\nThe database was already the system of record, which makes enqueuing a job part\nof the same transaction as the data it concerns. What the platform gained is\nretries and a history someone can open and read; the stated limit is that a\nqueue inside the database will not scale forever.\n\nIn a macOS application built over a C library, the queue was there to keep\norder. The library's blocking event loop emits events in a sequence that\nmatters, because an edit has to arrive after the message it edits. Starting one\ntask per event lets the runtime schedule them in any order, so an edit can\novertake the message it refers to. The design that shipped has\n[one producer thread, one consumer and one queue between them](/portfolio/adopted-swift-6-strict-concurrency-over-c/),\nfollowed by a single hop onto the main actor. Order holds because there is\nexactly one path through. The price is parallelism: every event passes through\nthat one consumer.\n\nA queue can also do harm. In one macOS application, a burst of core log events was retained\nin an unbounded main stream and copied to subscribers whose consumers did not use those\nevents. Diagnostic reports recorded 111 GB of compressed pages on a machine with 36 GB;\na separate footprint sample recorded growth of about 41 MB per second during a connection\nretry loop. The\n[event path was given explicit bounds](/portfolio/an-app-that-ate-41-mb-a-second-stopped/),\nwith subscriptions filtered by event type and an explicit overflow policy. Repetitive\ndiagnostic events may be dropped and affected state refreshed instead of keeping unlimited\nevents. Architecture checks and application-budget tests defend that design, but the record\ndoes not establish a measured post-fix memory figure or a production benchmark.\n\n## Where it goes wrong\n\nQueue trouble surfaces late and somewhere else, because the work is out of\nsight by design.\n\n**The backlog grows until memory or disk gives out.** A queue with no limit\naccepts work faster than it is done and keeps the difference. Give every queue a\nmaximum length and decide ahead of time what happens at that length: slow the\nproducer, refuse new work, or drop the oldest. Then watch two numbers, the depth\nand the age of the oldest message, because a short queue of very old messages is\nas bad as a long one.\n\n**The same message is processed twice.** A consumer finishes the work and dies\nbefore it acknowledges; the broker, correctly, delivers the message again.\nRabbitMQ's documentation tells consumers to expect redeliveries and to be\nwritten with idempotence in mind. Give each message an identifier and record the\nidentifiers already handled, in the same transaction as the work itself. A note\non this site about\n[selling limited stock without overselling](/notes/selling-limited-stock-without-overselling/)\nbuilds its writer around exactly that assumption.\n\n**Messages arrive in a different order from the one they were sent in.** Parallel\nconsumers, prefetching, and retries that put a failed message behind newer ones\nall reorder work. When order matters, route everything about one account or\none order through the same partition or the same consumer. Kafka does this by\nkey, at the cost that one busy key moves only as fast as one consumer.\n\n**One bad message blocks or loops forever.** A message whose payload crashes the\nconsumer every time is requeued, delivered again, and crashes it again. Cap the\nattempts, and move a message that has used them all to a separate place where a\nperson can look at it. RabbitMQ calls this dead-lettering and applies it to\nrejected messages, expired ones and messages that would overflow a length limit.\n\n**Work disappears on restart.** A channel, a list in memory or a background\nthread is a queue that keeps nothing across a restart. In AMQP, a durable queue\nalone is not enough either: the messages must be published as persistent too.\nDecide per kind of work whether losing it is acceptable.\n\n**The database and the queue disagree.** A service writes a row, then publishes\na message, and crashes between the two; now one exists without the other. Either\nkeep the queue in the same database, so that a job becomes visible to workers\nonly when its transaction commits, which is how River behaves, or write the\nmessage into an outbox table in that transaction and publish it from there.\n\n## Further reading\n\n- [AMQP 0-9-1 Model Explained](https://www.rabbitmq.com/tutorials/amqp-concepts)\n  — exchanges, queues, bindings and acknowledgements, set out by RabbitMQ.\n- [Consumer Acknowledgements and Publisher Confirms](https://www.rabbitmq.com/docs/confirms)\n  — when a broker considers a message delivered, and why redelivery happens.\n- [Dead Letter Exchanges](https://www.rabbitmq.com/docs/dlx) — the conditions\n  that move a message aside and where it goes.\n- [The Locking Clause of SELECT](https://www.postgresql.org/docs/current/sql-select.html#SQL-FOR-UPDATE-SHARE)\n  — `FOR UPDATE` and `SKIP LOCKED`, and the queue-like tables they suit.\n- [River: Getting started](https://riverqueue.com/docs) — a job queue that\n  stores its jobs in the application's own database.\n- [Apache Kafka: Introduction](https://kafka.apache.org/41/getting-started/introduction/)\n  — topics, partitions and retention in the log model.\n\n\u003chttps://engineer.company/glossary/queue/\u003e\n","date_modified":"2026-11-08T02:22:23Z","description":"How a queue hands work between programs, what acknowledgement and retries promise, when a database table is enough, and six ways queues misbehave.","formats":{"gemtext":"https://engineer.company/glossary/queue/index.gmi","gopher":"https://engineer.company/glossary/queue/gophermap","html":"https://engineer.company/glossary/queue/","json":"https://engineer.company/glossary/queue/index.json","markdown":"https://engineer.company/glossary/queue/index.md","pdf":"https://engineer.company/glossary/queue/index.pdf"},"kind":"page","language":"en","title":"What is a queue?","url":"https://engineer.company/glossary/queue/","version":1}