{"canonical_url":"https://engineer.company/glossary/protocol/","content_markdown":"# Protocol\n\n## What it is\n\nA protocol is a written agreement about a conversation between programs: what each message looks\nlike, what it means, which side speaks first, what may follow what, and what either side does when\nthe other says something unexpected. Two programs written by different people in different languages\ncan work together because each was built against the same document rather than against the other.\n\nNIST's glossary collects several definitions from its publications, and they share a core: rules\ncovering the format of messages and the procedures around them, followed by two or more parties to\nreach a joint objective. One of them names the half that is easy to forget: the rules describe the\norder of the messages as well as their structure. A message in the right format at the wrong moment\nbreaks the conversation as surely as a malformed one.\n\nSpecifications published as RFCs state their rules with capitalised keywords. MUST is an absolute\nrequirement. SHOULD allows a deviation only when its full implications are understood and weighed.\nMAY marks something truly optional. Since 2017 those words carry that meaning only in capitals, so a\nlower-case \"must\" in the same document is ordinary English. The gap between SHOULD and MUST is the\ngap between a program that is unusual and one that is wrong.\n\nProtocols are stacked. An application protocol, the part that knows about pages, menus or mail,\nusually rides on a transport protocol whose only job is to move bytes between two machines. TCP, the\ntransport most of them use, provides what its specification calls \"a reliable, in-order, byte-stream\nservice\": bytes arrive complete and in the order they were sent. It also promises no relationship\nbetween the chunks one side writes and the chunks the other side reads. A stream has no messages in\nit, so every protocol on top has to mark where its own messages end, with a line ending, a length\nsent up front, the connection closing, or some other agreed marker. UDP makes the opposite bargain.\nIt carries separate datagrams with a minimum of mechanism, and neither delivery nor protection\nagainst duplicates is guaranteed. Each transport tells the services on one machine apart by a port\nnumber, and the two number theirs separately.\n\nThe rest of a protocol's character comes from a few choices every design makes, on purpose or not:\n\n- **Who speaks first.** Some servers greet the client, others wait in silence for a request, and\n  when both sides wait, nothing happens until a timer runs out.\n- **What the server remembers.** In a stateless protocol every request carries everything needed to\n  answer it. In a stateful one, the meaning of a message depends on what came before it on the same\n  connection.\n- **How it grows.** Version numbers, option codes and fields that must be ignored when unknown are\n  the room a protocol leaves for its own future.\n\n## Why it matters\n\nA protocol is what makes either end replaceable. Any client written to the document can talk to any\nserver written to it, so a reader can change software and a server can be rewritten in another\nlanguage without the other side noticing. Programs that agree only because they were built together\ncannot change apart.\n\nThe protocol also settles much of the security posture before any code exists. Which side opens the\nconnection decides what has to be let in: a server that waits for clients needs a port open to the\nworld, while a program that only dials out needs none. What the messages carry decides what can be\nlogged, counted or leaked. A request that names a path can be logged by path, and hiding a reader's\nnetwork address does not prevent the server from counting requests. A protocol that\nruns without encryption shows every request to every network it crosses.\n\nFinally, protocols outlive the programs that speak them. Implementations are replaced every few\nyears while the conversation goes on for decades, and every behaviour deployed programs rely on\nbecomes part of it, whatever the document says. The Internet Architecture Board has written about\nboth halves of that problem: mistakes that are tolerated until they become the standard, and room\nfor growth that nobody uses until it no longer works.\n\n## What it looks like in practice\n\nThis site [answers on four protocols, out of one folder of files on one server](/cli/): HTTP,\nGemini, Gopher and an onion service over Tor. The page describes Gopher as a menu protocol from 1991\nwith plain TCP, no encryption and no scripting, and gives a whole request typed by hand:\n\n```sh\nprintf '/contact\\r\\n' | nc engineer.company 70\n```\n\nIt calls that line the entire protocol. Underneath, one deploy writes one folder and four front\ndoors read it, so none of them is a separately maintained copy that could drift.\n\n[Firewall rules are verified by position rather than presence](/portfolio/verified-firewall-rules-by-position/),\nbecause a packet filter evaluates an ordered list and stops at the first match. One rule of the\ncheck concerns protocols: exactly one rule exists per port, and always with a protocol, because a\nrule without one silently widens the surface. The declared public surface is a short list of five\nports, asserted as a whole, so a port that appears without being declared fails the check.\n\nThe fourth door is\n[an onion mirror](/portfolio/published-a-readable-tor-onion-mirror/), and its\nprotocol shaped both its firewall and its reporting. The anonymity daemon reaches the network by\nmaking outbound circuits, so nothing dials in, no port opens, and neither firewall layer has\nanything to synchronise. Gemini requests and Gopher connections are tallied per day with no address\nand no path. The onion has no counter and no flag to add one, because this site deliberately leaves that\naudience uncounted.\n\n## Where it goes wrong\n\n**A tolerated mistake becomes the standard.** The robustness principle, \"be conservative in what you\nsend, and liberal in what you accept\", was meant to help programs work together. Applied without\nlimit, it lets a flawed implementation ship and spread. The next implementation has to accept the\nsame flaw to work with it, and the flaw becomes a de facto standard that anything new must be\nbug-for-bug compatible with. RFC 9413 puts it plainly: the deployed implementations become a\nconformance test that can supersede the specification. Reject what the specification forbids, as\nearly in a protocol's life as possible, and make the error visible to someone who can fix it.\n\n**Two programs disagree about where a message ends.** On a byte stream the boundary is wherever the\nprotocol says it is. A protocol that offers two ways to say it, or a parser that accepts a sloppy\nversion, lets a proxy and the server behind it read the same bytes as different messages. What one\ntreats as the tail of a request, the other treats as a new request the proxy never checked. Allow\none framing rule, reject anything ambiguous, and make every hop parse it the same way.\n\n**A rule names a port but not a protocol.** A port number means nothing until a transport is named,\nand depending on the tool, a firewall rule written with only a number can open it on both TCP and\nUDP. The service that needed one of them gets it, and the other is open to whatever listens there\nlater. Write the protocol into every rule, and compare the whole open surface against a declared\nlist.\n\n**The room for growth stops working.** If no deployed sender ever uses a protocol's extension\npoints, receivers get written that reject what they have never seen, and the first real extension\nbreaks against them. The longer such a receiver stays deployed, the harder it is to correct.\nExercise the extension points from the start, including with values that mean nothing, so that\ntolerance is tested all the time rather than assumed.\n\n## Further reading\n\n- [NIST CSRC glossary: protocol](https://csrc.nist.gov/glossary/term/protocol) — the definitions\n  used in NIST security publications, each with the document it comes from.\n- [RFC 9293: Transmission Control Protocol](https://www.rfc-editor.org/rfc/rfc9293) — the current\n  TCP standard, including the byte stream with no message boundaries.\n- [RFC 768: User Datagram Protocol](https://www.rfc-editor.org/rfc/rfc768) — the whole of UDP in\n  three pages, from 1980.\n- [RFC 1436: The Internet Gopher Protocol](https://www.rfc-editor.org/rfc/rfc1436) — an\n  informational description of a whole application protocol, with sample sessions.\n- [RFC 9110: HTTP Semantics](https://www.rfc-editor.org/rfc/rfc9110) — the meaning of requests,\n  responses, methods and status codes, independent of the version on the wire.\n- [RFC 9413: Maintaining Robust Protocols](https://www.rfc-editor.org/rfc/rfc9413) — the Internet\n  Architecture Board on why tolerating errors entrenches them.\n\n\u003chttps://engineer.company/glossary/protocol/\u003e\n","date_modified":"2026-11-09T21:50:25Z","description":"What a network protocol pins down between two programs, why it outlasts every program that speaks it, and the faults that make two sides drift apart.","formats":{"gemtext":"https://engineer.company/glossary/protocol/index.gmi","gopher":"https://engineer.company/glossary/protocol/gophermap","html":"https://engineer.company/glossary/protocol/","json":"https://engineer.company/glossary/protocol/index.json","markdown":"https://engineer.company/glossary/protocol/index.md","pdf":"https://engineer.company/glossary/protocol/index.pdf"},"kind":"page","language":"en","title":"What is a protocol?","url":"https://engineer.company/glossary/protocol/","version":1}