{"canonical_url":"https://engineer.company/glossary/json/","content_markdown":"# JSON\n\n## What it is\n\nJSON is a plain-text notation for nested data, written with braces, brackets,\ndouble quotes and commas so that a program in almost any language can read back\nexactly what another program wrote. The letters stand for JavaScript Object\nNotation, because the syntax was taken from the way JavaScript writes literal\nvalues, yet a JSON document belongs to no language and needs no JavaScript to\nread. Files usually end in `.json`, and over HTTP the content is labelled with\nthe media type `application/json`.\n\nThe whole format fits on one page. Four primitive types (strings, numbers, the\nliterals `true` and `false`, and `null`) and two structured ones make up the\nvocabulary: an object, which is an unordered set of name and value pairs, and an\narray, which is an ordered list. Anything can nest inside anything else. Dates,\nbinary data, comments and references between parts of a document are all left to\nconventions outside the format.\n\nDouglas Crockford first registered it with the IETF in RFC 4627, in July 2006.\nThe current definition is RFC 8259 from December 2017, which agrees with Ecma's\nstandard ECMA-404 on the grammar and adds the rules that matter when two systems\nexchange it. Text that leaves a closed ecosystem must be UTF-8, and a sender\nmust not add a byte order mark in front of it.\n\nXML is older and more expressive, with attributes, namespaces and a long\ntradition of schema languages. YAML was revised in version 1.2 to be a strict\nsuperset of JSON, adding the comments and lighter punctuation people want in\nfiles they edit by hand. Binary encodings such as Protocol Buffers and CBOR\ntrade readability for size and speed.\n\n## Why it matters\n\nPicking JSON is really picking a contract between two systems: the shape of what\ncrosses the wire, and sometimes the shape of what lands in storage.\n\nOn the wire it is the default because everything already speaks it. A browser\nparses it without a library, every mainstream language has a mature parser, and\nan engineer reading a log can see what a request carried. The costs lie\nelsewhere. Every record repeats its field names, numbers travel as decimal text\nthat has to be parsed back, and the format itself says nothing about which\nfields are required or what type each one holds. A binary, schema-first encoding\nsuch as Protocol Buffers is smaller and faster to parse, and its contract is\nchecked by a compiler, but a person needs a tool to read a single message. For a\npublic or browser-facing API, JSON is the sensible choice. Between internal\nservices where volume or latency dominates, the trade can go the other way.\n\nA schema usually supplies the missing contract. JSON Schema declares which\nproperties an object must have, what types they hold and whether extra\nproperties are allowed, and OpenAPI builds on it to describe whole HTTP APIs. A\nschema is a separate document, and it stays accurate only while something forces\nit to match the code.\n\nIn storage the question is whether a structure belongs in a JSON column or in\nreal columns. PostgreSQL offers two types. `json` keeps an exact copy of the\ninput text and reparses it every time a function touches it, while `jsonb`\nstores a decomposed binary form that is slightly slower to write, much faster to\nquery and can be indexed; its documentation recommends `jsonb` for most\napplications. A JSON column suits data whose shape varies from row to row or\nbelongs to the client, such as a user's saved preferences. A field the database\nhas to join on, constrain or sum deserves a column of its own.\n\n## What it looks like in practice\n\nOn one platform the PostgreSQL functions assemble each complete response as JSON\nthemselves, and the Go API\n[forwards that JSON from the database to the browser untouched](/portfolio/designed-a-json-passthrough-from-postgres-to-api/).\nThe handler takes it as a `json.RawMessage` and passes it on without decoding or\nre-encoding it, and a small `QueryJSON` helper made that the default. The DTOs,\nmappers and response structs of a layered API fell away. When a function's\noutput changes, the new shape reaches the client with no edit to handler code.\n\nStored JSON brings a different problem, because it outlives the code that wrote\nit. One platform keeps notification settings and similar preferences as a user's\nsaved values laid over a set of defaults, originally merged one level deep. A\nkey added to the defaults later was absent from every row saved before it\nexisted, so client code read an undefined field and crashed, and it crashed for\nthe longest-standing users. The fix was\n[a deep merge that treats the defaults as a catalogue](/portfolio/implemented-a-catalogue-driven-deep-merge/):\nevery key in the catalogue comes out present, nested keys included, whatever the\nsaved blob holds. A migration applied it to existing rows at once, and the\nfrontend dropped its defensive checks.\n\nJSON is also simply how data arrives. For historical electricity data delivered\nas JSONL, one JSON event per line, the task was to turn raw events into hourly\nconsumption per zone, and\n[three variants were built, in Python, JQ and SQL](/portfolio/a-month-of-meter-readings-in-180-milliseconds/).\nPython was the fastest, at 34ms for one day of data and 180ms for thirty days.\n\n## Where it goes wrong\n\nThe grammar is small enough that parsers agree on what valid text looks like.\nTrouble comes from everything the grammar leaves open.\n\n**A large ID comes back one digit different.** The grammar puts no ceiling on\nnumbers, but most parsers, JavaScript's among them, read them into 64-bit\nfloating point, which holds integers exactly only up to 2⁵³ − 1. RFC 8259 names\nthe range from −(2⁵³ − 1) to 2⁵³ − 1 as the one implementations agree on. Past\nit, identifiers change without a warning:\n\n```js\nJSON.parse('{\"id\": 9007199254740993}').id;\n// 9007199254740992\n```\n\nSend 64-bit identifiers as strings, and send money as a string or as an integer\ncount of the smallest unit, never as a float.\n\n**Two services read the same object differently.** RFC 8259 says the names in an\nobject should be unique and leaves the rest open: given a repeated name, many\nparsers keep the last value, others raise an error, and some hand back every\npair. When a gateway checks the first `role` and the backend acts on the second,\nthe check protected nothing. Reject duplicate names where data enters the\nsystem. I-JSON, defined in RFC 7493, is a profile of JSON that forbids them, and\na contract can cite it by name.\n\n**A hand-edited file stops parsing.** Comments and trailing commas are outside\nthe grammar, so the first person to annotate a config file, or to leave a comma\nafter the last item, breaks it. Some tools accept a relaxed dialect, which makes\nthe same file valid for one reader and broken for the next. Keep JSON for data\nmachines write, and give hand-maintained files a format made for editing, such\nas YAML or TOML.\n\n**Text arrives garbled, or one parser rejects what another accepts.** RFC 8259\nrequires UTF-8 between systems and forbids a byte order mark at the start. A\nproducer that writes Latin-1, or an editor that prepends a mark, yields text\nwhose fate depends on the parser. The grammar also allows an escape such as\n`\"\\uDEAD\"`, half of a surrogate pair with its partner missing, which no valid\nUnicode string can hold, and libraries treat it in different ways. Encode as\nUTF-8 at the source and reject invalid sequences at the boundary.\n\n**One small request ties up the parser.** A few kilobytes of nested brackets can\ndrive a recursive parser thousands of levels deep, and an oversized body is\nparsed in full before any schema sees it. RFC 8259 lets an implementation limit\nboth text size and nesting depth, and an API that accepts JSON from strangers\nshould use both: cap the request size before reading, cap the depth while\nparsing, and check the shape only after that.\n\n## Further reading\n\n- [RFC 8259: The JSON Data Interchange Format](https://www.rfc-editor.org/rfc/rfc8259)\n  — the current IETF definition, with its notes on numbers, duplicate names and\n  encoding.\n- [ECMA-404: The JSON Data Interchange Syntax](https://ecma-international.org/publications-and-standards/standards/ecma-404/)\n  — Ecma's standard for the same grammar, which RFC 8259 cites as normative.\n- [RFC 7493: The I-JSON Message Format](https://www.rfc-editor.org/rfc/rfc7493)\n  — a stricter profile that requires UTF-8 and forbids duplicate names.\n- [JSON Schema specification](https://json-schema.org/specification) — the\n  vocabulary for declaring which properties a document must have.\n- [PostgreSQL: JSON Types](https://www.postgresql.org/docs/current/datatype-json.html)\n  — `json` against `jsonb`, and what each keeps of the original text.\n- [Go: json.RawMessage](https://pkg.go.dev/encoding/json#RawMessage) — the type\n  that lets a Go program delay decoding or pass JSON along untouched.\n\n\u003chttps://engineer.company/glossary/json/\u003e\n","date_modified":"2026-10-20T21:59:50Z","description":"How JSON works, when an API or a database column should use it, and the duplicate-key, big-number and encoding traps that break it in production.","formats":{"gemtext":"https://engineer.company/glossary/json/index.gmi","gopher":"https://engineer.company/glossary/json/gophermap","html":"https://engineer.company/glossary/json/","json":"https://engineer.company/glossary/json/index.json","markdown":"https://engineer.company/glossary/json/index.md","pdf":"https://engineer.company/glossary/json/index.pdf"},"kind":"page","language":"en","title":"What is JSON?","url":"https://engineer.company/glossary/json/","version":1}