{"canonical_url":"https://engineer.company/glossary/large-language-model/","content_markdown":"# Large language model\n\n## What it is\n\nA large language model is a neural network trained on a very large body of text\nto do one thing: given the text so far, give a probability to every token that\ncould come next. A token is a short piece of text, often a whole word, sometimes\npart of a word or a single punctuation mark. Writing an answer is that one step\nin a loop. The software picks a token according to those probabilities, appends\nit to the text, and asks again, until the model produces a token that means stop\nor a length limit is reached. NIST's profile for generative AI, published in\nJuly 2024, puts it in one line: “LLMs predict the next token or word in a\nsentence or phrase.” The abbreviation LLM, which the profile itself uses, stands\nfor the same thing.\n\nThe architecture behind the models of recent years is the Transformer, described\nin a 2017 paper on machine translation. It dropped the recurrent and\nconvolutional layers that earlier models were built from and relied on attention\nalone, a mechanism that lets every position in the text look directly at every\nother. That made training far easier to spread across many processors, and the\npaper's model trained in much less time than its predecessors.\n\n“Large” refers to the number of parameters, the weights the network learns\nduring training. A 2020 paper described a model with 175 billion of them, ten\ntimes more than any earlier language model that was not sparse, and showed that\nat that size a model could take on a new task from a handful of worked examples\nplaced in its input, with its weights left exactly as they were.\n\nA model trained only to continue text does exactly that: it continues the text,\nwhich is not the same as doing what it was asked. A 2022 paper described the\nstep that turns such a model into an assistant. People wrote the answers they\nwanted to a set of requests and the model was fine-tuned on them; then people\nranked the model's own answers, and it was trained further on those rankings, a\nmethod called reinforcement learning from human feedback. In the paper's\nevaluations, people preferred the answers of a 1.3-billion-parameter model\ntrained this way to those of the original model with a hundred times as many\nparameters.\n\n## Why it matters\n\nA language model takes on work that is difficult to cover with fixed rules:\nsorting a free-text email into the right folder, pulling the fields out of a\nscanned delivery note, drafting a reply in the customer's own language,\nsummarising a long thread. Work that used to need a person reading every item\ncan now be drafted by software and checked by a person.\n\nThe property that makes it useful is also what makes it hard to build on. Its\noutput is usually sampled, so the same input can give different answers on\ndifferent runs, and it is fluent whether or not it is right. NIST calls\nconfidently stated wrong output confabulation, better known as hallucination,\nand describes it as a natural result of how these models are built: they produce\ntext that matches the statistics of the text they were trained on, and that is\noften accurate and sometimes not. NIST also notes that a model can confabulate\nthe reasoning or the citations that seem to justify an answer. A wrong answer\nlooks exactly like a right one, so the only way to know how often a model is\nright on your work is to measure it on your work.\n\nNor is there one model to commit to. New models appear regularly, two of them\ncan behave quite differently on the same input, and the provider of the one an\napplication calls decides when it changes. A team building on a language model\nis building on a part it does not control, and the work that lasts is everything\naround the model: the cases that measure it, the tools it may call and the rules\nfor what it may do on its own.\n\nNIST's profile names twelve areas of risk for generative AI, among them\nconfabulation, data privacy, information security and intellectual property. It\nwas written for organisations managing risk rather than for developers, but it\nis a usable checklist for a team deciding what a feature built on a model may\ntouch.\n\n## What it looks like in practice\n\nA feature built on a language model starts with written cases, not with a\nprompt. Each case is a real input and the answer a person who knows the work\nwould accept, kept in a file under version control beside the code:\n\n```yaml\n# cases/delivery-notes.yaml\n- id: note-0142\n  input: scans/note-0142.txt\n  expect:\n    supplier: \"Example Freight\"\n    reference: \"DN-58213\"\n    pallets: 6\n- id: note-0157\n  input: scans/note-0157.txt\n  expect:\n    supplier: \"Sample Haulage\"\n    reference: \"DN-59002\"\n    pallets: 14\n```\n\nWhich model handles which kind of job is a setting, so a small, fast model can\ndo the extraction while a larger one drafts the replies that need judgement:\n\n```yaml\n# routing.yaml\njobs:\n  extract-delivery-note:\n    model: small-fast\n    cases: cases/delivery-notes.yaml\n  draft-claim-reply:\n    model: large\n    cases: cases/claim-replies.yaml\n```\n\nEvery change runs against all the cases before it ships, whether the change is a\nnew prompt, a new tool or a new model, and the result is a score for each case,\nkept over time so a fix for one case cannot quietly break another. When a new\nmodel is released, trying it is a one-line change to `routing.yaml` and a run\nagainst the cases, and the decision to switch rests on the scores rather than on\nthe announcement.\n\nThat is how this site describes its [custom AI agents](/ai-agents/). The choice\nof model is configuration, not code, with a small, fast model for sorting and\nextracting and a large one for judgement. An engagement begins by writing down\nthirty to fifty real examples with the right answers, and every change, whether\nto a prompt, a tool or the model, runs against all of them before it ships. The\npage also says that the model is kept replaceable, and that the code, the cases\nand the data belong to the client, so the agent stays theirs when the model\nunder it changes.\n\n## Where it goes wrong\n\n**Confident answers that are wrong.** A model asked about a contract clause that\nis not in front of it may describe one anyway, in the same assured tone it uses\nfor a correct answer, and may cite a source that does not exist. Give the model\nthe documents it should answer from, ask it to cite them, and check in code that\nevery citation points at a real document and passage. Measure the error rate on\nwritten cases, and route answers with consequences to a person before anything\nacts on them.\n\n**Instructions hidden in the data.** An application that lets a model read\nemail, web pages or uploaded files lets the authors of that text speak to the\nmodel. A 2023 paper showed this against real applications: instructions planted\nin content the model was likely to read made it leak data and change what the\napplication did, and NIST's profile lists indirect prompt injection among the\nsecurity risks of generative AI. Treat everything the model reads as untrusted\ninput, enforce permissions in the tools rather than in the prompt, give the\ntools only the rights of the person the model acts for, and require a person's\napproval for any action that cannot be undone.\n\n**A model that changes under you.** A new version, or a different provider, can\nanswer the same input differently, and an answer that passed last month fails\ntoday with no change to your own code. Name an exact model version wherever the\nprovider offers one, and run the whole case file on every change, a change of\nmodel included, before it reaches users.\n\n**Output that code cannot parse.** A model asked for JSON returns JSON followed\nby a friendly sentence, leaves out a field, or writes a number as a string.\nDescribe the expected shape in JSON Schema, check every answer against it, and\ntreat an answer that fails as an error, to be retried or handed to a person,\nnever as data.\n\n**Personal data in prompts and logs.** Whatever is sent to a hosted model leaves\nyour own systems, and a log of prompts and answers keeps personal data long\nafter the request is done. NIST adds that models can repeat sensitive material\nfrom their training data, a problem it calls data memorisation. Send only the\nfields a job needs, keep prompt logs under the same retention and access rules\nas the data in them, and read the provider's terms on retention and training\nbefore any personal data reaches it.\n\n## Further reading\n\n- [NIST AI 600-1: Generative Artificial Intelligence Profile](https://doi.org/10.6028/NIST.AI.600-1)\n  — NIST's July 2024 companion to its AI Risk Management Framework: twelve risks\n  of generative AI, confabulation among them, and suggested actions for each.\n- [Attention Is All You Need](https://arxiv.org/abs/1706.03762) — the 2017 paper\n  that introduced the Transformer, the architecture behind today's language\n  models.\n- [Language Models are Few-Shot Learners](https://arxiv.org/abs/2005.14165) —\n  the 2020 paper on a 175-billion-parameter model that takes on new tasks from\n  examples in its input.\n- [Training language models to follow instructions with human feedback](https://arxiv.org/abs/2203.02155)\n  — fine-tuning on written answers and ranked outputs, the step from text\n  predictor to assistant.\n- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://arxiv.org/abs/2302.12173)\n  — instructions planted in data a model reads, demonstrated against real\n  applications.\n\n\u003chttps://engineer.company/glossary/large-language-model/\u003e\n","date_modified":"2026-11-12T05:34:23Z","description":"What a large language model actually computes, why its answers need tests rather than trust, and five ways a feature built on one goes wrong.","formats":{"gemtext":"https://engineer.company/glossary/large-language-model/index.gmi","gopher":"https://engineer.company/glossary/large-language-model/gophermap","html":"https://engineer.company/glossary/large-language-model/","json":"https://engineer.company/glossary/large-language-model/index.json","markdown":"https://engineer.company/glossary/large-language-model/index.md","pdf":"https://engineer.company/glossary/large-language-model/index.pdf"},"kind":"page","language":"en","title":"What is a large language model?","url":"https://engineer.company/glossary/large-language-model/","version":1}