{"canonical_url":"https://engineer.company/glossary/backup/","content_markdown":"# Backup\n\n## What it is\n\nA backup is what the data gets restored from when the original is deleted, corrupted or lost along\nwith the machine that held it, so it has to live somewhere those events do not reach. The NIST\nglossary defines it as a copy of files and programs whose purpose is to make recovery easier. The\nnoun, plural backups, names the copies, and the verb, written as two words, names the act: a nightly\njob backs up the database and leaves a backup behind.\n\nNeighbouring things get mistaken for it. A replica receives every change within seconds, which\nprotects against a dead disk and copies a mistaken `DELETE` just as faithfully. A snapshot freezes a\nvolume at one moment, and becomes a backup only once it is stored where the original's failures\ncannot follow. A backup is defined by history and separation: it returns the data as it stood at an\nearlier point, from a place that survived whatever took the original.\n\nDatabases make this concrete. The PostgreSQL manual describes three fundamentally different\napproaches. An SQL dump, made with `pg_dump`, writes out the statements that rebuild the database,\nas a consistent snapshot of the moment the dump began. A file system level backup copies the data\ndirectory itself. Continuous archiving combines one such base backup with the write-ahead log (WAL)\nthe server produces afterwards, and replaying that log can restore the database to any moment since\nthe base backup. That ability is called point-in-time recovery.\n\n## Why it matters\n\nTwo numbers turn a general wish to be safe into a design. NIST defines the recovery point objective,\nRPO, as the point in time to which data must be recovered after an outage: in practice, how much\nrecent work the business can afford to lose. The recovery time objective, RTO, is how long a system\ncan stay in recovery before the outage harms what the organization is there to do.\n\nThe two pull on different parts of the design. RPO is set by how often a copy is taken: a nightly\ndump can lose a day of writes, while WAL archiving shrinks the loss to the last few moments of log\nstill on their way to the archive. RTO is set by how long a restore takes. The PostgreSQL manual\nnotes that the server replays every WAL segment since the last base backup, which can take a while\nwhen that backup is old, so how often to take base backups is an RTO decision as much as a storage\none.\n\nBoth numbers are claims about a restore, which is why a backup is judged by restoring it. A job that\nexits with status zero shows that bytes were written somewhere. Whether they rebuild a working\nsystem inside the RTO is answered by restoring on a schedule, somewhere other than production, and\ncomparing what comes back.\n\nSeparation is the other half. A copy in the same account, under the same credentials, shares the\nfailures it is meant to survive: a closed account, a stolen key, a script run against the wrong\ntarget. The useful question for any copy is which single event would take it out with the original.\n\n## What it looks like in practice\n\nAt the smallest scale, the company's\n[backup and restore-test implementation](/portfolio/backups-that-have-actually-been-restored/)\nwas designed to extend recovery beyond a provider snapshot. An Ansible role installs restic\nscripts and systemd units for a database dump where configured, encrypted deduplicated snapshots,\nretention pruning and integrity checks. A separate monthly workflow restores a known file to a\nscratch location and compares its bytes. Backup and restore have separate success signals, and\nthe timers require a configured repository. The passphrase comes from a protected file rather\nthan a systemd-parsed environment value. Those are implemented controls and activation\nprerequisites: the implementation record dated 2026-10-27 says the timers were disabled without\na repository and records no completed live restore. It supports the automation work, not a\nclaim that recovery has succeeded or that the host can already be lost safely.\n\nAt a larger scale,\n[a hundred critical data backups were automated with Barman and Google Cloud](/portfolio/automated-100-critical-data-backups-using-barman/)\nfor a product that lives on its GIS data. Barman handled PostgreSQL, Google Cloud held the offsite\ncopies, and Bash and Python orchestrated and validated the runs. Integrity checks confirmed that\neach backup was good rather than merely present.\n\nOn a third product the work was to define the recoverable position instead of assuming it.\n[Backup retention lives in the Bicep template beside the database](/portfolio/implemented-backups-and-disaster-recovery-as-code/),\nso point-in-time restore is a property of the template rather than a setting someone once clicked in\na portal, and the environment around it is code too. The audit wrote the gaps down: geo-redundancy\nis switched off, and the RTO is proposed rather than measured, with the drill that would measure it\ndescribed and waiting to run.\n\n## Where it goes wrong\n\nBackup problems tend to stay hidden until a restore is needed, so each is worth looking for on a\nquiet day.\n\n**A file copy of a running database restores as a broken cluster.** Copying PostgreSQL's data\ndirectory with `tar` while the server runs collects files from different moments, and the manual\nstates that the server must be shut down to get a usable backup this way. A filesystem snapshot\ntaken at one atomic instant does work if it includes the WAL, which the server replays on start as\nit would after a crash; with data and WAL on different filesystems, a simultaneous snapshot may be\nimpossible. `pg_dump`, or a base backup for continuous archiving, gives a consistent copy while the\nserver keeps running.\n\n**A gap in the WAL archive ends point-in-time recovery at the gap.** Recovery needs an unbroken\nsequence of archived WAL files reaching back to the start of the base backup. When the archive\ncommand keeps failing, PostgreSQL keeps retrying and WAL piles up in `pg_wal/`, so the first symptom\nis often a filling disk. The manual's advice is to set up and test archiving before the first base\nbackup, and to monitor it for delays.\n\n**The job stops, and the silence looks like success.** A timer job that breaks writes its error to a\nlog nobody reads, and each quiet day passes for a good one. Monitoring has to treat silence as the\nalarm: a check-in sent only after a successful run, to a service that pages when it is late, catches\na job that has stopped as well as one that fails loudly.\n\n**The key goes missing, or arrives changed.** Encryption makes access depend on the keys. restic\nsupports multiple passwords for one repository; its warning about irrecoverable data loss applies\nwhen no usable repository key remains. The key has\nto live somewhere that survives the machine. In the restic setup above, the change could have come\nfrom the supervisor itself: it interprets escape sequences in environment values, so a passphrase\nwith a backslash, handed over that way, would have reached restic as different bytes from the ones\nthe repository was created with, and without a word of warning. Reading it from a file closes that\nroute.\n\n**Retention removes the copy that was needed.** Corruption is often noticed weeks after it happened,\nand a policy that keeps seven daily copies has by then replaced every clean copy with a damaged one.\nA tiered policy keeps a long tail at low cost, and restic's `--dry-run` shows what a policy would\nremove before anything goes:\n\n```bash\nrestic forget --keep-daily 7 --keep-weekly 5 --keep-monthly 12 \\\n  --prune --dry-run\n```\n\n`forget` removes snapshots and `prune` removes the data only they referenced, so after a prune the\nold copies are gone for good.\n\n**The integrity check passes and the restore still breaks.** By default, `restic check` verifies the\nrepository's structure without reading every pack file, since that means downloading all of them;\n`--read-data` reads everything, and `--read-data-subset=1/5` reads one fifth, so five runs from\n`1/5` to `5/5` cover the lot. A full read proves only that the stored bytes are intact. Whether they\nrebuild a working database, with the right roles and settings, is something a restore shows.\n\n## Further reading\n\n- [NIST glossary: backup](https://csrc.nist.gov/glossary/term/backup) — the definition used here and\n  the publications behind it.\n- [NIST SP 800-34 Rev. 1](https://csrc.nist.gov/pubs/sp/800/34/r1/upd1/final) — the contingency\n  planning guide that defines RPO and RTO.\n- [PostgreSQL: Backup and Restore](https://www.postgresql.org/docs/current/backup.html) — the three\n  approaches and what each one costs.\n- [PostgreSQL: Continuous Archiving and Point-in-Time Recovery](https://www.postgresql.org/docs/current/continuous-archiving.html)\n  — base backups, WAL archiving and restoring to a chosen moment.\n- [restic: Preparing a new repository](https://restic.readthedocs.io/en/stable/030_preparing_a_new_repo.html)\n  — passwords, password files and the warning about losing them.\n- [restic: Removing backup snapshots](https://restic.readthedocs.io/en/stable/060_forget.html) —\n  retention policies, `forget`, `prune` and `--dry-run`.\n\n\u003chttps://engineer.company/glossary/backup/\u003e\n","date_modified":"2026-11-09T21:50:25Z","description":"What a backup is, how RPO and RTO shape its schedule, why only a tested restore proves it works, and the quiet ways backups let people down.","formats":{"gemtext":"https://engineer.company/glossary/backup/index.gmi","gopher":"https://engineer.company/glossary/backup/gophermap","html":"https://engineer.company/glossary/backup/","json":"https://engineer.company/glossary/backup/index.json","markdown":"https://engineer.company/glossary/backup/index.md","pdf":"https://engineer.company/glossary/backup/index.pdf"},"kind":"page","language":"en","title":"What is a backup?","url":"https://engineer.company/glossary/backup/","version":1}