Logging

What it is

Logging, also called event logging, is a program keeping a running
account of its own behaviour, one entry per event: a request served,
a password refused, a retry, a crash. Each entry, a log line or log
record, carries at least a time and a message, so that someone can
later reconstruct what happened and in what order. The OpenTelemetry
project defines a log as a timestamped text record, structured or
unstructured, with optional metadata, and notes that of all the
telemetry signals, logs have the biggest legacy.

That legacy has a standard. Syslog, the long-standing protocol for
passing event messages to a collector, was standardised as RFC 5424
in March 2009, which obsoletes the older RFC 3164, and it gives
every message one of eight severities, from 0 for Emergency to 7 for
Debug. Most logging libraries carry a shorter ladder of the same
idea: debug, info, warning, error. OpenTelemetry's log data model
maps them all onto one number from 1 to 24, in six bands from TRACE
to FATAL.

An unstructured line is free text: easy for a person to read, and
something a machine can only take apart with a regular expression
written for that exact wording. A structured line follows a fixed
layout, so every entry has the same named fields with the same types
and meanings. OpenTelemetry prefers structured logs in production
and adds a warning that is easy to miss: a line encoded as JSON is
not structured just because it parses; what makes it structured is a
stable layout. Shown across several lines here, a structured entry
is a single line on disk:

```json
{
    "time": "2026-09-14T03:12:07.412Z",
    "severity": "WARN",
    "service": "checkout",
    "event": "payment.retry",
    "attempt": 3,
    "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736"
}
```

Metrics and traces are the signals usually collected beside it.
Metrics are numbers sampled over time, cheap to keep and good at
saying how often. Traces follow one request through every service it
touches. A log keeps the detail of single events, which makes it
what someone reads after an alert to find out why. OpenTelemetry
ties logs to traces by attaching the IDs of the active trace and
span to each log record.

Why it matters

A log is the only account of the past that most systems leave
behind. A metric can say that errors rose at 03:12; the log says
which request, from which service, with which value, and what the
program did next. When something breaks at night and has recovered
by morning, the log is the witness still there.

That account has a price, paid three ways. Every line costs disk,
input and output, and processor time, and more again if it is
shipped to a central store, where a hosted service typically bills
by the volume taken in. Every line is also data held about someone:
a web server's access log holds the addresses and browsers of
everyone who called. And every line has to be guarded. The OWASP
Logging Cheat Sheet asks that each event record when, where, who and
what, and that the logs themselves be protected against tampering
and against unauthorised access, modification and deletion, because
they may contain personal and other sensitive information.

At the scale of an organisation, NIST Special Publication 800-92,
the Guide to Computer Security Log Management by Karen Kent and
Murugiah Souppaya, published in September 2006, treats log
management as infrastructure and processes run across the whole
enterprise rather than a setting in one program.

Logs also work best next to numbers. The Prometheus project's
instrumentation guide suggests a counter for every line of logging
code, so that when an interesting message turns up, it is possible
to tell how often it has occurred and over how long a period, and
suggests comparing the totals of info, warning and error lines from
one release to the next.

What it looks like in practice

The site's own access log is described line by line on the privacy
page. Each request records the page and time, the response status,
the browser's self-description, the name of the referring site and a
blurred network address: the last part is replaced with zeroes at
the moment the line is written, before anything touches the disk. A
line lives about thirty days. The server software closes each file
once a day, or sooner at about 10 MB, and deletes a closed file
twenty-eight days later, and every night a check reads every file
back and fails on any line older than thirty days. It has gone wrong
twice, and the page records both. In September 2026 the quiet
addresses filled

10 MB so slowly that their files had never been closed and still
held every line since logging began, so a daily job was added to
close them. On 26 October

2026 seven of the thirteen addresses that keep a log were still
writing into files that job had marked as closed, and one kept a
closed file forty-seven days old. The job is gone, and the server
software now closes and deletes the files itself.

A full test run of a macOS application recorded 610,996 core log
lines. A burst of core log events was retained in an unbounded main
stream and copied to subscriber streams whose consumers did not use
them. The incident record separately documents diagnostic reports of
111 GB of compressed pages on a 36 GB machine and a footprint sample
showing growth of about 41 MB per second during a connection retry
loop. Bounded buffers, subscriptions filtered by event type and a
logging rate budget addressed that event path. The recorded full
test-run comparison reduced core log lines to

1,411. It is not a production benchmark or a measured post-fix
memory figure. The budget discards information by design: repetitive
diagnostic events can be dropped and affected state refreshed
instead of retaining unlimited events. Separate shutdown timeouts
required their own correction.

On a production host, a hardening pass over thirteen systemd units
turned up one directive misspelled in three unit templates for as
long as those roles had existed. The service manager answers a key
it does not recognise by logging that fact and starting the unit
anyway, so the only trace of the mistake was a warning in a log; it
came to light while the units were being rewritten. Now every unit
goes through the service manager's own verifier during each
converge, so the same typo fails a run.

Where it goes wrong

The log grows fastest when the system is already failing. A loop
that logs each retry writes a handful of lines on a good day and
thousands a second on a bad one, taking disk, memory and processor
time from a system with none to spare. OWASP asks teams to verify
that logging cannot be turned into a way to exhaust system
resources, for example by filling the disk. Give every hot path a
budget, collapse repeated messages into one line with a count, and
keep a counter beside the message so the rate survives when lines
are dropped.

Secrets and personal data get written down. A request logged whole
carries whatever was in it. OWASP's list of what should usually be
removed, masked, hashed or encrypted before it reaches a log
includes session identifiers, access tokens, passwords, database
connection strings, encryption keys and payment card data. Do it
when the line is written, as the access log above does with
addresses, because by the time anyone cleans up, the line may sit in
a backup or an archive.

Input from outside writes a fake line. A user-supplied value
containing a carriage return or newline, written unescaped into a
text log, can start a new line that looks exactly like one the
application wrote. OWASP calls this log injection and asks that such
characters be sanitised and that data be encoded for the format it
is written in. A structured logger that writes each value as an
escaped JSON string does this by construction.

A log that goes quiet passes for a healthy system. A full disk, a
rotated file the program never reopened or a collector that went
down all look the same from outside: the lines stop arriving. OWASP
asks for a way to detect that logging has stopped. Count lines per
minute for each source and alert when a count drops to zero, so a
gap in the log becomes an alert of its own.

Further reading

  * Logs — OpenTelemetry on what a log is, why structured logs are
preferred and how logs are tied to traces.

  * Logs Data Model — the fields of a log record, its two timestamps
and the severity scale from 1 to 24.

  * RFC 5424: The Syslog Protocol — the syslog message format and
its eight severities.

  * NIST SP 800-92 — the Guide to Computer Security Log Management.

  * Logging Cheat Sheet — OWASP on which events to record, which
data to keep out and how to protect the result.

  * Instrumentation — the Prometheus project's practices for
instrumenting code, logging included.

hThis page on the web	URL:https://engineer.company/glossary/logging/	engineer.company	70
