{"canonical_url":"https://engineer.company/glossary/opentelemetry/","content_markdown":"# OpenTelemetry\n\n## What it is\n\nOpenTelemetry is an open-source standard, with libraries and tools to match,\nfor producing traces, metrics and logs from running software and handing them\nto whatever system stores and displays them. Most engineers call it OTel. It is\na project of the Cloud Native Computing Foundation, formed when two earlier\nefforts, OpenTracing and OpenCensus, merged into one.\n\nThe most useful thing to know about it is what it leaves out. OpenTelemetry is\nnot an observability backend: it keeps no data, draws no dashboards and sends\nno alerts. Jaeger, Prometheus and the commercial platforms do that part. So the\nquestion “OpenTelemetry or Prometheus?” compares a pipe with a destination, and\nthe usual answer is both: OpenTelemetry where the data is produced, a backend of\nthe team's choosing where it is kept.\n\nThe project is several pieces that fit together:\n\n- a **specification** that defines the API, the SDK and the data, across every\n  language;\n- **OTLP**, the protocol that carries telemetry over gRPC or HTTP, encoded with\n  Protocol Buffers;\n- **semantic conventions**, shared names for shared things, so an HTTP route or\n  a database statement is labelled the same way in Go, Java and Python;\n- an **API and SDK per language**, with instrumentation libraries and automatic\n  instrumentation for popular frameworks;\n- the **Collector**, a separate process that receives telemetry, processes it\n  and exports it onward.\n\nOf the signals it carries, traces take the most explaining. A trace follows one\nrequest through every service it touches. Each step is a span: a name, a start\ntime, a duration, a set of attributes, and the identity of the span that caused\nit. When the request crosses from one service to the next, that identity\ntravels with it in a header, by default the one W3C Trace Context defines:\n\n```text\ntraceparent: 00-a0892f3577b34da6a3ce929d0e0e4736-f03067aa0ba902b7-01\n```\n\nThe four fields are the format version, the trace ID shared by every span of the\nrequest, the ID of the calling span, and a set of flags, where a final `01`\nmarks the trace as sampled. Metrics and logs complete the picture, and a fourth\nsignal, baggage, carries small key-value facts along with the request so that a\nservice three hops away can still see them.\n\n## Why it matters\n\nStorage is the visible cost of observability. The stickier one is\ninstrumentation: someone has to add the calls to every service, in every\nlanguage, and keep them current as the code changes. When each vendor shipped\nits own agent and its own API, changing vendor meant doing all of that again.\nWith OpenTelemetry the code calls one API, and the choice of backend moves into\nconfiguration, either an exporter in the SDK or a route in the Collector. The\nproject states the principle outright: the data a system generates belongs to\nwhoever runs it.\n\nThat ownership is practical as well as contractual. The Collector sits between\nthe applications and the backend, which makes it the place to filter, sample,\nrename or strip attributes before anything leaves, and to send the same stream\nto two backends while a migration is under way. Moving from one vendor to\nanother becomes a change to a Collector configuration rather than to a hundred\nservices.\n\nVolume is the cost to plan for. Each request can produce dozens of spans, and at\nreal traffic the bill for keeping all of them can outgrow the service being\nwatched. The SDK's default sampler, `ParentBased(root=AlwaysOn)`, samples every\nnew trace and follows the caller's decision for the rest, so out of the box\neverything is kept. How much to keep is a decision a team makes on purpose.\n\nMaturity differs by signal and by language. OTLP is stable for traces, metrics\nand logs, while each language's SDK moves at its own pace, and the project's\nstatus page says where each one stands. Read it before committing a service\nwritten in a less common language.\n\nAnd it is not always worth it. A single application with structured logs and a\nhandful of metrics gains little from distributed tracing. OpenTelemetry pays for\nitself when a request crosses process boundaries, through a frontend, an API, a\nqueue and a database, and the question “where did the time go?” takes a meeting\nto answer.\n\n## What it looks like in practice\n\nOn one platform the monitoring was built rather than bought, and OpenTelemetry\nis its tracing half.\n[Tracing runs over OTLP, so a slow request is followed across the frontend, the API and the database](/portfolio/built-first-party-error-monitoring-and-tracing/)\nrather than guessed at. Beside it runs an in-house error pipeline, an errmon\nservice and an ingest service, that sanitises payloads before they are stored,\ngroups errors into recurring problems, detects spikes and regressions with a\ncooldown so one bad deploy does not page anyone forty times, symbolicates\nminified frontend stack traces, and runs a synthetic heartbeat to prove that the\npipeline itself is alive.\n\nThe results land in the platform's own database as error events, error groups,\nan inbox, API latency and stack samples. Operators work from 11 views over that\ndata, one of them for service level objectives. The page is plain about the\nprice: building cost real time and added one more thing to maintain, and a\nbought tracker would have been running the same afternoon. In return, sensitive\ndata stays on the platform. The alerting rules, too, were written for this\nplatform rather than a generic one.\n\n## Where it goes wrong\n\n**The trace stops at a boundary.** Context crosses HTTP calls easily, because\nthe instrumentation libraries write and read the header on both sides. A message\nqueue, a scheduled job or a proxy that strips unknown headers breaks the chain\nunless someone carries the context across by hand. The trace splits in two, and\nthe slow half looks unrelated to the request that caused it. Put the context in\nthe message metadata, and check that every service runs the same propagator.\n\n**Attributes leak or explode.** A span attribute is easy to add, and the easiest\nvalues to add are the wrong ones: email addresses, full URLs with tokens in the\nquery string, SQL with the literal values still in it. On metrics the danger is\ncardinality. An attribute with millions of distinct values, a user ID for\ninstance, multiplies the number of time series and the bill along with it.\nFollow the semantic conventions, keep identifiers off metrics, and redact in the\nCollector as a second line of defence.\n\n**Sampling happens by accident.** Head sampling decides at the start of a\nrequest, before anyone knows whether it will be slow or fail, so keeping 1% of\ntraces also keeps about 1% of the rare slow ones. Tail sampling decides after\nseeing a trace's spans together and can keep every error and every slow request,\nbut the component doing it must hold all that data while it waits. It is a\nstateful service with capacity of its own to plan and to monitor.\n\n**The pipeline is trusted blindly.** OTLP acknowledges delivery one hop at a\ntime; end-to-end delivery is outside its scope. A Collector that runs short of\nmemory or falls behind on its export queue drops telemetry, and the applications\nsending to it carry on as if all were well. Watch the Collector's own metrics,\nrun more than one, and treat a gap in the traces as a fault to explain rather\nthan a quiet hour.\n\n## Further reading\n\n- [What is OpenTelemetry?](https://opentelemetry.io/docs/what-is-opentelemetry/)\n  — the project's own definition, its components, and what it deliberately is\n  not.\n- [OTLP Specification](https://opentelemetry.io/docs/specs/otlp/) — the wire\n  protocol: transports, encoding, compression and acknowledgement.\n- [Trace Context](https://www.w3.org/TR/trace-context/) — the W3C\n  Recommendation behind the `traceparent` header.\n- [Context propagation](https://opentelemetry.io/docs/concepts/context-propagation/)\n  — how a trace follows a request across process boundaries.\n- [Semantic conventions](https://opentelemetry.io/docs/concepts/semantic-conventions/)\n  — the shared names for attributes, metrics and resources.\n- [Sampling](https://opentelemetry.io/docs/concepts/sampling/) — head and tail\n  sampling, and what each one costs to run.\n\n\u003chttps://engineer.company/glossary/opentelemetry/\u003e\n","date_modified":"2026-10-20T21:59:50Z","description":"How OpenTelemetry lets code be instrumented once for any backend, what OTLP and the Collector do, and where tracing projects usually stall.","formats":{"gemtext":"https://engineer.company/glossary/opentelemetry/index.gmi","gopher":"https://engineer.company/glossary/opentelemetry/gophermap","html":"https://engineer.company/glossary/opentelemetry/","json":"https://engineer.company/glossary/opentelemetry/index.json","markdown":"https://engineer.company/glossary/opentelemetry/index.md","pdf":"https://engineer.company/glossary/opentelemetry/index.pdf"},"kind":"page","language":"en","title":"What is OpenTelemetry?","url":"https://engineer.company/glossary/opentelemetry/","version":1}