{"canonical_url":"https://engineer.company/glossary/container-runtime/","content_markdown":"# Container runtime\n\n## What it is\n\nA container runtime is the software between a container image on disk and an isolated process in the\nkernel. The phrase covers two layers, and people often mean different ones: the low-level runtime,\nalso called the OCI runtime, and the high-level runtime, more often called a container engine.\n\nThe low-level runtime does one narrow job. It reads a bundle, which the Open Container Initiative's\nruntime specification defines as a directory holding a `config.json` file and the container's root\nfilesystem, asks the kernel for the namespaces, cgroups and mounts that the configuration describes,\nand starts the process inside them. The specification names the operations every runtime must\nsupport, namely state, create, start, kill and delete, and leaves the command line to each\nimplementation. Version 1.0 was published in July 2017. runc, maintained by the OCI itself, calls\nitself \"a low level tool not designed with an end user in mind\". crun does the same job in C with a\nsmaller memory footprint and is the default Podman's documentation names. youki is a third, written\nin Rust.\n\nThe engine sits above it. containerd describes itself as a daemon that manages the complete\ncontainer lifecycle of its host: image transfer and storage, container execution and supervision,\nand the storage and network attachments around them. CRI-O does a similar job built for Kubernetes\nalone. An engine pulls and unpacks the image, writes the bundle, hands it to the low-level runtime,\nand keeps watching the process after that runtime has exited. Docker Engine sits one step higher\nstill: its documentation says it uses containerd for the container lifecycle, and that containerd\nuses runc by default. Podman takes the other route and keeps no long-running daemon. Each command\nprepares the bundle itself, calls crun or runc, and leaves a small monitor called conmon in charge\nof the running container.\n\nKubernetes adds one more piece of vocabulary. The kubelet on each node talks to the engine over the\nContainer Runtime Interface, CRI, a gRPC protocol that containerd and CRI-O both implement. Docker\nEngine predates CRI, so for years Kubernetes carried an adapter for it called dockershim, and\nremoved it in v1.24.\n\n## Why it matters\n\nChoosing a container runtime is really choosing which layer to change, and the two layers buy\ndifferent things.\n\nThe engine decides how containers live from day to day: who pulls and stores images, who restarts a\ncrashed container, where its logs go, which API other tools speak, and who owns the process tree.\nSwapping Docker Engine for Podman on a server, or for containerd on a Kubernetes node, changes all\nof that and leaves the containers themselves alone, because the image format and the bundle are the\nsame OCI standards underneath. The Kubernetes documentation on the dockershim removal says it\nplainly: images produced by `docker build` work with all CRI implementations.\n\nThe low-level runtime decides how isolated the process is. runc and crun start an ordinary process\non the host's kernel, which is fast and fully compatible, and means every container on the machine\nshares that one kernel. gVisor's runtime, runsc, puts an application kernel written in Go between\nthe container and the host and answers the container's system calls itself; its own documentation\nnames the price as reduced application compatibility and higher overhead per system call. Kata\nContainers boots each workload inside a lightweight virtual machine with a dedicated kernel, using\nhardware virtualization as a second layer of defence, and needs a hypervisor on each node that runs\nit.\n\nKubernetes lets the second choice be made per pod rather than per cluster. A RuntimeClass names a\nhandler configured in the engine, and a pod that sets `runtimeClassName` runs under that handler, so\nan untrusted workload can sit in gVisor or Kata while everything else stays on runc. The\ndocumentation frames this as a balance of performance against security.\n\n## What it looks like in practice\n\nIn everyday speech the phrase drifts towards the engine layer, and the work on this site uses it in\nthat sense.\n\nTwenty Docker-packaged applications\n[spread over two clouds and a few container runtimes](/portfolio/ran-20-docker-apps-on-google-cloud-and-aws/)\nhad to deploy consistently and be diagnosable fast when something broke. Docker packaged them,\ntroubleshooting went through Podman, and orchestration debugging went through Kubernetes. Working\nacross engines like that leans on the layering above: the engines differ, and the OCI image and\nruntime standards beneath them are shared.\n\nOn a newer host the choice was made at the engine layer.\n[The container plane runs on Podman and Quadlet in place of Docker](/portfolio/ran-containers-on-podman-and-quadlet/),\nbecause the default engine publishes container ports above the firewall rules the hardening role\ninstalls, and because it would add a second supervisor beside systemd. What the choice changed is\nwho starts each container and who watches it afterwards: containers became services that start the\nway services start, under the supervisor the host already trusted.\n\nThe same reasoning carries over to\n[a 512 MB host for the company's public services](/portfolio/a-whole-company-on-one-small-server/),\nwhere memory settled every argument. Its container plane is Quadlet units under systemd rather than\na daemon with its own supervisor, a choice made on a machine where about 464 MB of memory is usable.\n\n## Where it goes wrong\n\nRuntime trouble tends to surface at the seams between layers, where each side assumes the other is\nset up the way it expects.\n\n**Node tooling goes blind after a cluster upgrade.** From Kubernetes v1.24 the kubelet talks to\ncontainerd or CRI-O directly, so containers the cluster schedules never pass through Docker Engine.\n`docker ps` on the node lists only what Docker itself started, and monitoring agents, startup\nscripts and privileged pods that call the Docker CLI or mount `docker.sock` lose their view. The\nworkloads keep running. It is the tools around them that need moving to `crictl` or the engine's own\nAPI.\n\n**The kubelet and the runtime manage cgroups two different ways.** Both must use the same cgroup\ndriver, `cgroupfs` or `systemd`, and the kubelet defaults to `cgroupfs`. On a host where systemd is\nthe init system, that leaves two cgroup managers with two views of the same resources, and the\nKubernetes documentation warns that such nodes can become unstable under resource pressure. The fix\nis the systemd driver on both sides, which is also the recommended driver with cgroup v2. For\ncontainerd 2.x with runc, the runtime side looks like this:\n\n```toml\n# /etc/containerd/config.toml, containerd 2.x\n[plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.runc.options]\n  SystemdCgroup = true\n```\n\nThe kubelet side is `cgroupDriver: systemd` in its configuration, and containerd needs a restart\nafterwards. containerd 1.x spells the section differently, and the Kubernetes page gives both forms.\nChanging the driver on a node that has already joined a cluster is delicate: pods created under one\ndriver can hit errors when they are recreated under the other, so the documented advice is to\nreplace or reinstall the node with the new configuration.\n\n**A pod moved to a sandboxed runtime stops at startup.** A RuntimeClass whose handler the node's\nengine lacks sends the pod straight to the Failed phase, with the reason in an event. Past that, the\nworkload has to suit the sandbox. gVisor answers system calls in its own kernel, so software that\nleans on less common kernel features may behave differently, and its documentation names\nsystem-call-heavy workloads as a poor fit. Kata needs hardware virtualization on the node, which on\na cloud virtual machine means nested virtualization. A `scheduling` node selector on the\nRuntimeClass keeps such pods on the nodes that can run them.\n\n**An image imported by hand stays invisible to the kubelet.** containerd keeps image names and\nmetadata per namespace. Its `ctr` client works in the namespace `default` unless told otherwise,\nwhile the CRI plugin the kubelet talks to works in `k8s.io`. An image loaded with a plain\n`ctr images import` therefore lands where the kubelet never looks, and the pod goes back to the\nregistry or stops with an image pull error. Importing with `ctr -n k8s.io images import`, and\nchecking with `crictl images`, which sees what the kubelet sees, avoids the gap.\n\n## Further reading\n\n- [OCI Runtime Specification](https://github.com/opencontainers/runtime-spec) — the standard every\n  low-level runtime implements: the bundle, `config.json` and the lifecycle operations.\n- [runc](https://github.com/opencontainers/runc) — the OCI's own runtime, with notes on how\n  higher-level tools drive it.\n- [Container Runtimes](https://kubernetes.io/docs/setup/production-environment/container-runtimes/)\n  — cgroup drivers, CRI versions and the containerd and CRI-O settings Kubernetes expects.\n- [Container Runtime Interface (CRI)](https://kubernetes.io/docs/concepts/containers/cri/) — the\n  gRPC protocol between the kubelet and the engine.\n- [Runtime Class](https://kubernetes.io/docs/concepts/containers/runtime-class/) — picking a\n  different runtime handler per pod, and keeping those pods on nodes that support it.\n- [Alternative container runtimes](https://docs.docker.com/engine/daemon/alternative-runtimes/) —\n  how Docker Engine hands containers to gVisor, Kata Containers, youki or Wasmtime.\n\n\u003chttps://engineer.company/glossary/container-runtime/\u003e\n","date_modified":"2026-11-08T02:22:23Z","description":"What a container runtime is, how runc, containerd, CRI-O and Docker stack up, which layer a runtime choice changes, and where clusters trip over it.","formats":{"gemtext":"https://engineer.company/glossary/container-runtime/index.gmi","gopher":"https://engineer.company/glossary/container-runtime/gophermap","html":"https://engineer.company/glossary/container-runtime/","json":"https://engineer.company/glossary/container-runtime/index.json","markdown":"https://engineer.company/glossary/container-runtime/index.md","pdf":"https://engineer.company/glossary/container-runtime/index.pdf"},"kind":"page","language":"en","title":"What is a container runtime?","url":"https://engineer.company/glossary/container-runtime/","version":1}