{"canonical_url":"https://engineer.company/glossary/end-to-end-test/","content_markdown":"# End-to-end test\n\n## What it is\n\nAn end-to-end test checks a whole running system the way a user meets it. It\nstarts the application with its real front end, back end, database and\nconfiguration, walks through a journey such as signing up or paying for an\norder, and asserts on what the user would see at the end. The usual shorthand\nis E2E test, and the practice is called end-to-end testing.\n\nIn _Software Engineering at Google_, Adam Bender sorts tests along two axes.\nSize is about resources: a small test stays inside one process; a medium test\nmay use several processes on one machine; a large test may span machines. Scope\nis about how much code a test exercises, and tests of large scope go by several\nnames, among them functional tests, end-to-end tests and system tests. Joseph\nGraves, in the book's chapter on larger testing, adds that the tests with the\nwidest scope usually bring in several real dependencies and use fewer test\ndoubles.\n\nWhat those real dependencies buy is fidelity, which Graves defines as how well a\ntest reflects the real behaviour of the system under test. A unit test of a\ncheckout form proves that the form's code is right in a sandbox; an end-to-end\ntest of the same form proves that the deployed page, the route behind it, the\npricing rules and the database agree with one another. The cost is hermeticity:\nthe more an environment resembles production, the harder it is to seal it off\nfrom everything else, and Graves notes that the two are often in direct\nconflict.\n\nFor a web application the user's interface is the browser, so that is where an\nend-to-end test stands. Graves puts it plainly: for end users, \"the public API\nis the application itself\". Playwright Test, which the example below uses, is\nthe runner the Playwright project builds for end-to-end tests of web\napplications. It bundles the runner, assertions, isolation and parallel runs,\nand drives Chromium, Firefox and WebKit, headless or headed.\n\n## Why it matters\n\nSmaller tests leave gaps that only a test of the assembled system can close, and\nGraves lists the common ones. A test double can drift away from the thing it\nreplaces, and nothing tells the test that the real implementation has changed. A\nunit test covers the code inside one binary, while the deployment configuration,\nstart-up scripts and settings of the production instances sit outside it.\nProblems that appear only under load are a third gap, and the fourth is\nbehaviour nobody anticipated, which is how most problems reach users in the\nfirst place. Bender writes that Google keeps large tests mostly for end-to-end\nruns of the whole system, which check its configuration more than any one piece\nof code.\n\nThey also check the assembled interface that the user meets. A form that\nposts to a route the back end has just renamed, or a release whose\nconfiguration points at the wrong service, passes every unit test on both sides\nand fails the first time a real person tries it.\n\nThe price is speed and reliability. Graves notes that by default Google gives a\nlarge test 15 minutes or an hour before it times out, and that they may be\nnonhermetic, sharing resources with other tests and traffic, and therefore\nnondeterministic. Bender names the usual sources of that nondeterminism: the\nclock, network delays, thread execution order, hardware interrupts and the\nbrowser's renderer. That is why the book's rough guideline is a pyramid: about\n80 per cent narrow-scoped unit tests, 15 per cent integration tests and 5 per\ncent tests that exercise the complete system from end to end. Larger tests,\nBender writes, serve as a sanity check while the product grows and should not be\nthe main way a team catches bugs, and Google teams often reserve them for builds\nand releases, keeping their cost out of the faster development loop.\n\n## What it looks like in practice\n\nTake a shop that sells a teapot and asks a carrier's API for a delivery\nestimate. A Playwright spec for a guest's first purchase reads like the journey\nitself:\n\n```ts\n// e2e/checkout.spec.ts\nimport { test, expect } from \"@playwright/test\";\n\ntest.beforeEach(async ({ page }) =\u003e {\n    // The carrier's API belongs to someone else: answer for it here.\n    await page.route(\"https://api.carrier.example/**\", (route) =\u003e\n        route.fulfill({ status: 200, body: '{\"days\": 2}' }),\n    );\n});\n\ntest(\"a guest can buy a teapot\", async ({ page }) =\u003e {\n    await page.goto(\"/shop/teapot\");\n    await page.getByRole(\"button\", { name: \"Add to basket\" }).click();\n    await page.getByRole(\"link\", { name: \"Basket\" }).click();\n    await expect(page.getByText(\"Delivery in 2 days\")).toBeVisible();\n    await page.getByLabel(\"Email\").fill(\"buyer@example.com\");\n    await page.getByRole(\"button\", { name: \"Place order\" }).click();\n    const thanks = page.getByRole(\"heading\", { name: \"Thank you\" });\n    await expect(thanks).toBeVisible();\n    await page.getByRole(\"link\", { name: \"View your order\" }).click();\n    await expect(page.getByText(\"Teapot\")).toBeVisible();\n});\n```\n\nEach locator names what the user sees: a button called Add to basket, a field\nlabelled Email, a heading that says Thank you. Playwright recommends role\nlocators first, and its locators are strict: when more than one element matches,\nthe action throws instead of guessing. Before each click or fill, Playwright\nwaits for the element to be actionable, and an assertion such as `toBeVisible()`\nkeeps retrying until the condition holds, so the spec contains no sleeps. The\n`page.route` call answers the browser's requests to the carrier with a fixed\nbody, so the test never depends on a server the team does not run; the glob has\nto match the whole URL, and `**` matches across slashes. Routing reaches only\nthe requests the page itself makes: when the shop's server asks the carrier\ninstead, the browser never sees that request, and a test double has to stand in\non the server's side. Every test also gets a fresh browser context, which\nPlaywright likens to a brand-new browser profile, so the cookies and basket of\none test cannot leak into the next.\n\nThe configuration starts the application and tells Playwright where it lives:\n\n```ts\n// playwright.config.ts\nimport { defineConfig } from \"@playwright/test\";\n\nexport default defineConfig({\n    testDir: \"e2e\",\n    retries: process.env.CI ? 2 : 0,\n    use: {\n        baseURL: \"http://localhost:3000\",\n        trace: \"on-first-retry\",\n    },\n    webServer: {\n        command: \"npm run start\",\n        url: \"http://localhost:3000\",\n        reuseExistingServer: !process.env.CI,\n    },\n});\n```\n\nThe `webServer` block runs `npm run start` and waits until the URL answers\nbefore the first test; on a laptop it reuses a server that is already running,\nand on CI it always starts its own. `baseURL` lets the spec say `/shop/teapot`\ninstead of a full address. `npx playwright test` runs the suite headless,\n`--headed` shows the browser window, and `npx playwright show-report` opens the\nHTML report. Playwright retries no failing test by default; here CI allows two\nretries, a test that fails and then passes is reported as flaky rather than\npassed, and a trace is recorded on the first retry to show what was different.\n\nAt a larger scale the browser layer is the smallest share of the work. The\n[test suite built across Go, SQL and the browser](/portfolio/built-a-test-suite-across-go-sql-and-browser/),\nfrom 2025, has 981 Go test functions across 310 files, 543 Vitest and Playwright\nspecs including 48 end-to-end files and 30 browser specs, and 494 SQL\nbehavioural test files. The task was to check each layer's behaviour where that\nbehaviour lives rather than everything from the outside through a browser, so a\nstored function is tested in the database instead of through three layers of\napplication above it. The page is plain about the cost: the suite is slow, it\ntaxes every change, and at that size it needs maintenance of its own.\n\n## Where it goes wrong\n\n**The suite is an ice cream cone.** Bender's name for a suite heavy on\nend-to-end tests and light on integration and unit tests. A suite of that shape\nis usually slow, unreliable and awkward to work with, and the shape is common in\nprojects that went from prototype to production in a hurry. Graves adds that\nwhen the code is hard to unit test, end-to-end tests are the only ones anyone\ncan write, and the project has produced legacy code within days. Each new\nservice also multiplies the paths through the system, so an end-to-end suite can\nnever cover them all. Push each case down to the smallest test that can catch\nit, keep the browser for the journeys only the whole system can show, and build\ntowards the pyramid from the first days of a project.\n\n**Selectors follow the markup.** A locator such as\n`button.buttonIcon.episode-actions-later`, Playwright's own example of what to\navoid, breaks the next time someone renames a class, although the user sees\nnothing different. Graves warns that UI tests have a reputation for being\nunreliable and expensive, partly because changes to look and feel break them\nwhile the behaviour stays the same. Locate elements by role, label or text, the\nway a user finds them, or by a test id the team owns. Where the back end has a\npublic API, split the journey at the boundary between UI and API, and drive the\nrest of the end-to-end test through the API.\n\n**The test sleeps.** A fixed wait is too short on a slow day and too long on\nevery other day. Graves notes that tests relying on sleeps and timeouts all\nstart failing when the machines that run them are overloaded, and the reruns\nadd more load. Playwright's best practices show a subtler version of the same\nmistake:\n`expect(await page.getByText(\"welcome\").isVisible()).toBe(true)` checks once\nand moves on, while\n`await expect(page.getByText(\"welcome\")).toBeVisible()` waits for the\ncondition. Use web-first assertions, and when a test waits on the back end,\npoll for the state change with a timeout.\n\n**Tests share data.** One test creates an account that the next one expects, or\ntwo parallel workers buy the last teapot in stock. The suite passes in one order\nand fails in another, or only when it runs in parallel. Playwright's advice is\nto keep each test completely isolated, with its own storage, session, data and\ncookies, and to control the data in any database the tests use. Create what\neach test needs, with unique names, and never rely on what an earlier test left\nbehind.\n\n**Someone else's server is in the test.** A real payment provider, carrier or\nmaps service makes the suite depend on a system the team cannot control, and\nsending it traffic may cost money. Graves advises against letting automated\ntests call a real third-party API, and calls such a dependency an important seam\nalong which to split tests; Playwright's own advice is to test only what you\ncontrol. Answer for the third party with `page.route` when the browser makes the\ncall, as the spec above does, or with a test double the server is configured to\ncall when the server makes it, and let that dependency be where the test ends.\n\n**Retries hide the flakiness.** Retrying on CI turns a red build green, and the\ncost compounds. Bender works through the arithmetic: at 10,000 tests a day, a\n0.1 per cent chance of a false failure in each means 10 flakes a day to\ninvestigate, and as flakiness approaches 1 per cent the tests begin to lose\nvalue. Rerunning trades machine time for engineering time and only postpones the\nroot cause. The pytest documentation makes the same point from another angle:\ntests at a higher level depend on more state, and so fail intermittently more\noften. Treat every test Playwright reports as flaky as a bug, open the trace\nfrom the retry, and fix the test or delete it.\n\n## Further reading\n\n- [Testing Overview](https://abseil.io/resources/swe-book/html/ch11.html)\n  — Adam Bender's chapter of _Software Engineering at Google_: test size and\n  scope, the pyramid, the ice cream cone and the cost of flaky tests.\n- [Larger Testing](https://abseil.io/resources/swe-book/html/ch14.html)\n  — Joseph Graves on fidelity, the gaps unit tests leave, UI and third-party\n  boundaries, and driving out flakiness.\n- [Installation](https://playwright.dev/docs/intro)\n  — Playwright's own introduction: installing Playwright Test, running the\n  example test, headed mode and the HTML report.\n- [Best Practices](https://playwright.dev/docs/best-practices)\n  — testing user-visible behaviour, isolating tests, leaving third-party\n  servers out, locators and web-first assertions.\n- [Retries](https://playwright.dev/docs/test-retries)\n  — how Playwright retries failing tests and labels each one passed, flaky or\n  failed.\n- [Flaky tests](https://docs.pytest.org/en/stable/explanation/flaky.html)\n  — pytest's explanation of why tests flake, from uncontrolled system state and\n  test order to thread safety, and what to do about it.\n\n\u003chttps://engineer.company/glossary/end-to-end-test/\u003e\n","date_modified":"2026-11-12T05:34:23Z","description":"Where an end-to-end test sits beside unit tests, a worked Playwright checkout spec and config, and the habits that make browser suites slow and flaky.","formats":{"gemtext":"https://engineer.company/glossary/end-to-end-test/index.gmi","gopher":"https://engineer.company/glossary/end-to-end-test/gophermap","html":"https://engineer.company/glossary/end-to-end-test/","json":"https://engineer.company/glossary/end-to-end-test/index.json","markdown":"https://engineer.company/glossary/end-to-end-test/index.md","pdf":"https://engineer.company/glossary/end-to-end-test/index.pdf"},"kind":"page","language":"en","title":"What is an end-to-end test?","url":"https://engineer.company/glossary/end-to-end-test/","version":1}