{"canonical_url":"https://engineer.company/glossary/penetration-test/","content_markdown":"# Penetration test\n\n## What it is\n\nA penetration test, often shortened to pentest, is security testing in which the\ntesters mimic real-world attacks to find ways around the security features of an\napplication, a system or a network. NIST's Technical Guide to Information\nSecurity Testing and Assessment, SP 800-115, notes that this often means\nlaunching real attacks on real systems and data, with the tools and techniques\nthat attackers use. The shorter definition in SP 800-53 Rev. 5, NIST's catalogue\nof security and privacy controls, adds that the assessors typically work under\nspecific constraints, and that is what separates a test from an attack: it is\nauthorised, bounded and agreed in writing before it starts.\n\nWhat sets it apart from other security testing is the aim. A vulnerability\nscanner compares what it sees with descriptions of known flaws and reports what\nmight be there. A penetration tester tries to use the flaw. SP 800-115 puts the\ndifference plainly: scanners check only for the possible existence of a\nvulnerability, while the attack phase of a penetration test exploits it to\nconfirm that it exists. Most tests also look for combinations, weaknesses on one\nor several systems that together give more access than any one of them would\nalone.\n\nThe OWASP Web Security Testing Guide (WSTG) notes that the technique also goes\nby the names black-box testing and ethical hacking. The team tests from the\noutside, without prior knowledge of the target's internal design, and behaves as\nan attacker would, although it is often given one or more valid user accounts.\n\nTwo choices shape every test. The first is the starting point. An external test\nbegins outside the network with little more than the addresses in scope; an\ninternal test begins inside it, usually with the access of an ordinary employee,\nand tries to climb from there by privilege escalation. The second is who knows.\nSP 800-115 calls overt testing white hat testing: it runs with the knowledge and\nconsent of the IT staff. Covert testing, or black hat testing, runs without the\nIT staff knowing but with the full knowledge and permission of upper management.\n\nSP 800-115 divides the work into four phases, and adds in a footnote that this\nis one of many acceptable ways to group it. In planning, the rules are agreed,\nmanagement approval is documented and the goals are set; the testing itself\nstarts in discovery. Discovery gathers information about the target, scans it,\nand compares what it finds with vulnerability databases such as the US National\nVulnerability Database and with the testers' own knowledge. The attack phase,\nwhich the guide calls \"at the heart of any penetration test\", tries to exploit\neach candidate flaw. When an exploit opens new access, the testers loop back\ninto discovery from the new position. Reporting runs alongside all three and\nends with a report that lists the vulnerabilities found, rates their risk and\nsays how to fix them. Other methods split the same work differently: the\nPenetration Testing Execution Standard (PTES), which the WSTG lists among its\nreference methodologies, uses seven phases, from pre-engagement interactions to\nreporting.\n\n## Why it matters\n\nA scanner's report is a list of possibilities. Some of them cannot be exploited\nin your setup, and a scanner cannot see a flaw that nobody has described, or\none that appears only when two harmless-looking weaknesses are used together.\nA penetration test answers a different question: what could an attacker\nactually do here, and how far could they get? SP 800-115 adds what else it can\nshow: how well a system stands up to real attack patterns, how skilled an\nattacker would need to be, which further countermeasures would help, and whether\nthe defenders notice the attack and respond.\n\nThat is why its findings carry weight with people who never read scanner\noutput. A finding that says an outsider without an account read the customer\ntable is evidence, not a probability. SP 800-115 describes penetration testing\nas an iterative process that uses minimal access to gain greater access, and a\ngood report shows that chain step by step, so the fix can break it at its\ncheapest link.\n\nThe same realism makes it costly and risky. NIST calls it labour-intensive,\nsays it needs great expertise, and warns that systems may be damaged or made\ninoperable during the test; experienced testers reduce that risk, but it can\nnever be fully eliminated. Many of the things a tester does, such as attacking\nsystems to compromise them, are activities an organisation's own policies\nnormally forbid. The rules of engagement (ROE) exist for that reason: a\ndocument, agreed before the test begins, that gives the test team authority to\ncarry out defined activities and nothing beyond them.\n\nIt is also one technique among several, not a verdict on its own. The WSTG\nobserves that many teams make penetration testing their main way of checking a\nweb application's security, and argues that it should be neither the main check\nnor the only one. Web applications are almost always bespoke, so testing them\nfrom outside is closer to research than to finding known flaws in standard\nproducts, and the test arrives late in the development life cycle, after the\ndesign and code are settled. When the testers have the finished application but\nno access to its source code, the guide adds, a penetration test is still\nclearly better than leaving the application untested.\n\nFrequency follows from the cost. SP 800-115 suggests that, because of its high\ncost and potential impact, testing an organisation's network and systems once a\nyear may be sufficient, with regular, lighter scanning in between. Some\nstandards also ask for it directly: the WSTG notes that PCI DSS, the payment\ncard industry's data security standard, defines its own penetration testing\nrequirement, covering both external and internal testing.\n\n## What it looks like in practice\n\nMost of a penetration test is decided before the first request is sent.\nAppendix B of SP 800-115 is a template for the rules of engagement, with\nsections for the purpose and scope, the people and the test schedule, how the\nparties communicate and when testing halts, the systems in scope and those\nexcluded, the activities allowed and forbidden, how test data is handled, how\nresults are reported, and a page of signatures. A trimmed version for an\nexternal test of a web shop might read as follows. It was written for this\narticle, and the names and addresses come from the ranges reserved for\ndocumentation.\n\n```text\nRULES OF ENGAGEMENT: external test of shop.example.com\n\nPurpose       Find which flaws an outsider can exploit, and how far\n              each one leads.\nTargets       203.0.113.10 to 203.0.113.20, ownership checked in WHOIS\nExclude list  203.0.113.15 (payment provider, not ours to test)\n              Every third-party service the shop calls\nAccounts      Two customer accounts and one support account\nSchedule      3 to 14 November, 19:00 to 06:00 CET, outside trading peaks\nTester IPs    198.51.100.7 and 198.51.100.8 only\nAllowed       Exploiting a confirmed flaw to show its impact; reading\n              data that belongs to the test accounts; uploading one\n              marked test file, removed at the end\nForbidden     Denial of service; changing or deleting customer data;\n              social engineering; physical entry\nHalt          Outage of the live shop, real customer data in view, or\n              signs of a real intruder: stop, phone the security lead\nResume        Only on written word from the security lead\nTest data     Sensitive; kept on the test team's laptops, deleted 30\n              days after the final report\nReporting     Daily status note; final report within ten working days\nSigned        Test team leader; chief information security officer\n```\n\nEach line answers a question that SP 800-115 raises. The guide says to verify\nthat every public address in the target list belongs to the organisation before\ntesting begins, and points to WHOIS registration data for that. It suggests\ntesting an operational site in the evening rather than during peak business\nhours. It asks for the addresses the testers work from, so that administrators\ncan tell test traffic from a real attack. It calls the list of systems that may\nnot be touched the exclude list, and says the rules should give criteria for\nhalting the test and a process for resuming it. It also asks what happens if\nthe testers find an actual intruder: should testing stop, and when can it start\nagain, on whose authority? At a minimum, the test team leader and senior\nmanagement sign, stating that they understand the scope and its limits.\n\nUnder rules like these, the attack phase follows the pattern the WSTG gives for\nSQL injection. A scan of the shop's search page fingerprints a possible flaw.\nThe first evidence is a database error when a single quote is typed into the\nfield. The tester then sends crafted input that changes the grammar of the\nquery, and confirms the flaw only when the query returns data it should not.\nIf that data included the support account's session, the tester would sign in\nas support, look again from there, and the report would show the whole path: one\nunfiltered field, one leaked session, one support view of every order.\n\nA test like this sits beside other checks, not in place of them. The write-up of a\n[continuous security scanning programme](/portfolio/ran-a-continuous-security-scanning-programme/)\nfrom the founder's work under NextMariner\ndescribes static analysis, dynamic testing against a live instance and\ndependency checks running automatically, alongside written security audits that\ninclude a threat model and an assessment against the OWASP testing guide. The\npage gives the reason for that pairing: scanners find the classes of problem\nthat someone has already described, and a threat model is where the problems\nspecific to one platform get named.\n\n## Where it goes wrong\n\n**Testing without written authorisation.** A verbal \"go ahead\" from a team lead\ndoes not cover attacks that company policy forbids, and it protects neither the\ntesters nor the people who asked for the test. Write the rules of engagement\nfirst, and have the test team leader and senior management sign them before\nany traffic is sent.\n\n**Testing addresses that are not yours.** A target list copied from an old\ndiagram can include addresses that now belong to someone else, and shared\nplatforms put other customers next to yours. Check that every public address in\nscope belongs to the organisation, as SP 800-115 asks, and put each third-party\nservice on the exclude list by name.\n\n**Selling a scan as a penetration test.** A scanner report with a new cover\npage lists possibilities, not proof. Ask for evidence of exploitation for every\nfinding: what the tester did, what access it gave, and what the next step would\nhave been.\n\n**Making it the only security testing.** A single test sees a sample of the\nrisks, at one moment, from outside. The WSTG places threat modelling in design,\ncode review and source analysis during development, and penetration testing in\nthe test stage that follows. Run all of them, and use the penetration test to\nconfirm what the earlier checks could not.\n\n**Starting with no rule for stopping.** If the testers take down a live service\nor find signs of a real intruder, and nobody has agreed who halts the test and\nwho restarts it, people improvise under pressure. Put halt criteria, a call\ntree and a resume process in the rules of engagement.\n\n**Treating the results as ordinary data.** A penetration test report is a map\nof what an adversary could exploit, and SP 800-115 says test results should be\ntreated as sensitive. Agree how findings are gathered, sent, kept and destroyed\nbefore the test starts, and limit who receives the full report.\n\n**Closing findings without a re-test.** A fix can be partial, or land in a\ndifferent place from the flaw. The WSTG calls testing that tries to exploit\nvulnerabilities found in earlier reviews focused penetration testing, and notes\nthat it shows whether specific flaws are actually fixed in the deployed code.\nRe-run the original exploit against every fix before the finding is closed.\n\n**Expecting a covert test to find everything.** According to SP 800-115,\ncovert testing examines the damage an adversary could cause; it does not test\nevery control or find each vulnerability, and it usually gets in through the\nmost basic weaknesses. Choose an overt test when the aim is coverage, and a\ncovert one when the aim is to learn whether the defenders notice.\n\n## Further reading\n\n- [NIST SP 800-115: Technical Guide to Information Security Testing and Assessment](https://csrc.nist.gov/pubs/sp/800/115/final)\n  — NIST's guide to planning and running security tests, with the four phases\n  of a penetration test and a template for the rules of engagement.\n- [OWASP Web Security Testing Guide v4.2: Introduction](https://wstg.owasp.org/v4.2/2-Introduction/)\n  — where penetration testing fits beside review and source analysis, and why\n  it should not be the only technique.\n- [OWASP WSTG v4.2: Penetration Testing Methodologies](https://wstg.owasp.org/v4.2/3-The_OWASP_Testing_Framework/1-Penetration_Testing_Methodologies/)\n  — a short overview of PTES, the PCI DSS guidance, PTF, NIST SP 800-115 and\n  OSSTMM.\n- [NIST SP 800-53 Rev. 5: Security and Privacy Controls for Information Systems and Organizations](https://csrc.nist.gov/pubs/sp/800/53/r5/upd1/final)\n  — NIST's catalogue of security and privacy controls, where control CA-8 covers\n  penetration testing and its rules of engagement.\n\n\u003chttps://engineer.company/glossary/penetration-test/\u003e\n","date_modified":"2026-11-09T21:50:25Z","description":"What a penetration test is, how its four phases run, what the rules of engagement must settle, and where it fits beside scanning and code review.","formats":{"gemtext":"https://engineer.company/glossary/penetration-test/index.gmi","gopher":"https://engineer.company/glossary/penetration-test/gophermap","html":"https://engineer.company/glossary/penetration-test/","json":"https://engineer.company/glossary/penetration-test/index.json","markdown":"https://engineer.company/glossary/penetration-test/index.md","pdf":"https://engineer.company/glossary/penetration-test/index.pdf"},"kind":"page","language":"en","title":"What is a penetration test?","url":"https://engineer.company/glossary/penetration-test/","version":1}