Case Study: How 16 Penetration Tests Missed a Vulnerability
This Could Have Cost One Company Over $103 Million in PCI Fines
The interesting fact about this study is that the company had been getting “penetration testing” quarterly every quarter since 2012 by various notable companies. We uncovered the information in the 4th quarter of 2016.
That is a total of 16 penetration tests by 7 different vendors that missed the vulnerability.
How Did 16 “Pen Tests” Miss This Vulnerability?
Because of the way they are being tested. Each penetration test prior to ours had relied heavily on automated tools to identify vulnerabilities. The pen testing teams would run automated scans and then perform manual tests of the results. The problem with that is automated tools only look for publicly known vulnerabilities in systems – leaving vulnerabilities in custom applications or undiscovered “zero day” vulnerabilities unidentified.
Most cyber risks are hidden.
Similar to an iceberg, most vulnerabilities are hidden from automated and compliance-driven vulnerability scanning and penetration testing. Taking an enhanced red teaming approach to advanced penetration testing finds risks “below the surface” by manually emulating the aggressive actions of a hacker. The Withum Cyber approach involves human cyber operations experience, tools, tactics, and procedures at each stage of the test. It has been determined, by comparing test results for organizations that have employed multiple testing methodologies, that applying deep hands-on technical experience towards finding organization-specific vulnerabilities is a truly comprehensive way of identifying and analyzing a network’s level of security.
What is Enhanced Teaming?
An enhanced blue team approach to advanced penetration testing emulates the activities that advanced persistent threat actors (such as nation-state threats or organized crime) would carry out against your organization. Beyond a scan for vulnerabilities, this advanced level of testing takes advantage of the training, experience, and adaptability of our penetration testing specialists in finding, exploiting, and leveraging vulnerabilities to gain access and determine the impact of that access on the organization.
| Vulnerability Assessment | Traditional Penetration | Enhanced Blue Teaming/ Advanced Penetration Testing | |
| Scoping | Limited | Limited to scan results | Comprehensive |
| Skill Level Required | Tutorial Needed | Training Required | Advanced Degree |
| Objective | Broad scanning for information gathering | Utilize broad scanning to manually test a network for compliance driven needs | Uncover as many vulnerabilities as possible using the resources leveraged by real attackers |
| Techniques | Fully automated using software which identifies publicly known vulnerabilities | Driven by automation with penetration testers manually testing the findings uncovered by automated scanning | Human driven with a team of hackers focused on your network identifying vulnerabilities unique to your network |
| Threat Emulsion | None | Partial | Advanced Persistent Threat Emulation |
| Reporting | Computer generated report with unverified information and no determination of business impact | Computer generated report which is verified by penetration tester reducing the amount of false positives | Narrative report with actionable remediation steps and verified intelligence determining the business impact of all findings |
It is important to understand the difference in the complexity and depth of testing levels, and why Withum Cyber uses an enhanced red team approach to penetration testing.
Key Learnings
- There is a vase difference in definitions of “penetration testing.” Make sure you understand the difference in the level of testing you are receiving.
- As cybercrime continues to grow and being an increasing threat, you must start to conduct more comprehensive testing in order to truly remain secure and build your cyber resilience.
- Becoming a “want to know” organization and proactively looking for threats and vulnerabilities is imperative.
- An enhanced blue teaming approach to penetration testing is the only way to uncover organization specific vulnerabilities.
If you have any additional questions or would like more information around penetration testing or Withum’s Cyber & Information Services team, please fill out the form below.
How Can We Help?
