The most critical stage when using offensive security to improve defenses comes
after obtaining leadership approval for a testing exercise. Current industry definitions have significant overlap, with the same term used to describe different underlying services, and with the added complication of AI-based offensive tools. Overshadowed by years’ worth of penetration tests exploiting the same set of techniques, or red teamers waltzing through the front door, driving lasting and impactful security improvements based on testing results continues to become less straightforward.
As an industry, we have accepted that using offensive testing is a good way to find gaps in our defenses. However, less attention is given to
whether the type of testing chosen actually helps to systematically fix the gaps identified. This leads to problems like:
- Penetration tests continuing to surface the same class of findings as previous years, or the same finding in a different location.
- Organizations paying for advanced red team exercises while not having implemented foundational security controls.
The types of problems mentioned above arise because defenders often select offensive testing solutions based on the service "name". This leads to a
mismatch between the type of offensive testing conducted and the defensive technologies that need to be validated. In this session, I will first provide a framework for defenders to categorize types of offensive security testing based on
what their security controls will be tested against (attacks vs. adversaries) and
how they will be tested (emulation vs. simulation). This framework helps defenders to:
- Understand what the core value proposition of each offensive security service is, independent of what terminology is used to describe it.
- Work bottom-up from the defenses you have to identify the most appropriate testing methodology.
Next, I will demonstrate how to use this model within attendees’ organizations to plan out an offensive testing program based on their threat model, security goals, and maturity.
The goal of this session is to encourage attendees to think about offensive security from a new standpoint. By introducing a framework to categorize offensive testing methodologies with a primary focus on the security controls being validated, defenders will understand how to distinguish between the various offensive security services on the market, select the most appropriate solution for their organization, and progress between offerings as their security program matures.
