Measuring Success: KPIs for Red Team Programs

Level: Advanced

The Problem with "Number of Findings"

How do you measure if a Red Team is doing a good job? The lazy metric is "Number of Critical Vulnerabilities Found."

This is a terrible metric. * If the Blue Team is doing a great job, the Red Team finds less. Does that mean the Red Team failed? * If the Red Team finds 50 bugs, but none of them lead to a breach, did they add value? * It incentivizes "Vuln Farming" (finding shallow bugs) instead of complex attack paths.

We need metrics that measure Security Improvement, not just activity.

1. Mean Time To Detect (MTTD) & Respond (MTTR)

These are arguably the most important metrics for the SOC, and the Red Team is the yardstick to measure them.

  • MTTD: Time from the first malicious action (e.g., Phishing email lands) to the first alert generated in the SIEM.
  • MTTR: Time from the alert to the containment (e.g., Host isolated).

Goal: Reduce these numbers over time. * Q1 Exercise: MTTD = 4 Days. * Q2 Exercise: MTTD = 4 Hours. * Result: The program is working.

2. Attack Path Coverage

Use the MITRE ATT&CK Framework. There are ~200 techniques. How many have you tested? * "We have tested 15% of the ATT&CK matrix this year." * "We have validated detection for 80% of Ransomware-related techniques."

This shows Breadth of coverage.

3. The "Silent Check" Pass Rate

Sometimes, we run tests just to see if the alarms work. * Run 10 known malicious behaviors (that should trigger alerts). * How many triggered? * Metric: Detection Efficacy Rate (e.g., 7/10 detected).

4. Remediation Effectiveness

When the Red Team finds a hole, and Blue says "Fixed," verify it. Metric: Recidivism Rate (How often do fixed bugs reappear?). If you find the same "weak password" issue every quarter, the remediation (Policy? Control?) is failing.

5. Cost of Attack (The Economic Metric)

Make it expensive for the attacker. * Low Maturity: Attacker needs $5 phishing kit and 1 hour. * High Maturity: Attacker needs $50,000 zero-day and custom C2 dev capabilities.

If you force the attacker to burn expensive tools to get in, you have reduced the pool of adversaries who can afford to hack you.

Reporting to the Board

Don't show the Board a list of CVEs. Show them the Trend Line.

"Last year, a ransomware simulation took 2 hours to compromise the network. This year, it took 5 days and was detected 3 times. We are becoming resilient."

That is a slide that gets budget approved.