Accuracy misleads because most vulnerabilities are never exploited in a given 30 day window. FIRST says a rule that predicts zero exploitation for everything scores around 97 to 98 percent accuracy and is useless. Judge a prioritization rule with coverage, effort and efficiency instead.
The trap
Imagine two rules. Rule A flags nothing. Rule B flags a set of CVEs it thinks are dangerous. If only 2 to 3 percent of CVEs see exploitation in a period, Rule A is right about the other 97 percent, and its accuracy looks excellent. Yet it catches nothing. FIRST makes exactly this point: the metric rewards the prediction that needs no model.
This applies to any rule, including "patch all Critical", "patch all KEV" or "patch above a score". Each one makes a claim about the future, and each should be measured.
The three measures FIRST uses
| Measure | Meaning | Statistics name |
|---|---|---|
| Coverage | Fraction of exploited vulnerabilities your rule captured | Recall |
| Effort | Fraction of all vulnerabilities you must act on | Workload |
| Efficiency | Fraction of what you acted on that was actually exploited | Precision |
FIRST's examples: if a rule flags 500 CVEs and those include 80 percent of the CVEs exploited in the next 30 days, coverage is 80 percent. Acting on 500 of 200,000 is an effort of 0.25 percent. If 100 of the 500 were exploited, efficiency is 20 percent.
How to test your own rule
You can run a version of this on your own history.
- Pick a past date, such as 90 days ago, and list the findings your rule would have flagged that day.
- Decide what counts as exploited later, such as a KEV listing within the next 30 days, using a source you can verify.
- Compute coverage: of the later exploited items, how many were on the list.
- Compute effort: how many findings did you flag out of all open ones.
- Compute efficiency: of the flagged, how many turned out exploited.
- Run the same test for one or two alternative rules, and compare.
Your sample will be small, so treat the result as a guide and not a verdict. FIRST itself warns against judging a model by a single CVE.
What the trade off looks like
Lowering the line catches more of what is exploited but costs more effort and lowers efficiency. Raising it does the opposite. FIRST says a threshold is a local choice depending on your capacity, and recommends picking a point you can sustain. See choosing an EPSS threshold.
A random pick is the baseline. If your rule is no better than flagging a random share of CVEs, it is not helping, whatever it looks like.
Report it honestly
When you present to leadership, show coverage and effort side by side, not a single percent correct. Example wording: "This rule would have flagged 4 percent of open findings and captured most of the ones later listed as exploited, in our own history." Say how small the sample is, and which source defined exploited.
Beyond scores
Reachability and presence checks are rules too. Test them the same way, by asking later whether an item they dismissed turned out exploited. A dismissed item that later appears on KEV is a sign to inspect the check.
Safeguard's reachability analysis and policy gates give you rules you can test against your own history. You can try a free scan at gold.safeguard.sh/scan.
Try it yourself
Click through it at your own pace, or open it full screen.
Look at how findings are grouped, and consider which grouping rule you could test with coverage and effort.
Quick check
What is coverage?
Per FIRST, the fraction of exploited vulnerabilities that a given threshold captures. In statistics it is recall.
What is efficiency?
The fraction of the vulnerabilities you acted on that were actually exploited. In statistics it is precision.
Sources
Last reviewed: 19 October 2026.
