A vanity security metric looks good on a slide but does not change any decision. Test a metric with seven questions: who acts on it, what changes if it moves, can it be gamed, does it have a denominator, is the trend meaningful, is it comparable over time, and is it tied to risk.
The seven questions
- Who acts on it? If no named role would do something differently when it moves, it is decoration.
- What decision does it change? Examples: where to invest, which team to help, whether to ship.
- Can it be gamed cheaply? If closing tickets or lowering scanner sensitivity moves it, it will be gamed.
- Does it have a denominator? "Found 4,000 vulnerabilities" means little. "4,000 across 300 services, 12 of them reachable and exploited" starts to.
- Is the trend meaningful? If the number rises when you improve detection, a rise may be good news. Say so on the chart.
- Is it comparable over time? Changing scanner, rules or scope breaks the series. Annotate the date.
- Is it tied to risk or outcome? Activity measures (scans run, training hours) are not outcomes.
Usual suspects
Total vulnerabilities found. Rewards noisy tools and punishes thorough ones. Replace with open items by exploit evidence and age.
Number of scans run. Measures tool activity. Replace with coverage: share of assets scanned successfully in the last 7 days.
Attacks blocked. Depends on internet noise more than your posture. Replace with incidents and near misses reviewed.
Percent of staff trained. Completion is not behaviour. Pair with reported suspicious emails or secure practice checks. See phishing simulation metrics that mislead for a close cousin of this problem.
Dashboard views. Nobody should celebrate this one.
Replace, do not just delete
Executives still want a number. Offer a better one in the same slot. Instead of "scans run", show "services with a successful scan this week". Instead of "vulnerabilities found", show "reachable, exploited vulnerabilities open past SLA".
Keep the metric card short
For every metric you keep, write four lines: definition, data source, owner, and the action it triggers. If you cannot fill in the last line, remove the metric. See a metric definition template for the full version.
When a vanity number is fine
Some counts are useful for context, such as assets in inventory. They are not wrong, they are just not goals. Show them as a footnote.
Apply the checklist to one example
Take the figure "4,000 vulnerabilities found this quarter". Who acts on it? Nobody in particular. What decision does it change? None. Can it be gamed? Yes, by turning scanner rules down. Does it have a denominator? No. So it fails four of seven questions.
Now take "3 exploited vulnerabilities open on internet facing services, oldest 9 days, owner named". It has an owner, triggers an action, cannot be improved without fixing something, and has a clear scope. Run your own metrics through the same test during the next review and drop the ones that fail more than three questions.
Quick check
How many metrics should a security dashboard show?
Few enough that someone reads each one. Five to eight, each with an owner, is a reasonable start.
Is a big vulnerability count always bad news?
No. It may mean you recently improved coverage. Always read it next to coverage and exploit evidence.
Last reviewed: 19 October 2026.
