Retire a security metric when no decision depends on it, when it has been gamed or has saturated, or when its definition has drifted. Keep a note of why, replace it with something closer to the decision, and run the old and new together for a short overlap so trends are not lost.
Signs a metric has run its course
- Nobody acts on it. Ask who changed something last quarter because of it. If no one can say, it is decoration.
- It is saturated. It sits at the ceiling or floor and cannot show improvement or decline.
- It has been targeted into meaninglessness. Behaviour has bent to the number. Goodhart's law, as commonly quoted, says a measure that becomes a target ceases to be a good measure.
- Its definition drifted. The tool, scope or rules changed so the series is no longer comparable.
- The risk changed. The metric tracks something that mattered two years ago.
- It costs more than it informs. Collecting it takes manual effort with little payoff.
Metric lifecycle
Treat metrics as having a lifecycle, not a permanent seat on the dashboard.
| Stage | Question to ask |
|---|---|
| Introduce | Which decision will this inform, and who owns it? |
| Operate | Is the definition stable and documented? |
| Review | Does anyone act on it? Has it been gamed? |
| Retire or replace | What closer measure takes over? |
NIST SP 800-55 Volume 1, published in December 2024, covers identifying and selecting information security measures and explains a prioritisation and evaluation process for them. It supersedes the 2008 revision, which is a reminder that measurement guidance itself ages.
Steps to retire a metric
- Name the decision it was meant to support and check whether that decision still exists.
- Check for gaming: has the number improved without the underlying outcome improving?
- Pick a replacement that is closer to the outcome and harder to bend.
- Run both for one or two reporting cycles and explain the difference in the report.
- Archive the old series with its definition and the reason for retiring it.
- Tell the audience in the report itself, not just in a changelog.
Replacing, not just removing
Dropping a flattering metric quietly looks like hiding a problem. Dropping an unflattering one looks the same. Always state the reason, and keep the data available. If a metric is retired because it was gamed, say so, since that finding is useful for the next metric design.
A quarterly review rhythm
Once a quarter, list each dashboard metric with its owner, its last decision and its gaming risk. Remove or rework the bottom few. A dashboard that grows without pruning becomes unreadable, and the important number gets lost.
Do not retire for the wrong reason
Do not retire a metric because it is bad news. Retire it because it no longer informs. If the number is bad and still meaningful, fix the problem.
Safeguard offers SBOM and dependency scanning, reachability analysis and policy gates. You can run a free scan at gold.safeguard.sh/scan.
Quick check
How long should old and new metrics overlap?
One or two reporting cycles is common, enough to explain the difference.
Should we delete the old data?
No. Archive it with its definition and the reason for retirement.
Sources
Last reviewed: 19 October 2026.
