Keep a dated change log for anything that can move a vulnerability metric without changing real risk, such as a scanner upgrade, new scope or a new SLA rule, and annotate charts at those dates. Otherwise readers will mistake definition changes for good or bad performance.
What changes numbers without changing risk
| Change | Typical effect |
|---|---|
| Adding repositories or assets to scanning | Open count rises |
| Upgrading or replacing a scanner | Counts move in either direction |
| Changing severity or priority rules | Items shift between tiers |
| Changing the SLA clock rule | Time to fix shifts |
| Adding a reachability filter | Headline count falls, the unfiltered count does not |
| Deduplicating findings | Count falls |
| Importing a backlog from another system | Aging jumps |
Any of these can produce a step change that looks like a result.
The change log
A change log is a short table kept next to the dashboard. Each row has a date, what changed, which metrics it touches, and its expected direction.
| Date | Change | Metrics affected | Expected effect |
|---|---|---|---|
| date | description | list | up, down, shift |
Treat it like a code change record. Anyone adjusting scanning or rules adds a row before the change goes live.
Steps to put it in practice
- Create the log in a shared place the security and engineering leads can edit.
- Require an entry for scanner, scope, rule and definition changes.
- Mark the chart with a vertical line and a short label at each date.
- For major changes, show both old and new definitions for one overlap period.
- Mention relevant entries in the monthly update, so executives hear the explanation first.
Show like for like when you can
If scope grew, show the trend for the original scope alongside the total. If a reachability filter was added, show filtered and unfiltered series. This avoids the suspicion that the change is a way to improve the numbers. See vulnerability management metrics before and after a tool and why two scanners report different counts.
Pitfalls
- Making a change right before a board meeting with no note
- Rewriting history after a definition change, with no record of the old values
- Letting the log fall out of date
What to tell executives
"The open count rose this month because we added [scope]. On the original scope, the count fell." Always name the cause and give the like for like direction.
Safeguard's policies and compliance evidence automation can help keep dated records of rule changes. You can run a free scan to record a starting baseline before you change anything.
Quick check
Do small changes need an entry?
If a change could move any reported metric, add one. It takes a minute and saves a long meeting.
Should the log be public to customers?
Not usually. Keep it internal, but be ready to explain definitions if you share metrics externally.
Last reviewed: 19 October 2026.
