When a security metric jumps, check what changed in the pipeline before you decide risk changed. Keep a dated change log of scanner versions, rules, thresholds and pipeline edits, and mark those dates on every chart. Most sudden shifts have a boring explanation.
The usual suspects
- A scanner upgrade that adds rules or changes how severity is assigned.
- A new data feed or changed feed refresh timing.
- A threshold change in the gate, such as raising or lowering what blocks.
- A new repository or branch brought into scope, which adds its backlog at once.
- A change in how results are counted, such as dropping duplicates.
- A runner or cache change that affects timing.
- A holiday week or release freeze that changes how much code moves.
None of these mean the code got riskier or safer overnight, yet each can move a chart sharply.
Keep a change log
Use a simple table in a shared file or wiki, with one row per change.
| Date | Change | Owner | Expected effect |
|---|---|---|---|
| Dated | Scanner upgraded to a named version | Platform | New findings may appear |
| Dated | Gate threshold changed | Security | More or fewer blocks |
| Dated | Repositories added to scope | Security | Coverage up, backlog up |
The expected effect column is the useful one. Writing the prediction down before the change lets you compare it with what happened and learn.
Show annotations on charts
Put a vertical marker on each chart at the change date, labelled with a short note. Many dashboards support annotations from a table or API. If yours does not, a link to the change log under each chart is a fine second best.
How to compare across a change
- Pick equal windows before and after the change, such as four weeks each, avoiding holidays.
- Compare the same repositories in both windows.
- Where possible, re-run the old scanner version on recent code to see its output next to the new one. That separates tool effect from code effect.
- State the conclusion with its uncertainty. Small samples and short windows can mislead.
Rules for the team
- No scanner or gate change without a log entry.
- Announce threshold changes to developers before they take effect.
- Pin versions so changes happen on purpose, not as a side effect.
- Review the log when someone asks "why did this change?" and add anything missing.
Why this builds trust
Executives and developers lose faith in metrics that jump for unexplained reasons. A change log turns a surprise into a one line answer, and it shows that the program is run with care.
Quick check
Should I restate past data after a rule change?
If you can re-run the old data under the new rules, yes, and say so. If not, keep the old data and mark the break clearly.
Who owns the change log?
Whoever owns the pipeline template, with security adding entries for rules and thresholds.
How Safeguard fits
Safeguard provides policies and gates and continuous monitoring, so changes to what blocks can be set deliberately and reviewed. Try a free scan at gold.safeguard.sh/scan.
Last reviewed: 19 October 2026.
