Compare scanners with all suppressions turned off first, so every tool sees the same raw state. Then run a second pass that tests how well each one imports your past decisions. Otherwise your current scanner looks quiet only because years of ignore rules are silencing it.
The hidden handicap
Most teams that already use a scanner have accumulated exceptions: ignored advisories, accepted risks, path exclusions, and fixes deferred with a note. Those decisions live in the current tool's configuration. A new candidate starts with none of them and reports everything.
The comparison then looks unfair to the newcomer. It shows hundreds of findings where the incumbent shows dozens. Part of that is noise, and part of it is just the absence of memory. You need to separate the two.
Pass one: raw versus raw
Disable suppressions in the incumbent for the test, or export its raw output and ignore its exception list. Run all candidates against the frozen repositories. Compare raw findings, with the same settings, using the methods in the parity checklist.
This tells you what each tool sees, with no history applied. Note how many findings the incumbent's suppressions were hiding. That number is worth knowing on its own, since many suppressions go stale. See why suppression files rot.
Pass two: carry over decisions
Now test migration. For each candidate, ask:
- Can it import an exception list, or do you have to re-enter each one?
- What fields travel with a decision: reason, author, expiry date, scope?
- Does an exception apply to the advisory only, to one package, to one path, or to everything?
- Can exceptions expire so they get reviewed?
- Does the tool keep an audit trail of who accepted what?
Record the effort. If importing 120 exceptions takes a day of manual copying, that is a switching cost to write into the decision.
| Question | Tool A | Tool B |
|---|---|---|
| Bulk import | Yes or no | Yes or no |
| Reason and owner kept | Yes or no | Yes or no |
| Expiry supported | Yes or no | Yes or no |
| Scope control | Granular or broad | Granular or broad |
| Audit trail | Yes or no | Yes or no |
Review the old exceptions first
Before importing anything, review the list. Sort exceptions by age and check each old one: is the package still present, is the reason still true, did a fix appear? Many exceptions can simply be deleted. A migration is a good moment to clean up, and a comparison that forces the review is delivering value on its own.
Treat exceptions without a recorded reason with caution. Do not import them. Re-triage instead. The question of where exceptions should live is covered in scanner, ticket or GRC.
What fair reporting looks like
State both numbers in the readout: raw findings per tool, and findings after carrying over reviewed exceptions. If the candidate shows far more than the incumbent in pass one but they converge in pass two, the gap was history, not quality. If it still shows many more after import, adjudicate to see whether it finds real issues the incumbent missed, using the process in adjudicating one sided findings.
Pitfall: suppressions as evidence of tool quality
A tool with many suppressions is not necessarily a noisy tool. It may simply be older. Avoid blaming the scanner for the debt, or crediting it for silence. Silence earned through ignoring is not security.
Safeguard's policies and gates let you record decisions and enforce them, and the free scan at gold.safeguard.sh/scan shows a raw first view to use as pass one.
Quick check
Should I import all old exceptions?
No. Review them first, import the ones that are still true, and drop the rest.
Do exceptions need expiry dates?
They should. An expiry forces a re-check, which keeps the list honest.
Last reviewed: 19 October 2026.
