A percent reduction claim measures the gap between two numbers the vendor chose: a starting count and an ending count. It tells you nothing until you know what was counted, over what period, in whose codebase, and who did the counting. Ask for those four things before the number enters your business case.
Why the number is slippery
Percent changes hide their inputs. If a customer had 10,000 alerts and now has 2,000, that is an 80 percent drop. If the customer had 50 alerts and now has 10, it is also 80 percent, and a very different story. The claim works the same for both.
Three other things move the result without any product improvement:
- Baseline choice. Counting every raw scanner finding, including duplicates, makes the starting figure large.
- Time window. A quarter after a big cleanup looks better than a quarter during one.
- What counts as an alert. Some products stop showing items rather than resolving them. Hidden is not fixed.
The four questions
| Question | A useful answer sounds like | A weak answer sounds like |
|---|---|---|
| What was the baseline? | Unique findings per repository, from the customer's previous tool, on a stated date | Total alerts across the estate |
| What was counted at the end? | Findings still open and visible to developers | Findings after filters the product applied |
| How was it measured? | Exported counts from both states, available for review | A quote from a customer success call |
| Does my stack look like theirs? | Same languages, similar repo count and age | A different industry, scale or tech stack |
If the vendor cannot answer the second row, ask whether suppressed items are still reachable in the product with a reason attached. A good filter leaves a trail.
Turn the claim into a test
You can check a claim like this in your own trial without trusting the case study at all.
- Pick two or three repositories you know well.
- Record your current open findings, deduplicated, before the trial begins.
- Run the tool and record what it surfaces to developers, plus what it suppressed and why.
- Sample 20 suppressed items and 20 surfaced items. Ask an engineer to judge each one.
- Compare. A real reduction keeps the genuinely risky items on the surfaced side.
The sample matters more than the percentage. A tool that drops 90 percent of alerts by hiding a few real problems has made things worse, and the headline would still read well.
Try it yourself
Click through it at your own pace, or open it full screen.
Watch which findings leave the list and what reason is attached to each. That reason is the part a percentage cannot show you.
What to put in writing
If a reduction figure matters to your decision, ask the vendor to restate it as a trial success criterion, with your baseline and your repositories. Safeguard publishes a free scan at gold.safeguard.sh/scan and offers reachability analysis, so you can see how findings are narrowed on your own manifest and judge the reasoning yourself.
Quick check
Is a large percent reduction a bad sign?
No. It is a reason to ask what the baseline and the end state were. A large drop from a noisy baseline can be real, and the sample test above will tell you.
Can I ask a vendor for the customer's raw data?
You can ask. Most customers will not allow it, so a better request is that the vendor reproduce the same measurement on your repositories during the trial.
Last reviewed: 19 October 2026.
