Safeguard
Security Guides•

How to judge a scanner's false positives before you buy it

A simple sampling method to measure how noisy a vulnerability scanner is on your code, plus the signals that cut noise and how to read vendor claims.

Safeguard Security Team
3 min read

Measure noise yourself: sample 20 findings from your own repositories, classify each as correct and actionable, correct but not relevant, or wrong, and compare tools on that split. A vendor claim of low false positives means little without a method you can repeat.

Why the term is slippery

Teams use false positive for three different things. A wrong match, such as the wrong package or version. A correct match that does not affect you. And a finding that is real but not worth fixing now. Only the first is a bug in the scanner. The other two are prioritisation gaps. Separate them before you compare.

The sampling method

  1. Export all findings from a scan of one real repository.
  2. Randomly pick 20. Do not choose the obvious ones.
  3. For each, check the package in the lockfile, the affected version range in the advisory, and whether your code uses the affected feature.
  4. Put each in a bucket and count.

Twenty is small, so treat results as directional. If two tools differ by a lot, the signal is real. If they differ by a little, it is noise.

What reduces noise

  • Accurate package matching by exact ecosystem and version, not by name alone
  • Reachability context for the code path
  • Exploit signals, such as known exploitation or a public proof of concept
  • Suppression with reasons, owner and expiry
  • Deduplication across repositories sharing a package

Red flags in sales material

A single percentage with no method. A comparison on a sample app. Claims of zero false positives. Any of these should lead you back to your own sampling.

The cost of getting it wrong

High noise teaches developers to ignore the tool. Low noise from a scanner that misses real issues is worse. Always test the seeded known vulnerability case alongside the noise test.

Suppression is part of the answer

Even a good scanner will be wrong sometimes, so judge how cheaply you can say so. Can a developer mark a finding as not applicable, with a reason, in a few clicks? Does the decision carry across branches and repositories? Does it expire? A tool with mediocre matching and good suppression can be easier to live with than a tool with great matching and a clumsy way to record disagreement.

Try it yourself

Click through it at your own pace, or open it full screen.

Look at how exploit context changes the order of findings.

Where Safeguard fits

Safeguard combines reachability analysis with dependency scanning. Try a free scan on a repository and run the sampling method on the result.

Quick check

Is a lower finding count always better?

No. A scanner that finds fewer issues might be missing real ones. Check both noise and coverage.

Last reviewed: 19 October 2026.

Never miss an update

Weekly insights on software supply chain security, delivered to your inbox.

Self-healing security runs on Safeguard.

Your first fix PR is minutes away.

No sales call required, even your agent can complete the purchase over MCP.