To find out what a scanner misses, plant known vulnerable components in a test project and see which ones it reports. Most trials count false positives only. False negatives are harder to see and more costly, so give them equal time in your evaluation.
Why false negatives get skipped
A false positive wastes an engineer's afternoon, so everyone notices. A false negative produces no output at all. You only learn about it after an incident or an audit. That asymmetry is why a seeded test is worth an afternoon.
Build the seed project
Create a throwaway repository, never a production one. Add a dozen dependencies with publicly documented advisories, chosen to vary: a direct dependency, a transitive one two levels deep, one pinned in a lockfile only, one declared in a different manifest format, one in a container base image, and one vendored as source. Record the advisory identifier for each beforehand.
Vary the shape, not just the count
Coverage gaps usually hide in shape: monorepos with nested manifests, private registries, unusual lockfiles, languages with weak advisory data. Include the shapes your own estate uses.
Run, then score
Run each candidate against the same seed project. Score recall as found divided by planted. Then inspect the misses. Did the tool skip a manifest type, fail to resolve a transitive version, or lack the advisory? Each cause points to a different risk.
Add a freshness check
Choose one advisory published in the last few days and see how quickly each tool reports it against the seed. Report latency matters during active exploitation.
Keep expectations realistic
No tool reaches perfect recall, and advisory databases differ. The goal is to learn where each tool is weak so you can compensate, not to find a flawless one.
Safety note
Only plant packages in an isolated test repository, and do not install or run known malicious packages. Vulnerable but non malicious versions are enough.
Keep the seed project afterward. Rerun it at each renewal, and whenever you add a language or a package manager to your estate. It becomes a cheap regression test for your own tooling and shows if coverage slips over time.
Where Safeguard fits
Safeguard covers SBOM and dependency scanning, reachability analysis, policies and gates, and fix pull requests that stay off until you turn them on. You can try a free scan on a repository before you commit to anything.
Try it yourself
Open the inventory and check whether direct and transitive packages are both listed, the same way you would check your seed project.
Click through it at your own pace, or open it full screen.
Quick check
How many seeded items do I need?
A dozen varied ones tell you more than a hundred similar ones.
Should I share the seed list with vendors?
No. Share the project only if you want the tool to scan it, and keep the answer key private.
Last reviewed: 19 October 2026.
