A fair comparison of dependency scanners starts with a weighted scorecard written before any demo. List eight to ten criteria, assign weights that add up to one hundred, score every tool on the same repositories, and record evidence for each score. The weights are the real decision, so set them first.
Choose criteria that you can test
Good criteria are things you can observe in a week. A starting set:
- Ecosystem coverage for the languages and package managers you use.
- Accuracy on a sample: missed packages and false matches.
- Transitive dependency depth and lockfile handling.
- Ability to prioritise by exploitability or reachability.
- Fix guidance and automated update options.
- CI and IDE integration.
- SBOM export in open formats.
- Policy and exception controls.
- Reporting and audit history.
- Data handling and deployment options.
Drop criteria nobody will test, such as vague claims about "AI". A criterion you cannot verify only adds a place for bias.
Set weights first
Hold a short session with security, engineering and compliance and agree weights before looking at products. A team that relies on a polyglot monorepo may weight ecosystem coverage heavily. A team under audit may weight evidence. Write the weights down and do not change them after seeing results.
Test on your own code
Pick three repositories: a typical service, your oldest legacy project, and the one with the most dependencies. Build a ground truth for one by hand using the lockfile. Then compare each tool's inventory against it. Score recall, which is how many real packages the tool found, and precision, which is how many of its findings are correct.
Keep scoring honest
- Score independently, then compare.
- Require a note of evidence for any score above or below the middle.
- Treat ties as ties. A one point gap is noise.
- Have someone who did not run the test review the totals.
What a scorecard cannot tell you
It will not capture how the team feels after a month of use. Add a short developer survey at the end of the pilot and treat it as a veto, not a number to average away.
A caution about "free" as a column
If an open source option scores close to a paid one, that is a result, not a failure. Weigh the cost of your own staff time to run and maintain it.
Try it yourself
Open the dependency inventory demo and check the package list against what you would expect from your own lockfile. Notice how transitive packages are shown, and whether the depth is visible.
Click through it at your own pace, or open it full screen.
Where Safeguard fits: it offers SBOM and dependency scanning with reachability analysis, and the free scan at https://gold.safeguard.sh/scan lets you add it to a scorecard test using your own repository.
Quick check
How many tools should be on the scorecard?
Three or four is manageable. More than that and the test corpus work grows faster than the insight.
Can we change weights mid-way?
Avoid it. If you must, rescore every tool and note why, so the change does not favour one result.
Last reviewed: 19 October 2026.
