Choose a software supply chain security tool by testing five things on your own code: how complete its inventory is, how actionable its findings are, whether it checks packages before install, what evidence it gives auditors, and where your source code goes. Rank nothing by feature count. Rank by what survives a trial on your repositories.
What problem are you buying for?
"Supply chain security" covers at least four jobs, and tools differ in which they do well. OWASP's 2025 Top 10 defines the risk broadly: breakdowns or compromises in building, distributing or updating software, often through third-party code, tools and dependencies. That includes the dependencies you declare, the ones that arrive transitively, the packages you install, and the build tools that assemble them.
Write down which job hurts most today. Is it vulnerability noise? A customer asking for SBOMs? A fear of malicious packages? An audit? The answer changes your weighting.
The five criteria
1. Inventory completeness. Does it resolve lockfiles, transitive dependencies and container layers, and does it produce standard SBOM formats such as CycloneDX and SPDX? Test on a repository where you know the dependency count. A missing 15 percent is a finding.
2. Actionable findings. Does each finding show the dependency path, a fix target, and whether your code actually calls the vulnerable function? Reachability analysis varies a lot by language and by how it treats uncertain cases, so ask what happens with reflection and dynamic dispatch. A tool that hides uncertain cases is guessing. Also ask whether it uses exploitation data such as CISA's Known Exploited Vulnerabilities catalog, which CISA says should be an input to prioritization.
3. Before-install checks. A vulnerability scanner cannot flag a malicious package that has no CVE. Ask whether the tool checks typosquats, dependency confusion and known-malicious packages at install time or in the IDE, which registries it covers, and whether it can run in audit mode first.
4. Evidence for auditors. Can it export a dated record of what was scanned and what was done? Ask which compliance requirements it decides automatically and which still need your evidence. Be wary of any tool that implies it certifies you. Certification comes from an auditor.
5. Where code goes. Does scanning happen in a vendor cloud, on your machine, or in your environment? Some tools run locally and upload only results. Some work with no AI model at all, while others require one. If you are regulated, ask about model options, including bringing your own.
What are the honest trade-offs?
Wider coverage usually means more integration work. Deeper reachability usually means more language-specific limits. Local scanning protects source but needs an agent on your machines. No tool does everything, and a vendor who claims otherwise is selling.
Ask about maturity labels too. Many products mix generally available features with early access ones. A good vendor will tell you which is which in writing.
How should you run the trial?
Pick two or three repositories of different shapes: one old and sprawling, one new, one in a language you worry about. Give every candidate the same ones. Before looking at results, write your own list of ten known issues and three planted bad packages. Then score each tool on misses, noise, explanation quality and setup time. Fix the pass mark before the first result arrives.
What should you ask in the first call?
- Which ecosystems and languages are generally available, and which are preview?
- What does "unknown" look like in your results?
- What runs locally, and what is uploaded?
- Which parts of a compliance framework are decided by scan and which need my evidence?
- What can I turn off per team, and who can switch features off tenant-wide?
- How do I export everything if I leave?
What to do
- Write the job you are hiring the tool for, in one sentence.
- Pick three repositories and build an answer key before any trial.
- Score the five criteria on the same repositories for each candidate.
- Get maturity labels and data-handling answers in writing.
- Decide the pass mark first, and stick to it.
Quick check
Should you pick the tool with the most features?
No. Feature lists hide gaps. Test inventory completeness, finding quality and install-time checks on your own repositories.
Does a scanner catch malicious packages?
Only if it has malicious package detection. A vulnerability scanner keyed to CVEs will miss a malicious package that has no CVE.
Sources
- OWASP Top 10 2025, A03 Software Supply Chain Failures
- CISA, Known Exploited Vulnerabilities catalog
- NIST SP 800-218, Secure Software Development Framework
- Safeguard docs, Reachability analysis
Last reviewed: 18 October 2026.
