Ask which databases the tool reads, how it matches a record to an exact package and version, how fast new advisories appear, and what it does when a record is incomplete. The answers explain most of the differences in accuracy you will see between tools in a pilot.
Why the data matters more than the interface
Two tools can read the same lockfile and produce different findings. The gap is almost always in the vulnerability data or the matching logic. A polished dashboard cannot fix a stale feed.
Questions about sources
- Which public databases do you ingest? Ecosystem advisories, a general national database, an open source aggregator?
- Do you add your own research, and can you tell me which findings come from it?
- Can I see the source of each finding?
The OSV project, for example, collects databases that use the OpenSSF Vulnerability format and presents them together, and its page says it uses version analysis to represent affected versions in each entry. A tool that can show you the originating record lets you check its claim.
Questions about matching
- Do you match on exact package name, ecosystem, and version range, or on looser identifiers?
- How do you handle packages with the same name in different ecosystems?
- What happens with vendored or renamed code?
Loose matching creates false positives. Strict matching can create misses when the data lacks the fields. A good vendor can explain the trade-off, not deny it.
Questions about freshness
- How long between a public advisory and the finding appearing for me?
- Do you rescan existing projects when new data arrives, or only when I push code?
- Can I be notified when something changes in a build I shipped last month?
The last one matters most. A scan on commit tells you about the day you built. New advisories arrive afterward, so you need rescanning against stored inventories.
Questions about incomplete records
Public records sometimes lack severity scores, affected version ranges, or fix information. Ask what the tool displays then. Hiding the record is risky. Showing it with a clear "not yet analyzed" label is better. See also a clean scan can mean the data is not mapped yet.
Questions about ranking
Ask whether the tool uses exploit signals. CISA describes its Known Exploited Vulnerabilities catalog as one input to a prioritization framework, and FIRST describes EPSS as a probability of exploitation in the next 30 days. See EPSS, KEV, and CVSS in SCA triage.
Safeguard's dependency scanning shows findings with their sources, and the free scan at https://gold.safeguard.sh/scan lets you check this on your own code.
How to run these questions in a pilot
Pick three recent advisories for packages you use. Note the date each was published. Check when each tool first reported it against your repository. The gap between publication and appearance is your freshness number, and it is easy to measure. Then take one advisory with an unusual affected range and see whether each tool flagged the right versions and left the fixed ones alone. Two short experiments like these reveal more about data quality than a feature matrix will.
Quick check
Is more databases always better?
Not automatically. More sources can mean duplicate records. Ask how the tool merges them into one finding.
Sources
Last reviewed: 19 October 2026.
