A scanner is only as good as the advisory data behind it. To test coverage and freshness, pick a small set of known advisories, time how long each takes to appear, and compare results across tools on the same inputs. A one week experiment tells you more than any claim about database size.
Why data quality is hard to see
Two scanners can both say they check "all known vulnerabilities" and still disagree. They may draw on different public sources, add their own research, normalise package names differently, or map versions with different rules. The count of entries is a weak signal. What counts is whether the right advisory matches the right version of your package, quickly.
Test one: coverage
Choose ten to twenty packages from your real lockfiles, including a few obscure ones and a few from less common ecosystems. For each, look up advisories in public sources such as the national vulnerability database, the repository advisory databases for your ecosystem, and the package maintainers' own notices. Then check which advisories each scanner reports. Record:
- Advisories found by the scanner and by public sources.
- Advisories found only by public sources, which are misses.
- Findings the scanner raises that you cannot verify, which may be false matches.
Test two: freshness
When a new advisory is published for a package you use, note the time of publication, then note when each tool first reports it. Do this across three to five new advisories during the pilot. If you cannot wait for real ones, review recent entries and check whether the tool already knows them.
Freshness matters most for exploited issues. Check how quickly a tool picks up entries flagged as actively exploited by public catalogues.
Test three: quality of the record
For a handful of findings, check:
- Does it give the fixed version?
- Is the affected range correct, or does it flag versions that are fine?
- Does it carry a severity score and an exploit signal, or only one?
- Is there a link back to the original source?
Test four: the long tail
Some malicious package incidents and early disclosures appear before formal advisory entries. Ask the vendor how they handle them and how they label unconfirmed items.
Read the results sensibly
Perfect agreement is unlikely. Differences in severity scores are normal. Focus on misses and delays for packages you really use.
Try it yourself
Open the zero day watch demo and see how an early signal is handled before a formal advisory exists. Check whether the tool tells you how confident it is.
Click through it at your own pace, or open it full screen.
Where Safeguard fits: it includes zero-day discovery alongside SBOM and dependency scanning, and you can run the free scan at https://gold.safeguard.sh/scan and judge its data against your own checks.
Quick check
Is a bigger database always better?
No. Accuracy of matching and speed of updates matter more than raw count.
Should I trust one source of advisories?
Cross-check at least two. Each public source has gaps.
Last reviewed: 19 October 2026.
