How to test whether a scanner's advisory data is fresh

Test advisory freshness by choosing five recent public advisories in ecosystems you use, noting when each was published, and checking when your scanner first reported it against a project that uses the package. The gap in hours or days, not the vendor's database size, is the number to compare.

Why this matters more than database size

Vendors often quote how many vulnerabilities they track. That says little about how quickly a new advisory reaches your scan. When a serious flaw is announced, the first days decide whether you patch before exploitation starts. A large database that updates weekly can lose to a smaller one that updates hourly.

The test, step by step

1. Pick the advisories. Use public sources such as ecosystem advisory feeds and the national vulnerability database. Choose five from the last month, spread across the ecosystems you use. Include at least one for a smaller package, not only the famous ones.
2. Record the public time. Write down when each advisory first appeared publicly.
3. Set up a canary. Create a scratch repository that pins a vulnerable version of each package. Keep it out of production.
4. Watch the tool. Scan the canary daily, or on the schedule the tool uses, and record the first time each finding appears.
5. Compute the lag. Subtract the public time from the detection time for each advisory.

Because this uses past advisories, you can also run it retrospectively: add the package to the canary now and see whether the tool already knows. That checks coverage but not speed, so do both when you can.

What to look for

• Median lag and worst case. The worst case matters for emergencies.
• Lag by ecosystem. A tool may be fast for one package manager and slow for another.
• Missing entirely. An advisory that never appears is a coverage gap, more serious than delay.
• Severity at first sight. Some tools show a finding with no score at first and update later. Ask how this is shown.

Questions for the vendor

• Which sources do you ingest, and how often?
• Do you add your own research, and how is it labelled?
• What happens when an advisory is withdrawn or corrected?
• Is there a status page or log showing the last data update?
• How do you handle notifications for newly published advisories against already scanned projects, so we do not wait for the next scheduled scan?

That last question is key. A freshness test that only looks at scans misses the case where new data arrives after your last scan. Continuous matching of stored inventories against new advisories closes that gap.

Where Safeguard fits: it provides continuous monitoring of dependency inventories and zero-day discovery, and the free scan can be added to a canary comparison.

Try it yourself

Click through it at your own pace, or open it full screen.

Look for how a new advisory is matched against an existing inventory without a fresh scan.

Quick check

Last reviewed: 19 October 2026.