Ask any vulnerability management vendor how they find assets, how they rank risk with exploitation data, whether they check reachability, what a fix looks like, where your data goes, and how you leave. Good answers are specific and testable on your own code during a trial. Vague answers usually mean a longer list of findings, not fewer problems.
Why do the questions matter more than the feature list?
Every vendor in this category will say it finds vulnerabilities. The hard part is what happens after: who gets the ticket, whether it is real, and whether it closes. NIST describes enterprise patch management as identifying, prioritizing, acquiring, installing and verifying patches, and notes that business owners and security teams often disagree about how much effort it deserves. A tool should reduce that argument, not feed it.
So test the whole loop. Bring one real repository, one container image and one vendor SBOM to the demo.
What coverage questions should you ask?
- What do you scan? Ask for the list: source dependencies, container layers, build artifacts, vendor SBOMs, infrastructure code. Then ask what is not covered.
- How do you find assets you do not know about? An inventory you must fill by hand is out of date by Friday.
- How fresh is the advisory data, and which feeds do you read? Ask whether it includes the CISA Known Exploited Vulnerabilities catalog, which CISA describes as an authoritative list of vulnerabilities exploited in the wild and recommends using as an input to prioritization.
What prioritization questions should you ask?
- What signals feed the ranking beyond severity score? FIRST describes EPSS as a model estimating the probability a CVE will be exploited in the next 30 days. A good vendor can show you how that sits next to KEV status and your own context.
- Do you check reachability? Ask how: call graph analysis, runtime data, or neither. Then ask what the tool does when it cannot decide. A tool that says unknown is more honest than one that always says safe.
- Can I see why a finding is ranked where it is? If the answer is a number with no explanation, your engineers will not trust it.
What remediation questions should you ask?
- What is the fix path? Ask for an example of a pull request the tool opened, and what it does when no fixed version exists.
- How do you handle transitive dependencies? Many real fixes sit two or three levels down.
- What are the limits? Every product has some: unsupported ecosystems, languages with weak analysis, scans that time out on large monorepos. Ask for them in writing.
What about data and exit?
- Where does my code go, and can analysis run locally? Some products upload source, some upload only an SBOM and findings. Ask which, and what is stored and for how long.
Also ask about export. Can you take findings, history and SBOMs out in open formats such as SPDX or CycloneDX if you leave?
What do honest answers sound like?
They contain numbers and conditions. "Reachability works for these languages, and for others we mark the result as unknown." "We reduce noise but do not remove it." If the answer to every question is yes, ask for a demo of the hard one.
Watch for these:
- Counts of vulnerabilities found used as the success metric.
- Claims about prevented breaches with no method.
- A trial that only works on the vendor's sample project.
What to do
- Write down your top three outcomes, such as shorter time to fix criticals or fewer false alarms.
- Pick one repository, one image and one vendor SBOM with known issues.
- Run the same inputs through every shortlisted product.
- Score each on the ten questions above, and note what each vendor refused to answer.
- Check data handling with your privacy and security leads before the trial, not after.
- Ask for contract terms on data export and deletion.
Quick check
Is a higher CVSS score enough to prioritize?
No. Severity describes impact if exploited, not the chance it will be. Pair it with exploitation signals such as KEV and EPSS, and with whether your code reaches the vulnerable function.
How many vendors should I trial?
Two or three on identical inputs is enough to see real differences. More than that tends to produce spreadsheet fatigue, not better decisions.
Sources
- NIST SP 800-40 Rev. 4, Guide to Enterprise Patch Management Planning
- CISA Known Exploited Vulnerabilities Catalog
- FIRST, Exploit Prediction Scoring System
Last reviewed: 18 October 2026.
