Ask who ran the test, who paid, what was measured, on which version and date, and whether you can read the method and raw results. A real independent result survives these questions easily, while a weak one gets vague very fast.
The eight questions
- Who ran it? Name the organization and the people. Is there a public record?
- Who paid? Vendor funded, buyer funded, grant funded, or self funded? Funding is not disqualifying but it must be stated.
- What exactly was measured? Detection, speed, noise, coverage, usability? A score for one is not a score for another.
- On what inputs? Test application, open source projects, or customer code? How were correct answers decided?
- Which version and date? Products change often. A result about an old version says little about the current one.
- Who else took part? Were other tools tested under the same conditions? Were any removed from the report?
- Can I see the method and raw data? If they cannot share it, ask why.
- Does it match my environment? Languages, frameworks and scale matter.
Watch the wording
Be careful with phrases such as "validated by," "recognised by" or "tested against" without a name and a link. Participation in a public exercise is not the same as winning it. For example, NIST states that its SATE study does not aim to find the best tool and should not be read as a ranking, so a claim that cites SATE as proof of superiority deserves a follow up question.
Check whether the claim is a ranking
Ask whether the source actually ranks anything. Some programs publish descriptive results only. If a vendor converts descriptive results into "number one" language, the source did not say that.
Ask for the limits
A good source states what the test does not show. Ask the vendor to read the limitations section aloud. If there is none, say so.
Replace the claim with your own evidence
Whatever the answers, run your own test on your own code. A cited result is a reason to consider a tool, not a reason to skip testing. See how to run your own bake-off.
What good answers sound like
A confident vendor will name the tester, link the report, say who funded it, and tell you where the product did poorly. They will also say what they did after the test, such as fixing a weakness it exposed. Be wary of answers that change the subject to features, offer a summary slide instead of the document, or say the report is confidential but shareable under a sales process. None of those proves anything bad, but each is a reason to lean harder on your own test.
Where Safeguard fits
Safeguard offers SBOM and dependency scanning, reachability analysis and zero-day discovery. The free scan at gold.safeguard.sh/scan gives you your own evidence in minutes.
Quick check
What if the vendor cannot share the method?
Treat the claim as marketing, not evidence, until you can read how the test was done.
Sources
Last reviewed: 19 October 2026.
