Test a reachability tool with a repository where you already know the answers. Plant or pick vulnerable dependencies, mark which vulnerable functions your code calls and which it does not, and score the tool on misses first. A tool that hides a reachable flaw is worse than one that is noisy.
Why build the answer key first?
Vendors demo on their own sample projects. You need your code, with answers you wrote down before the trial. Without an answer key you are judging by how a dashboard looks, and every reachability dashboard looks good when it shows fewer red items.
Spend a day building the key. It is the most valuable part of the trial.
How do you build the key?
Pick one or two real services. For each, choose eight to twelve vulnerable dependencies already flagged by any scanner. Then read the code and label each finding:
- Called: your code reaches the vulnerable function on a normal path.
- Called only behind a flag or route: reachable, but only with a configuration enabled.
- Not called: the package is installed but the vulnerable function is never invoked.
- Hard to tell: reflection, dynamic dispatch or plugin loading is involved.
Include at least two "hard to tell" cases on purpose. How a tool treats them shows its character.
What categories does Safeguard document?
As a reference for what a good answer looks like, Safeguard's reachability documentation uses four verdicts: reachable, conditionally reachable, unreachable and unknown. It states that unknown results, such as those caused by reflection or dynamic dispatch, are treated conservatively as potentially reachable rather than suppressed. It also lists weak spots: reflection, eval, data-driven vulnerabilities, and paths that exist only under configurations the analyzer cannot enumerate.
Whatever tool you test, ask for the same honesty. A tool with no "unknown" bucket is guessing somewhere.
How do you score the trial?
Score in this order.
- Misses. Any "Called" item marked unreachable is a failure. Count them. One is a conversation; three is a no.
- Noise cut. Of the "Not called" items, how many did it correctly deprioritize? This is the benefit you are paying for.
- Explanation. For five findings, can the tool show the call path, not just a label? You will need it to convince a developer.
- Languages. Does it cover the languages you run? Safeguard's page, for example, marks some languages GA and others Preview, which is the kind of detail to ask every vendor for.
- Speed and fit. Time a full scan, and see whether results land where people work, such as pull requests or the IDE.
Open the exploitability check demo
What should you not believe?
Percentages in marketing. Safeguard's documentation itself says its reported filtering figures are aggregates across customer tenants and not guarantees. Treat any such number, from any vendor, as a reason to run your own test.
Also watch for a tool that treats reachability as a verdict on exploitability. Reachable means code can call the function. Exploited in the wild is a separate signal, and CISA's Known Exploited Vulnerabilities catalog is the usual place to check it.
What to do
- Choose one or two services and build the answer key before you open any vendor trial.
- Include at least two "hard to tell" cases.
- Run each tool on the same commit and save raw output.
- Score misses, then noise cut, then explanations, then speed.
- Decide in writing what miss count ends the trial.
Quick check
What is the worst failure in a reachability trial?
Marking a function your code really calls as unreachable. That hides a real risk.
Should unknown results be hidden?
No. Safeguard's docs treat unknown as potentially reachable, which is the cautious default.
Sources
Last reviewed: 18 October 2026.
