To validate a reachability tool, build a small repository where you already know the right answers. Include a vulnerable function you call directly, one you call through an interface, one you call by reflection, and one you never call. Then check whether the tool agrees with you on each, and whether it shows its evidence.
Why seed your own tests
A vendor demo uses code that works well for the vendor. Your code has your frameworks and your habits. A seeded test takes an afternoon and tells you more than a slide deck.
Pick a known vulnerable package
Choose a published advisory that names an affected function and has a fixed version. Pin the vulnerable version in a scratch project. Check the advisory text yourself so you know which function is the flawed one.
The six cases
- Direct call. Your code calls the vulnerable function by name. Expected result: reachable.
- Never called. The package is imported or listed but the function is not used. Expected: not reachable.
- Through an interface. The call goes through an interface with one implementation. Expected: reachable, and see whether the path is shown.
- Through a callback. Pass the function as a value to another function that calls it. Expected: reachable.
- By reflection or string lookup. Invoke by name from a config string. Expected: reachable in truth, and a static tool may miss it. Record what it does. Documentation of one Go tool says calls through reflect are not visible to static analysis, so a miss here is a known class of gap.
- Transitive. A package you import calls a second package that holds the flaw. Expected: reachable if your code uses the part of the first package that calls the second.
Score the results
| Case | You expect | Tool says | Path shown |
|---|---|---|---|
| 1 to 6 | fill in | fill in | yes or no |
Count misses in each direction. A tool that finds all reachable cases but flags the never called one is over approximating. A tool that clears case 5 may be under approximating dynamic calls. Neither is automatically a reason to reject it. You are learning how it behaves.
Add two questions
- What does it say when the advisory has no function data?
- Does the report separate package level matches from function level ones?
Keep the repository
Keep the scratch repo and re-run it when you change tools or upgrade a version. It becomes a small regression suite. For measuring noise across real projects, see how to measure SCA false positives in a pilot.
Quick check
How many cases are enough?
Six to ten, covering your main language and frameworks. More cases add little compared with realism.
What if the tool fails the reflection case?
That is common for static analysis. Decide whether your critical code relies on reflection and add review steps if it does.
Safeguard users can compare results with the free scan on the same scratch repository.
Sources
Last reviewed: 19 October 2026.
