Ask a SAST vendor whether it supports your exact languages and frameworks, how it reports accuracy, whether it needs a buildable project, how findings integrate with your pipeline, and what classes of bug it cannot find. Then verify the answers on your own code. Brochure numbers rarely transfer to your repositories.
This is a how-to-choose guide. We do not rank products.
What does OWASP say to weigh?
OWASP's page on source code analysis tools lists selection criteria that still hold up: support for your programming language, ability to detect the OWASP Top Ten and other vulnerability classes, accuracy as measured by false positive and false negative rates, understanding of your libraries and frameworks, whether the code must be buildable, whether it can run against binaries, availability in developer IDEs, ease of setup, CI integration, license cost, and interoperability of output through SARIF.
The same page is blunt about weaknesses. It says SAST tools can automatically identify only a relatively small percentage of application security flaws, produce high numbers of false positives, often cannot see configuration issues because they are not in the code, and find it hard to prove that a flagged issue is a real vulnerability. Keep that in mind: your goal is a useful, noisy-but-bounded signal, not completeness.
The ten questions
1. Which languages and frameworks, at what depth? "Supports Java" can mean syntax parsing or full taint tracking through Spring. Ask for the framework list, and which are modeled for sources and sinks.
2. How do you measure accuracy? Ask for method. NIST's SATE studies explicitly aim to explore relevance, correctness and prioritization of warnings and state that they are not meant to rank tools. If a vendor cites a benchmark, ask which version and whether the vendor tuned for it.
3. Does it need a build? Some analyzers need compilable code, libraries and instructions. Others read source directly. A build requirement affects monorepos, legacy code and acquired code.
4. What does a finding contain? Look for file, line, the data flow from source to sink, a CWE mapping and a fix suggestion. A bare warning makes a developer hunt for the cause. A traced path shows it.
5. How does it handle suppression? Inline comments, baselines and policy files. Ask who can suppress and whether it is logged.
6. Where does the code go? Cloud analysis, your runner, or on-prem. Match this to your source residency rules and verify with network logs in the trial.
7. Which output formats? The OASIS SARIF standard lets results flow into code scanning views and issue trackers. Ask whether the SARIF is complete or a thin export.
8. How does it fit the workflow? IDE feedback, pull request comments, CI gating and incremental scans on changed files. Scan time on your largest repo decides adoption.
9. What can't it find? A good vendor names authentication logic, access control design, business logic flaws and runtime configuration. A vendor who says everything is covered is selling.
10. What does a new rule take? Can you write custom rules for internal frameworks, and how long does the vendor take to add a rule for a newly published issue class?
How do you run a bake-off that means something?
Pick three repos: one clean, one with known issues you have already triaged, and one huge. Run each tool on all three with default settings, then with tuning. Count true findings against your known list, count false alarms in a random sample of fifty, and record scan time and setup effort. Let two developers triage a day of results and ask what they would act on.
CWE, the MITRE catalog of weakness types, gives you a neutral vocabulary for comparing results across tools, so map each finding to a CWE before you compare.
What to do
- List your languages, frameworks and build systems before any call.
- Send the ten questions in writing and compare answers side by side.
- Pick three trial repos, including one with a known-issue list.
- Define pass marks: true positive rate, false alarm ceiling, scan time.
- Check SARIF output in your own code scanning view.
- Confirm where code runs and verify with network logs.
- Pair SAST with dependency scanning and review, since it covers one slice.
To see what a traced finding looks like, run the demo.
Quick check
Can SAST replace code review?
No. OWASP notes SAST finds a relatively small percentage of flaws and misses configuration and design issues. Use it alongside review and testing.
Is a low false positive rate enough?
No. A tool with few alerts may also miss real bugs. Measure both false positives and false negatives on code you know.
Sources
- OWASP: Source Code Analysis Tools
- NIST: Static Analysis Tool Exposition (SATE) VI
- OASIS SARIF 2.1.0 specification
- MITRE CWE
Last reviewed: 18 October 2026.
