Ask every SAST vendor the same twelve questions, in writing, and compare answers side by side. Cover languages and frameworks, build requirements, accuracy evidence, workflow integration, output formats and licensing. Vague answers are a signal too. The list below follows the selection criteria OWASP publishes for source code analysis tools.
Coverage questions
- Which languages and versions do you support, and which frameworks inside them? A language listed on a page is not the same as the framework you use being understood. OWASP calls out framework understanding as its own criterion.
- Which vulnerability classes do you cover? Ask for a mapping to the OWASP Top Ten or to CWE identifiers, and ask what is explicitly not covered.
- What do you not detect well? OWASP notes that static tools are weak on authentication problems, access control issues and insecure cryptography use. A vendor who says "nothing" has not thought hard about it.
Build and workflow questions
- Does the tool need to build the code? OWASP points out that many tools need the right libraries, build instructions and complete code. Ask what happens with partial code, generated code and monorepos.
- Where does it run? IDE, pull request, CI, scheduled scan. Ask which of these are included in the quoted price.
- How is the scan time on a repository of our size? Ask for a number from a comparable customer or run it yourself in a pilot.
- How do we suppress a finding, and does the suppression survive refactoring? Suppression handling decides how noisy the tool feels in month six.
Evidence and accuracy questions
- What accuracy evidence can you share? OWASP lists false positive and false negative rates and OWASP Benchmark scores as criteria. The Benchmark is a runnable test application with planted, exploitable vulnerabilities, so ask whether scores are published and how they were produced. Treat any single score as one input, since one test suite does not represent your code.
- Can we run it on our own code before signing? If the answer is no, note it.
Output and cost questions
- Which output formats do you export? SARIF is an OASIS standard for static analysis results, and its stated goal is letting teams combine results from different tools without converting each one.
- How is licensing counted? OWASP notes that pricing may vary by user, organization, application or lines of code. Ask for a worked example using your numbers so you can compare quotes on the same basis.
- What happens to our data? Ask where code is processed, what is retained and how long.
How to use the answers
Put the vendors in columns and the questions in rows. Mark each cell green, amber or red. Do not add numeric weights until you have talked about what matters to you, because weights tend to hide disagreements instead of settling them.
Follow up on any amber cell by asking for evidence, not reassurance. A screenshot, a documentation page or a trial run counts. A sentence on a call does not.
Quick check
Is a published benchmark score enough to choose a tool?
No. A benchmark measures detection on one test application. Use it as one input beside a pilot on your own code.
Why ask for SARIF support?
It lets you move findings between tools and dashboards without writing converters, which reduces lock-in.
Sources
Last reviewed: 19 October 2026.
