To test a SAST tool yourself, create a branch, plant a small set of known flaws in the style of your own code, scan it, and record which ones are reported, with what location and what explanation. Ten to twenty cases give you a smoke test, not a detection rate.
Why plant your own
Published benchmarks are useful. The OWASP Benchmark is a runnable web application with deliberately planted vulnerabilities, intended to be exploitable so scoring is fair, and it has scorecard generators for many tools. But it is a different codebase from yours. A seeded branch uses your frameworks, your coding style and your build.
Design the cases
Write cases from your own incident history and from common classes. For each, decide in advance what a good result looks like.
- Injection into a database query built by string concatenation.
- The same query written with parameters, as a negative control that should not be flagged.
- A user input that flows through a helper function in another file before reaching the sink.
- The same flow with a proper validator or encoder in between.
- A hardcoded credential in a config file.
- A weak cryptographic choice in a utility class.
The negative controls matter as much as the bugs. A tool that flags everything looks perfect on detection and awful on noise.
OWASP's injection guidance recommends keeping untrusted data separate from commands and queries and preferring parameterized queries, so your safe versions should follow that pattern.
Run and record
Scan the branch exactly as you would in production: same pipeline, same configuration. Fill in a table with one row per case: planted, flagged, flagged correctly, message clarity, time to understand. Also note anything flagged that you did not plant, and judge it honestly.
Do not tune the rules until after the first run. You want to see out-of-the-box behavior, because that is what most of your developers will live with.
What the result tells you
A caught bug with a clear explanation and exact line is a good sign. A miss on the cross-file flow tells you something about depth. A false alarm on a negative control tells you about noise.
What it does not tell you: overall accuracy. Twenty cases is a tiny sample. OWASP also warns that static tools are weak on access control, authentication and cryptography problems, so do not be surprised if those cases are missed. Use the result to ask sharper questions of the vendor.
Keep the branch
Store the seeded branch and the results table. When rules update or you evaluate a second tool, rerun the same cases. Over time it becomes your own small regression suite for scanning quality.
Quick check
Is it safe to plant flaws in a real repository?
Use a throwaway branch or a separate private repository, and never merge or deploy it.
Why include safe code in the test?
To see whether the tool avoids false alarms. Detection without precision is just noise.
Sources
- OWASP Benchmark project
- Source code analysis tools, OWASP Community
- A03 Injection, OWASP Top 10 2021
Last reviewed: 19 October 2026.
