How to judge automatic fix suggestions from a SAST tool

Take twenty real findings from your pilot, apply each suggested fix on a branch, and count how many compile, pass your tests, and actually remove the flaw. The share that survives all three tells you whether suggestions save time or create review work. Judge on your code, not on a showcase.

What a fix suggestion can get wrong

A suggested change can fail in four ways.

• It does not compile or breaks a test.
• It removes the warning but not the flaw, for example by hiding a value from the scanner instead of validating it.
• It fixes the flaw but changes behaviour that something depends on.
• It is correct but in a style your team would reject, so every suggestion needs rewriting.

Only the first is easy to catch automatically. The second and third need a human who understands the code.

The twenty finding test

Choose findings that differ in type, language and difficulty. Include a few that you know are hard, such as ones that need a design change. For each:

1. Apply the suggestion on a branch.
2. Run the build and your test suite.
3. Rescan and confirm the finding is gone and no new one appeared.
4. Have a developer read the diff and rate it: merge as is, merge with edits, reject.
5. Record the minutes spent.

Compare minutes against fixing the finding by hand from a short description. A suggestion that takes as long to review as to write has not helped.

Questions for the vendor

• Where does the suggestion come from: a rule based template, a model, or both? Ask what happens to your code during generation, and whether it leaves your environment.
• Does the tool verify its own suggestion, by rescanning or running tests, before showing it?
• Can it open a pull request for you, and can you require review before merge? Auto merging security changes is a policy decision, not a default.
• Can you turn suggestions off per rule or per repository?
• What does it say when it cannot propose a fix? An honest "no suggestion" beats a confident wrong one.

Where fixes tend to work

Mechanical changes tend to go well: parameterising a query, swapping an unsafe function for a safe equivalent, adding an escape call, upgrading a dependency version. Changes that need business knowledge, such as access control logic, rarely suit automation.

Be sceptical of any claim that every finding can be fixed automatically. OWASP's description of static analysis points out that authentication, access control and cryptography problems are hard to detect automatically. Fixing them is harder still.

Keep humans in the loop

Treat generated fixes as pull requests from a new colleague. They get a normal review, normal tests, and a record of who approved. For compliance reasons, make sure the change history shows that a person approved it.

Try it yourself

Click through it at your own pace, or open it full screen. See how a fix is proposed as a pull request rather than applied silently.

Quick check

Safeguard's autonomous remediation opens fix pull requests and is off by default, with every category set to Manual. Try the free scan at gold.safeguard.sh/scan.

Sources

• OWASP, Source Code Analysis Tools

Last reviewed: 19 October 2026.