You cannot tell from the diff alone. Prove it by reproducing the problem on the old code, showing the same steps fail on the new code, rescanning, and searching for the same weak pattern elsewhere. A fix that only passes review has not been shown to work.
Reproduce first
Before reading the AI suggestion, write down how the flaw shows up. For a dependency issue, that may be the advisory's affected version range. For a code issue, it may be a request, an input string or a failing test. If you cannot describe the bad behavior, you cannot confirm it is gone.
This step also protects you from a subtle trap. A suggestion can be built to satisfy the scanner's rule rather than the underlying weakness. Reproducing the behavior tests the weakness itself.
Retest after
Run the same steps against the branch with the fix. Three outcomes are possible.
- The bad behavior stops. Good. Keep the check as a regression test.
- The behavior changes shape. A different payload or path still works. The fix was narrow.
- Nothing changes. The suggestion did not touch the real cause.
For code flaws, a test that fails on the old code and passes on the new code is the strongest evidence you can attach to the pull request. Write it yourself or review it closely, because the same tool that wrote the fix may write a test that asserts the new behavior rather than the correct one. OWASP's guidance on AI assisted coding makes this point about test fabrication directly.
For dependency fixes, check the data
A dependency fix is easier to verify. Vulnerability records in the OSV format describe affected ranges with events such as introduced and fixed, so the fixed version is a published fact you can compare to the version in the diff. Check that the new version is at or above the fixed event for your release line, and that the lockfile resolves to it. See the post on verifying the version an AI fix recommends.
Rescan and look sideways
Rescan the branch with the tool that raised the finding. Then search the repository for the same pattern. If one handler built a query by string concatenation, others may too. A good fix often prompts a wider cleanup, and a fix that changes one line while the pattern spreads across forty files is a partial fix.
Try it yourself
Click through it at your own pace, or open it full screen. Look at how exploitability evidence is separated from a bare finding, the same separation you want when judging a fix.
What to attach to the pull request
- The reproduction steps and their result before and after.
- The rescan result.
- The list of other places checked for the same pattern.
Safeguard's scans and fix pull requests can feed this evidence, and autonomous remediation stays off until you turn on a category. Reviewers still decide.
Quick check
Is a clean rescan proof that the fix works?
It is good evidence, not proof. Scanners have blind spots, so pair it with a reproduction of the original behavior.
What if I cannot reproduce the flaw?
Say so in the pull request, lean on the advisory or finding details, and give the change a heavier review before merging.
Sources
Last reviewed: 19 October 2026.
