Treat the patch like input from an untrusted user and run deterministic checks before a person reviews it. Check which files changed, which packages were added, how large the diff is, and whether your own tests and scanners pass. Code makes these decisions, not another model.
The principle
OWASP's guidance on improper output handling is to treat the model as any other user, with a zero trust approach, and validate its responses before they reach other components. A patch is a model response headed for your repository and then your build. The same logic applies.
Deterministic means the same input gives the same verdict every time, with no judgment call. That makes the checks auditable and hard to talk around.
A validation pipeline
Run these in order. Cheap checks first, so bad patches fail fast.
- Path allowlist. The diff may touch only files in the area of the finding, plus manifest and lockfile for that area. Reject changes to workflow files, deployment configuration, protection settings and secrets files.
- Size limits. Cap changed files and lines. A one line dependency fix that returns two hundred changed lines is a signal.
- Dependency diff. List every package added, removed or version changed. Reject additions the finding does not explain. Check that each added name exists in the official registry and is the package you expect.
- Install behavior. Flag new install scripts or post install hooks.
- Build and test. Run your required checks. Failures stop the pipeline.
- Re-scan. Run the scanner on the patched tree. The original finding should be gone and no new high severity finding should appear.
- License check. If policy restricts licenses, run the check on any new dependency.
Only a patch that clears all seven reaches a human reviewer.
What the table of checks catches
| Failure | Caught by |
|---|---|
| Fix edits a pipeline file | Path allowlist |
| Unexpected extra dependency | Dependency diff |
| Typosquatted or invented package name | Registry existence check |
| Fix breaks behavior | Build and tests |
| Fix does not remove the finding | Re-scan |
| Oversized rewrite | Size limit |
What deterministic checks cannot catch
A patch can pass every automated check and still be wrong in a way your tests do not exercise. It can also be subtly weaker than the original, for example by loosening validation to make a test pass. Automated checks narrow the field. They do not replace judgment. See what green CI does not prove about a fix PR for the gap.
Why not use a second model as the checker
A model reviewing a model's output can help a reviewer, but it is not a gate. It can be wrong in correlated ways and can be steered by the same planted text. Keep model review advisory and let code decide pass or fail.
Make failures useful
When a patch fails a check, record which check and why. A pattern, such as repeated path allowlist failures in one repository, tells you where scope or ownership is wrong. Count rejections per check each month.
Log the verdict
Store the result of each check with the pull request: which ran, which version of the rule set, pass or fail. That record is what an auditor or an incident reviewer will want later.
Safeguard offers autonomous remediation with fix PRs, off by default with every category set to Manual, so these checks sit in front of a human decision rather than replacing it.
Try it yourself
Click through it at your own pace, or open it full screen. Watch how a change moves from finding to a gated decision.
Quick check
Is passing all automated checks enough to merge?
No. It is enough to reach a reviewer. A person still decides whether the change is correct and appropriate.
Why check that a package name exists?
Generated code and fixes can name packages that do not exist or that imitate real ones, and an attacker can register such names. Confirm each addition.
Sources
Last reviewed: 19 October 2026.
