Veracode's 2025 GenAI Code Security Report tested more than 100 large language models across over 80 coding tasks and found that 45% of the resulting code samples contained at least one OWASP Top 10 vulnerability class. Cross-site scripting failed 86% of the time, log injection failed 88% of the time, and Java was the worst-performing language at over 70% failure — and, notably, newer and larger models did not produce meaningfully more secure output than older ones. This lands on top of an earlier and more troubling finding: a Stanford study presented at ACM CCS 2023 ("Do Users Write More Insecure Code with AI Assistants?", Perry, Boneh, et al.) found that developers using a Codex-based AI assistant wrote significantly less secure code than a control group with no assistant — and were simultaneously more likely to believe, incorrectly, that their code was secure. Put those two findings together and you get the actual risk of AI-assisted coding: it is not just that assistants introduce bugs, it is that they introduce bugs while eroding the human skepticism that used to catch them. This post lays out the guardrails that keep that dynamic from reaching production — review gates, scanning gates, provenance tracking, and least-privilege for the agents themselves.
Why can't code review alone catch AI-introduced vulnerabilities?
Human review alone struggles because the Stanford study's central finding was specifically about false confidence: participants using the AI assistant rated their own insecure code as secure more often than the control group rated theirs. That is a reviewer problem, not just an author problem — a diff that reads fluently and follows conventional patterns invites less scrutiny, even when it silently drops input validation or reintroduces a pattern a linter would catch instantly. This is why mandatory human review before merge has to be paired with automated gates rather than treated as sufficient on its own: branch protection rules that require at least one approval, combined with SAST, SCA, and secrets scanning running as required CI checks on every pull request, catch classes of bugs — hardcoded credentials, injection patterns, vulnerable dependencies — that a confident-sounding AI-authored diff makes easier for a human to wave through. Treat AI-authored PRs as a higher-scrutiny category, not an equivalent one.
What does provenance tracking actually buy you?
Provenance tracking buys you the ability to apply differentiated scrutiny instead of uniform trust across every commit. If a PR is flagged as AI-assisted — whether by a commit trailer, a CI-detected pattern, or an IDE plugin recording which suggestions were accepted — a policy engine can route it through stricter checks: mandatory second reviewer, mandatory SAST re-scan even on files that normally skip it, or a required security-team sign-off above a certain diff size. This is a natural extension of the software supply chain provenance work already standardized around SLSA (Supply-chain Levels for Software Artifacts) and in-toto attestations, which prove what build system produced an artifact from what source and materials. Extending that same attestation model — a signed, auditable predicate — to say "this diff was AI-assisted, here is what was generated versus human-written" gives auditors and incident responders something to query later, the same way an SBOM lets you query "are we affected" the moment a new CVE drops, instead of re-deriving the answer from scratch.
Where should scanning gates sit relative to AI-assisted commits?
Scanning gates should sit at commit and CI time, not just at merge, because AI assistants generate code at a volume and velocity that makes end-of-cycle scanning too late to be useful. A pre-commit or IDE-level check that flags a vulnerable package or an obvious secret the moment an assistant suggests it costs a developer seconds; the same finding discovered in a security review three sprints later costs a rewrite. Concretely, this means running SAST and SCA as blocking CI steps — not advisory ones — on every PR regardless of authorship, with policy language that can express severity thresholds ("block on any CWE-89 or CWE-502 finding above CVSS 7.0") so a security team isn't manually re-triaging every AI-flagged diff by hand. Given that Veracode found XSS and log-injection failures in the majority of samples across the model set they tested, teams that only scan quarterly, or only scan release branches, are structurally unable to catch what AI assistants are producing daily.
Should AI-authored PRs be allowed to auto-merge?
No — auto-merge should be disabled for any PR identified as AI-authored or AI-assisted, full stop, because auto-merge assumes the checks that ran are sufficient to certify correctness and security, and neither the Veracode nor the Stanford findings support that assumption for AI-generated code at current model quality. This applies symmetrically to AI agents used by security and platform teams themselves, not just to developer-facing assistants: an auto-remediation agent that opens a dependency-upgrade PR should still require the same human approval and CI gate as a human-authored change, with its write scope limited to the specific repositories, package ecosystems, and severity bands it was explicitly authorized for — not standing write access it could use unpredictably. Least-privilege for the agent's CI token or API credential matters here as much as the review gate itself; an assistant with broad merge or deploy permissions turns a single bad suggestion into a shipped vulnerability with no human in the loop at all.
How Safeguard Helps
Safeguard's guardrail engine enforces exactly this kind of differentiated policy without requiring a separate tool for each stage: policy-as-code rules evaluate at the IDE, commit, CI, registry, admission, and runtime checkpoints, and every decision — block, warn, or auto-fix — produces a signed, auditable record that a security team or auditor can replay later. For AI-assisted code specifically, Safeguard's attestation model already includes a Safeguard Review predicate, a Griffin-generated, in-toto-formatted record of what changed, what was patched, and what was blocked on a given artifact — a concrete mechanism teams can extend into provenance tracking for AI-generated diffs, alongside the SLSA provenance and Sigstore signing Safeguard produces for every build. And because Griffin itself is an AI agent that opens auto-fix pull requests, Safeguard applies the same discipline to its own output: auto-fix guardrails require a pre-approved scope — specific repos, ecosystems, and severity bands — and still route through the standard review and CI gates rather than merging unsupervised. The guardrail that catches a risky suggestion from a third-party coding assistant is the same one that keeps Safeguard's own remediation agent honest.