In 2025, "AI coding assistant" mostly just means "the model behind it." GitHub Copilot leans on OpenAI's GPT-4.1/Codex family, Cursor lets developers pick between Claude, GPT-4-class, and Gemini models, Amazon Q Developer runs Amazon's own models, and Tabnine offers self-hosted options. That distinction matters because insecure output rates track the underlying model and its guardrails far more than the product wrapper around it. Meta's CyberSecEval 3 benchmark found roughly 30-35% of tested model completions contained at least one exploitable coding weakness. Veracode's 2025 GenAI Code Security Report, which ran 80 security-relevant coding tasks against more than 100 LLMs, found 45% of generated code samples failed at least one security test — a number that hasn't meaningfully improved since its 2024 report despite large jumps in functional accuracy. This piece walks through what the numbers actually say, where the risk concentrates, and how to catch it before it ships to production.
What Does "Insecure Output Rate" Actually Measure?
Insecure output rate is the share of AI-generated code samples that contain at least one exploitable weakness — usually mapped to a CWE or OWASP Top 10 category — when the model is run against a fixed set of coding tasks. The methodology differs by study, and that's why headline numbers vary so much between reports. Veracode's approach scores 80 curated tasks per language with static analysis tuned to specific CWEs, then reports the percentage of tasks where the generated solution introduced a flaw. Meta's CyberSecEval 2 and 3 combine static analysis with an "LLM-as-judge" pass to catch logic-level issues static tools miss. The earlier and still widely cited Stanford study by Perry, Sridhar, and colleagues (published at CCS 2023, "Do Users Write More Insecure Code with AI Assistants?") took a different, arguably more realistic approach: it had human participants complete security-relevant programming tasks with and without AI assistant access, then had security experts manually review the resulting code. None of these numbers translate directly into "how often will code my team ships be vulnerable" — that depends heavily on your review process — but they're the best public signal we have for comparing risk across models and, by extension, the assistants built on top of them.
How Do Insecure Output Rates Compare Across Leading AI Coding Assistants?
They cluster by underlying model family and programming language far more than by vendor brand. In Veracode's 2025 test set, failure rates ranged from roughly 38% for the best-performing model family up to 72% for Java tasks specifically, regardless of which company shipped the model. That's a meaningful data point for buyers because most commercial assistants are thin interfaces over a small number of foundation models: GitHub Copilot Enterprise defaults to OpenAI's models with an option to route to Claude or Gemini, Cursor exposes a model picker so the same seat can produce very different security outcomes depending on which model a developer selects that day, Amazon Q Developer runs Amazon's proprietary models tuned for AWS-centric code, and Tabnine's enterprise tier supports self-hosted models specifically so regulated teams can pin a known-quantity model version. The practical takeaway is that "which assistant is safest" is really "which model-and-guardrail combination is safest for this language and task," and picking the newest, most capable flagship model (GPT-4.1, Claude Opus 4, Gemini 2.5) does not automatically buy you a lower insecure output rate.
Which Vulnerability Types Show Up Most Often in AI-Generated Code?
The same handful of CWEs dominate across nearly every published study, led by injection flaws and hardcoded credentials. Meta's CyberSecEval work consistently flags CWE-89 (SQL injection), CWE-79 (cross-site scripting), and CWE-798 (use of hardcoded credentials) as recurring categories across the model set it tests. The Stanford/Perry study found the sharpest security gap between AI-assisted and unassisted participants on tasks involving raw string concatenation into SQL queries — exactly the pattern that produces SQL injection — and also found AI-assisted participants were more likely to use outdated or insecure cryptographic primitives when a task touched encryption. Veracode's 2025 breakdown adds insecure deserialization and path traversal to the recurring list, and notes that these categories persist even in models that score well on functional-correctness benchmarks like HumanEval, meaning a model can reliably produce code that compiles and passes unit tests while still being trivially exploitable.
Does a Newer or Larger Model Produce Safer Code?
Not reliably, and the trend line is the most counterintuitive finding in this space. Veracode's year-over-year comparison found insecure output rates held flat or slightly worsened across newer model generations even as those same models posted sharp gains on functional-correctness benchmarks such as HumanEval and SWE-bench. The explanation is straightforward once you look at how these models are trained: reinforcement learning and fine-tuning pipelines optimize heavily for "does the code run and pass the given tests," not "does the code resist an attacker supplying adversarial input." A model can get dramatically better at solving LeetCode-style problems from 2023 to 2025 without any corresponding pressure to stop hardcoding a database password or to start parameterizing a query. Buyers evaluating assistants on benchmark leaderboards or "coding ability" scores should treat those numbers as orthogonal to security, not a proxy for it.
Why Do Developers Trust Insecure AI Suggestions Anyway?
Because AI assistants are fast and persuasive, and the evidence shows developers calibrate trust in exactly the wrong direction. The Stanford/Perry study's most cited result isn't the security gap itself — it's that participants who used the AI assistant were more likely to rate their own, objectively less secure code as secure than participants who worked without it. Snyk's 2023 AI code security survey adds a second data point: a majority of developers reported shipping AI-suggested code without a dedicated security review step, and a notable share admitted bypassing existing security policy specifically to move faster with AI-generated suggestions. Autocomplete-style interfaces compound the problem by keeping developers in a "flow state" where accepting a suggestion is a single keystroke, review fatigue sets in over long PRs, and the sheer volume of code an assisted developer ships per day goes up — so even a constant per-line risk translates into more vulnerable lines reaching production per sprint.
How Safeguard Helps
Insecure output rate is a property of your whole pipeline, not a single vendor's safety marketing, which is why Safeguard doesn't ask you to bet your security posture on picking the "right" assistant. Most engineering orgs already run a mix — Copilot in one team, Cursor in another, Amazon Q for AWS-heavy services — and developers switch models inside the same tool from one session to the next. Safeguard scans commits and pull requests for the exact vulnerability classes these benchmarks flag as most common: SQL and command injection, hardcoded credentials and secrets, insecure deserialization, and path traversal, regardless of which assistant or model produced the diff. It ties findings back to provenance and SBOM data so you can see which vulnerable lines trace to AI-assisted commits versus hand-written code, giving security teams a real answer to "how much of our exposure is coming from AI-generated code" instead of a guess. Policy gates block merges on high-severity, AI-attributable findings before they reach main, and continuous monitoring keeps working as your team's assistant-and-model mix changes — so your security posture doesn't depend on any single vendor's benchmark score staying good.