GitHub disclosed in 2023 that Copilot already generates up to 46% of code in files where it's enabled, and Stack Overflow's 2024 Developer Survey found 76% of developers are now using or planning to use AI coding assistants. That adoption curve is running well ahead of security validation. Veracode's 2025 GenAI Code Security Report tested leading LLMs against 80 coding tasks spanning multiple languages and found that 45% of the resulting code samples contained at least one OWASP Top 10 vulnerability, with the failure rate climbing above 70% for Java. Separately, a 2025 USENIX Security paper found that code-generating models "hallucinate" non-existent open-source package names in 5.2% to 21.7% of samples, and that attackers can register those exact hallucinated names to distribute malware — a technique now called slopsquatting. AI-generated code isn't inherently insecure, but it inherits every weakness in its training data, ships faster than review capacity can absorb, and opens attack surfaces that traditional SAST and dependency scanning were never built to catch.
How much AI-generated code actually ships with vulnerabilities?
Close to half of it, according to controlled testing. Veracode's 2025 GenAI Code Security Report ran more than a dozen large language models through 80 discrete coding tasks designed to surface OWASP Top 10 issues — SQL injection, cross-site scripting, insecure cryptographic calls, and improper input validation — and found that 45% of the generated code samples introduced at least one exploitable flaw. The distribution wasn't even across languages: Java code failed security checks more than 70% of the time, while Python and JavaScript fared somewhat better but still failed roughly 38-40% of the time. Critically, the report found no meaningful improvement in vulnerability rates across newer, larger model versions released between 2023 and 2025 — bigger context windows and better benchmark scores on coding-correctness leaderboards did not translate into more secure output. That matters for any team measuring AI-tool quality purely by whether the code compiles and passes unit tests, because those are exactly the checks that miss injection flaws and logic-level authorization bugs.
Can AI coding assistants invent software packages that don't exist?
Yes, and attackers are already exploiting it. Researchers publishing "We Have a Package for You!" at USENIX Security 2025 ran 16 commercial and open-source code-generation models across roughly 576,000 generated code samples in Python and JavaScript and found that models recommended non-existent package names at an average rate of 5.2% for commercial models (like GPT-4 series and Claude-era models tested at the time) and 21.7% for open-source models. The dangerous part wasn't the hallucination itself — it was the repetition: when researchers re-ran the same prompts, 43% of hallucinated package names reappeared consistently across runs, and 58% reappeared more than once across ten repetitions. That predictability is exactly what makes "slopsquatting" viable: an attacker registers the hallucinated name on PyPI or npm before a real developer's AI assistant confidently suggests npm install <hallucinated-package> in a generated snippet, and the developer installs it without checking. Unlike typosquatting, which relies on a human mistyping a name, slopsquatting relies on the AI making the same wrong recommendation over and over.
Has AI-generated code caused a real-world security incident?
Yes — in July 2025, a malicious prompt injected into Amazon's Q Developer extension for VS Code was merged and shipped to end users before Amazon caught it. A hacker submitted a pull request to the extension's public GitHub repository containing hidden instructions directing the AI coding agent to run destructive commands designed to wipe local files and attempt to clear connected AWS resources. The pull request passed review and was published in version 1.84.0 of the extension on the VS Code Marketplace, reaching an unknown number of developer machines before Amazon pulled the release and issued a corrected build. The attacker later told reporters the goal was to demonstrate that "AI security theater" — reviewing code for what it looks like rather than what an embedded agent will actually execute — is not sufficient. The incident is a direct illustration of prompt injection as a supply chain vector: the vulnerability wasn't in a library or a CVE-numbered dependency, it was in natural-language instructions smuggled through a code review process tuned to catch syntax, not intent.
Why do developers trust AI-written code more than they should?
Because controlled research shows AI assistance breeds overconfidence, not just insecure output. A Stanford University study published in 2023 ("Do Users Write More Insecure Code with AI Assistants?" — Perry, Srikant, et al.) had 47 participants complete five security-relevant programming tasks, including building a Python function to encrypt a string and writing SQL query logic resistant to injection. Participants who had access to an AI code assistant during the study wrote significantly less secure code on tasks involving SQL injection and string encryption than a control group working without AI help — and were simultaneously more likely to rate their own code as secure. In other words, the assistant didn't just introduce bugs; it made developers less likely to go looking for them. This is the mechanism behind why AI-assisted code often skips a manual security review entirely: the developer's confidence went up exactly as the actual risk went up, a mismatch that no amount of "just review the AI's output" policy fully solves without tooling that checks the code independent of how sure the author feels about it.
Are open-source maintainers being overwhelmed by AI-generated reports and pull requests?
Yes, and it's consuming real triage capacity. curl maintainer Daniel Stenberg wrote in early 2025 that AI-generated vulnerability reports submitted through curl's HackerOne bug bounty program had become a significant share of total submissions, describing them as confident-sounding, plausible-looking write-ups that describe vulnerabilities that don't actually exist in the codebase. Because the reports are grammatically clean and cite real function names, they take a maintainer real time to disprove — time that used to go to reviewing actual patches. The same pattern shows up on the contribution side: maintainers across GitHub have reported a rise in pull requests generated by AI coding agents that pass CI and look complete but introduce subtle logic errors or unnecessary new dependencies pulled in to solve a problem that a one-line fix would have handled. For a security team, this means the volume problem isn't limited to code your own engineers write — it extends to every open-source dependency in your SBOM, where AI-generated contributions are now mixed into the commit history with no label distinguishing them from human-authored code.
How Safeguard Helps
Safeguard is built for exactly this shift — code that ships faster than it can be manually reviewed and carries risk that generic linting won't surface. Griffin AI, Safeguard's security-focused model, reviews AI-generated and human-written code alike for the injection, authorization, and dependency-confusion patterns that tools like Veracode's research show LLMs reproduce at scale, including flagging package references that don't resolve to a known, maintained registry entry — the exact signal that catches slopsquatting before it reaches a lockfile. Reachability analysis then determines whether a flagged vulnerability sits in code your application actually executes, so teams aren't drowning in theoretical findings from AI-hallucinated or dead-code paths. Safeguard generates and ingests SBOMs continuously as AI agents add and swap dependencies, keeping your bill of materials accurate even when a coding assistant introduces a new package mid-sprint. When a real issue is confirmed, Safeguard opens an auto-fix pull request with the corrected code and the reachability context attached, so the fix moves through review as fast as the AI-generated vulnerability did.