GitHub Copilot now writes an estimated 46% of code in files where it's enabled, according to GitHub's own 2023 disclosure, and Copilot has more than 1.3 million paid subscribers across 50,000+ organizations. That scale is the problem. When an autocomplete engine trained on public repositories — bugs, secrets, and all — suggests code to a developer, it reproduces whatever patterns were common in its training data, insecure ones included. A 2021 Stanford/NYU study found that 40% of programs generated by Copilot across 89 security-relevant scenarios contained exploitable vulnerabilities. A 2023 follow-up found developers using an AI assistant not only wrote less secure code, they were more confident it was safe. Individually, each of these is a bad suggestion accepted by one developer. Multiplied across a 200-person engineering org shipping dozens of PRs a day, it's a systemic, self-reinforcing supply chain risk that traditional code review wasn't built to catch.
How often does Copilot suggest insecure code?
Roughly 40% of the time in security-relevant contexts, based on the most cited academic study on the subject. Researchers Pearce, Ahmad, Karnwal, Dolan-Gavitt, and Karri published "Asleep at the Keyboard?" at IEEE S&P 2022, testing Copilot against 89 scenarios mapped to MITRE's top 25 CWEs, across C, Python, and Verilog. Of the roughly 1,700 programs Copilot generated, 39.33% were vulnerable to at least one CWE in scope — with especially high rates for CWE-798 (hardcoded credentials), CWE-476 (NULL pointer dereference), and CWE-79 (cross-site scripting). The study's authors noted Copilot doesn't distinguish "secure" from "commonly written" — it optimizes for what's statistically likely to follow a given prompt, and a lot of what's statistically likely on GitHub is code nobody security-reviewed before committing.
Does using an AI assistant make developers overconfident in bad code?
Yes — a controlled study found users of an AI coding assistant wrote measurably less secure code while rating their own code as more secure than a control group. NYU researchers Perry, Srivastava, Kumar, and Boneh ran a 2023 user study (published at IEEE S&P) with 47 participants split between an AI-assisted group and an unassisted control group, tasked with writing code for five security-relevant programming problems including SQL query building and string encryption. The AI-assisted group produced more insecure solutions on most tasks and was significantly more likely to self-report that their code was secure when it wasn't. This is the dangerous part for engineering orgs: it's not just that Copilot introduces bugs, it's that it erodes the human skepticism that normally catches them, because a fluent, confident-looking suggestion reads as "reviewed" even when it's the opposite.
Can an AI coding assistant recommend a package that doesn't exist?
Yes, routinely — and attackers are already exploiting it. Large language models "hallucinate" plausible-sounding package names that were never published, a phenomenon researchers call package hallucination and that's now commonly referred to as slopsquatting. A 2024 USENIX Security study by Spracklen et al. tested 16 code-generating LLMs across 576,000 code samples and found an average hallucination rate of 5.2% for commercial models and 21.7% for open-source models, with some models hallucinating the same fake package name repeatedly and predictably across runs. Attackers have taken notice: security researchers at Vulcan Cyber (now part of Tenable) demonstrated in 2023 that they could register a hallucinated package name Copilot-style assistants suggested, publish malicious code under it, and have it get installed by developers who trusted the AI's suggestion enough to skip verifying the package existed on the real registry first. Once that package lands in a lockfile, it ships to production with every downstream build.
Does AI-generated code increase duplicated and unreviewed code across a codebase?
Yes — code churn and copy-pasted code have both risen sharply since AI assistants went mainstream. GitClear's 2024 "AI Copilot Code Quality" report, which analyzed 211 million lines of changed code from 2020 through 2023, found that the share of code classified as "copy-pasted" (versus moved, refactored, or newly authored) rose from 8.3% in 2021 to a projected 12.3% in 2024 — the first sustained increase GitClear had measured in its multi-year tracking. The same report found "code churn" — code reverted or substantially rewritten within two weeks of being committed — on pace to double compared to its pre-Copilot baseline. More copy-pasted, less-refactored code correlates directly with more duplicated vulnerabilities: the same insecure pattern Copilot suggested once tends to reappear across files and services rather than getting centralized into a single, reviewable, patchable function.
Which vulnerability classes show up most often in AI-suggested code?
Injection flaws, hardcoded secrets, and weak cryptography dominate, based on both the Pearce et al. and subsequent industry testing. In the Stanford/NYU dataset, the highest-risk CWEs by suggestion volume were CWE-798 (use of hardcoded credentials), CWE-259 (hardcoded password), CWE-327 (broken or risky crypto algorithm), CWE-79 (XSS), and CWE-89 (SQL injection) — the same categories that have topped OWASP's Top 10 and MITRE's CWE Top 25 for over a decade. That's not coincidental: these are the patterns most densely represented in the public GitHub repositories, Stack Overflow snippets, and tutorial code that trained the underlying models, meaning Copilot isn't inventing new vulnerability classes so much as it's industrializing the reproduction of old, well-understood ones at a pace and volume no manual review process was sized for. A single accepted suggestion with a hardcoded API key or a string-concatenated SQL query can propagate into a dozen services before a security team ever sees a pull request.
How Safeguard Helps
Safeguard is built for exactly this shift — where the volume of AI-generated code has outpaced what manual security review can triage. Our reachability analysis determines whether an AI-suggested vulnerable pattern, like a hardcoded secret or an injection-prone query, is actually exploitable in your running application, so teams stop drowning in findings for dead code paths and focus on what attackers can actually reach. Griffin AI, Safeguard's detection engine, is trained specifically to recognize the insecure patterns common in Copilot- and other LLM-generated code — hallucinated dependencies, weak crypto defaults, and hardcoded credentials among them — and flags them before merge rather than after deployment. Safeguard also generates and ingests SBOMs across your build pipeline, so a hallucinated or slopsquatted package introduced by an AI suggestion is caught against known-good dependency graphs instead of silently entering your supply chain. When an issue is confirmed and reachable, Safeguard opens an auto-fix PR with the remediated code, closing the loop between "an AI wrote something insecure" and "a human has to remember to fix it" — because in a world where nearly half of new code is AI-suggested, that loop has to be automated, too.