In September 2023, researchers at Stanford ran a controlled study that should have set off alarms across the industry. They gave developers a set of programming tasks — some with access to an AI coding assistant, some without. The group using AI wrote measurably less secure code across most tasks, including classic flaws like SQL injection and improper input sanitization. But here's the part that matters more than the vulnerability count: the AI-assisted developers were also more confident their code was secure than the developers who wrote everything by hand. That gap between how secure code actually is and how secure a developer believes it to be has a name now: the AI code trust gap. It's showing up in dependency files, pull requests, and production incidents, and it's growing faster than most security teams can track it.
Why do developers trust AI-generated code more than they should?
Developers trust AI-generated code more than they should because fluency reads as correctness. Large language models produce code that is syntactically clean, well-commented, and confidently formatted — it looks like the work of a careful senior engineer, even when the underlying logic is wrong. The Stanford study (Perry et al., "Do Users Write More Insecure Code with an AI Assistant?") found this effect directly: participants who used OpenAI's Codex-based assistant were significantly more likely to say their code was secure, while actually introducing more vulnerabilities than the control group across five of six programming tasks. This isn't a one-off finding. GitHub's own 2022 research on Copilot found the tool reproduced insecure coding patterns roughly 40% of the time in security-relevant scenarios, a figure that has only partially improved as models have scaled. The trust gap persists because nothing in the developer experience signals uncertainty — no wavy red underline for "this authentication check is subtly wrong."
What does the research actually show about AI-written code quality?
The research shows AI-written code compiles and runs more often than it gets reviewed properly. A 2024 GitClear analysis of over 150 million changed lines of code found that the share of code added via "copy-paste" (rather than moved or refactored) had roughly doubled since 2021, coinciding with the mainstream adoption of AI coding assistants — a strong proxy for developers accepting suggestions wholesale instead of integrating them thoughtfully. Snyk's 2023 AI Code Security Report surveyed over 500 developers and found that while 96% were using AI coding tools in some capacity, roughly 80% admitted to bypassing security policies specifically to ship AI-generated code faster, and more than half said they rarely or never checked AI suggestions against a known vulnerability database before merging. The tools are fast. The review discipline hasn't caught up to the pace they enable.
Can AI actually invent dependencies that don't exist?
Yes, and it's already being weaponized. A 2024 academic study from researchers at the University of Texas at San Antonio, Virginia Tech, and the University of Oklahoma tested 16 popular code-generating models against 576,000 code samples and found that commercial models hallucinated non-existent package names in about 5.2% of generated code, while open-source models did so in roughly 21.7% of cases — collectively producing over 200,000 unique hallucinated package references. Attackers have already caught on. The technique, now called "slopsquatting," involves registering the exact hallucinated package name on npm or PyPI so that the next developer who copies an AI suggestion and runs npm install or pip install pulls down malicious code instead of a 404. Because the hallucinated names are often plausible-sounding variants of real, popular libraries, they slip past casual review — the trust gap doesn't stop at logic errors, it extends into the supply chain itself.
Why do AI models keep repeating the same insecure patterns?
AI models repeat insecure patterns because they were trained on the internet's actual code, including the mistakes. Public repositories are full of hardcoded credentials, deprecated cryptographic functions, unparameterized SQL queries, and outdated dependency versions — and a model trained to predict "what code plausibly comes next" has no built-in mechanism for preferring the secure minority pattern over the common majority one. NYU researchers analyzing Copilot's suggestions across 89 different scenarios in 2021 found that about 40% of the completions contained vulnerabilities from the MITRE CWE top-25 list, including CWE-798 (hardcoded credentials) and CWE-89 (SQL injection). Newer models have narrowed this gap through reinforcement learning and safety fine-tuning, but they haven't closed it, because the underlying training distribution hasn't fundamentally changed — the internet is still full of code written before anyone thought about it being used to train a model that ships to production.
How is the trust gap showing up in real incidents, not just studies?
It's showing up as a measurable rise in secrets and vulnerable dependencies actually reaching production. GitGuardian's 2024 State of Secrets Sprawl report found over 12.7 million hardcoded secrets exposed on public GitHub in 2023 alone, a 28% year-over-year increase that tracks closely with the growth curve of AI-assisted commits. Separately, in early 2024 several security researchers publicly demonstrated that asking mainstream coding assistants for "a simple login function" or "a script to connect to this database" frequently produced code with plaintext password storage or missing TLS verification — problems a junior developer might also introduce, but now happening at the volume and speed of an entire team's output, funneled through a handful of models. The result isn't a new category of vulnerability. It's the old categories, arriving faster, in more repositories, reviewed by fewer human eyes per line, because the code "looks done."
Is slower, more careful AI adoption actually the answer?
No — the answer is verification infrastructure that doesn't depend on developer vigilance holding up under deadline pressure. Every study cited above points to the same root cause: humans are bad at maintaining skepticism toward output that looks polished, and AI coding assistants are exceptionally good at producing polished output regardless of correctness. Telling developers to "review AI code more carefully" is the security equivalent of telling users to "pick better passwords" — true, and largely ineffective at scale, because it fights human psychology instead of building around it. The organizations closing the trust gap aren't the ones with the strictest AI usage policies; they're the ones that put automated, non-negotiable checks between "AI suggests it" and "it ships."
How Safeguard Helps
Safeguard treats every AI-generated contribution the same way it treats any other untrusted input to the software supply chain: verify before it merges, not after it ships. That means scanning every commit and pull request — regardless of whether a human or a model wrote it — for hardcoded secrets, known-vulnerable dependency versions, and CWE-mapped insecure code patterns before it reaches a protected branch. It means validating every new package reference against real, resolvable registry metadata, so a hallucinated dependency name gets flagged as suspicious the moment it appears in a lockfile, closing the exact door that slopsquatting attacks rely on. And it means giving security and engineering leaders visibility into how much of their codebase is AI-assisted and where that code sits relative to their actual risk posture, so the confidence gap between "this code looks fine" and "this code is verified" gets closed with evidence instead of vibes. AI coding assistants aren't going away, and they shouldn't — they're a genuine productivity gain. But productivity gains that outrun verification just move risk downstream. Safeguard exists to make sure that trade doesn't happen silently.