Safeguard
Best Practices

Everybody's shipping code they can't read (AI-generated c...

AI coding assistants ship code fast, but studies show nearly half contains vulnerabilities, hallucinated packages, and leaked secrets nobody reviewed.

Aman Khan
AppSec Engineer
7 min read

In April 2024, security researcher Bar Lanyado published proof that ChatGPT regularly recommends npm and PyPI packages that don't exist — so he registered one of the hallucinated names himself. Within weeks it had racked up thousands of downloads from developers who never questioned the suggestion. That's not a hypothetical edge case. It's a preview of what happens when code generation outpaces code review. GitHub reports over 1.8 million paid Copilot subscribers, and Google's 2024 DORA report found more than 75% of developers now use AI tools daily. Y Combinator's own president said publicly in March 2025 that a quarter of that batch's startups had codebases that were roughly 95% AI-written. Teams are shipping more code, faster, from a tool with no memory of their threat model, their compliance obligations, or their last incident. The real question isn't whether AI is writing your code — it's whether anyone is still reading it before it ships.

Why is everyone suddenly shipping AI-generated code?

Because AI coding assistants went from novelty to default tooling in under three years. Stack Overflow's 2024 Developer Survey put AI-tool usage at 76% of respondents, with 82% specifically using it to write code. GitHub's own telemetry shows Copilot now suggests roughly 46% of the code in files where it's enabled, and internal GitHub research claims developers accept close to 30% of those suggestions without edits. That adoption curve outran every prior tooling shift in software history — faster than cloud migration, faster than open source itself. The result is a huge and growing share of production code that was never typed by a human who understood the full context of the system it's landing in, only accepted by one who was moving quickly and trusted the tool.

Does AI-generated code actually introduce more vulnerabilities?

Yes, measurably, according to multiple independent studies rather than vendor marketing. Veracode's 2025 GenAI Code Security Report tested over 100 large language models across 80 curated coding tasks and found that 45% of the generated code samples failed baseline security tests — with Java the worst performer at over 70% failure. Critically, the models weren't ignorant of secure alternatives; when a secure and insecure pattern were both available, they picked the insecure one nearly half the time. Stanford researchers reached a related but distinct conclusion in their widely cited 2023 study "Do Users Write More Insecure Code with AI Assistants?": developers given Codex-style assistants produced measurably less secure solutions across most of the study's tasks, and were significantly more confident their code was safe. That combination — more vulnerabilities, more confidence — is the exact recipe for flaws that skip review entirely.

Why can't code review catch what AI writes?

Because pull request volume has grown faster than reviewer attention, so review has become skimming rather than reading. GitClear's 2024 analysis of 153 million changed lines across enterprise repos found that copy-pasted code exceeded refactored code for the first time since the study began tracking it in 2020 — a direct byproduct of developers pasting AI output rather than composing it. At the same time, average pull request size has grown while the time reviewers spend per line has not; reviewers increasingly approve large, AI-authored diffs on the strength of "it compiles and passes tests," not because they traced the logic. A human reviewer who wrote every line of a function understands its assumptions. A human reviewer approving an AI's 200-line function they didn't write is, functionally, rubber-stamping a black box.

Can AI code introduce dependencies and secrets nobody approved?

Yes, in two distinct and well-documented ways: hallucinated packages and leaked training-data patterns. A 2025 USENIX Security paper, "We Have a Package for You!", tested 16 code-generation models across more than 576,000 generated samples and found that 19.7% of recommended packages didn't exist at all — a phenomenon researchers now call "slopsquatting." Open-source models hallucinated fake package names 21.7% of the time versus roughly 5.2% for commercial models, and the same hallucinated names recurred across repeated queries often enough that attackers can predict and pre-register them, exactly as Lanyado demonstrated. Separately, GitGuardian's 2024 State of Secrets Sprawl report logged 12.8 million secrets exposed on public GitHub in 2023 alone, and flagged AI-generated boilerplate as a growing contributor, since assistants trained on public repositories reproduce credential-shaped patterns — placeholder-looking keys, hardcoded tokens, sample connection strings — that get shipped as-is because they read as "just example code."

Where do AI-focused scanners like Aikido fit — and where do they stop?

Tools like Aikido Security are useful precisely because they consolidate SAST, SCA, and secrets scanning into one lightweight dashboard built for fast-moving dev teams, and that convenience is real — but scanning code after it's written only tells you what's wrong with a snapshot, not whether the pipeline that produced and shipped it can be trusted. A point-in-time scanner can flag a vulnerable function or a leaked key; it generally can't tell you whether a given commit came from a human, an agent, or an AI tool with elevated permissions, whether the build artifact matches the source it claims to, or whether that artifact carries a verifiable attestation by the time it reaches production. As AI-authored commits become a larger share of every repo, "did the scanner find a bug" is a narrower question than "can we prove what shipped and how it got there" — and that provenance gap is where scanning-only tools consistently run out of runway.

That gap matters more than it sounds like on paper. A dashboard full of open findings is still a snapshot of code that has already merged, already built, and in many pipelines already deployed — the scanner is telling you what to fix, not preventing what shouldn't have shipped in the first place. For a team relying on AI assistants to write a third or more of its codebase, after-the-fact detection increasingly means after-the-fact incident response.

How Safeguard Helps

Safeguard is built around the assumption that a growing share of your code was never fully understood by the human who merged it, and treats that as a supply chain problem, not just a code quality problem. Concretely, Safeguard:

  • Flags AI-authored and AI-assisted commits at the PR level so reviewers and policy gates know when extra scrutiny is warranted, instead of treating every diff as equally human-reviewed.
  • Ties SBOM and build provenance together, verifying that what actually built and deployed matches signed, attested source — closing the gap that pattern-matching scanners leave between "we scanned the code" and "we know what shipped."
  • Cross-checks new dependencies against real registry data in CI, catching hallucinated or newly-registered packages — the exact slopsquatting pattern researchers demonstrated — before they're pulled into a build.
  • Enforces identity-aware secrets and policy gates in the pipeline itself, so credential-shaped strings and risky patterns are blocked at merge and build time, not discovered in a dashboard after the fact.
  • Maintains the audit trail SOC 2 and enterprise buyers actually ask for, mapping every artifact back to its source, its build, and its approvals — evidence that scanning findings alone don't provide.

AI is not going to slow down, and neither should your team. But "we ran a scanner" and "we can prove what we shipped" are different guarantees, and only one of them survives an audit, an incident, or a supply chain attack traced back to a package nobody remembers approving. Safeguard is built for teams that need the second guarantee — full provenance and trust across the pipeline, not just a list of findings after the code is already written.

Never miss an update

Weekly insights on software supply chain security, delivered to your inbox.