In June 2024, researchers from the University of Texas at San Antonio, Virginia Tech, and the University of Oklahoma published "We Have a Package for You!," a study later accepted at USENIX Security 2025 that generated 576,000 code samples across 16 LLMs in Python and JavaScript and checked every recommended package against real registries. The result: 19.7% of recommended packages did not exist — 205,474 unique hallucinated names — with open-source models hallucinating at 21.7% on average versus 5.2% for commercial ones, and GPT-4 Turbo posting the lowest rate of any tested model at 3.59%. JavaScript code hallucinated packages more often than Python (21.3% versus 15.8%). That gap between "code that compiles and looks idiomatic" and "code that is safe to merge" is now a daily review problem, not a hypothetical one. This post is a checklist — organized around the three failure modes that show up most often in AI-generated pull requests: injection flaws, insecure defaults, and hallucinated dependencies — for reviewers who need something more concrete than "read it carefully."
Why do hallucinated packages matter more than a typo?
They matter more because hallucinations are exploitable at scale, not one-off mistakes. The USENIX study found that 58% of hallucinated package names recurred when the same prompt was run again — meaning the hallucination isn't random noise, it's a predictable output an attacker can pre-register. Of the hallucinated names, roughly 51% were pure fabrications, about 38% were conflations that blended two real package names into something plausible-sounding, and around 13% were typo variants of real packages. Python Software Foundation developer-in-residence Seth Larson coined the term "slopsquatting" (popularized after a conversation with Ecosyste.ms creator Andrew Nesbitt) for the resulting attack: register the exact name an LLM is known to hallucinate, publish malicious code under it, and wait for pip install or npm install to pull it in automatically. Socket.dev and Aikido Security have both documented real slopsquat registrations riding on this pattern. A reviewer's job is to treat every new dependency in an AI-authored diff as unverified until proven otherwise — not just plausible-looking.
What should a reviewer actually check before approving a new dependency?
Before approving any package a diff introduces, verify it exists on the actual registry (not just that the import statement looks syntactically correct), check its publish date and maintainer history, and confirm it isn't a near-miss of a popular name. A genuinely useful heuristic from the hallucination research: fabricated and conflated names often sound like they should exist — python-json-logger-utils blending two real packages, for instance — which is exactly why lexical familiarity is a bad signal. Concretely: run pip index versions <package> or check npmjs.com directly rather than trusting the AI assistant's or IDE's autocomplete, which may be trained on the same hallucination-prone data. Flag anything published in the last 30 days with no repository URL, sparse version history, or a name within edit distance of 1-2 from a top-1000 package — these are the same signals Safeguard's Eagle malware classifier and typosquat-similarity scoring apply automatically, and they map directly onto what a human reviewer should eyeball by hand when Eagle isn't in the loop.
What insecure defaults show up most often in AI-generated code?
The most common defaults are missing input validation, disabled TLS verification, overly permissive CORS or IAM policies, and weak or absent parameterization in database queries — patterns that compile cleanly and pass a quick glance because they're syntactically standard, not because they're safe. This isn't speculation: a 2023 Stanford study by Perry, Srivastava, Kumar, and Boneh ("Do Users Write More Insecure Code with AI Assistants?," presented at ACM CCS) found that developers using an AI coding assistant produced measurably more insecure code across tasks like string encryption, SQL sanitization, and path handling than a non-AI-assisted control group — and, more troublingly, those developers were more likely to rate their own code as secure. Reviewers should specifically scan generated code for verify=False in HTTP clients, string-concatenated SQL instead of parameterized queries, eval() or exec() on any input that traces to a request body, and default-open network policies — because an LLM optimizing for "code that runs" has no signal telling it that "code that runs" and "code that resists exploitation" are different bars.
Where does injection risk get introduced that a diff alone won't show?
Injection risk often lives outside the diff entirely — in the prompts, system instructions, and retrieved context that produced the code, not just the code itself. OWASP's Top 10 for LLM Applications lists Prompt Injection (LLM01) as its top risk precisely because an AI coding assistant that ingests untrusted content — a README, an issue comment, a scraped API response — during code generation can be steered into emitting attacker-influenced logic without any human noticing the instruction was ever there. For a code reviewer, this means treating AI-authored code that touches external content ingestion, templating, or command construction with the same suspicion as hand-written code in those categories: check that user- or third-party-controlled strings never reach subprocess.run(shell=True), raw SQL, or template rendering without explicit escaping, regardless of how clean the surrounding code looks. The fact that a human didn't type the vulnerable line doesn't change whether CWE-78 (OS command injection) or CWE-89 (SQL injection) applies to it.
How should a checklist be enforced instead of just followed?
A checklist only works if failing it blocks the merge automatically, because manual review discipline degrades under deadline pressure exactly when AI-generated volume is highest. That means wiring dependency-existence checks, typosquat-similarity scoring, and known-malicious-package classification into CI as a gate, not a suggestion. Safeguard's guardrails engine ships a built-in "Typosquat detected" rule with a BLOCK effect enforceable at CI, registry, and Kubernetes admission — catching a package name within edit-distance 2 of a top-1000 library before it ever reaches a lockfile — and Eagle, Safeguard's malware classification model, scores newly-published, no-repository-URL, aggressively-versioned packages as suspicious or malicious automatically. Neither is a hallucination detector by name, but both catch the exact artifact a slopsquatting attack depends on: a real, malicious package sitting at a name an LLM was likely to suggest. Reachability analysis then confirms whether the AI-introduced dependency is even called, so a flagged package that never executes doesn't block a release unnecessarily.
What's the minimum viable review process for teams shipping AI-generated code today?
The minimum viable process is three gates applied to every AI-touched diff: verify every new dependency against the live registry before merge, run a taint-aware SAST pass that traces external input to injection sinks rather than pattern-matching function names, and require a human sign-off on any change to authentication, authorization, or network-egress defaults. None of these are exotic — they're the same controls mature teams already run on human-written code — but AI-generated code needs them applied with less trust in surface plausibility, because fluency and correctness are no longer the same signal. The USENIX findings make the economics explicit: with 205,474 known-hallucinated names already catalogued and 58% of them reproducible on demand, the cost of an attacker pre-registering a handful of high-probability names is trivial compared to the blast radius of even one of your build pipelines resolving to it.
How Safeguard Helps
Safeguard's deep dependency scanning resolves every package in an AI-generated diff against real registries up to 100 levels of transitive depth, cross-referencing resolved names against internal registries and known typosquat patterns via Eagle before a hallucinated or slopsquatted name ever reaches a lockfile. Eagle's typosquat-similarity scoring (edit-distance ≤2 against the top-1000 packages per ecosystem) and metadata-anomaly detection — newly-published, no repository URL, aggressive versioning — catch the exact publishing pattern a slopsquat attack relies on, whether Eagle was updated before or after the malicious package landed. And guardrails enforcement lets a team turn "Typosquat detected" into a BLOCK effect at CI, registry, or admission, so a hallucinated dependency introduced by an AI coding assistant is stopped automatically rather than depending on a reviewer noticing an unfamiliar package name in a fifty-file diff.