A package hallucination happens when a coding assistant recommends a library that does not exist, or invents an import for one that does. It is harmless until someone installs it. If an attacker has registered that name, the install runs their code. The defence is checking packages before they enter the build.
Why do language models invent packages?
A language model predicts likely text. It does not look up a registry. When you ask for code that parses a niche file format, it produces a name that fits the pattern of real package names, such as a plausible prefix joined to a plausible noun. OWASP describes hallucinations in its LLM misinformation entry as the model filling gaps in training data with statistical patterns, producing answers that sound correct but are unfounded. It lists unsafe code generation, including suggestions of non-existent libraries, as a named example.
That makes the failure different from a normal bug. The code looks reasonable, compiles in the reader's head, and the only signal that something is wrong is a registry lookup that nobody ran.
Why is this a supply chain problem and not just an annoyance?
Public registries let anyone publish a name that is free. If a model repeatedly suggests the same missing name, a publisher can claim it and wait. The next developer or agent that follows the suggestion installs the attacker's package and any install scripts it carries.
Agents raise the stakes. A person sees a failed install and stops. An autonomous coding agent may read the error, pick another name, and keep going, or may install first and test afterwards. The more steps a workflow does without a human reading, the more important an automatic gate becomes.
Be careful with numbers. Rates of hallucinated names vary by model, prompt and language, and published figures come from specific studies. Do not quote a percentage as a general truth. Measure your own environment instead.
What does a hallucinated dependency look like?
A constructed example. A developer asks an assistant for a Python helper that converts a proprietary log format. The reply includes import logfmt_convert_pro and the line pip install logfmt-convert-pro. The name looks right, the description in the reply is confident, and it has never been published by anyone you trust.
Five checks would expose it in two minutes. The package is brand new or absent. It has no source repository link. It has one release and no history. Its maintainer account is new. Its name sits one character away from a well-known project.
How to check or apply this
- Treat every package name from a model as untrusted input, the same as a name from a forum post.
- Before the first install, look the name up on the registry. Check publish date, release history, maintainers, linked source and download trend.
- Compare against what your project already uses. A real need usually maps to an existing, well-known library.
- Pin versions and keep a lockfile with hashes so a name cannot silently resolve to something new.
- Use an internal registry or proxy that only serves approved packages, and let new names require review.
- Add a policy gate in the pull request that flags packages younger than a set age or with no verifiable source.
- Scan the dependency graph on every change, including transitive packages that the new one pulls in.
- For agents, require human approval for any dependency addition, and block install scripts by default where your tooling allows it.
- Keep a list of known-good internal names reserved on public registries to prevent confusion attacks.
Can tools help?
Yes. Software composition analysis can inventory what was added and compare it against vulnerability and malware data. Safeguard offers dependency scanning, policies and gates that can block unreviewed packages in a pull request, and a free in-browser scan at gold.safeguard.sh/scan covering up to 100 dependencies. A scan cannot tell you that a name is a hallucination, but it can show you a package that is new, unknown or flagged before it ships.
Quick check
Is a hallucinated package the same as a typosquat?
No. A typosquat imitates an existing name with a small spelling change. A hallucinated package is a name the model invented that never existed. The attack on both is the same: register the name and wait for installs.
Does lockfile pinning prevent this?
It prevents silent drift after you have vetted a package, but not the first install of a bad name. Vetting the new dependency is still required.
Sources
- OWASP Top 10 for LLM Applications 2025, LLM09 Misinformation
- NIST Secure Software Development Framework, SP 800-218
- SLSA supply chain framework
Last reviewed: 18 October 2026.
