OWASP's LLM09 (Misinformation) describes a model producing false or misleading content that looks credible. The controls that work are grounding answers in verified sources, validating high-stakes output automatically, and keeping trained humans in the loop. A disclaimer in the footer, or a bigger model on its own, does not fix it.
What does LLM09 actually cover?
OWASP defines it as false or misleading information that appears credible. Hallucination is the main cause, where the model fills gaps with statistical patterns and produces something that sounds right but is unfounded. Biased or incomplete training data adds to it.
The entry also names overreliance. People trust the output, skip verification, and carry the error into a decision or a build. The risk is a pair: the model is wrong, and the reader does not check.
OWASP lists four example types: factual errors, unsupported claims, misrepresented expertise, and unsafe code generation. The last one matters most to engineering teams. A model can suggest an insecure library, or one that does not exist.
Why is this a security problem and not just a quality problem?
Because a wrong answer can be turned into an access path. OWASP's first attack scenario describes people probing coding assistants for package names the models commonly invent, then publishing malicious packages under those names. A developer who trusts the suggestion installs it.
No clever prompt is needed. The assistant did what it normally does, and the supply chain absorbed the result.
The second scenario has no attacker at all. A company ships a medical chatbot without checking accuracy and gets sued. Reputational and legal exposure arrive without anyone attacking you.
Which controls work?
OWASP lists eight mitigations. In practice they sort into three tiers.
Controls that change the output. Retrieval-augmented generation pulls answers from trusted, verified sources, which narrows the room to invent. Fine-tuning and prompting techniques can reduce incidence. These lower the rate but never take it to zero, so they cannot be your only layer.
Controls that catch errors before they land. Automatic validation of key outputs is the strongest one for high-stakes flows. If the model proposes a dependency, check that it exists in your registry and passes policy before anything installs. If it cites a regulation or a contract clause, check the citation against the source text. OWASP also calls for secure coding practices so incorrect suggestions do not become vulnerabilities.
Controls that shape the human. Human oversight works only if reviewers are trained to avoid overreliance. Interface design helps: label AI-generated content, state the intended field of use, and say plainly where reliability is limited. OWASP also lists risk communication and user training.
Which controls do not work on their own?
These are our judgment calls, not OWASP findings, but they follow from its text.
- A generic disclaimer. OWASP asks for specific limits and field-of-use statements, not a blanket line nobody reads.
- A rubber-stamp reviewer. A person who approves everything adds delay, not assurance. That is the overreliance problem with a name attached.
- Trusting a newer model to be fixed. Hallucination comes from how models generate text, per OWASP. Treat it as a standing property and design around it.
- Checking only at the end. If a wrong package name is already installed, review comes too late.
What to do
- List every place a model output drives an action: a dependency install, a ticket, a customer reply, a compliance answer.
- Rank them by cost of being wrong. Start controls at the top of the list.
- Add automated validation where an output can be checked against a source of record, such as a registry, a policy, or a document.
- Ground answers in verified sources and show which source was used.
- Train reviewers on what typical failures look like, and sample their approvals to confirm they are really checking.
- Label AI-generated content and state its limits for the specific use.
- Gate package installs suggested by assistants through the same policy as any other dependency.
Quick check
Does retrieval-augmented generation remove hallucination?
No. OWASP lists it as a mitigation that improves reliability by using verified external sources. It lowers the risk but does not remove it, so pair it with validation and review.
How is LLM09 connected to the software supply chain?
OWASP's example describes attackers publishing malicious packages under names that coding assistants tend to invent. Verifying that a suggested package is real and passes policy before install closes that path.
Sources
Last reviewed: 18 October 2026.
