Wrong answers become a security issue when someone acts on them. OWASP lists Misinformation as LLM09: false or misleading output that appears credible, which can lead to security breaches, reputational damage and legal liability. The controls are grounding, verification, human review and honest interface design.
What OWASP describes
The main cause is hallucination, described as content that seems accurate but is fabricated. Biased training data and incomplete information also contribute. A related risk is overreliance, where users trust output without checking.
OWASP's examples include an airline chatbot that gave travellers wrong information and was successfully sued, fabricated legal citations that caused problems in court, and models that suggest insecure or nonexistent libraries.
The security angle
The library example is the sharpest. OWASP describes attackers publishing malicious packages under names that coding assistants commonly hallucinate. A developer who installs the suggested name without checking pulls in attacker code.
The other examples show the business side: an answer that sounds official can commit a company to something, expose it to a claim, or mislead a user on a safety topic. No attacker is needed for any of that.
Controls that reduce it
OWASP lists several:
- Ground answers with retrieval from verified sources.
- Cross-verify and use human oversight, especially for critical information.
- Validate key outputs automatically where stakes are high.
- Communicate limits and label AI generated content.
- Use secure coding practices so incorrect code suggestions do not become vulnerabilities.
- Design the interface with content filters, reliability notes and a defined field of use.
- Train users on limits and the need to verify.
Practical rules for engineering teams
- Verify any package name a model suggests exists, is the one you intend, and is maintained, before installing it.
- Require citations for factual claims in customer facing answers, and check that the cited text supports the claim.
- Do not let a model make commitments, such as refunds or contractual statements, without a defined approval path.
- Measure accuracy on your own questions and track it over time.
What this is not
Grounding reduces hallucination. It does not remove it. Treat it like a quality control with an error rate, and design for the cases where it fails.
Safeguard's dependency scanning and SBOM tooling can flag unexpected or unknown packages in a project, which is one safety net against hallucinated dependencies.
Where to start this week. Pick one place where your organisation relies on model output and trace what happens if it is wrong. Add one check: a package existence test in the developer workflow, a citation check in the support bot, or a human review step for commitments. Small, concrete checks fit into delivery. A vague policy asking people to be careful does not.
Quick check
Does RAG solve hallucination?
It helps by grounding answers in retrieved sources, but OWASP pairs it with verification and human oversight, so it is not a complete fix.
Sources
Last reviewed: 19 October 2026.
