For OWASP LLM09 misinformation, log each consequential answer with its prompt, retrieved sources, model version and whether a person reviewed it. Alert on answers with no supporting source, on package names that do not exist, and on high-impact actions taken without review. Misinformation is a reliability failure that can become a security one.
What does OWASP say?
LLM09:2025 describes misinformation as an LLM producing false or misleading content that appears credible. Hallucination is a major cause, bias and incomplete data add to it, and overreliance magnifies the harm. OWASP lists factual inaccuracies, unsupported claims, misrepresentation of expertise and unsafe code generation among the risks. Its mitigation list includes grounding answers with retrieval augmented generation, human review for high-stakes content, automatic validation of key outputs, labelling AI content, secure coding practices for AI-suggested code and user training.
Which decisions need logging?
Not every chat does. Start with answers that someone may act on: advice shown to customers, summaries used in decisions, code suggestions merged into a repository and generated text sent outward under your name. For each, you want to reconstruct what the model saw and what happened next.
What to log
- The request, with the user or service identity and the feature.
- The model name and version, plus the system prompt version.
- The retrieved sources, with identifiers, if you use retrieval.
- The answer, or a reference to it.
- Any validation result, such as a citation check or a package existence check.
- Whether a person reviewed it, who and the outcome.
- The downstream action: sent, merged, published or discarded.
Keep these where the application cannot alter them.
Which alerts are worth having?
No support for a factual claim. For features that must cite sources, alert when an answer contains claims with no matching retrieved passage. This needs a grounding check, which can itself be imperfect, so tune and sample.
Package that does not exist. OWASP describes attackers who find package names that coding assistants frequently hallucinate and publish malicious packages under those names. Alert when a suggested dependency is missing from your approved registry or has very recent creation, and require review before it enters a manifest.
High-impact action without review. An answer that triggered a refund, a configuration change or an outbound message with no human step where policy requires one.
Drop in grounded answers. A sudden rise in ungrounded answers after a model or index change.
Repeated user corrections. Users flagging the same answer as wrong, which signals a source or prompt problem.
Label missing. Generated content shown to users without the AI label your policy requires.
What should you test beyond alerts?
Keep a fixed set of questions with known answers and rerun it after every change in model, prompt or data. Track the share answered correctly and the share where the model said it did not know. Add a set of questions about your own products, since that is where wrong answers do the most harm.
What about coding assistants?
Treat AI suggested code as untrusted input. Run the same scans, reviews and dependency checks as for any contribution, and block unknown packages. Safeguard's Gold registry and free in-browser scan at gold.safeguard.sh/scan can help you check a package name before you install it.
What to do
- List the features whose answers people act on.
- Add the log fields above for those features.
- Add a grounding check or citation requirement where facts matter.
- Block dependencies that are not in your approved registry until reviewed.
- Require human review for high-impact actions and log it.
- Rerun a fixed evaluation set after every model, prompt or data change.
Quick check
Can we eliminate hallucinations?
No. OWASP's mitigations reduce and catch them. Design for detection and review.
Is misinformation really a security issue?
It can be. A hallucinated package name can lead to malicious code, and a wrong answer can expose a company to harm.
Sources
Last reviewed: 18 October 2026.
