LLMs are useful reviewers when you treat them as fast, fallible assistants. They are not a replacement for human review, a prompt cannot make them immune to injection, and a confident tone says nothing about correctness. Here are five myths that cause trouble, with what the published guidance says.
Myth 1: a good enough prompt removes hallucinations
Fact. Anthropic's guidance lists techniques that reduce hallucinations: permission to say "I don't know", quote grounding, citation checks, and comparing runs. It also states that these do not eliminate them and that critical information should still be validated. Build a verification step into the workflow, as in the checklist for verifying LLM findings.
Myth 2: telling the model to ignore malicious instructions stops prompt injection
Fact. OWASP states that no fully reliable prevention method is known, and its measures are aimed at reducing impact. The UK NCSC explains why: unlike SQL, a model has no hard boundary between data and instructions, so the risk may never be fully mitigated and should be managed through design. It also advises being skeptical of vendors who claim to stop prompt injection.
Myth 3: a deny list of attack phrases is a reasonable defense
Fact. The NCSC advises against relying on deny lists of known attack phrases because they are easy to bypass. OWASP says input filters are useful but not sufficient on their own.
Myth 4: if the AI wrote it and the AI reviewed it, it is double checked
Fact. Two passes by similar systems share blind spots. A Stanford study found that participants with an AI assistant wrote significantly less secure code and were more likely to believe their code was secure. A model reviewing a model's output can repeat the same wrong belief. Independent checks such as tests, static analysis, and human review are what add information. For more on judge style checks, see is an LLM judge guardrail enough.
Myth 5: if the model is accurate in a demo, it is safe to automate
Fact. Accuracy and safety are different questions. OWASP's excessive agency risk is about what the system can do when the model is wrong or manipulated: too much functionality, permission, or autonomy. A very accurate triage bot that can also close tickets and change settings is still dangerous if one input can steer it. Constrain the actions, and put human approval on consequential ones.
What the fair version sounds like
- LLMs are good at breadth, summarizing, drafting, and prompting you to look at places you might skip.
- They are weak at knowing your environment, confirming reachability, and being consistent run to run.
- Use them to propose, and use tools and people to decide.
- Measure them on your own code and keep a record.
A way to test any claim
When someone says an AI tool is accurate or safe, ask three things. What was it measured on, and is that like your code? What can it do if it is wrong? What happens if text it reads is hostile?
Quick check
Are these myths specific to one vendor's model?
No. They come from how language models work and from general guidance, so they apply across providers.
Sources
- Reduce hallucinations (Anthropic)
- LLM01:2025 Prompt Injection (OWASP GenAI)
- LLM06:2025 Excessive Agency (OWASP GenAI)
- Prompt injection is not SQL injection (UK NCSC)
- LLM Prompt Injection Prevention Cheat Sheet (OWASP)
- Do Users Write More Insecure Code with AI Assistants? (arXiv)
Last reviewed: 19 October 2026.
