To reduce invented findings from an LLM security review, give it explicit permission to say it is unsure, make it quote the code before it reasons, ask it to find supporting evidence for each claim and retract what it cannot support, and compare several runs. None of this removes false positives, but it makes them cheaper to catch.
Why models invent findings
A model asked to "find vulnerabilities" is being asked to produce vulnerabilities. If the code is clean, the easiest way to satisfy the request is to stretch. Phrases such as "potential issue" and "could be exploited if" fill the gap.
Anthropic's hallucination guidance lists the countermeasures, and they map well onto review work.
Technique 1: allow "I don't know"
Anthropic says that explicitly giving the model permission to admit uncertainty can drastically reduce false information. In a review prompt, that looks like: "If the code does not clearly contain a problem in scope, answer no findings. Do not speculate."
Also define the empty result. A prompt that never mentions the possibility of zero findings quietly demands at least one.
Technique 2: quote first, then analyze
For long inputs, the guidance recommends extracting word for word quotes first and basing the analysis only on those quotes. In practice, split the task into two steps in one prompt: list the lines that handle untrusted input, then say which of them are unsafe and why.
A finding that points at a quote can be checked in seconds. A finding that describes code in general terms cannot.
Technique 3: make each claim auditable
Anthropic describes a retraction pass: after the first answer, ask the model to find a direct supporting quote for every claim, and remove any claim it cannot support. You can run this as a second prompt over the first output. It is cheap and it catches the findings that rest on code that is not there.
Technique 4: run it more than once
Best-of-N verification means running the same prompt several times and comparing. Findings that appear in every run deserve attention first. Findings that appear once may still be real, but inconsistency is a signal to read that spot yourself.
Keep the cost in mind. Repeat runs multiply token use, so reserve them for high risk code such as authentication, payment, and file handling.
What none of this fixes
A confident wrong answer that is consistent across runs is still wrong. The model may misunderstand a framework behavior, or not know that a middleware already sanitizes the input. Only a human, a test, or a deterministic tool can settle those.
Track your own numbers. For a month, record each LLM finding as confirmed, rejected, or unclear. You will learn which categories the model handles well and which it does not, and you can then narrow the scope accordingly.
For dependency findings, whether a vulnerable function is actually called is a reachability question that a language model should not be asked to guess. Reachability analysis in Safeguard answers it from the code and call graph, and you can try a free scan at gold.safeguard.sh/scan.
Quick check
Is a lower temperature enough to stop invented findings?
No. It makes output more repeatable, not more correct. The OWASP cheat sheet lists lower temperature among partial mitigations for repeated-attempt attacks, and it is not a complete fix there either.
Sources
Last reviewed: 19 October 2026.
