Treat every identifier an AI assistant cites as unverified until you have opened the primary record and checked that its title and description match the claim. Models can produce identifiers that look valid but point to something else, or to nothing. A two-minute check per reference prevents wrong tickets and wrong advice.
The failure mode
A language model generates plausible text. An identifier such as a CVE number or a CWE number is a pattern the model has seen often, so it can produce one that fits the shape without being correct. Common slips: a real identifier attached to the wrong product, a correct weakness class with the wrong number, or a made-up advisory that reads convincingly.
OWASP names this risk category LLM09:2025 Misinformation, where a model produces false or misleading content that appears credible.
A four-step check
- Open the primary record. For a weakness, use the MITRE CWE page. For a vulnerability, use the CVE record or the national database entry for that identifier.
- Compare the name. Does the title match what the assistant said? For example, CWE-89 is "Improper Neutralization of Special Elements used in an SQL Command ('SQL Injection')", and CWE-798 is "Use of Hard-coded Credentials". If an assistant calls CWE-798 an injection flaw, something is off.
- Compare the affected product and versions. A real record for a different product is the commonest error. Match the package name, the version range and the fixed version.
- Compare the severity source. If the assistant quotes a score, find the source and the scoring version. CVSS v4.0 groups metrics into Base, Threat, Environmental and Supplemental, and the specification says the Base score reflects intrinsic severity, not complete risk.
What a mismatch tells you
| Mismatch | Likely cause | Action |
|---|---|---|
| Identifier does not exist | Invented | Discard the claim, ask for a source |
| Exists, wrong product | Confused with another record | Search by product and version yourself |
| Right product, wrong version range | Out of date or guessed | Use the record's range |
| Right record, wrong summary | Paraphrase drift | Read the record, not the summary |
| Score differs from the record | Different version or source | Cite the record's score and date |
Make the assistant easier to check
- Ask it to give a link to the record next to every identifier, then open the link yourself. Links can be invented too, so do not trust a link you did not open.
- Give it the facts instead of asking it to recall them. Paste the advisory text, or connect it to a tool that returns real scan data, and ask it to explain, not to remember.
- Ask what it is unsure about. Models often answer honestly when asked directly.
Build it into the workflow
If a ticket or report is created from an assistant's answer, add a required field for the verified record link. A reviewer can then see at a glance that someone checked. For reports that go to customers, see AI incident summaries: verify before you send.
Test your assistant
Ask ten questions where you know the answers, such as "what is CWE-798?" and "which CWE covers SQL injection?" Count wrong answers. Then ask about three obscure identifiers and see whether it admits uncertainty or invents detail. That tells you how much to trust it on harder ones.
Quick check
Is an identifier that exists in the database proof the claim is right?
No. It proves the identifier exists. You still need to check that the record describes the product and flaw the assistant claimed.
Can I trust the assistant if it gives a link?
Only after you open the link and check the content. Links can be wrong as well.
Safeguard offers an MCP server so an assistant can query real scan data instead of recalling it, and a free scan at gold.safeguard.sh/scan.
Sources
Last reviewed: 19 October 2026.
