CWE-1427 is the weakness behind prompt injection. The product uses externally provided data to build prompts for a large language model, and the way they are built makes the model fail to distinguish user supplied input from the developer's system directives.
If you build a chatbot, a retrieval augmented tool, an agent or any feature that sends text to an LLM, this is yours.
How it happens
A system prompt carries the developer's guidance. When external data lands in the same prompt, an attacker can insert instructions in plain human language, or use special characters or tags, to make the model ignore the original guidance. MITRE says LLMs treat all instructions as legitimate, so often there is no way for the model to tell which language is malicious.
It also reaches beyond the chat box. Many LLM systems pull in data from adjacent products or external sources such as Wikipedia through API calls and retrieval augmented generation (RAG). MITRE says any external source that may hold untrusted data should be considered potentially malicious.
# Vulnerable pattern: user text spliced into the instructions.
prompt = (
"You are a support bot. Never reveal the discount code.\n"
f"Customer says: {user_text}"
)
# Better: keep roles separate and treat user text as untrusted data.
messages = [
{"role": "system", "content": "You are a support bot. Never reveal the discount code."},
{"role": "user", "content": user_text},
]
Role separation is not a complete defense. MITRE rates differentiating input from system prompting as moderate, so it needs company.
The record's observed examples:
- CVE-2023-32786: an LLM integration framework where prompt injection can force the service to retrieve data from an arbitrary URL, effectively SSRF.
- CVE-2024-5184: an ML based email analysis product that lets a malicious user inject a direct prompt and take over the service logic, forcing it to leak its hard coded system prompt.
- CVE-2024-5565: a library that generates SQL through LLMs using RAG, where the prompt function used to present results could be altered by prompt injection.
- CVE-2024-48746: an AI based business intelligence dashboard integration where prompt injection through the natural language component allowed arbitrary code execution.
What an attacker gains
MITRE says consequences are entirely contextual, depending on the system around the model.
- Confidentiality. Extracting sensitive information from the model, such as the system prompt.
- Integrity. Modifying application data or running unauthorized code or commands, to the extent the use case allows.
- Access control. Reading or modifying data and assuming privileges, again depending on the use case.
The lesson from the examples: the model's reach is the blast radius. SSRF, code execution and SQL all came from what the model was allowed to trigger.
How to find it
MITRE recommends dynamic analysis using known prompt injection techniques, adjusted to the model or system, with manual or automated interpretation. It also says design review helps but works best with dynamic analysis. Because LLMs are not deterministic, run the same test several times, and repeat testing whenever you switch models or update weights.
How to fix it
MITRE's mitigations, with ratings where given:
- Sanitize and label user input. Design the application so user controllable input is identified as untrusted and cannot carry misleading or dangerous characters. Rated high.
- Differentiate user input from system prompting when building prompts. Rated moderate.
- Train with examples that avoid leaking secrets and disregard malicious inputs, and label training data. Test repeatedly.
- Monitor from outside. During operation, use components outside the model, called supervisors or guardrails, to moderate its output.
- Fine tune the model to better control and neutralize dangerous inputs.
Pair these with least privilege for anything the model can call. A guardrail that sits between the model and its tools limits what a successful injection can do.
Try it yourself
This demo shows a guardrail between an AI assistant and its tools.
Click through it at your own pace, or open it full screen.
Quick check
Why can an LLM not just ignore malicious instructions?
MITRE says LLMs are designed to treat all instructions as legitimate, so the model often cannot tell developer directives from attacker text.
Is retrieved content safe if it comes from a trusted site?
MITRE says any external source that may contain untrusted data should be considered potentially malicious, including data fetched through RAG.
Why test the same prompt more than once?
Prompting is non deterministic, so a single pass proves little. MITRE also says to retest when the model changes or its weights are updated.
