Security teams keep discovering that the fastest way into an AI-powered application isn't a buffer overflow or a SQL injection payload — it's a sentence. In June 2024, JFrog researcher Tong Liu disclosed CVE-2024-5565, a prompt injection flaw in the Vanna.AI text-to-SQL library rated 8.1 (High) by NVD, that let an attacker turn a natural-language question into arbitrary Python execution on the host server. Two months later, security firm PromptArmor showed that Slack AI could be manipulated into leaking data from private channels using instructions hidden inside a public message. Neither exploit touched a traditional software bug. Both exploited the same root cause: large language models can't reliably distinguish trusted developer instructions from untrusted user or document content. That confusion is called a prompt injection vulnerability, and OWASP now ranks it the number-one risk facing LLM applications. Here's how it works, why it's so hard to patch, and what to do about it.
What Is a Prompt Injection Vulnerability?
A prompt injection vulnerability exists when an LLM application processes untrusted text as if it were trusted instructions, letting an attacker override the developer's intended behavior. Every LLM application works by concatenating a system prompt (the developer's rules), user input, and often retrieved external content (documents, emails, web pages, API responses) into a single text blob before sending it to the model. The model has no structural way to tell these three sources apart — it just sees tokens. If an attacker can get their text into any of those three inputs, they can potentially write instructions the model will follow with the same authority as the developer's original prompt. OWASP's Top 10 for LLM Applications 2025 codifies this as LLM01:2025 Prompt Injection, and MITRE ATLAS tracks the underlying technique as AML.T0051. Unlike SQL injection, there's no parameterized-query equivalent that eliminates the class entirely — the model itself is the parser, and it was trained on natural language, not a strict grammar.
How Does Direct Prompt Injection Differ From Indirect Prompt Injection?
Direct prompt injection means the attacker types the malicious instruction straight into the chat box; indirect prompt injection means the attacker plants it in content the model retrieves later, such as a webpage, PDF, or email. The clearest public example of direct injection is Microsoft's Bing Chat launch in February 2023, when Stanford student Kevin Liu typed "Ignore previous instructions" into the chat and got the bot ("codename Sydney") to print its confidential system prompt in full. Indirect injection is more dangerous because the victim never sees the payload. Security researcher Johann Rehberger demonstrated this in 2023 against early ChatGPT plugins: a hidden instruction embedded in a webpage, rendered as a markdown image URL, caused the model to silently exfiltrate the user's conversation history to an attacker-controlled server the moment the user asked the assistant to summarize that page. The Slack AI case from August 2024 followed the same pattern — an attacker posted an innocuous-looking message in a channel they had access to, containing instructions that Slack AI later executed when a victim with access to private channels ran a query, causing it to surface private data in its answer.
Why Is Prompt Injection Ranked the Top LLM Security Risk in 2025?
Prompt injection sits at #1 on the OWASP Top 10 for LLM Applications 2025 list because it is the entry point for nearly every other downstream attack against an AI system, not a self-contained bug. Once an attacker controls what the model "believes" its instructions are, they can chain that control into data exfiltration (as in the Slack AI and ChatGPT plugin cases), remote code execution (as in Vanna.AI), privilege escalation through overly-trusted tool calls, or denial of wallet by forcing expensive tool loops. The category held the #1 spot in OWASP's original 2023 list too, and two years of public disclosures — Vanna.AI, Slack AI, EmailGPT, Google Bard's indirect injection via Google Docs (reported by Embrace The Red in late 2023), and dozens of CTF-style jailbreaks against production chatbots — have not produced a general fix. Vendors have shipped mitigations like input/output filtering and instruction hierarchies (OpenAI's "system > developer > user" precedence introduced in 2024), but independent red teams continue to bypass them within days of release, which is why the risk remains rated as high-severity and unresolved industry-wide.
What Real-World Exploits Show Prompt Injection Escalating to Code Execution?
The clearest escalation path is CVE-2024-5565, where a prompt injected into a natural-language query to Vanna.AI's SQL-generation library caused the application to execute attacker-supplied Python inside its plotly/matplotlib chart-rendering step, giving the attacker code execution on the backend server. The library was designed to convert questions like "show me sales by region" into SQL and a chart; because the generated visualization code was passed to Python's exec() without sandboxing, an attacker could simply ask a question crafted to make the model generate malicious rendering code instead of a chart. NVD scored it 8.1 (High) precisely because no authentication was required beyond normal application access — a low bar most internal analytics tools don't defend against. This is the pattern security teams should generalize from: anywhere an LLM's output feeds into a code interpreter, shell command, SQL execution engine, or file write operation, a successful prompt injection converts a chat feature into an RCE primitive.
How Can Security Teams Detect and Reduce Prompt Injection Risk?
Security teams reduce prompt injection risk by treating every LLM input — user text, retrieved documents, tool outputs, API responses — as untrusted, and by removing the model's ability to directly trigger high-impact actions. Concretely, that means: never pass LLM-generated code, SQL, or shell commands to an execution function without sandboxing and allowlisting (the exact control missing in CVE-2024-5565); apply least-privilege API keys and scoped OAuth tokens to any tool the model can call, so a successful injection can't reach data or systems beyond what that specific feature legitimately needs; run adversarial red-team prompts — including indirect payloads hidden in test documents — against every LLM feature before it ships, the way Embrace The Red and PromptArmor did against production systems; and audit the dependency chain of LLM orchestration frameworks (LangChain, LlamaIndex, Vanna.AI-style wrappers) for known CVEs, since many of 2023-2024's disclosed prompt injection escalations lived in third-party glue code rather than the model itself. Static prompt filtering and "ignore instructions that contradict the system prompt" guardrails help but are bypassable; the durable fix is architectural — assume injection will succeed sometimes, and limit what a successful injection can actually do.
How Safeguard Helps
Safeguard treats LLM orchestration libraries and AI-agent frameworks the same way it treats any other dependency in your software supply chain: something to inventory, assess for exploitability, and fix fast when a flaw like CVE-2024-5565 lands. Our SBOM generation and ingest pipeline flags vulnerable versions of libraries such as Vanna.AI, LangChain, and LlamaIndex the moment they enter your codebase, while reachability analysis determines whether your application actually invokes the vulnerable code path — including unsandboxed exec() calls or tool-execution functions — so your team isn't stuck triaging every CVE with equal urgency. Griffin AI correlates that reachability signal with how the LLM feature is wired into your infrastructure to prioritize the injection-to-RCE chains that matter most. When a fix is available, Safeguard opens an auto-fix PR with the patched dependency version pinned, so remediation ships in a pull request instead of a backlog ticket.