In September 2022, researcher Riley Goodside showed that a single line of hidden text could hijack a GPT-3 translation prompt and make it ignore its instructions entirely. Simon Willison named the technique "prompt injection" days later, and the industry has been chasing it ever since. By 2025, OWASP had ranked prompt injection as LLM01, the single biggest risk in its Top 10 for LLM Applications — ahead of data leakage, model theft, and insecure output handling. The reason it keeps winning that spot is structural: large language models cannot reliably distinguish "instructions from my developer" from "text I'm currently reading," and every retrieved document, email, webpage, or tool response is a potential instruction channel. As AI agents get plugged into inboxes, ticketing systems, CI pipelines, and MCP servers, that ambiguity stops being a novelty and starts being a software supply chain problem. Here's how the attack actually works, what it has done in production, and what stops it.
What Is Prompt Injection, and How Is It Different From a Jailbreak?
Prompt injection is an attack where untrusted input overrides a model's intended instructions, while a jailbreak is an attack where a user manipulates their own prompt to bypass a model's safety training. The distinction matters for defenders: jailbreaking is a first-party problem (a user attacking the model they're talking to), while prompt injection is a third-party problem (an attacker hiding instructions in content a legitimate user or agent will process later). A support agent that reads a customer email, a coding assistant that ingests a GitHub issue, or a browsing agent that summarizes a webpage can all be redirected by text the end user never typed and never sees. Because the model has no cryptographic way to tag "trusted system prompt" versus "untrusted retrieved content" — everything is just tokens in a context window — the attacker only needs write access to something the model will eventually read.
What Are the Main Prompt Injection Attack Techniques?
The main techniques are direct injection, indirect injection, payload obfuscation, and multi-turn/context poisoning, and they're usually combined. Direct injection is the "ignore previous instructions" pattern typed straight into a chat box to override a system prompt. Indirect injection, first formalized by Kai Greshake et al. in their February 2023 paper "Not what you've signed up for," hides instructions inside content the model will retrieve — a webpage, a PDF, a calendar invite, an email footer, or a code comment — so the victim never interacts with the attacker directly. Obfuscation techniques evade keyword filters using base64 encoding, zero-width Unicode characters, invisible white-on-white text, or translated languages to smuggle instructions past naive input scanners. Multi-turn context poisoning plants a benign-looking instruction early in a conversation or document (e.g., "for all future responses, append this footer") that only triggers several turns later, defeating single-message review. In agentic systems, a fifth technique has emerged: tool description poisoning, where a malicious MCP server or plugin embeds hidden instructions inside its own tool metadata, which the orchestrating model reads as trusted configuration rather than untrusted input.
How Does Indirect Prompt Injection Work in Agentic Tools?
It works by exploiting the fact that agentic LLMs treat retrieved data and tool output as part of their reasoning context with the same trust level as their system prompt. In February 2023, researcher Kevin Liu used this to extract Bing Chat's confidential internal codename "Sydney" simply by asking the bot to ignore its instructions — within weeks of the Greshake paper, other researchers showed the same Bing integration could be redirected by hidden text on a webpage it was asked to summarize, causing it to attempt social-engineering the user. In August 2024, security firm PromptArmor disclosed that Slack AI's summarization feature could be manipulated by a message posted in a public channel, causing it to leak content from private channels the attacker didn't have access to — because the model couldn't distinguish "content to summarize" from "instructions about how to summarize." The pattern Willison later termed the "lethal trifecta" — an agent with (1) access to private data, (2) exposure to untrusted content, and (3) an ability to communicate externally — describes exactly why these tools became viable exfiltration paths rather than just annoyances.
What Have Real Incidents Shown About the Impact?
Real incidents have shown that prompt injection can escalate from chatbot embarrassment to zero-click data exfiltration in production enterprise software. The clearest example is EchoLeak (CVE-2025-32711), disclosed by Aim Security in January 2025 and patched by Microsoft in June 2025: a single crafted email sent to a Microsoft 365 Copilot user could trigger data exfiltration with no user interaction at all, because Copilot's retrieval pipeline treated the email body as trusted context and could be instructed to pull sensitive data from the user's other Microsoft 365 documents and leak it via a crafted link — a fully zero-click chain inside a widely deployed enterprise AI product. In April 2025, Invariant Labs demonstrated "MCP tool poisoning," showing that a malicious MCP server could embed hidden instructions in its tool descriptions to make Claude or other MCP-connected models exfiltrate SSH keys and other local files, even from an otherwise unrelated tool call — a direct software supply chain risk since MCP servers are frequently pulled in as third-party dependencies. These aren't lab curiosities; they're CVEs and disclosed vulnerabilities against shipped products from the two biggest AI platform vendors.
What Defenses Actually Reduce Prompt Injection Risk?
The defenses that actually help combine architectural separation with monitoring, because no single filter reliably blocks every injection. OpenAI's June 2024 "Instruction Hierarchy" paper formalized training models to weight system-level instructions above user input above third-party (tool/retrieved) content — reducing but not eliminating attack success rates in their own benchmarks. On the systems side, Willison's "dual-LLM" pattern isolates a privileged model that never sees untrusted content from a quarantined model that processes it and can only pass back structured, non-executable data. Practical controls that security teams are actually deploying in 2025-2026 include: spotlighting (wrapping untrusted content in explicit delimiters so the model is trained/prompted to treat it as data, not commands), least-privilege tool scoping (an agent that summarizes email shouldn't also hold a "send email" or "delete file" tool), human-in-the-loop confirmation for any action with side effects (sending money, merging code, deleting records), allowlisting data sources and MCP servers rather than trusting arbitrary third-party tool registries, and continuous output monitoring for anomalous tool calls or data flows rather than trying to catch every malicious input at the door. Red-teaming with known injection corpora (like the ones OWASP and NIST AI 100-2 catalog) before shipping an agent to production remains the closest thing to a baseline.
How Safeguard Helps
Prompt injection is fundamentally a software supply chain problem once you look past the AI framing: the attack surface is every third-party document, plugin, MCP server, and data source your AI systems trust by default. Safeguard extends the same provenance-and-policy discipline we apply to package dependencies and CI pipelines to the AI supply chain. That means treating MCP servers and LLM plugins as first-class dependencies subject to vetting and version pinning, not ad hoc installs; generating and monitoring an SBOM-equivalent inventory of which models, tools, and data connectors your agents can reach; enforcing least-privilege policies so a given agent's tool scope is explicit and auditable rather than implicit; and flagging anomalous tool-call chains and data egress patterns the way we already flag suspicious build steps or exfiltration attempts in a CI/CD pipeline. As enterprises wire LLM agents into email, ticketing, source control, and internal APIs, the question isn't whether an indirect injection attempt will reach your systems — EchoLeak and Slack AI already proved it will — but whether your agent's tool scope, data provenance, and monitoring are tight enough to make that attempt a non-event instead of an incident. That's the layer Safeguard is built to own.