In September 2024, security researcher Johann Rehberger showed that a single malicious document could hijack ChatGPT's long-term memory. By exploiting a prompt-injection flaw in ChatGPT's browsing tool, he got the assistant to write attacker-controlled instructions into its persistent memory store — instructions that survived across sessions and quietly exfiltrated every future conversation to a remote server. OpenAI shipped a partial fix months later, but the underlying pattern has since shown up everywhere agents keep state: RAG knowledge bases, vector databases, MCP tool servers, and coding-assistant config files. This is memory and context poisoning — the practice of corrupting the data an AI agent trusts as its own history, knowledge, or instructions so that it behaves maliciously without ever having its model weights touched. As enterprises wire agents into CI/CD, ticketing, and codebases, poisoned memory is becoming a supply chain problem, not just a chatbot quirk.
What Is Memory and Context Poisoning in AI Agents?
Memory and context poisoning is an attack that injects false or malicious data into the information an AI agent retrieves, stores, or trusts, rather than attacking its model weights directly. Modern agents don't just respond to a single prompt — they pull context from retrieval-augmented generation (RAG) indexes, vector databases, conversation history, tool outputs, and increasingly from persistent "memory" features that carry facts across sessions (ChatGPT Memory, Anthropic's memory tool, LangChain's ConversationBufferMemory, MemGPT/Letta, and similar frameworks). If an attacker can write even a small amount of data into any of those stores — a poisoned support ticket, a manipulated PDF, a booby-trapped GitHub issue, a tampered vector embedding — the agent will treat it as ground truth the next time it plans an action. Unlike a one-shot prompt injection that dies at the end of a session, memory poisoning persists: it re-triggers every time the agent recalls that memory, which is what makes it a distinct and more dangerous category than classic jailbreaking.
How Did Attackers Actually Poison ChatGPT's Memory?
Attackers poisoned ChatGPT's memory in 2024 by hiding instructions inside a document the user asked the assistant to summarize, a technique Rehberger named "SpAIware." Because ChatGPT's bio tool could write to persistent memory as a side effect of processing untrusted content, the injected text told the model to remember a fake "instruction" to send a copy of all future chats, including uploaded files and images, to an attacker-controlled URL via a markdown image request. Because the payload lived in memory rather than in the current prompt, it survived across every new conversation the victim started afterward — no repeated exploitation needed. Rehberger disclosed this to OpenAI in May 2024; OpenAI initially closed it as "not a security issue," then quietly patched the exfiltration vector in the ChatGPT macOS app that September after the researcher published a proof of concept. The episode is now the canonical case study cited in nearly every enterprise AI red-team engagement involving memory-enabled assistants.
How Little Data Does It Take to Backdoor an Agent's Knowledge Base?
It takes remarkably little: the AgentPoison research published in early 2024 showed that poisoning as little as 0.1% of a RAG knowledge base — a handful of crafted documents among tens of thousands — achieved an attack success rate above 80% across agents built for autonomous driving, healthcare QA, and general-purpose tool use. The technique works by optimizing poisoned entries so their embeddings land in a distinct, low-density region of the vector space near a trigger phrase, guaranteeing they get retrieved whenever that trigger appears in a user query, while leaving the agent's behavior on benign queries almost unchanged (accuracy on clean inputs dropped by less than 1% in the study). That asymmetry — huge blast radius from a tiny, hard-to-detect payload — is exactly what makes RAG poisoning attractive to attackers and hard to catch with manual review: nobody is going to read all 50,000 documents in a support knowledge base looking for the five that were engineered to matter.
Why Are Long-Running, Tool-Using Agents Especially Exposed?
Long-running agents are more exposed because every additional tool call, memory write, and multi-step plan is another opportunity for untrusted data to become part of the agent's trusted context. A single-turn chatbot forgets everything when the session ends; an autonomous coding agent, SRE bot, or customer-support agent built on frameworks like LangChain, AutoGPT, or MemGPT is explicitly designed to carry state forward — reading its own prior notes, cached tool outputs, and retrieved documents as authoritative input to its next decision. In April 2025, Invariant Labs documented "tool poisoning attacks" against the Model Context Protocol (MCP), showing that a malicious MCP server could embed hidden instructions inside a tool's description field — text the human developer never sees in most clients, but that the agent reads and obeys when deciding how to call that tool. Because agentic frameworks chain multiple tool calls and memory lookups per task, a single poisoned tool description or memory entry can influence dozens of downstream actions, compounding the blast radius far beyond what a single bad prompt could do in a stateless chat.
What Attack Vectors Are Showing Up in Production Right Now?
The most active vectors today are indirect prompt injection through documents/emails, poisoned developer tooling, and cross-session memory contamination. In March 2025, Pillar Security disclosed the "Rules File Backdoor" attack, in which invisible Unicode characters and crafted instructions hidden inside .cursorrules and similar AI-coding-assistant configuration files caused agents like Cursor and GitHub Copilot to silently insert vulnerable or backdoored code into a project — a form of context poisoning aimed squarely at the software supply chain rather than the end user. Other observed and demonstrated vectors include: web pages and search results that an agent browses being crafted to plant instructions in its scratchpad memory; shared team knowledge bases (Confluence, Notion, Slack exports) that get indexed into a company RAG pipeline without content vetting; and multi-agent systems where one compromised agent's output becomes "trusted" input memory for a second agent, letting the poison spread laterally across an automation pipeline. All of these share the same root cause: agents are architected to treat retrieved and remembered content with the same trust level as a system prompt, and very few pipelines validate that content before it becomes memory.
Can Poisoned Memory Be Removed Once an Agent Has Learned It?
Removing poisoned memory is harder than removing a malicious dependency because most agent architectures have no equivalent of a lockfile or audit log for what entered memory and when. Deleting the offending row from a vector database or memory store stops future retrieval, but it doesn't tell you which past agent actions, generated code, tickets, or emails were already influenced by the poisoned entry while it was live — and if the agent summarized or paraphrased the poisoned content into a new memory of its own (a behavior observed in MemGPT-style architectures), the malicious instruction can survive the deletion of its original source. This is why security teams increasingly treat agent memory the way they treat build artifacts: something that needs provenance tracking, integrity verification, and rollback capability built in from day one, not bolted on after an incident.
How Safeguard Helps
Safeguard treats agent memory, RAG indexes, and MCP tool metadata as first-class supply chain assets — the same way it treats packages, containers, and CI pipelines — because a poisoned vector store or tool description is functionally a malicious dependency your agent pulls in at runtime. Our platform gives security teams the controls that memory-poisoning incidents like SpAIware and AgentPoison have shown are missing by default:
- Provenance tracking for memory writes: Safeguard fingerprints and logs every document, tool output, and conversation turn that gets written into an agent's persistent memory or vector store, so you can trace any downstream action back to the source content that produced it — closing the "what did the agent learn, and from where" gap that made the ChatGPT memory incident hard to fully scope.
- Ingestion-time scanning for RAG and vector pipelines: before content is embedded and indexed, Safeguard screens it for prompt-injection patterns, hidden Unicode payloads, and anomalous embedding placement consistent with techniques like AgentPoison's trigger optimization, flagging low-density or statistically unusual entries for review instead of letting them silently join the knowledge base.
- MCP and tool-definition integrity checks: Safeguard inventories connected MCP servers and tool schemas, diffs tool descriptions against known-good baselines, and alerts on hidden instructions or unexpected changes — directly addressing the tool-poisoning class of attack disclosed against MCP in 2025.
- Config and rules-file scanning in the SDLC: for coding-assistant configuration files (
.cursorrules, agent system prompts, CI-invoked agent configs), Safeguard runs the same supply chain integrity checks it applies to dependency manifests, catching Rules-File-Backdoor-style payloads before they reach a developer's assistant. - Rollback and quarantine for compromised memory: when a poisoned entry is identified, Safeguard supports quarantining the affected memory or index segment and rolling back agent state, rather than leaving teams to manually reconstruct what changed.
As agentic AI moves from demo to production across engineering, support, and security operations, the software supply chain no longer ends at the last line of code — it now includes everything an agent remembers and retrieves. Safeguard extends supply chain security to that layer so a single poisoned document can't quietly become a standing instruction your agents follow indefinitely.