Secrets reach a model's context window the same ways they reach chat or tickets: someone pastes a log, an agent reads an environment file, a tool returns a config dump, a retrieved document contains a key. Once there, they can persist in transcripts, logs and caches. Prevent entry, limit retention, and rotate fast.
Why is this a distinct problem?
A prompt looks like a private conversation, so people paste freely. In practice a prompt may be stored by the application, forwarded to a hosted provider, written to debug logs, or replayed into later requests. OWASP lists sensitive information disclosure among its top risks for LLM applications, including exposure of credentials through inputs, outputs and training or retrieval data.
The UK NCSC and partner agencies' guidelines for secure AI development make a related point: protect the data your AI system handles across design, development, deployment and operation, not only at one stage.
Where do secrets slip in?
Four routes cover most cases.
Human paste. A developer drops a stack trace containing an authorization header into an assistant.
Agent file access. A coding agent reads a project directory and picks up an environment file or credentials file.
Tool output. A tool that prints configuration, runs a command with verbose logging or returns an HTTP response with tokens.
Retrieved content. A wiki page, ticket or repository chosen for retrieval holds a key someone committed years ago.
Each route has a different fix. That is why one policy sentence rarely works.
What persists, and where?
Look beyond the model provider. Check conversation history in your own application, observability tools that capture request bodies, error trackers, evaluation datasets built from real traffic, and shared transcripts. A secret that is gone from the chat window may still sit in a log index with a long retention period.
A constructed example
An engineer asks a coding assistant to debug a failing deployment and pastes the pipeline log. The log contains a cloud access key, printed by a script with debug output on. The assistant's answer is fine. The key is now in the assistant's history, in the team's shared transcript export and in the application's request log. The cleanup is not deleting one message. It is revoking the key, finding every copy and fixing the script that printed it.
What to do
- Assume anything that enters a prompt may be stored, and set retention for transcripts and request logs accordingly.
- Scan inputs before they reach the model and outputs before they are stored, using the same secret patterns you use on repositories.
- Keep credentials out of agent reach: exclude environment and credential files from default access.
- Replace long lived keys with short lived credentials scoped to one task.
- Turn off verbose or debug output that prints headers and tokens in pipelines.
- Redact secrets in tool results before returning them to the model.
- Write a one page response plan: revoke, find copies, rotate, review access logs.
- Train people on the plan, not just the rule, because speed of rotation limits harm.
Safeguard includes secrets scanning and an MCP Guardrail that can sit between agents and tools. Use them as one layer, together with the habits above.
Quick check
Is deleting a chat message enough after pasting a secret?
No. Copies may exist in logs, exports and caches. Revoke and rotate the secret first.
Should agents read environment files?
By default, no. Exclude them and grant specific, short lived access when a task needs it.
Sources
- OWASP GenAI, LLM02 Sensitive Information Disclosure
- UK NCSC, Guidelines for secure AI system development
Last reviewed: 18 October 2026.
