Log metadata for every request and store full prompt text only for a small sample, a flagged subset or an opt-in debug window. Redact before writing, restrict who can read the logs, and set a short retention. Full capture of all prompts turns your logs into a copy of your customers' data.
Three levels of detail
Metadata only. User or tenant ID, feature, model, token counts, latency, status, provider request ID, filter decisions. Safe to keep for a long time and enough for cost and reliability work.
Sampled content. Store the full prompt and reply for a small percentage of requests, chosen at random, plus all requests that failed or were flagged. Enough to find most quality problems.
Full capture. Every prompt and reply. Useful during a launch or an investigation, and risky as a standing setting.
Start at the first level and move up only with a reason and an end date.
Why full capture is a risk
Prompts hold whatever users type: names, health details, source code, credentials pasted by mistake. OWASP's sensitive information disclosure page lists personal data, financial and health records, business data and security credentials among what an LLM application can expose, and includes sanitization and access controls in its prevention list. Logs are one of the places that data ends up and is exposed from.
Logs also tend to flow into other tools: search indexes, dashboards, tickets and backups. Each copy needs the same controls.
Redact at write time
Strip sensitive patterns before the log line is written, not when someone reads it. Typical targets: email addresses, phone numbers, card numbers, government IDs, API keys and tokens, and anything your product knows is sensitive, such as account numbers. Pattern-based redaction misses things, so treat it as a reducer, not a guarantee. See what log data to redact before sending to an LLM and what never to log.
Redaction at read time has a weakness: the raw data still sits in storage.
Control who reads
- A named group, not everyone with log access
- Access requests that expire
- An audit trail of who opened prompt content
- No prompt content in shared chat channels or tickets, which copy it beyond your control
Customer data in logs may also be subject to your contract, including any promise about staff access. Check your data processing terms.
Retention
Set separate retention for the levels. Metadata may stay for months. Content should expire in days or weeks. Write the number down, enforce it with automation and test that deletion works, including in backups and search indexes.
Debug sessions with consent
For hard bugs, let a support engineer request full capture for one account, for one day, with the customer's agreement. Log that the session happened. Turn it off automatically.
Quick check
Can I use logged prompts to build an evaluation set?
Only with care. Use synthetic or consented, redacted examples, and check that your terms allow that use.
Sources
Last reviewed: 19 October 2026.
