To test an LLM app for sensitive information disclosure, plant known canary values in the places the model can read, ask questions as users who should not see them, and check outputs, logs and error messages for leaks. OWASP LLM02 gives you the categories to cover. Test with your own data and permissions, never someone else's.
What does LLM02 cover?
OWASP defines sensitive information in LLM systems as personal, financial, health, business and credential data, plus proprietary model details. It lists three common vulnerabilities: personal information leaking during interactions, exposure of proprietary algorithms or training data, and confidential business data appearing in generated responses.
OWASP's example scenarios are simple. A user receives another user's personal data because of inadequate sanitization. A user gets around input filters to extract sensitive information. Sensitive data included carelessly in training later appears in output.
OWASP also warns that system prompt restrictions help but are not sufficient, since they may not always be honored and could be bypassed through prompt injection or other methods. So the test plan should assume instructions alone do not hold.
How do you build a test plan?
Think in four surfaces, and make a canary for each.
Training and fine-tuning data. If you fine-tune, seed a unique harmless string into a test dataset and check whether the model repeats it to unrelated users. OWASP's mitigation of keeping user data out of the training model supports this check.
Retrieval and connected data. Create documents with different permission levels, each holding a unique marker phrase. Ask questions as users without access and confirm the marker never appears.
Conversation memory and logs. Check whether one user's earlier chat can surface in another user's session, and whether logs store sensitive input in plain text.
System configuration. Review whether error messages, debug output or prompts reveal keys, internal URLs or settings. OWASP points to API security guidance on avoiding configuration leaks through errors.
For each, run the same questions as at least three roles: an unauthenticated user, a normal user and an admin. Compare results against what policy says each role may see.
What do you do with findings?
Rank them by what leaked and to whom. A model repeating a user's own data to that user is a minor design point. A model repeating one customer's data to another is a major incident. Then fix at the right layer: permissions in retrieval, sanitization before training, redaction on output, and shorter retention.
What to do
- Write down the classes of sensitive data your app touches and where each enters.
- Create canary values for each class, using fake data only.
- Test across roles, including a user who should see nothing.
- Check training data intake for personal data and credentials before any fine-tune.
- Apply least privilege on every data source the model can reach.
- Redact or tokenize sensitive fields on output where the app does not need them.
- Tell users not to enter sensitive information, and publish retention and training use terms, as OWASP suggests.
- Review logs and error messages for secrets and personal data.
- Repeat after every model, prompt or data source change.
Quick check
Does a system prompt saying "never reveal secrets" protect us?
Not on its own. OWASP says such restrictions may not always be honored and can be bypassed. Keep secrets out of the model's reach and enforce access in code.
What is a canary value?
A unique fake string you plant in a data source so you can tell if it escapes. If the canary shows up where it should not, you have found a leak path without exposing real data.
Sources
- OWASP GenAI: LLM02:2025 Sensitive Information Disclosure, fetched 18 October 2026.
Last reviewed: 18 October 2026.
