To keep an LLM app from leaking personal or confidential data, sanitise what goes in, give the model only the data the current user may see, lock down system level settings, tell users what not to enter, and test for cross-user exposure. These map to OWASP's guidance on Sensitive Information Disclosure.
What counts as sensitive
OWASP lists PII, financial details, health records, confidential business data, security credentials and legal documents. It also includes proprietary training methods and source code. If your model can see it, assume your model can say it.
The checklist
Data going in
- Scrub or mask sensitive content before it is used in training or fine tuning.
- Apply input validation to catch sensitive or harmful inputs.
- Consider tokenisation or pattern based redaction of sensitive fields before processing.
Access
- Apply least privilege to the data the application can reach.
- Limit the model's access to external data sources.
- Retrieve data in the context of the requesting user, so another person's records never enter the prompt.
Configuration
- Limit users' ability to override or read system level settings.
- Check that error messages and configuration do not expose internals. OWASP points to its API security misconfiguration guidance for this.
People and policy
- Tell users not to enter sensitive information.
- Publish clear data retention and usage policies.
- Let users opt out of training where that applies.
Do not rely on the prompt to hide data
OWASP notes that system prompt restrictions on what the model may return are not always honoured and can be bypassed with prompt injection. A line telling the model not to reveal addresses is not data protection. If the address should not be revealed, it should not be in the context.
Test for the scenarios OWASP names
- A user receives another user's data because of inadequate sanitisation. Test with two accounts and shared caches.
- An attacker bypasses input filters to extract information. Run adversarial prompts.
- Training data leaks. Review what was included in tuning sets and who approved it.
Record the results. A launch review that produces evidence is easier to defend than one that lives in a meeting.
Safeguard's compliance and evidence automation can collect and store that kind of evidence alongside your other controls.
Who signs off. Assign owners. Data access is usually an engineering call, retention terms a legal or privacy call, and user notices a product call. A checklist with no names beside the items tends to be ticked by whoever is closest. Hold a short review before launch where each owner states what they verified and how, and keep that record with the release.
Quick check
Is anonymising training data enough?
It helps, but OWASP also lists access controls, configuration, user education and retention policy. Treat it as one control of several.
Sources
Last reviewed: 19 October 2026.
