What are the main AI data security risks and which frameworks help?

AI systems create data risks at three points: data going in (prompts, uploads, training sets), data stored (embeddings, logs, fine-tuned models) and data coming out (answers that reveal more than they should). The NIST AI RMF, NIST's adversarial machine learning taxonomy and the OWASP LLM Top 10 each help organise controls.

Where does data move in an AI system?

Trace one request. A user types a question and attaches a document. The application adds instructions and retrieves passages from an internal index. All of that goes to a model, possibly hosted by a third party. The response comes back, is stored in a log, and may be fed into later training or evaluation.

Every arrow in that picture is a place for data to leak, be altered or be kept longer than intended. Teams that only think about "the model" miss most of them.

What are the main risks?

Leakage in prompts. Staff paste customer records, code or credentials into tools without approved terms. Once the data is with a third party, retention and training use depend on their agreement and your settings.

Sensitive information in outputs. A model trained or tuned on private data can repeat it. A retrieval system may return documents the asking user should not see if access control is applied to the index, not the user.

Poisoned or tampered data. NIST's AI 100-2 taxonomy covers attacks on training data and model privacy, including attempts to learn what was in the training set.

Over-trusted outputs. OWASP names improper output handling, where model output goes unchecked into other systems, and misinformation, where wrong output is trusted.

Weak inventory. If you do not know which datasets feed which models, you cannot respond to a deletion request or a breach notice accurately.

Which frameworks help, and with what?

The NIST AI Risk Management Framework is voluntary and organises work into governance, mapping context, measuring and managing risks. It suits setting policy and accountability.

NIST AI 100-2 is a taxonomy of attacks and mitigations, useful for threat modelling.

The OWASP Top 10 for LLM Applications is developer-oriented and lists concrete vulnerability classes with mitigations.

Privacy and security law sits alongside these. If personal data is involved, your privacy obligations still apply to how AI systems process it. Ask counsel what applies to you.

A constructed example

A company builds an internal assistant over its document library. It indexes everything, including HR files, because that was quicker. Access control exists on the file shares, but the index returns any matching passage to any employee. A staff member asks about "salary bands" and receives a manager's compensation notes.

The fix is to carry permissions into the retrieval layer, filter results by the asking user's rights, exclude restricted collections from the index, and log which documents were returned.

How to check or apply this

1. Draw the data flow for each AI feature: inputs, stores, models, outputs and logs.
2. Classify data and decide which classes may go to which models or providers.
3. Read the provider's terms on retention and training use, and set the account options accordingly.
4. Enforce user-level permissions at retrieval time. Never index data users cannot see.
5. Redact or tokenise sensitive fields before they reach a model when the task allows.
6. Control and log changes to training, tuning and evaluation datasets.
7. Set retention for prompts, outputs and embeddings, and make deletion cover them.
8. Treat model output as untrusted before it touches other systems.
9. Review the setup against NIST AI RMF functions once a year or when the design changes.

Quick check

Sources

• NIST AI Risk Management Framework
• NIST AI 100-2 E2025, Adversarial Machine Learning
• OWASP Top 10 for LLM Applications
• OWASP LLM05 Improper Output Handling
• OWASP LLM09 Misinformation

Last reviewed: 18 October 2026.