Before you ship a model or a RAG feature, ask fifteen questions about where its data came from, who can change it, how you would notice tampering and how you would roll back. OWASP LLM04 describes poisoning as an integrity attack. Answers should come with evidence, not confidence.
What is the risk, in OWASP's words?
OWASP defines data poisoning as manipulation of pre training, fine tuning or embedding data to introduce vulnerabilities, backdoors or biases. It calls it an integrity attack, because tampering undermines the model's ability to predict accurately. Risks include degraded performance, biased or toxic output, exploitation of downstream systems and hidden backdoors that stay dormant until a trigger appears, which makes them hard to test for. Models shared through open repositories can also carry malicious pickled code that runs on load.
The checklist
Answer yes, no or unknown. Unknown counts as no.
Origin
- Do you have a record of where each dataset came from, and every transformation applied? OWASP suggests tracking origins with CycloneDX or ML BOM.
- Is each data vendor vetted, and do you validate their outputs against trusted sources?
- Can you verify a dataset has not changed since you approved it? Data version control can help detect manipulation.
Access
- Do you know who can write to training data, fine tuning data and embedding stores?
- Do infrastructure controls prevent the pipeline from reading unintended data sources?
- Is user supplied content kept apart from approved data until it is reviewed?
Pipeline
- Is untrusted data processed in a sandbox before it joins a training or indexing job?
- Do you filter for anomalies in new data?
- Do you monitor training loss and model behavior, with thresholds that raise an alert?
Testing
- Have you red teamed the model for resilience to adversarial inputs, within your rules of engagement?
- Does your evaluation set include cases your users care about, run on every new model version?
- Are model files loaded in formats and environments that cannot run unexpected code?
Recovery
- Can you roll back to the previous model or index version within hours?
- Is user supplied information stored in a place that can be corrected without retraining the whole model? OWASP notes a vector database can serve this purpose.
- Does your incident plan name who decides to pull a model and who tells customers?
How should you read the results?
Count the no and unknown answers by group. Origin and access gaps are the cheapest to close and give the most protection. Testing gaps matter, but testing alone cannot prove a model is clean, since dormant backdoors are built to evade tests.
What does the evidence look like?
A dataset register with origin, hash, owner and approval date. Access lists for data stores. Pipeline logs. Evaluation results per model version. A rollback runbook with a date of the last drill. These also help in audits and customer questionnaires. Safeguard's AI model scanning and component inventory can feed part of this record. It does not replace your own evaluation.
What to do
- Create a dataset and model register this week.
- Add hashes and approval dates to every entry.
- Restrict write access to data stores and log changes.
- Move untrusted data to a quarantine step.
- Define alert thresholds for training and runtime behavior.
- Run an evaluation set on every release and keep results.
- Rehearse a rollback and record how long it took.
Quick check
Can testing prove a model is not poisoned?
No. OWASP notes that backdoors can stay dormant until triggered, which makes them hard to detect. Combine testing with provenance and access controls.
Is RAG data in scope?
Yes. OWASP includes embedding data in its definition of poisoning.
Sources
Last reviewed: 18 October 2026.
