Before you train on supplied data, ask the vendor where it came from, what rights cover it, who can touch it, how quality is checked and how you will be told of changes. OWASP lists vetting data vendors rigorously among its mitigations for data poisoning.
Why vendors need their own questions
A data or labeling supplier sits upstream of your model. Their errors or compromises become yours, and a poisoned or mislabeled set can change model behavior in ways that are hard to see later. OWASP's data and model poisoning entry names rigorous vendor vetting, verifying data legitimacy and validating outputs against trusted sources as mitigations. Its supply chain entry adds that dataset licenses can restrict usage, distribution or commercialization.
Questions by theme
Provenance
- Where did each part of the data come from?
- Is any of it scraped, and from which sources?
- Is any of it generated by a model, and which one?
A good answer is specific and documented. A weak one is "public sources".
Rights 4. What license or permission covers our intended use, including commercial use? 5. Does it contain personal data, and what is the basis for holding it? 6. What happens if a source asks for removal?
You may want your own counsel to review the answers, since this post is not legal advice.
Access and handling 7. Who at the vendor can read or edit the data, and how is that access controlled and logged? 8. If annotators are contractors or crowd workers, how are they vetted and how is their work separated? 9. Where is the data stored and processed?
Quality and tampering 10. How do you check label quality, and what error rate do you measure? 11. Do you detect duplicates, outliers and planted examples? 12. Can you provide file hashes so we can verify what we received?
Change and incident 13. How will you tell us if a source is found compromised or a batch is recalled? 14. Do you keep earlier versions so we can compare? 15. What is your notice period for a security incident?
How to use the answers
| Answer quality | Next step |
|---|---|
| Specific, with documents | Record it in your inventory and proceed |
| Partial | Ask follow up questions, limit use until answered |
| Vague or refused | Treat the data as untrusted, sample heavily or decline |
Do not let a missing answer sit. Write the gap into the risk record with an owner and a date.
After you receive data
Vendor answers are claims, not proof. Hash the files on arrival, sample records yourself, and keep the vendor's version identifier with your record. See how to verify a downloaded dataset with checksums and what to record for a dataset in an AI bill of materials.
Quick check
Do I need all fifteen questions for every vendor?
No. Use the full list for data that will train a model with customer facing impact, and a shorter set for low stakes data.
Can a questionnaire replace my own checks?
No. Treat answers as claims and verify with hashes, sampling and review.
Sources
Last reviewed: 19 October 2026.
