You test for LLM supply chain risk by checking five things: where each model and dataset came from, whether files match their published hashes, whether the packages around the model are patched, whether the licenses allow your use, and how the model behaves under your own evaluations. Each test below is defensive and needs no attack code.
What does LLM03 cover?
OWASP defines LLM supply chain vulnerabilities as those affecting the integrity of training data, models and deployment platforms. They go beyond ordinary dependencies to third party pre trained models and data, and open access models, LoRA and PEFT fine tuning, and on device models add more exposure.
Its risk list includes vulnerable third party packages, licensing risks, outdated or deprecated models, vulnerable pre trained models that are hard to inspect statically, weak model provenance, vulnerable LoRA adapters, abuse of collaborative model merging, on device model risks and unclear provider terms.
What are the tests?
1. Provenance test. For every model, adapter and dataset in use, write down the publisher, the source URL, the version and the date you fetched it. Then try to answer: could someone have impersonated this publisher? OWASP notes that published models lack strong origin guarantees. Pass condition: every artifact has a recorded, verifiable origin.
2. Integrity test. Compare file hashes and signatures against what the publisher states. OWASP recommends verifying integrity with signatures and file hashes and signing externally supplied code. Pass condition: a script does this on every download and on every deployment.
3. Component test. Generate an SBOM for the application, including the serving stack and training code, and scan it. OWASP calls for a current, signed SBOM and suggests evaluating AI BOMs and ML SBOMs, for example via CycloneDX. Pass condition: no unreviewed critical findings, and an owner for each component.
4. License test. Inventory licenses of models, datasets and packages, since OWASP notes dataset and model licenses may restrict use or commercialization. Pass condition: each license is recorded and checked against your use.
5. Behavior test. Run your own evaluation set on the third party model. OWASP says benchmarks alone can be bypassed by fine tuning, so test for your use case. Include checks for unexpected outputs on rare inputs and for data your application must never reveal. Pass condition: results stored for each model version, with a threshold for rollback.
6. Format test. Models shared in open repositories may use serialization formats that execute code on load. OWASP's poisoning page mentions malicious pickled code. Prefer formats that do not run code, and load unknown files only in an isolated environment. Pass condition: no unreviewed model file is loaded outside a sandbox.
7. Change test. Watch for silent changes: a new version under the same name, a changed provider policy, a deprecated model. OWASP advises auditing suppliers and their terms regularly. Pass condition: an alert or a calendar review exists.
How do you track results?
Keep a register with one row per artifact: origin, version, hash, license, last test date and owner. It doubles as evidence for audits and customer questionnaires. Safeguard scans AI models and can inventory components, and Gold lists AI models and MCP servers, so some of this can be automated. The behavior tests remain yours to design.
What to do
- List every model, adapter, dataset and AI package in production.
- Record origin, version and hash for each.
- Add hash and signature verification to your download and deploy scripts.
- Generate and scan an SBOM, including the serving stack.
- Record licenses and check them against your use.
- Build a small evaluation set and run it on each new model version.
- Review suppliers and terms twice a year.
Quick check
Do I need an AI BOM?
OWASP suggests evaluating AI BOMs and ML SBOMs, such as CycloneDX, as part of managing model and data components. It gives you a record to test against.
Can benchmarks replace my own tests?
No. OWASP notes benchmarks alone can be bypassed by fine tuning, so evaluate on your use case.
Sources
Last reviewed: 18 October 2026.
