Treat model weights, datasets and tokenizers like any third party dependency. Prefer formats that cannot run code on load, pin each artifact by hash, record where it came from, scan before loading, and run unfamiliar artifacts in isolation. A model download is code execution risk and a data quality risk together.
What can go wrong?
OWASP lists supply chain among its top LLM risks, covering third party models, datasets and components, and lists data and model poisoning separately: tampering with training, fine tuning or embedding data to change behavior.
There is also a concrete, older problem. Hugging Face's documentation explains that pickle, the default serialization format for PyTorch weights, can execute arbitrary code when a file is loaded, because a pickle is a stack of instructions run in sequence. Its advice includes loading models from trusted users and organizations, relying on signed commits, and using alternative formats. The Hub also shows the list of imports in pickled files.
Which format should you prefer?
The safetensors project describes itself as a simple format for storing tensors safely, as opposed to pickle, while remaining fast. Choosing a format that stores only tensor data removes the code execution route at load time. It does not tell you whether the weights are the ones you intended, or whether the model behaves badly, so it is one control among several.
How do you record provenance?
Write down, per artifact: the source location, the exact revision or commit, the hash, who approved its use and when, the license, and the intended use. For datasets add collection method and known gaps. The NIST profile SP 800-218A adds AI specific practices to the Secure Software Development Framework, including attention to the provenance of data and models.
Keep it machine readable and next to the code that loads the artifact, so a build can fail if the hash does not match.
A constructed example
A team fine tunes a classifier from a community model. Their loading script pulls the latest revision of the repository by name. One day the repository owner pushes a new weights file. Nothing in the team's pipeline notices, and results shift subtly. The fix pins the commit and the file hash, mirrors the approved artifact to internal storage, and makes the loader refuse anything else. When the team wants an update, it goes through the same review as a library upgrade.
Checklist
- Inventory every model, dataset, tokenizer and adapter used in production and training, with owners.
- Prefer safe tensor formats, and avoid loading pickle based files from untrusted sources.
- Pin each artifact by revision and hash, never by a floating name or tag.
- Mirror approved artifacts to storage you control, and load only from there.
- Record provenance, license and approval for each artifact in a reviewed file.
- Scan artifacts before loading, and load unfamiliar ones in an isolated, network restricted environment.
- Review training and fine tuning data sources for poisoning risk, and log changes.
- Evaluate behavior on a fixed test set after every artifact change, and keep the results.
- Define who can add or update artifacts, and require review.
Safeguard offers dependency scanning and the Gold registry for open source packages, and policies and gates for what gets approved. Use the same discipline for models that you use for packages.
Quick check
Why is pickle risky for model files?
Loading a pickle runs instructions in the file, so a malicious file can execute code. Hugging Face documents this.
Does a safe format prove a model is trustworthy?
No. It removes one code execution route. You still need provenance, hashes and behavior testing.
Sources
- Hugging Face Hub, pickle scanning
- Safetensors documentation
- OWASP GenAI, LLM03 Supply Chain
- OWASP GenAI, LLM04 Data and Model Poisoning
- NIST SP 800-218A
Last reviewed: 18 October 2026.
