A software engineer who would never pipe an unknown shell script into bash will, without a second thought, call from_pretrained on a model they found five minutes ago. The ceremony is different, so the instinct is different. But the risk profile is closer than it looks: a pretrained model is executable-adjacent code from a third party, and the pipeline that delivers it has all the same weak links as any other software supply chain — plus a few of its own.
Model supply chain attacks are not hypothetical. Security researchers have repeatedly found weaponized models on public hubs — models that run an attacker's payload the moment they are deserialized, before a single inference happens. As models became the crown jewels of AI applications, the supply chain that delivers them became a target. Here is how the attacks work and how to shut them down.
The most direct route: malicious serialization
Many model files are still serialized with Python's pickle format, and unpickling is code execution by design — the format can carry arbitrary callables that run during load. A malicious model file exploits this directly: the weights are real enough to pass a casual check, and the payload fires when you load them. This is the model-world equivalent of a package with a malicious install script, hidden inside an artifact teams download casually.
The fix is largely a format choice. The safetensors format stores weights as pure data with no code-execution path, and it has become the default for good reason. Prefer it. Treat pickle-based weights from any source you do not fully trust the way you would treat an unknown executable, and if you must load one, do it in a sandbox with no credentials and no network egress until you have verified it.
Attacks on the naming and delivery layer
The weights do not have to be malicious for the supply chain to be. Several attacks target how you find and fetch a model rather than the model itself.
- Typosquatting. An attacker publishes a model under a name a hair's breadth from a popular one — a transposed letter, a plausible fork, an org name that looks official. A hurried engineer pulls the wrong one.
- Rug pulls. A model or the repository behind it is benign when you evaluate it, then updated with a malicious payload after it has earned trust and been pinned to a floating reference like
latestormain. - Dependency confusion. The model's loading code pulls helper packages, and an attacker publishes a malicious package with the same name on a public index that resolves ahead of your intended source. The December 2022
torchtritonincident — a malicious package shadowing a legitimate PyTorch dependency and exfiltrating system information on install — is the canonical example, and the pattern applies cleanly to the tooling around models. - Compromised publishers. An attacker takes over the account or CI of a legitimate model publisher and ships tainted weights under a trusted name. Provenance you cannot verify is provenance you do not have.
Attacks on the model's behavior
Deeper in the chain sit attacks that leave the file format clean but corrupt what the model does. A backdoored model behaves normally until it sees a trigger — a specific phrase, token, or pattern — at which point it flips to attacker-chosen behavior. These can be introduced by poisoning the training or fine-tuning data, or by directly tampering with weights. They pass functional tests because the trigger is rare by design. Defending against them means caring about where a model was trained and by whom, not just whether it loads safely.
Defenses that work
The playbook is the one the software world already learned — pin, verify, inventory, scan — applied to a new class of artifact.
- Pin to hashes, not names. Reference models by a specific, content-hashed version from a known publisher. A model that changed silently under the same name is a rug pull, exactly like a compromised package version.
- Verify provenance and signatures. Prefer models with verifiable provenance and cryptographic signing. Model-signing efforts built on transparency-log infrastructure are maturing in 2026; use them where available so "who built this and has it changed" is a check, not a hope.
- Scan model files. Inspect downloaded models for unsafe serialization and known-malicious patterns before they enter your environment, the same way you scan a dependency.
- Sandbox untrusted loading. No credentials, no egress, isolated compute, until verified.
- Inventory everything. Maintain an AI bill of materials listing every model, adapter, and dataset you depend on, with source and version. Without it, a disclosed malicious model is a frantic hunt; with it, it is a query.
How Safeguard helps
Safeguard brings AI artifacts under the same supply-chain discipline as your code. Software composition analysis inventories the components your AI application depends on — including the packages your model-loading code pulls in — and reconciles them against vulnerability and reputation data, so typosquats and dependency-confusion attempts surface before they resolve into your build. The Griffin AI detection engine inspects the loading and integration code where an unsafe deserialization or an unvalidated model source turns a downloaded artifact into code execution, and auto-fix remediation proposes the corrected version — a safe load path, a pinned dependency, a verified source.
Provenance is not an add-on at Safeguard; it is the philosophy. As a builder of its own Griffin model family, Safeguard applies a provenance-first approach to how AI artifacts should be sourced and shipped, and that same discipline is what the platform helps you enforce across the models you consume. To see how this compares to traditional dependency scanners, look at Safeguard versus Snyk.
The model you did not train is a dependency you did not write. Pin it, verify it, scan it, and stop treating it as inert data. Create a free account or read the documentation to bring your model supply chain under control.