Safeguard
AI Security

Zero-day risk in third-party AI model dependencies

Malicious Hugging Face models, a trojanized Ultralytics PyPI release, and a torch.load bypass show AI model dependencies now carry real zero-day risk.

Safeguard Research Team
Research
6 min read

On December 4, 2024, two versions of the ultralytics PyPI package — a computer-vision library with tens of millions of lifetime downloads — were silently swapped for builds containing an XMRig cryptominer, injected through a poisoned GitHub Actions cache. Two months earlier, JFrog researchers had already catalogued more than 100 malicious models sitting on Hugging Face's public hub, including one that opened a reverse shell to a South Korean IP address the moment a data scientist called torch.load(). Neither incident required a phishing email or a stolen credential. Both required only that someone pip install a package or load a pretrained checkpoint — the exact workflow every ML team runs dozens of times a day. AI model dependencies now carry the same blast radius as a compromised npm package, but with far less tooling watching them. This post breaks down why model artifacts behave like unvetted executables, walks through two real 2024–2025 incidents, and explains what actually closes the gap.

What makes an AI model dependency different from a normal software dependency?

A model dependency is different because the artifact itself can execute code on load, not just at import time of some surrounding library. A requirements.txt entry for flask==2.3.0 pulls source code that a maintainer wrote and a package index scans for typosquats. A line like AutoModel.from_pretrained("some-org/some-model") pulls a multi-gigabyte binary checkpoint — frequently serialized with Python's pickle module — that can embed a __reduce__ method calling os.system() the instant it's deserialized. There is no compiler, no static review, and often no code diff to inspect: the "dependency" is a blob of tensors plus an optional payload. Hugging Face alone hosts over 1.5 million public models as of mid-2025, most uploaded by third parties with no code-signing requirement, and organizations routinely pull base models, LoRA adapters, and fine-tunes from that same pool into production inference pipelines.

How many malicious models have actually been found on public model hubs?

At least 100 were confirmed on Hugging Face in a single February 2024 sweep by JFrog's security research team, using a scanner built specifically to flag unsafe pickle opcodes. The most-cited example, uploaded under the name "baller423," was a PyTorch model whose deserialization payload opened a reverse shell to the IP 210.117.212.93 on port 4242 — fully functional malware disguised as a text-classification checkpoint, sitting on the hub for anyone to download and run. Hugging Face has since added automated pickle scanning and pushed the safetensors format as a safer default, but pickle-based .bin and .pt checkpoints remain widely uploaded, widely downloaded, and load-and-execute by default in older tutorial code and internal scripts that predate the migration.

What happened in the Ultralytics YOLOv8 supply chain compromise?

On December 4, 2024, attackers published two trojanized releases — versions 8.3.41 and 8.3.42 — of the ultralytics PyPI package, one of the most widely used object-detection libraries in production computer-vision pipelines. The compromise didn't touch PyPI's servers directly; it exploited a GitHub Actions cache-poisoning technique that let the attacker inject a cryptomining payload into the build artifact before it was signed and published as an official release. Anyone who ran pip install ultralytics or had automated dependency updates enabled during that window pulled a package that quietly mined cryptocurrency on their infrastructure — GPU boxes running inference workloads being an especially attractive target given their compute density. The maintainers pulled the releases within hours, but the incident showed that CI/CD compromise upstream of a model-serving library is functionally identical to a compromised model: the trust boundary that matters is the artifact you load, not just the vendor name on it.

Can a model-loading function itself be a zero-day, even when you think you're being careful?

Yes — CVE-2025-32434 proved that the "safe" loading path in PyTorch wasn't safe. For years, the guidance to defend against pickle-based model attacks was to call torch.load(file, weights_only=True), which was documented as blocking arbitrary code execution by restricting what could be unpickled. Security researcher Ji'an Zhou demonstrated in early 2025 that this restriction could be bypassed, allowing remote code execution even with weights_only=True set — meaning every team that had followed the official mitigation was still exposed. PyTorch patched the flaw in version 2.6.0, and Hugging Face pushed out advisories urging users to pin to patched PyTorch versions before loading any untrusted checkpoint. The lesson: a mitigation that was correct on the day it was written can become a zero-day the moment someone finds the bypass, and "we already fixed that" is not a permanent state for model-loading code.

Why don't existing SCA and container scanners catch these risks?

They don't catch them because software composition analysis tools are built to match code dependencies against CVE databases, and model weights are neither code nor a package-manifest entry in the way SCA tooling expects. A scanner reading requirements.txt or a package-lock.json can flag a vulnerable version of transformers or torch, but it has no visibility into which specific checkpoint a data science team pulled from Hugging Face at runtime, what base model that checkpoint was fine-tuned from, or whether the file was serialized as pickle versus safetensors. Models are also frequently fetched dynamically inside application code — via from_pretrained() calls, S3 URLs, or MLflow registries — rather than declared in a manifest a scanner parses at build time, so the dependency exists entirely outside the artifact inventory most AppSec teams already maintain.

What should a security or platform team actually do about this risk today?

Start by generating an inventory of every model artifact in production, not just the code around it — an AI-BOM that records model source, revision hash, serialization format, and lineage back to its base model, the same way an SBOM tracks a software package. From there, pin model references to a specific commit hash rather than a mutable tag like main, reject pickle-format checkpoints from untrusted sources in favor of safetensors, and load any third-party model inside a sandboxed, network-isolated process before it ever touches a production inference path. Finally, treat model-loading libraries (PyTorch, TensorFlow, ONNX Runtime) like any other dependency with a CVE history — CVE-2025-32434 will not be the last torch.load bypass — and prioritize patching them based on whether an untrusted-model-loading code path is actually reachable in your deployment, not just whether the CVE exists somewhere in your dependency tree.

How Safeguard Helps

Safeguard extends software supply chain security to the AI model layer instead of stopping at source code. Griffin AI continuously maps which model-loading and inference code paths are actually reachable from a deployed service, so a torch.load or from_pretrained call buried three layers deep in a monorepo gets prioritized correctly instead of triggering the same alert as dead code. Safeguard generates and ingests AI-BOMs alongside standard SBOMs, tracking model provenance, revision hashes, and serialization format so teams can prove exactly which checkpoints are running where. When a known-malicious model, an unpatched loader CVE like CVE-2025-32434, or an unpinned model reference is detected, Safeguard opens an auto-fix PR that pins the safe revision or upgrades the vulnerable library — turning a multi-week remediation cycle into a reviewable pull request.

Never miss an update

Weekly insights on software supply chain security, delivered to your inbox.