Machine learning pipelines have become high-value targets for supply chain attacks, and the security community is still catching up. The attack surface for an ML pipeline is fundamentally different from a traditional software project -- it includes not just code and dependencies, but training data, pretrained models, feature stores, and inference infrastructure.
We have been tracking supply chain incidents specific to AI/ML systems throughout 2025, and the trend is alarming. Here is a breakdown of the attack vectors and practical defenses.
The ML Supply Chain is Different
A traditional software supply chain involves code, dependencies, build tools, and deployment infrastructure. An ML supply chain adds several new components, each with its own trust model:
Training data. Data flows in from external sources -- public datasets, purchased data, web scraping, user-generated content. Poisoning this data can subtly alter model behavior without changing any code.
Pretrained models. Most ML projects start from a pretrained model rather than training from scratch. These models are downloaded from registries like Hugging Face, PyTorch Hub, or TensorFlow Hub. A compromised pretrained model can include hidden behaviors that persist through fine-tuning.
ML frameworks and libraries. The Python ML ecosystem (PyTorch, TensorFlow, scikit-learn, and hundreds of smaller libraries) is a massive dependency surface with the same risks as any other package ecosystem -- plus unique risks from native code compilation and GPU driver interactions.
Model serialization formats. Many ML model formats (Python pickle files, TensorFlow SavedModel) can execute arbitrary code when loaded. This is not a bug -- it is a design choice that prioritizes flexibility over safety. It is also a massive attack vector.
Attack Vector 1: Malicious Model Files
The most direct attack on ML supply chains is distributing malicious model files. Python's pickle format, which is still the default serialization for PyTorch models, can execute arbitrary code during deserialization. Load a poisoned .pkl file, and you are running the attacker's code.
In 2025, we have documented:
- 23 malicious models on Hugging Face that contained code execution payloads in pickle files
- 8 cases where legitimate models were re-uploaded with modified weights that included steganographically embedded payloads
- 5 incidents where model files on GitHub were modified through compromised CI pipelines
The Hugging Face team has been proactive about scanning for malicious models and promoting the SafeTensors format (which cannot execute code), but pickle-based models are still widely used.
Defense: Use SafeTensors or ONNX format instead of pickle whenever possible. Scan model files before loading. Run model loading in sandboxed environments with no network access and limited filesystem permissions. Verify model checksums against trusted sources.
Attack Vector 2: Training Data Poisoning
Data poisoning is the supply chain attack that most ML practitioners know about in theory but rarely defend against in practice. The attack works by injecting carefully crafted samples into training data that cause the model to learn a hidden behavior -- a backdoor that activates when the model encounters a specific trigger pattern.
The supply chain angle is critical here. Most training datasets are not created from scratch -- they are curated from public sources, purchased from data providers, or scraped from the web. Each of these sources is a potential injection point.
In 2025, we tracked several notable data poisoning campaigns:
- A coordinated effort to inject biased examples into a widely-used NLP dataset through automated Wikipedia edits
- Poisoned images in a popular computer vision dataset that caused models to misclassify specific objects
- Manipulated code examples in a code generation training set that taught models to introduce subtle security vulnerabilities
Defense: Implement data provenance tracking. Validate training data against known-clean reference sets. Use techniques like spectral signatures and activation clustering to detect poisoned samples. Maintain SBOMs that include data sources, not just code dependencies.
Attack Vector 3: Compromised ML Packages
The Python ML ecosystem is particularly vulnerable to supply chain attacks because:
- ML packages often include compiled native code (CUDA kernels, C++ extensions) that is harder to audit
- Installation frequently requires running setup.py, which executes arbitrary code
- ML practitioners typically install packages in privileged environments with GPU access
- Version pinning practices in ML projects are notoriously poor -- "just pip install the latest" is common
We tracked 67 malicious packages targeting ML practitioners in H1 2025, compared to 23 in all of 2024. The most common pattern is packages that mimic popular ML libraries with additional "utility" features -- extra data loaders, visualization tools, or model optimization helpers. These packages work as advertised while silently exfiltrating training data, model weights, or API keys.
Defense: Pin all dependencies with hash verification. Use virtual environments with minimal permissions. Audit new dependencies before adding them. Run ML workloads in isolated environments with network controls.
Attack Vector 4: CI/CD Pipeline Compromise
ML pipelines often have CI/CD systems that are more complex and less hardened than traditional software pipelines. A typical ML CI/CD system handles:
- Data ingestion and preprocessing
- Model training (often on GPU infrastructure with high costs)
- Model evaluation and validation
- Model deployment to inference endpoints
- A/B testing and canary deployments
Each stage has credentials, access to data, and execution capabilities that make it an attractive target. Training pipelines are especially dangerous because they have access to training data (potential IP), GPU resources (cryptomining target), and model registries (deployment access).
We have seen cases where compromised training pipelines were used to:
- Inject backdoors into models during training
- Exfiltrate training data to external servers
- Use GPU resources for cryptocurrency mining
- Modify model artifacts before deployment
Defense: Apply the same CI/CD security practices used for software pipelines -- ephemeral build environments, minimal credentials, audit logging, and pipeline integrity verification. Add ML-specific controls: validate model checksums between pipeline stages, monitor GPU utilization for anomalies, and verify that training data has not been modified.
Attack Vector 5: Model Registry Poisoning
Model registries (Hugging Face, MLflow, internal registries) are the ML equivalent of package registries. They store, version, and distribute models. And they face the same supply chain risks -- account compromise, typosquatting, namespace confusion.
We documented 11 cases in 2025 where legitimate model repository names were targeted with lookalike models. For example, a model named meta-llama/Llama-2-7b-chat-hf being mimicked by meta_llama/Llama-2-7b-chat-hf (underscore vs hyphen). The imposter models were modified to include subtle behavioral differences -- not outright malicious, but biased in specific ways.
Defense: Verify model signatures when available. Use organization-verified model sources. Implement model scanning in your deployment pipeline. Maintain an approved model registry with vetted, checksummed models.
Building an ML SBOM
Traditional SBOMs capture code dependencies. ML SBOMs need to capture more:
- Code dependencies (standard SBOM)
- Training data sources and versions
- Pretrained model provenance
- Training configuration and hyperparameters
- Hardware and framework versions used for training
- Model file checksums and format
The ML SBOM space is still evolving. CycloneDX 1.6 added machine learning BOM (MLBOM) support, which is a good starting point. We are working on tooling that generates ML-aware SBOMs automatically from common training frameworks.
How Safeguard.sh Helps
Safeguard.sh extends supply chain security to ML pipelines. Our SBOM generation captures ML-specific dependencies, our vulnerability tracking covers ML frameworks and libraries, and our policy gates can enforce controls on model provenance and dependency health. With Safeguard, you can apply the same rigor to your ML supply chain that you apply to your traditional software supply chain -- because attackers are already treating them as a single target.