The AI/ML boom in early 2023 brought massive investment in model development, training infrastructure, and deployment pipelines. What it didn't bring was security. While organizations raced to build and deploy AI systems, the supply chain security of those systems remained an afterthought — if it was thought about at all.
AI/ML pipelines have their own supply chains, distinct from traditional software, and those supply chains have unique vulnerabilities that most security teams aren't equipped to address.
The AI/ML Supply Chain
A typical ML pipeline includes:
Training Data
- Public datasets (ImageNet, Common Crawl, etc.)
- Purchased datasets from third-party providers
- Scraped data from the internet
- Internal organizational data
Pre-trained Models
- Models downloaded from Hugging Face, TensorFlow Hub, or PyTorch Hub
- Foundation models accessed via API (OpenAI, Anthropic, Google)
- Open-source models from research papers
Libraries and Frameworks
- TensorFlow, PyTorch, scikit-learn, JAX
- Data processing libraries (Pandas, NumPy, Apache Spark)
- Model serving frameworks (TFServing, Triton, ONNX Runtime)
Infrastructure
- GPU cloud instances (AWS, GCP, Azure)
- Container images for training and inference
- Orchestration tools (Kubeflow, MLflow, Airflow)
Each of these is a supply chain link that can be compromised.
Specific Risks
1. Model Poisoning Through Training Data
If an attacker can influence the training data, they can influence the model's behavior. This is known as data poisoning, and it can happen at several points:
Public dataset manipulation: Many ML models are trained on public datasets. If an attacker can inject malicious samples into a public dataset before it's used for training, the resulting model will incorporate those malicious patterns.
Scraped data poisoning: Models trained on web-scraped data can be influenced by anyone who controls web content. An attacker could publish content specifically designed to be ingested by web scrapers and influence model behavior.
Label manipulation: Supervised learning depends on correctly labeled data. If an attacker can corrupt the labels, the model will learn incorrect associations.
2. Malicious Pre-trained Models
Downloading a pre-trained model from a public repository is equivalent to downloading a binary from the internet. That model can contain:
Backdoors: A model with an embedded backdoor behaves normally for most inputs but produces attacker-chosen outputs when triggered by specific inputs.
Trojan models: Models that have been modified to include hidden functionality, such as exfiltrating data or executing code during inference.
Serialization exploits: Many model formats (particularly Python's pickle format) allow arbitrary code execution during deserialization. Loading a malicious model file can execute any code on your system.
The pickle deserialization risk is particularly acute. PyTorch models are commonly distributed as pickle files. Loading a malicious PyTorch model with torch.load() executes arbitrary Python code.
3. Dependency Vulnerabilities
ML frameworks have deep dependency trees. A typical PyTorch installation pulls in hundreds of packages. These packages have the same supply chain risks as any other software dependencies, plus additional risks:
- Native code: ML libraries often include compiled C/C++ code for GPU acceleration, which can contain memory safety vulnerabilities
- Platform-specific binaries: GPU-enabled packages include platform-specific compiled code that's harder to audit
- Rapid release cycles: ML libraries are updated frequently, and breaking changes can force developers to pin to older, vulnerable versions
4. Model Distribution Attacks
How models are distributed creates additional risks:
Hub repositories: Hugging Face and similar hubs host user-uploaded models with minimal vetting. Anyone can upload a model, and many models include pickle files that could execute arbitrary code.
Model format vulnerabilities: Serialized model formats can embed executable code. SafeTensors was developed specifically to address this by providing a safe serialization format.
Transfer learning attacks: When you fine-tune a pre-trained model, you inherit any backdoors or biases in the base model. If the base model was poisoned, your fine-tuned model will carry that poison.
5. Pipeline Infrastructure Attacks
The infrastructure running ML pipelines is often less hardened than production application infrastructure:
- Jupyter notebooks running with elevated privileges on GPU instances
- Training scripts that download data and models from the internet without verification
- GPU instances with broad network access for downloading large datasets
- Experiment tracking systems with stored credentials for cloud resources
Practical Defenses
Verify Model Sources
- Use SafeTensors format instead of pickle where possible
- Verify model checksums against known good values
- Download models from verified organizations on model hubs
- Scan model files for embedded executable code before loading
Secure Training Data
- Maintain provenance records for all training data
- Validate dataset integrity using checksums
- Monitor for unauthorized modifications to training data
- Use data validation tools to detect anomalous samples
Lock Down ML Dependencies
- Pin all ML library versions to specific releases
- Use lockfiles for Python dependencies
- Scan ML dependencies for known vulnerabilities
- Test model outputs when upgrading library versions (model behavior can change with library updates)
Harden Pipeline Infrastructure
- Run training jobs in isolated environments
- Limit network access for training instances
- Apply least-privilege principles to ML infrastructure
- Monitor GPU instances for unauthorized access and data exfiltration
Generate SBOMs for ML Systems
ML systems need SBOMs that go beyond traditional software:
- Standard software dependencies
- Model provenance (where the base model came from)
- Training data provenance
- Framework and library versions used during training
- Hardware and infrastructure configurations
How Safeguard.sh Helps
Safeguard.sh extends supply chain security to AI/ML pipelines:
- ML Dependency Scanning: Safeguard.sh scans ML framework dependencies for known vulnerabilities, covering the unique dependency trees of TensorFlow, PyTorch, and other ML libraries.
- SBOM for ML Systems: Safeguard.sh generates comprehensive SBOMs that include ML-specific components, providing visibility into your AI supply chain.
- Container Security for ML: Safeguard.sh scans the container images used for training and inference, identifying vulnerabilities in base images, GPU drivers, and ML runtime environments.
- Continuous Monitoring: Safeguard.sh continuously monitors your ML infrastructure dependencies against new CVE disclosures, alerting you when vulnerabilities are found in any component of your AI pipeline.
The AI/ML supply chain is the newest and least understood attack surface in most organizations. Building security into your ML pipelines now — before they become critical production systems — is significantly easier than retrofitting security later.