When a traditional software application has a vulnerability, you can trace it. The CVE maps to a package, the package appears in your SBOM, and the SBOM tells you which products are affected. The remediation path is clear: update the dependency, rebuild, redeploy.
Now consider an AI model. It was trained on datasets sourced from multiple providers. The training pipeline used dozens of libraries -- PyTorch, Transformers, tokenizers, data loaders -- each with their own dependencies. The model weights were fine-tuned by a third party. The inference runtime adds another layer of dependencies. And the model's behavior is determined not just by code but by training data, hyperparameters, and stochastic processes that are fundamentally harder to audit than deterministic software.
When something goes wrong with an AI system -- a biased output, a data poisoning attack, a prompt injection exploit -- tracing the root cause requires a different kind of bill of materials. Traditional SBOMs cover the software dependencies. AI SBOMs need to cover the data, the models, and the training process itself.
The AI Transparency Gap
The software industry spent years building infrastructure for software supply chain transparency: SBOMs, vulnerability databases, package registries with provenance data, and standards like CycloneDX and SPDX. AI systems benefit from all of this for their software components, but they have an additional transparency layer that remains largely undocumented.
Consider what a standard SBOM captures for an AI application:
- Python runtime and version.
- PyTorch, TensorFlow, or JAX frameworks.
- Hugging Face Transformers and tokenizer libraries.
- ONNX Runtime or TensorRT for inference.
- FastAPI or Flask for serving.
- All transitive dependencies of the above.
This is necessary but insufficient. It tells you nothing about:
- Which training datasets were used and their provenance.
- The model architecture and its known limitations.
- Bias evaluations and fairness metrics.
- The training process, including hardware, duration, and hyperparameters.
- Fine-tuning history and who performed it.
- Known failure modes and adversarial vulnerabilities.
This gap is what AI SBOMs and model cards aim to fill.
Model Cards: The Original AI Transparency Document
Model cards were proposed by Mitchell et al. in 2019 as a standardized way to document machine learning models. The original paper recommended documenting:
- Model details: Architecture, version, training date, developer.
- Intended use: Primary use cases and out-of-scope applications.
- Factors: Demographic, environmental, and technical factors relevant to model performance.
- Metrics: Performance metrics across different evaluation conditions.
- Evaluation data: Datasets used for evaluation and their characteristics.
- Training data: Overview of training data (though detailed disclosure may not always be possible).
- Ethical considerations: Known risks, biases, and mitigations.
- Caveats and recommendations: Known limitations and deployment guidance.
Model cards gained significant adoption through Hugging Face, which integrated model card templates into their model repository. As of 2025, most models published on Hugging Face include at least a basic model card.
However, model cards are primarily human-readable documents. They are markdown files with varying levels of detail and no enforced schema. This makes them useful for human review but difficult to consume programmatically for supply chain automation.
AI SBOMs: Machine-Readable AI Transparency
AI SBOMs extend traditional SBOM formats to capture AI-specific components. The two major SBOM standards have both moved to accommodate AI:
CycloneDX Machine Learning BOM (MLBOM)
CycloneDX 1.5 introduced first-class support for machine learning components. An MLBOM can describe:
- Model components: Architecture type, quantization level, input/output specifications.
- Dataset components: Training and evaluation datasets with provenance, licensing, and content descriptions.
- ML framework dependencies: Standard software dependencies used in training and inference.
- Model properties: Hyperparameters, training metrics, evaluation results.
The MLBOM format is designed to be part of a larger CycloneDX BOM that also includes traditional software dependencies. This means a single document can describe both the Python packages an AI application depends on and the model and dataset components it uses.
SPDX AI and Dataset Profiles
SPDX 3.0 introduced the AI and Dataset profiles, which add AI-specific metadata to the SPDX format:
- AI Package: Describes a model with fields for type of model, information about training energy consumption, domain, and autonomy level.
- Dataset Package: Describes a dataset with fields for collection method, data preprocessing, known biases, and sensitive personal information indicators.
Both formats provide the machine-readable schema that model cards lack, enabling automated analysis of AI supply chains.
What Goes Into an AI SBOM
A comprehensive AI SBOM for a production AI system should include:
Software Dependencies
Everything a traditional SBOM captures: the Python environment, ML frameworks, serving infrastructure, and all transitive dependencies. This is the foundation -- every known vulnerability in these components should be trackable.
Model Provenance
Where did the model come from? Was it trained in-house, downloaded from a model hub, or provided by a vendor? What is the model's version, and what changed since the last version? If the model was fine-tuned from a base model, what is the base model's provenance?
Training Data Lineage
Which datasets were used for training? Where were they sourced? What licenses apply? Were they filtered, augmented, or preprocessed? Is there personally identifiable information (PII) in the training data? Has the data been evaluated for bias?
This is often the most challenging section to populate. Training data lineage for large language models can involve billions of web pages, books, and code repositories, and full disclosure may not be feasible for commercial or legal reasons. Even partial documentation is better than none.
Evaluation Results
What benchmarks was the model evaluated against? What were the results, disaggregated by relevant subgroups? Are there known failure modes or adversarial inputs?
Known Risks and Limitations
This includes AI-specific risks like hallucination rates, bias patterns, prompt injection susceptibility, and data extraction vulnerabilities. Unlike software CVEs, these risks may not have standardized identifiers, but they should be documented.
Security Implications
AI SBOMs have direct security applications:
Data poisoning detection: If you know what datasets a model was trained on, you can monitor those datasets for tampering. When a training data source is compromised, you can identify which models need retraining.
Model supply chain attacks: Pre-trained models downloaded from public hubs can contain backdoors. AI SBOMs with provenance information help organizations track which models they use and where they came from, enabling rapid response when a compromised model is identified.
Vulnerability management for ML frameworks: ML frameworks have had significant security vulnerabilities. PyTorch, TensorFlow, and ONNX Runtime have all had CVEs that could allow arbitrary code execution. An AI SBOM that includes software dependencies alongside model components enables vulnerability scanning across the full AI stack.
Regulatory compliance: The EU AI Act requires documentation of training data, model evaluation, and risk assessment for high-risk AI systems. AI SBOMs provide a structured, machine-readable format for this documentation.
Adoption Challenges
AI SBOM adoption faces several practical challenges:
Training data opacity: Many commercial AI providers treat training data as a trade secret. Even organizations willing to document their training data may not have complete lineage for data sourced from third parties.
Model versioning complexity: AI models evolve through fine-tuning, quantization, distillation, and other transformations. Each transformation creates a new version that should be tracked, but the model is not a source file you can diff.
Tooling maturity: SBOM generation tools for traditional software dependencies are mature. AI SBOM generation tools are emerging but not yet widely adopted. Most organizations would need to build custom tooling to generate comprehensive AI SBOMs.
Lack of AI vulnerability databases: There is no NVD equivalent for AI-specific vulnerabilities. Model biases, failure modes, and adversarial vulnerabilities are not cataloged in a standardized, queryable format.
How Safeguard.sh Helps
Safeguard supports CycloneDX MLBOM generation for AI applications, capturing both traditional software dependencies and AI-specific components in a single bill of materials. The platform scans ML framework dependencies against vulnerability databases, ensuring that known CVEs in PyTorch, TensorFlow, ONNX Runtime, and other AI infrastructure are identified and tracked alongside model metadata.
For organizations building or deploying AI systems, Safeguard provides a centralized view of the AI supply chain: which models are in use, what versions are deployed, which software dependencies support them, and what vulnerabilities exist across the stack. This visibility is the foundation for both security management and regulatory compliance as AI-specific regulations like the EU AI Act take effect.