In March 2024, security researchers at JFrog scanned the roughly 500,000 public models hosted on Hugging Face and found around 100 that executed malicious code the moment someone loaded them with pickle.load(). No CVE was filed. No vendor advisory went out. The models looked like ordinary fine-tuned checkpoints, and most of the teams that pulled them had no record of where the weights came from, what training data touched them, or which base model they were derived from. That gap — not knowing what is inside a model the way you'd know what is inside a container image — is exactly what an AI Bill of Materials (AI-BOM) is built to close. As foundation models, LoRA adapters, and third-party datasets get stitched into production pipelines as casually as npm packages once were, an AI-BOM gives security teams the same traceability for models that an SBOM gives them for code.
What Is an AI-BOM and How Is It Different from a Software SBOM?
An AI-BOM is a structured, machine-readable inventory of every component that goes into training, fine-tuning, and serving a machine learning model — not just the code that runs it. A software SBOM (per NTIA's 2021 minimum elements guidance) lists packages, versions, and licenses so you can answer "am I running Log4j 2.14?" An AI-BOM has to answer harder questions: which base model was this fine-tuned from, which datasets trained it, what preprocessing and quantization steps were applied, which serving framework loads it, and who signed off on each stage. CycloneDX addressed this directly by adding a dedicated ML-BOM object type in CycloneDX 1.5 (released February 2023), with fields for model parameters, datasets, and quantitative "considerations" like bias and fairness metrics. SPDX followed with an AI and dataset profile in SPDX 3.0 (April 2024). Neither replaces the other — most mature AI-BOM implementations nest a full SBOM inside the ML-BOM for the serving stack, since a model still runs on PyTorch, CUDA drivers, and Python packages that need their own vulnerability tracking.
Why Do Model Supply Chains Need Their Own Bill of Materials?
Because a model's attack surface and provenance chain look nothing like a regular application's, and treating them the same hides real risk. A typical production model today is assembled from parts sourced from at least three or four different parties: a base model pulled from Hugging Face or a vendor API, a fine-tuning dataset scraped or licensed from a third party, adapter weights (LoRA/QLoRA) trained in-house, and a serving stack (vLLM, Triton, TensorRT-LLM) with its own dependency tree. Each hop is a place where the SolarWinds-style "trusted update" problem recurs: Hugging Face alone hosts over 1.5 million models as of mid-2026, most uploaded by pseudonymous accounts with no code-signing requirement. Protect AI's huntr bug bounty platform has cataloged more than 3,500 vulnerabilities in ML-specific tooling since 2023, and a large share trace back to unsafe deserialization — pickle, joblib, and legacy .h5 formats that can execute arbitrary code on load, the same class of bug JFrog found actively weaponized in the wild. Without a component-level record, a compromised base model can propagate through every downstream fine-tune with nobody able to say which deployments are affected — there's no equivalent of grep-ing your dependency tree for a bad package version.
What Should an AI-BOM Actually Contain?
At minimum, an AI-BOM needs six categories of data: model identity and version, provenance (base model and training lineage), dataset manifests, dependency and runtime inventory, cryptographic hashes of weights and artifacts, and a signed attestation of who built and approved each stage. CycloneDX's ML-BOM schema operationalizes this with a modelCard object (framework, task type, license, hyperparameters), a datasets array (name, classification, sensitivity, whether it contains PII), and considerations fields for known limitations and ethical risks — fields that mirror the documentation Google's original 2019 "Model Cards for Model Reporting" paper proposed, now made machine-readable. The provenance chain matters as much as the inventory: if a model is a LoRA fine-tune of Llama 3.1 8B, the AI-BOM should record that lineage explicitly, because a vulnerability or backdoor discovered in the base weights months later needs to map to every derivative instantly. SHA-256 hashes of the actual weight files are non-negotiable — file names and version tags are trivially spoofed, and several of the malicious Hugging Face uploads JFrog identified used names nearly identical to legitimate, popular repositories.
What Regulations Are Pushing Companies Toward AI-BOM Today?
U.S. Executive Order 14110, signed October 30, 2023, required companies training dual-use foundation models above a compute threshold to report training runs and safety testing to the Department of Commerce — a government-facing precursor to AI-BOM disclosure, even though it was rescinded by a subsequent EO in January 2025 (guidance built around it, and the reporting habits it created, largely persisted in industry practice). The EU AI Act, which entered into force August 1, 2024, imposes technical documentation obligations on providers of general-purpose AI models starting August 2, 2025, and requires that documentation include training data summaries and model architecture details — data that's far easier to produce continuously from an AI-BOM than to reconstruct after the fact for an audit. NIST's AI Risk Management Framework (released January 26, 2023) and its 2024 Generative AI Profile both list "maintain provenance records for training data and model components" as a core control. For companies already carrying SOC 2 Type II attestations, auditors are increasingly asking for the same kind of change-tracking evidence for models that they've asked for in code deployments for years — and "we don't track that" is not an answer that clears a control test.
What Happens When a Team Skips AI-BOM Tracking?
The most common failure mode isn't a dramatic breach — it's a silent, expensive incident response problem. When a vulnerability or license violation surfaces in a widely-used base model (as happened when several Hugging Face repositories were pulled in 2024 after researchers found the license terms of their source datasets had been misrepresented), teams without an AI-BOM have to manually reconstruct which of their internal models and fine-tunes descend from the affected one. That process, done by memory and Slack archaeology, routinely takes security teams days to weeks across a dozen or more downstream services — compare that to the minutes it takes to run a dependency query against a modern SBOM when a CVE like Log4Shell drops. The same gap shows up in licensing risk: a dataset with a non-commercial or share-alike clause buried three fine-tunes upstream can put an entire commercial product's IP posture in question, and legal teams have no way to catch it without a manifest trail. And in security incidents specifically, unsafe deserialization formats mean a malicious model isn't just a bad prediction risk — loading it can mean remote code execution on the GPU host the moment torch.load() runs, which is why NVIDIA, Hugging Face, and PyTorch have all pushed safetensors as the safer weight format since 2023 without fully deprecating pickle-based loading.
How Safeguard Helps
Safeguard extends the same software supply chain security model teams already use for code — provenance, attestation, and continuous vulnerability tracking — to the model layer, so AI-BOM isn't a separate spreadsheet nobody maintains. Safeguard automatically generates CycloneDX ML-BOMs for models pulled into your pipelines from Hugging Face, internal registries, or vendor APIs, capturing base model lineage, dataset manifests, and weight hashes at the point of ingestion rather than reconstructing them after the fact. It flags unsafe serialization formats (pickle, .h5) before they reach a serving host, cross-references model and dataset licenses against your organization's approved-use policy, and continuously monitors upstream base models so that a newly disclosed issue in a foundation model automatically surfaces every internal fine-tune derived from it. For teams carrying SOC 2 or preparing for EU AI Act documentation obligations, Safeguard exports signed, audit-ready AI-BOM records alongside the software SBOMs it already produces — giving auditors and incident responders one system of record for everything that goes into shipping a model, instead of two.