If you run the ML platform, the fastest way to secure it is to treat four things as ordinary supply chain dependencies: model files, notebooks, AI SDKs and the MCP servers your agents call. Inventory all four in one place, refuse to load any model file you have not inspected without executing it, pin SDKs and model revisions the way you pin any other package, and put a gate between "it works in a notebook" and "it runs in production". None of that needs a new team; it needs the same intake, pinning and promotion rules your platform already applies to containers and Python packages, extended to the artifacts data scientists actually ship.
This guide is written for the person who owns the training and serving platform, the notebook environment and the model registry, and who has just been asked by security, audit or a customer to show that the AI parts of the stack are under control. The first two thirds work with whatever tools you have. Safeguard enters at "How Safeguard does this".
Key takeaways
- A model file is code you did not write. Pickle based formats can run instructions at load time, so "download, then inspect" is the wrong order; inspect without loading, then load.
- Prefer tensor only formats (safetensors, and GGUF for local inference) and load PyTorch checkpoints with
weights_only=True. Pin every hub model to a commit, not a branch name. - Notebooks are where unpinned installs and hidden dependencies enter. Require that anything scheduled from a notebook has a lockfile and passes the same scan as a service.
- AI SDKs (model provider clients, orchestration frameworks, inference servers) are fast moving packages with deep dependency trees. Treat them as tier one dependencies with an upgrade owner.
- MCP servers and coding agents are part of the supply chain too: they are packages that run with a person's privileges and decide which tools a model can call.
- Safeguard capabilities covered here, all GA: AI-BOM discovery, AI-SPM static scanning of model files, model scanning for LLMs, SLMs, skills and agents, software composition analysis with reachability, policies and gates, MCP Guardrail, AI Gateway, agent identity, AI agent discovery, the Safeguard CLI and autonomous remediation (which ships switched off).
The situation
The trigger usually looks like one of these. A security review of a new AI feature asks for "the list of models and AI libraries in production, where they came from and how they were checked". A customer questionnaire adds an AI section. An advisory lands for a popular inference server or orchestration library, and someone asks which services use it. Or an internal audit asks how a notebook ended up running a nightly job against production data.
The person asked is the ML platform lead, because you own the registry, the training cluster, the notebook hub and the serving layer. The deadline is short, often a week, because the request is attached to a launch or a renewal.
A bad week looks like this. You discover that the honest answer to "which models are in production" is a spreadsheet from last quarter plus whatever is in the registry, and that several services pull weights straight from a public hub at startup by repository name. Two notebooks that run on a schedule install packages inside cells with no versions. Three teams use three different versions of the same orchestration framework, and nobody owns upgrading it. Engineers have connected local coding agents to MCP servers you have never seen. None of this is negligence. It is what happens when the fastest path from experiment to production skips the controls that exist for everything else.
The fix is not a separate AI security programme. It is pulling AI artifacts into the controls you already run for code: inventory, intake checks, pinning, scanning, gates and evidence.
The rules and guidance behind the request
You will be asked to show alignment with a handful of documents. This section summarises what they ask of a platform owner. It is not legal or audit advice; read the primary text for anything you will sign.
NIST SP 800-218A: secure development for generative AI
NIST SP 800-218A, published in July 2024, is an SSDF Community Profile for generative AI and dual-use foundation models. It augments the tasks in SSDF version 1.1 (SP 800-218) with practices, recommendations and notes specific to AI model development, and NIST says it is meant for producers of AI models, producers of AI systems that use those models, and acquirers of those systems. If you serve or fine-tune a hub model, you are at least the second and third of those.
The tasks a platform lead will be asked about, from the profile's own table:
| Task | What it says (paraphrased) | What it means for the platform |
|---|---|---|
| PS.1.3 (new) | Protect model weights and configuration parameters from unauthorised access and change; monitor their integrity; least privilege on access | Registry permissions, hash checks, no write access from notebooks |
| PS.3.2 (modified) | Keep provenance for every component of a release; track the provenance of a model and its components, including the training libraries, frameworks and pipelines used to build it | An AI bill of materials that includes the SDKs, not only the weights |
| PW.4.4 | Verify the integrity, provenance and security of an acquired AI model or other AI components, including datasets | An intake check for every hub or vendor model |
| RV.1.2 | Scan and test AI models frequently, mainly through automation, with a human involved as needed | Scanning on ingest and on a schedule, not once |
If your organisation already maps to the SSDF for a federal secure software attestation, 800-218A is how a reviewer will extend that mapping to the AI parts of the build.
NIST AI 600-1: the generative AI profile
NIST AI 600-1, the Generative AI Profile of the AI Risk Management Framework (July 2024), lists twelve risks. Number 12, Value Chain and Component Integration (section 2.12), is the supply chain one: GAI value chains involve many third party components such as procured datasets, pre-trained models and software libraries, which may be improperly obtained or not properly vetted. The profile notes that the heavy reuse of a limited number of foundation models makes this worse.
Its suggested actions turn that into work. GV-1.6-001 asks you to enumerate GAI systems into the AI system inventory. GV-6.1-007 asks you to inventory third party entities with access to organisational content and to keep approved GAI technology and service provider lists. MG-3.1-001 asks you to apply your risk tolerances to third party datasets, fine-tuned third party models and existing models adapted to a new domain. The platform owner is the person who can actually produce those lists.
OWASP Top 10 for LLM Applications: LLM03 Supply Chain
OWASP's LLM03:2025 covers the supply chain of LLM applications. Its risk list includes outdated or deprecated components, licensing risks across software and datasets, outdated or unmaintained models, weak model provenance (model cards give information but no guarantee of origin) and vulnerable LoRA adapters. Its mitigations include applying the usual vulnerable component practices, keeping an up to date and signed inventory of components with an SBOM, and using BOMs to inventory the licences involved. In other words, the same work as for any software, extended to models and adapters.
CWE-502: deserialization of untrusted data
CWE-502, Deserialization of Untrusted Data, is the weakness behind most model file risk. When a program deserializes data it does not trust, and the format can describe objects with behaviour, whoever controls the data can influence what runs. Python's pickle documentation is blunt about it: the module "is not secure. Only unpickle data you trust." Many model checkpoint formats are pickle underneath, and Hugging Face's pickle scanning page explains that a pickle is a stack of opcodes executed in sequence on load, which is how a file can trigger a function call.
The MCP specification's security guidance
The Model Context Protocol specification publishes security best practices for implementers. Three parts matter to a platform owner. MCP proxy servers must guard against the confused deputy problem when they front third party APIs. Token passthrough, where a server accepts a token that was not issued to it, is explicitly forbidden: MCP servers must not accept tokens that were not explicitly issued for them. And local MCP servers are binaries downloaded and run on a user's machine with that user's privileges, so a server from an untrusted source is a direct route to the machine. If your engineers or agents use MCP servers, those servers are packages with access, and they belong in your inventory.
CISA's AI data security guidance
In May 2025 CISA, the NSA, the FBI and international partners published AI Data Security: Best Practices for Securing Data Used to Train and Operate AI Systems. It covers data security and integrity risks across every phase of the AI lifecycle, from development and testing to deployment and operation. It is guidance, not regulation, but security reviewers quote it, and it is the natural companion to the model and SDK controls in this guide.
What counts as an AI supply chain asset
Before any control, agree what you are counting. A workable definition for a platform team has six asset types.
| Asset | Examples | Where it hides | Main risk |
|---|---|---|---|
| Model files | .bin, .pt, .pth, .ckpt, .safetensors, .gguf, .onnx, adapters | Hub caches, registry, object storage, container layers, laptops | Code execution at load, tampering, unknown origin |
| ML and AI SDKs | Provider clients, orchestration frameworks, transformers, torch, inference servers | Requirements files, lockfiles, notebook cells, base images | Known CVEs, deep transitive trees, breaking upgrades |
| Notebooks | .ipynb files, scheduled notebook jobs | Notebook hub, repos, shared drives | Unpinned installs, stored outputs, path to production with no review |
| Prompts and templates | Prompt files, system prompts, few shot sets | Repos, config stores | Unreviewed change to behaviour, injection surface |
| Agents, skills and MCP servers | Agent configs, skill definitions, MCP server entries | Developer machines, agent config files, CI | Broad tool access, credentials, unvetted packages |
| Endpoints and stores | Managed inference endpoints, vector databases | Cloud accounts | Shadow usage, data exposure |
You do not need perfect coverage on day one. You need the list, an owner per row and a date.
Step by step, with or without Safeguard
These steps work with any tooling. Each one ends with something you can show a reviewer.
1. Build the AI inventory
Start from what you can search mechanically:
- Search repositories for AI SDK imports and model loading calls (
from_pretrained,torch.load,pickle.load,joblib.load,onnx.load, provider client constructors). - Search for model file extensions in repositories, object storage buckets, container images and the registry.
- List managed inference endpoints from each cloud account.
- List notebooks that run on a schedule, and who owns each one.
- Ask each team for the MCP servers and agent tools they use, then check agent configuration files on a sample of machines to see how close the answers are.
Record each asset with type, name, source (hub repository and revision, internal registry, vendor), format, owner, where it runs and the date checked. That record is your first AI bill of materials. When you need to share it in a standard format, CycloneDX defines a machine learning BOM that represents models, datasets and their configuration and provenance, and SPDX 3.0 defines an AI profile for AI packages, models and datasets. Both sit alongside ordinary software components, so one document can describe the service, its SDKs and its model. For a field by field walk through of the CycloneDX side, see our CycloneDX ML-BOM implementation guide.
2. Put an intake check in front of every model file
Use this decision table for any model file that enters your environment from outside the team that trained it.
| Format | Can it run code at load? | Intake rule |
|---|---|---|
Pickle based (.pkl, many .bin, .pt, .pth, .ckpt) | Yes, by design | Inspect without loading. Load only with a restricted loader (torch.load(..., weights_only=True)) or convert to a tensor only format first |
| safetensors | No; it is a tensor only format built as a safe alternative to pickle | Accept after a structure check and hash pin |
| GGUF | Not by design; it is a binary container for tensors plus a standard set of metadata | Validate the container, pin the hash |
| ONNX | No Python execution, but custom operators and external data references matter | Check for custom operators and external data paths that leave the model directory |
Anything with trust_remote_code=True | Yes, it imports Python from the model repository | Treat as a code dependency: review the code, pin the revision, or refuse |
The order matters. A scanner that loads the file to inspect it has already lost. Inspection has to read the bytes without deserializing them.
3. Pin models the way you pin packages
A model referenced by repository name and branch can change underneath you. Pin hub models to a specific commit revision, record the file hash you inspected, and fail the load if the hash differs. Mirror approved models into your own registry or object store so production never pulls from a public hub at startup. That one change also removes a runtime dependency on an external service.
4. Make tensor only formats the default
The current PyTorch documentation for torch.load shows weights_only defaulting to true, which restricts the unpickler to tensors and a limited set of allowed types. Keep that default and do not set it to false to make an old checkpoint load. For new work, save and publish in safetensors, which Hugging Face describes as a simple format for storing tensors safely, as opposed to pickle. For local and edge inference, GGUF is common and holds tensors and metadata rather than executable code. Our post on safe model loading with safetensors versus pickle goes further into the loader side. Converting an old pickle checkpoint to safetensors inside an isolated job, once, is cheaper than reviewing it every time it loads.
5. Treat AI SDKs as tier one dependencies
Provider clients and orchestration frameworks release often and pull in large transitive trees, including optional integrations you may not use. Practical rules:
- Every service and every scheduled notebook has a lockfile with exact versions and hashes.
- One owner per framework family decides the supported version range across teams.
- SDK upgrades go through the same scan and test pipeline as any dependency, and the vulnerability scan covers transitive packages, not only the top level.
- Remove integrations you do not use. An unused integration package is attack surface with no benefit.
6. Put a promotion gate between notebook and production
Notebooks are excellent for exploration and poor as deployment units. The rule that works is simple: anything that runs on a schedule or serves traffic must exist as code in a repository with a lockfile, pass the dependency and model checks, and be owned by a named team. A notebook can stay a notebook; the job that runs it cannot skip the pipeline. Also strip outputs before commit, because .ipynb files store outputs, and outputs can hold data samples and tokens printed during debugging.
7. Govern MCP servers and coding agents
Write down which MCP servers are approved, at which versions, with which credentials and for which agents. Prefer remote servers you operate over local servers installed ad hoc. For local servers, pin the package version. Give each agent its own identity and credentials rather than a person's personal token, so you can see and revoke what each agent can reach. Apply the MCP specification's rule that a server must not pass through tokens that were not issued to it.
8. Gate in CI, with exceptions that expire
Turn the rules into checks that run on every pull request and every model registration: no critical known vulnerabilities in AI SDKs, no unpinned model references, no pickle based files without an inspection result, no notebook jobs without lockfiles. Decide which checks block and which warn. Where a team needs an exception, require a reason, an approver and an expiry date.
9. Watch runtime traffic for the applications you serve
Supply chain checks stop bad artifacts from entering. They do not stop a good model from being steered by a prompt injection in retrieved content. If you serve LLM applications, put a monitoring layer on prompts, responses and tool calls, start in observe only mode to learn what normal looks like, and then enforce the checks you trust.
10. Produce the evidence pack
The reviewer wants five things: the inventory with owners and dates, the intake rule and proof it runs, the pinning rule and proof it runs, the gate policy with recent results and the exception list. Keep each as an export with a date, not a screenshot.
A one page template you can use today
| Control | Rule | Owner | Evidence | Next review |
|---|---|---|---|---|
| AI inventory | All six asset types listed with source and owner | Platform lead | Dated export | Monthly |
| Model intake | No pickle based file loads without inspection | Platform | Scan results per file | Per model |
| Model pinning | Hub models pinned to commit and hash, mirrored internally | Platform | Registry records | Per model |
| SDK hygiene | Lockfiles everywhere; one owner per framework | Framework owners | Lockfiles, scan results | Per release |
| Notebook promotion | Scheduled notebooks are repo code with lockfiles | Data science leads | Pipeline runs | Per job |
| MCP and agents | Approved list, pinned versions, agent identities | Platform plus security | Approved list, config checks | Monthly |
| CI gate | Block and warn rules defined, exceptions expire | Platform plus AppSec | Gate results, exception log | Quarterly |
How do you vet a model from a public hub before it loads?
Do it in this order, in an isolated job with no credentials and no network beyond the hub:
- Resolve the repository to a specific commit and record it.
- List the files. If there is a safetensors version, take that and ignore pickle files.
- If only pickle based files exist, inspect their opcodes and imports without loading them, and reject any file that references modules capable of running commands or importing arbitrary code.
- Check whether the model needs
trust_remote_code. If it does, the repository's Python files are now part of your code base; review them like any third party code or choose another model. The Transformers documentation itself recommends loading custom model code from a specific commit revision so the code cannot change under you. - Record the licence of the weights and any declared training data, and check it against your policy.
- Hash every file you accept and store the hashes with the commit.
- Mirror the accepted files into your registry and serve from there.
Hub side scanning helps. Hugging Face says it runs a pickle import scan on every pickled file pushed to the Hub and lists the imports it finds, and it suggests loading models from users and organisations you trust and relying on signed commits. Treat those results as a signal, not as your control, because you still need your own record of what you checked and when. For the research background on why pickle checks matter, see our write up of pickle backdoor research on Hugging Face.
How do you keep notebooks from becoming unscanned production?
The usual path is innocent. A data scientist builds a feature pipeline in a notebook, it works, and someone schedules it. Inside are cells that install packages without versions, a credential pasted for a quick test, and outputs containing sample rows.
Three controls cover most of it:
- Dependency capture. Installs typed into cells never reach a manifest, so dependency scanning cannot see them. Require a requirements file or lockfile next to every notebook that runs unattended, and fail the scheduler if it is missing.
- Output hygiene. Strip outputs before commit with a pre-commit hook, and keep the notebook server's own trust model in place. The Jupyter Server security documentation explains that untrusted HTML output is always sanitized and untrusted JavaScript is never executed; do not weaken that to make a shared notebook render.
- Promotion path. When a notebook starts running on a schedule, it becomes a service. Move the logic into a module in a repository, keep the notebook as the exploration surface, and let the pipeline own the job.
We covered the failure pattern in more depth in the notebook that became production.
How do you manage AI SDK upgrades without breaking teams?
Pick an owner for each framework family. That person publishes a supported version range and an upgrade calendar, and the platform enforces the range in CI with a warn first, then block. When an advisory lands, the owner answers three questions from the inventory: which services carry the affected version, is the vulnerable code path reachable from our code, and what is the smallest upgrade that fixes it. Reachability matters here more than usual, because AI frameworks ship large optional integrations that many teams never import.
How do you govern MCP servers and coding agents?
Separate two cases.
Agents you build and serve (support assistants, internal copilots) call tools through MCP servers you run. Register each server and each agent, give each agent its own credentials, scope tools per agent and put a policy point between agents and servers so calls are logged and can be blocked.
For a fuller threat model of the protocol itself, see our MCP threat model for 2026 and the step by step guide to hardening an MCP server deployment.
Coding agents your engineers run locally are harder, because they live on laptops. They read configuration files that grant shell access, file edits, network access and auto-approved tools, and they connect to MCP servers that may have been installed with one command from a README. What you can do: publish an approved MCP server list, ask teams to use managed configuration where the agent supports it, and collect agent configuration and installed servers from machines with the person's consent, so your list reflects reality.
What evidence does security need from the platform lead?
Reviewers want to see a control running, not a policy document. For each control in the template above, bring the rule, the last run, the failures and what happened to them, and the open exceptions with their expiry dates. Keep a dated export of the AI inventory for each review period so you can show change over time.
How Safeguard does this
Safeguard maps onto the steps above. Each capability below is GA. Commands are shown exactly as docs.safeguard.sh shows them; they were not run for this post.
-
Inventory AI assets with AI-BOM discovery (GA). AI-BOM discovery is part of the AI-SPM engine and is switched on by an administrator through a feature flag. Once on, it runs through the normal scan flow across connected repositories, cloud accounts and SaaS integrations. In code, the docs say it finds AI SDK imports (for example
openai,anthropic,langchain,transformers,llama_index), model weight files, prompt templates, notebooks and MCP server configuration. In cloud, it finds managed inference endpoints such as SageMaker, Bedrock and Vertex AI through your cloud connectors. In connected SaaS, it surfaces unsanctioned AI tools. Vector databases are recorded as vector stores and agent definitions as agents. The result is an AI-BOM per product, a point in time snapshot of your AI footprint, and Safeguard's offering material says it is produced as a CycloneDX ML-BOM and an SPDX AI profile. This is step 1 of the guide, done continuously. -
Inspect model files without loading them with AI-SPM (GA). AI-SPM statically analyses model files across pickle, PyTorch, safetensors, GGUF and ONNX. Per the docs, it flags hidden code execution instructions in pickle data (imports of modules such as
osorsubprocess, and load time steps that run code), unpacks PyTorch archives to inspect each part, confirms safetensors files are tensor only, validates GGUF containers, and inspects ONNX graphs for custom operators and for external data paths that escape the model directory. Findings name the artifact, the format, the risk and its CWE mapping (CWE-502, CWE-94 or CWE-22), and land in the unified findings model. Once enabled, model files are scanned as they are ingested through your registries and pipelines. To scan a local folder of model files, the docs show:
safeguard aisec scan --dir ./models
This is the intake check in step 2. It reports on file structure and embedded code; it does not grade what the model learned.
-
Scan models from Hugging Face, plus skills and agents (GA). Safeguard's model scanning covers LLMs and SLMs pulled from Hugging Face or supplied directly, agent skill definitions, and agent configurations with their tool access. Hugging Face is one of the inputs on the Integrations page, alongside source control, container registries and file uploads.
-
Scan the AI SDKs and their dependency trees with SCA (GA). Provider clients, orchestration frameworks,
transformers,torchand inference servers are packages, so software composition analysis reads them from your manifests and lockfiles, across the full transitive tree, along with the container layers they ship in. Vulnerability data comes from public sources including NVD, GHSA, OSV and CISA KEV, and reachability analysis (GA) ranks findings by whether your code calls the vulnerable path. That answers the "are we actually exposed" question from the SDK upgrade section. -
Fix SDK vulnerabilities with autonomous remediation (GA, ships off). When a fixed version exists, Griffin can open a tested pull request that moves the dependency, on one repository or in bulk. It ships switched off, with every category set to Manual, so you choose what runs on its own.
-
Enforce the rules with policies and gates (GA). Policies evaluate SBOMs and generate findings; gates enforce them at checkpoints. In the app, policies are created under Settings, Policies, with a visual builder or YAML, and policies can also be written from a plain English prompt. Each policy's action decides what a failure does to the gate: Block fails the gate, Warn passes with a warning naming the policy, and Notify passes and sends a notification. With policy outcomes on for your workspace, the docs show this CI command, which fails the job when the named gate fails:
safeguard ci scan --wait --fail-on-gate <gate>
Exceptions are requested against a policy and finding with a business justification and an expiry date, go to approvers, and are recorded in an audit trail. That is step 8.
-
Put a policy point in front of MCP servers with MCP Guardrail (GA). MCP Guardrail generates guardrail proxy URLs for MCP servers, so agent tool traffic passes through a point where policy applies. The Guard docs explain that servers and agents are registered explicitly as inventory records, which lets you write rules against a specific server or agent and block or quarantine an individual agent. Guard does not scan your network to create server records on its own, so registration is part of your onboarding step for each server.
-
Know which agent holds what access with agent identity (GA). Agent identity tracks which agent holds which access, at what level, which is the record a reviewer asks for once agents act on internal systems.
-
See what coding agents run on your engineers' machines with AI agent discovery (GA). The docs describe AI agent discovery as running inside the Safeguard CLI and desktop app on the computers people work on. For the supported coding agents, it reports which agents are installed and what each may do (shell commands, file edits, network access, auto-approved tools, permission mode), the MCP servers each agent is connected to, the skills and plugins installed into it, local model files in known model folders, credentials by name only, and session activity from the agents' own local files. It shows the results under Assets, Inventory, Asset discovery. Three switches must all be on: Safeguard enables the feature for the organisation, an administrator turns it on and sets the collection policy, and each person allows it on their own computer. Credential values, tool output and file contents are never read. Before allowing it, a person can see exactly what would be sent:
safeguard aidr collect show
Agents not on the supported list are not detected, so this complements your approved list rather than replacing it.
-
Watch LLM traffic with AI Gateway (GA). For applications you serve, the AI Gateway sits between your app and the model provider and checks prompts, responses and tool calls for direct and indirect prompt injection, jailbreak patterns, and PII or secret egress. Monitor mode is the default: every interaction is evaluated and recorded as a guardrail event while traffic passes unchanged. Enforce mode needs an explicit per tenant opt-in. The gateway fails open by default, so a gateway problem does not take your app down. Raw prompt and response text is not stored; events carry redacted metadata. That is step 9.
-
Keep model IP and code inside your boundary if you need to. Safeguard can run in Zero AI mode (GA), where the platform runs deterministically with no LLM in the loop, or with your own model, and it supports on-premises and air-gapped deployment (GA). Safeguard's position is that customer code, SBOMs and telemetry are not used to train shared models.
What the requirements ask, what Safeguard does, what you still own
| Requirement or guidance | What Safeguard does | What you still own |
|---|---|---|
| NIST AI 600-1: inventory third party GAI components | AI-BOM discovery across code, cloud and SaaS | Owners, approval decisions, assets in places no connector reaches |
| CWE-502 and OWASP LLM03: unsafe model artifacts | AI-SPM static scanning of model files with CWE mapped findings | Choosing formats, converting old checkpoints, refusing trust_remote_code |
| SP 800-218A: protect and track components | SBOMs and AI-BOMs, SCA with reachability, gates in CI | Signing your own weights, retention of training data records |
| OWASP LLM03: vulnerable packages | SCA across AI SDKs and transitive trees; fix pull requests (off by default) | Upgrade calendar, compatibility testing, merge decisions |
| MCP security guidance: tool access and tokens | MCP Guardrail, agent identity, AI agent discovery on enrolled machines | Server approval, credential scoping, the no-passthrough rule in your own servers |
| CISA guidance: verify integrity, monitor use | Gates on findings, AI Gateway monitoring | Incident response and the decision to enforce |
What Safeguard does not do
- It does not judge model behaviour. AI-SPM inspects file structure and embedded code. It does not evaluate accuracy, bias, safety behaviour or what a model learned from its data. Evaluation and red teaming of model outputs stay with your ML and safety teams.
- It does not see installs that never reach a manifest. Dependency scanning works from manifests, lockfiles and images. A package typed into a notebook cell and never written down is invisible to it until you capture it in a lockfile, which is why the notebook promotion rule exists.
- It does not find every MCP server by itself. Guard records are created explicitly. AI agent discovery covers supported coding agents on computers where the organisation, the administrator and the person have all switched it on. Servers used by unsupported agents, or on machines that are not enrolled, still depend on your approved list.
- It does not export AI-BOMs from the CLI yet. The docs say there is no CLI command for listing, verifying or exporting AI-BOMs today. Use the app or API for the AI-BOM itself; the CLI scans folders of model files.
- It does not decide for you. Which models are approved, which licences are acceptable, which exceptions are granted and when to move the AI Gateway from monitor to enforce are your decisions. Autonomous remediation ships off for the same reason.
- It does not sign your weights. Signing models you train, and keeping the keys, is your process.
Common mistakes
- Scanning by loading. Any check that calls
torch.loadorpickle.loadon an untrusted file to look inside it has already run whatever the file contains. Inspect bytes, not objects. - Pinning the name, not the revision. A hub reference by repository and branch can change. Pin the commit and the file hash.
- Turning off safe defaults to fix an error. Setting
weights_only=Falseortrust_remote_code=Trueto make a model load removes the protection that triggered the error. Fix the artifact instead. - Counting only the top level SDK. The vulnerable package is often three levels down in an optional integration. Scan the full tree.
- Letting notebooks schedule themselves into production. Scheduling is a promotion. Make it go through the pipeline.
- Sharing a person's token with an agent. You lose the ability to see and revoke what the agent can do. Give each agent its own identity.
- Enforcing on day one. Moving a runtime guard straight to block mode breaks things and teaches teams to route around it. Monitor first, then enforce what you understand.
- Treating hub scan badges as your control. They help, but your reviewer wants your record of what you checked.
FAQ
Are safetensors files safe to load?
Safetensors is a tensor only format designed as a safe alternative to pickle, so loading it does not run code the way unpickling can. That is why Hugging Face recommends it. You should still pin the file hash and check its structure, because a well formed file can still come from the wrong source or contain the wrong weights.
Is torch.load safe now that weights_only defaults to true?
It is much safer. With weights_only=True PyTorch restricts unpickling to tensors and a short allow list of types. The protection disappears if code sets it to false, so search for that and treat it as a review item.
What is trust_remote_code and why does it matter?
It is a Hugging Face Transformers option that lets a model repository supply Python code that runs when you load the model. With it on, the repository is a code dependency. Review the code and pin the revision, or choose a model that does not need it.
Do notebooks need their own security tool?
Usually not. They need the same rules as other code: lockfiles, scanning, output stripping and a promotion path. The risk comes from notebooks skipping those rules, not from the format itself.
What is an AI-BOM and how is it different from an SBOM?
An SBOM lists software components. An AI-BOM also lists models, datasets and AI services with their provenance. CycloneDX ML-BOM and the SPDX 3.0 AI profile are the two common formats.
How do I find which services use a vulnerable AI SDK?
Search your SBOMs or AI-BOM for the package and version, then check reachability to see which services call the vulnerable code. Without an inventory, you are back to asking every team.
Are MCP servers part of the software supply chain?
Yes. A local MCP server is a package that runs with the privileges of the person or agent that starts it, and a remote one is a service with credentials. Both belong in your inventory with a pinned version and an owner.
Can Safeguard scan model files in an air-gapped environment?
Safeguard supports on-premises and air-gapped deployment and a Zero AI mode with no LLM in the loop. Model file scanning is static analysis, so it does not need to send weights anywhere.
Does AI-SPM tell me whether a model is backdoored?
Do not rely on it for that. AI-SPM looks for code execution content and malformed or unsafe structure in model files. Whether a model's learned behaviour has been tampered with is a question for evaluation and testing.
Most of the urgent work for an ML platform after an advisory is ordinary dependency work: an inference server, an orchestration framework or a tokenizer library needs to move to a fixed version across every service that carries it. The block below explains how Safeguard handles that part when a fix exists, and what happens when it does not.
Fix it without waiting for upstream
Whatever the vulnerability, most of the risk sits in two gaps. The first is the time between a fix being published and that fix actually running in your production builds. The second, and worse, is the time before any fix exists at all, while a maintainer works out a patch and a release schedule. Waiting through both gaps is a habit, not a requirement. Safeguard gives teams three ways to close them.
1. Autonomous remediation: pull requests, not tickets
When a fixed version exists, Safeguard's Griffin models open a tested, policy gated pull request that moves the affected dependency to the safe version. You can run it on one repository or in bulk across every project that carries the vulnerable version. Remediation categories include dependencies, licenses, security posture and findings, and each one is set separately. Reachability analysis ranks the vulnerable code paths your application actually calls first, so the first pull requests you merge are the ones that matter most. Autonomous remediation ships switched off, with every category set to Manual, so your team decides how much runs on its own and what still needs a human approval.
2. Gold Open Source: the safe version in seconds, free
gold.safeguard.sh is free and needs no login. Search any CVE or package to see affected and fixed versions, CVSS, EPSS, CISA KEV status and a recommended safe version, or use the "Am I affected?" checker. The same data is available through a free read API, README status badges and RSS threat feeds.
3. Gold Registry: start from components that are already fixed
Gold Registry provides hardened, drop in replacements for upstream packages and container images. Every Gold artifact is held continuously to zero exploitable critical or high CVEs and a completed license check, with signed build provenance (signed with Safeguard's key and verifiable with cosign, SLSA Build Level 3 as the design target) and a public evidence page you can hand to an auditor. An artifact loses its Gold badge if it regresses. Teams can point their own package directory at the registry so hardened equivalents resolve without changing how developers install anything.
When upstream has no fix yet, or will not ship one, Safeguard first works to get the fix accepted upstream. If the issue is severe or upstream will not move, Safeguard builds a fork, fixes the vulnerability there and publishes a hardened Gold build, so you are not stuck on a maintainer's release schedule. You can also request a hardened, compatibility tested build of a specific package. Ask which packages it covers before you plan around it.
| Your situation | What to use |
|---|---|
| A fixed version exists | Autonomous remediation opens the upgrade pull request; Gold Open Source confirms the safe version |
| You need to know today whether you are affected | Gold Open Source lookup and "Am I affected?" checker, free and without login |
| You want vulnerable versions to stop shipping | Safeguard policies and gates in CI |
| Upstream has no fix yet, or will not ship one | A custom Gold request; fork and patch |
| You want to start clean on new projects | Pull hardened components and images from Gold Registry |
One honest limit: when the vulnerable code lives in a vendor appliance, a hosted service or an operating system runtime you do not build yourself, the fix still has to come from that vendor or distribution. The steps above apply to the open source dependencies and container images you build and ship, which is where most teams carry the largest and least tracked share of their exposure.
See how Safeguard would help
The first demo connects Claude to the Safeguard MCP server with OAuth and shows how you choose which tools Claude can use. It is the same idea as the MCP governance step in this guide: an agent gets a defined set of tools, chosen by a person.
Click through it at your own pace, or open it full screen.
The second demo is a real CLI session: install, open the interactive terminal, ask it to fix a lodash CVE in a repository, and watch it read the finding and advisory, ask before any command that changes something, edit the manifest, run install and tests with approval, and push nothing. An AI SDK advisory in a Python service follows the same pattern of finding, fix and test.
Click through it at your own pace, or open it full screen.
The third demo sets up the local runner on your own machine, approves it with its device code, and scans from the terminal. For model files and notebooks that should not leave your environment, a local runner keeps the scan next to the data.
Click through it at your own pace, or open it full screen.
Quick check
Why is "download the model, load it, then scan it" the wrong order?
Because pickle based model files can run instructions during loading. Inspection has to read the file without deserializing it, and loading should happen only after the file passes.
A notebook installs packages inside a cell and now runs nightly. What is the minimum fix?
Capture its dependencies in a lockfile next to the notebook, run the same dependency scan and gate as any service, and give the scheduled job a named owner.
Sources
- NIST SP 800-218A, Secure Software Development Practices for Generative AI and Dual-Use Foundation Models and the PDF
- NIST AI 600-1, Generative AI Profile and the PDF
- OWASP LLM03:2025 Supply Chain
- MITRE CWE-502: Deserialization of Untrusted Data
- Python documentation: pickle
- PyTorch documentation: torch.load
- Hugging Face Hub: Pickle Scanning
- Hugging Face: Safetensors
- Hugging Face Hub: GGUF
- Hugging Face Transformers: loading models and trust_remote_code
- CycloneDX: Machine Learning Bill of Materials
- SPDX 3.0.1 AI profile
- Model Context Protocol: Security Best Practices
- Jupyter Server: Security
- CISA: AI Data Security best practices
- Safeguard docs: AI-BOM Discovery
- Safeguard docs: AI Security Posture Management (AI-SPM)
- Safeguard docs: AI-BOM and Model Security
- Safeguard docs: AI agent discovery
- Safeguard docs: Policies and Gates
- Safeguard docs: AI Gateway (LLM Firewall)
- Safeguard docs: Guard Servers and Agents
Download Safeguard at safeguard.sh/download and read the documentation at docs.safeguard.sh/docs.
Last reviewed: 25 October 2026. Product capabilities change; check docs.safeguard.sh for the current state.