Auditing an AI agent's dependencies means enumerating and risk-scoring four layers — the package tree, the tool layer (MCP servers, plugins, APIs), the model layer (weights, endpoints, system prompts), and the context layer (rules files, retrieved documents) — because an agent will happily execute trust decisions from any of them. Standard dependency audits stop at layer one and declare victory. That worked when software only did what its code said. An agent also does what its tools, model, and context say, so the audit surface is bigger and, honestly, messier. Here is the method we run quarterly against our own agents, in the order that finds problems fastest.
Layer 1: the package tree (the part you already know)
Start conventional. Generate a full resolved inventory, not just the manifest:
# Node agent
npm ls --all --json > deps.json
# Python agent
pip freeze --all > deps.txt
uv pip compile pyproject.toml -o resolved.txt
Then run it through SCA and check three agent-specific things a generic scan can under-weight:
- Framework velocity. Agent frameworks cut releases weekly. LangChain alone had multiple 2024–2025 CVEs (SSRF in
Web Research RetrieverCVE-2024-3095, path traversal, SQL injection in community integrations). If you are more than 60 days behind on the framework, assume unpatched issues. - Install scripts.
npm query ":attr(scripts, [postinstall])"lists packages that execute code at install time. In an agent repo these deserve individual review. - Native and loader code. Anything that deserializes —
pickle,torch.load,PyYAMLwithLoader=yaml.Loader— is a remote-code-execution primitive waiting for a poisoned artifact.
Layer 2: the tool layer — MCP servers, plugins, and APIs
This is where most agent audits fall apart, because tool dependencies rarely live in a manifest. Enumerate them from configuration and runtime:
# Common MCP config locations
cat .mcp.json mcp.json 2>/dev/null
cat ~/.claude.json ~/Library/Application\ Support/Claude/claude_desktop_config.json 2>/dev/null
For each server, record: distribution source (npm package? Docker image? random GitHub clone?), version pinning status, transport (stdio or HTTP), credentials it holds, and a hash of its current tool manifest. Then score:
| Check | Red flag |
|---|---|
| Version pin | npx -y server@latest anywhere |
| Manifest stability | Tool descriptions changed since approval |
| Credential scope | One token spanning read and write, or multiple systems |
| Egress | Server can reach arbitrary hosts |
| Maintenance | Single maintainer, no release in 6 months |
The tool manifest hash matters because of rug pulls — a server that changes its own tool descriptions after approval. Diff tools/list output against your approval-time baseline every audit. Our MCP server security checklist covers the hardening side; the audit's job is just to catch drift.
Layer 3: the model layer — weights, endpoints, prompts
Models are dependencies with versions, licenses, and vulnerabilities, and they change under you.
- Hosted models: record the exact model ID and pin it (
claude-sonnet-4-5, not an alias that silently upgrades). Note the provider's data retention terms — that is a compliance dependency. - Self-hosted weights: record the source repo, revision hash, and file format.
safetensorsis fine; a.binor.pklpickle checkpoint from an unverified Hugging Face repo is executable code. Verify hashes:huggingface-cli download org/model --revision <commit>and comparesha256sumagainst the published values. - System prompts and fine-tune data: version them in git. An unreviewed system prompt change is functionally a dependency upgrade with no changelog.
Capture all of this in an AIBOM so the next audit diffs instead of rediscovers. CycloneDX 1.6 has first-class ML fields for exactly this; SBOM Studio can hold model components alongside the package inventory so one query answers "what changed since Q1."
Layer 4: the context layer — files the agent obeys
Anything the agent reads can steer it, so the audit must enumerate what it reads by default: .cursorrules, CLAUDE.md, AGENTS.md, RAG source buckets, connected drives, inbound webhooks. For each source, ask two questions: who can write to it, and does the agent treat it as instructions?
Fast checks worth scripting:
# Invisible Unicode in instruction files
grep -rPn "[\x{200B}-\x{200F}\x{2060}\x{FEFF}]" CLAUDE.md .cursorrules AGENTS.md
# World-writable RAG sources (S3 example)
aws s3api get-bucket-acl --bucket rag-corpus | jq '.Grants'
If a context source is writable by people outside the team that owns the agent, that is a finding, full stop. The GitHub MCP exfiltration in 2025 was, at root, a context-layer audit failure: public issue text was writable by anyone and treated as instructions.
Turning the audit into a cadence
A one-off audit decays in weeks. What holds up:
- Continuous: SCA and secrets scanning on every PR; manifest-hash checks on every agent session start.
- Weekly: diff the tool inventory and model IDs against baseline; alert on drift.
- Quarterly: full four-layer review, re-score credentials and egress, prune tools unused in 90 days (dormant tools are pure attack surface).
- On incident: the audit artifacts — inventories, hashes, AIBOM — become your forensic timeline. This is the payoff for keeping them versioned.
Teams running this on Safeguard wire layers 1 and 2 into policy gates so the quarterly review is mostly reading diffs, not rebuilding inventories. Budget roughly two engineer-days for the first full audit of a mid-sized agent and half a day per quarter after that.
Frequently asked questions
How is auditing an AI agent different from auditing a normal service?
A normal service has one trust boundary: its code and package tree. An agent adds three more — tools, model, and context — and each can independently cause the agent to take actions. Auditing only the package tree covers maybe a quarter of the real attack surface.
What should go in an AI agent's SBOM?
Everything resolvable: packages with versions, MCP servers with versions and manifest hashes, model IDs or weight hashes, and pointers to versioned system prompts. CycloneDX with ML extensions handles all four; the goal is that two audits can be diffed mechanically.
How often should tool manifests be re-verified?
Hash-check on every session start if your gateway supports it, otherwise weekly. Rug pulls are cheap for attackers and invisible in normal operation, so verification frequency is your only real detection control.
Can I automate the whole audit?
Layers 1 and 2 automate well (scanners, manifest hashing, config parsing). Layer 3 is semi-automated (pin checks, hash verification). Layer 4 still needs a human asking "who can write to this and should the agent obey it?" — that judgment hasn't been automated credibly yet.