Do AI Coding Agents Add Vulnerable Dependencies More Than Humans?

Yes, on the best evidence available today. A 2026 study of 117,062 dependency changes found AI coding agents chose known-vulnerable package versions 2.46% of the time, against 1.64% for human developers, and agent dependency work added 98 more vulnerabilities than it removed while human work removed 1,316 more than it added. The fix is not to ban agents but to check every dependency they add at pull request time and give them live advisory data before they pick a version.

The study has limits, covered below, but its direction matches what we know about how language models learn package names and versions. If agents now write a large share of your pull requests, the dependency manifest is where a quiet mistake turns into months of exposure. This post covers the evidence, the causes, package hallucination, and the commands, CI configuration and policy that turn agent pull requests into a net source of fixes.

Key takeaways

• The data points one way. Across 2,807 popular GitHub repositories, agent pull requests picked known-vulnerable versions at 2.46% against 1.64% for humans, and agent dependency work increased vulnerabilities on net (arXiv 2601.00205).
• The bad picks were avoidable. A safe version already existed in 86.58% of the agents' vulnerable picks (paper, v2).
• Agent mistakes cost more to undo. 36.8% of agent vulnerable picks needed a major version upgrade to fix, against 12.9% for humans.
• Hallucinated packages are a separate risk. USENIX Security 2025 research found 205,474 unique invented package names across 16 code models (arXiv 2406.10279).
• Working is not secure. On the SusVibes benchmark, the headline agent setup produced working code 57% of the time and secure code 11.8% of the time (PMLR).
• Control it at the pull request, inform it at the prompt. Gate every dependency change against live advisory data, and give the agent a lookup tool so it picks safe versions in the first place.
• Measure the net. Track vulnerabilities introduced minus fixed per pull request, split by author type.

What does the evidence say, and how strong is it?

The dependency study

The most direct evidence is a paper by Tanmay Singla, Berk Çakar, Paschal C. Amusuo and James C. Davis, first posted to arXiv on 1 January 2026 as "Understanding Security Risks of AI Agents' Dependency Updates" and revised on 30 January 2026. The listing describes a five page paper under review at MSR 2026 (arXiv abstract).

The authors used AIDev-pop, a dataset of agent and human pull requests on popular GitHub projects: 2,807 repositories, 33,596 agent pull requests and 6,618 human ones. They extracted 117,062 dependency changes across seven ecosystems, 53,277 of them from agent pull requests (paper, v2). The paper names npm, PyPI and Maven Central as examples. For every added or updated package version, they checked a vulnerability database for advisories disclosed before the pull request was opened, so the study counts versions that were already known to be vulnerable at the time.

Source: Singla et al., arXiv 2601.00205 v2.

Agents were busier with dependencies than humans, but that activity did not improve security. Human dependency work, on balance, removed vulnerabilities; agent dependency work added them. One nuance cuts the other way: human vulnerable picks skewed more severe (23.3% critical against 12.0% for agents). Agents, though, fixed far fewer critical issues (15.9% of their fixes against 33.3%). The picture is less "agents are reckless" and more "agents are not doing the security part of dependency management".

How strong is it?

Moderate. It is the largest direct measurement we know of, on real pull requests rather than a benchmark. Weigh these limits, most stated by the authors:

• Not like for like. The authors cannot tell whether agent and human pull requests solved comparable tasks, so the human numbers are only a contextual baseline.
• Small absolute rates. 2.46% against 1.64% is a 50% relative increase, but most changes by either group were clean.
• No view of necessity. The data cannot show whether an added dependency was needed at all.
• Some internal inconsistency. The severity tables in v2 total to different numbers than the headline counts, without explanation. The headline rates match across both versions.

That is enough to justify controls, not enough to justify banning agents from manifests. The authors land in the same place: judge agents on whether they avoid vulnerable versions and on the remediation burden they create, and add CI guardrails for dependency updates.

The wider evidence on agent code

SusVibes, published at ICML 2026, turned 186 real feature requests (where a human had originally shipped a vulnerable implementation) into agent tasks and tested 12 agent setups on frontier models. All performed poorly on security. The headline configuration produced correct solutions 57% of the time and secure ones 11.8% of the time, and adding vulnerability hints to the request did not fix it (Zhao et al., PMLR 306). It measures code flaws rather than dependency choices, but the pattern is the same: agents optimise for what is checked, usually "does it work". Newer models may score differently, so check the benchmark's own leaderboard for current numbers.

Why do agents pick stale or vulnerable versions?

Several mechanisms push the same way. Knowing which apply tells you which control to add.

Training data is frozen; advisories are not. A model learns versions from data that stops at a training cutoff. Advisories published later are invisible unless the agent's environment supplies them. What it "remembers" as a good version is what was popular in its training data, and popular older versions are exactly the ones researchers have had longest to find flaws in.

Copied pins. If tutorials in the training data pinned lodash@4.17.15 or requests==2.19.1, the agent may reproduce that pin. A human usually notices the audit warning. An agent in a loop may see it and move on because tests pass.

Downgrading to make the build green. Facing a peer dependency conflict, the fastest fix is often to move back a version. That is rational if the goal is passing tests and risky if nobody checks what the older version carries.

Avoiding major upgrades. Major versions bring breaking changes, so agents stay on old major lines. When a project only patches its current line, staying behind means staying vulnerable. The 36.8% major upgrade figure fits this.

No signal in the loop. This is the root cause. Most agent setups give the model a terminal, files and a test runner, but not a vulnerability database. Without a structured answer to "is this version safe, and which one is", the agent chooses blind.

Invented names. Sometimes the package does not exist at all, which is the next question.

What is package hallucination (slopsquatting), and how often does it happen?

Definition

Package hallucination is when a code model recommends or imports a package that does not exist. Slopsquatting is the attack on it: someone registers the invented name and waits for developers or agents to install it. Seth Larson, the Python Software Foundation's developer-in-residence, coined the term in April 2025 (Wikipedia). OWASP describes the same scenario under LLM09 Misinformation in its 2025 Top 10 for LLM applications (OWASP LLM09:2025). For attack mechanics, see AI hallucination and package confusion attacks.

Frequency

"We Have a Package for You!" by Joseph Spracklen and colleagues, accepted at USENIX Security 2025, tested 16 code models in Python and JavaScript. Its abstract reports average hallucination rates of at least 5.2% for commercial models and 21.7% for open source models, and 205,474 unique invented names (arXiv 2406.10279).

A Cloud Security Alliance note from April 2026 adds detail from the full paper: 19.7% of samples contained at least one hallucinated package, 43% of invented names recurred in all ten repeats of the same prompt, and 8.7% of invented Python names exist as real npm packages (CSA). The OpenSSF guide also cites 19.7% (OpenSSF). Two cautions: the abstract gives 576,000 code samples while the CSA note says 2.23 million, which we could not reconcile; and these are 2024 and early 2025 models. The recurrence finding matters most, because a predictable name is worth registering.

Is it being exploited?

Sources disagree. The best known case was a test: in 2023 a researcher uploaded an empty package under a name models kept suggesting, huggingface-cli, and it drew more than 30,000 downloads in three months. Wikipedia, as of July 2026, says no slopsquatting cyberattack has been reported. The CSA note, citing a security vendor's reporting, describes real malicious packages exploiting the vector with "tens of thousands of downloads". The control is the same either way: confirm a package exists, has a history and is the one you meant before it lands in a manifest. An agent with shell access can generate the import, install it and commit the lockfile in one turn, so the pull request, not someone's attention, is where this gets caught.

How do you check every dependency an agent adds at pull request time?

Every agent change ends up as a diff against a manifest and a lockfile. Here is a step by step setup; adapt it to your stack.

Step 1: See what changed

git fetch origin main
git diff --name-only origin/main...HEAD -- \
  '**/package.json' '**/package-lock.json' '**/pnpm-lock.yaml' \
  '**/requirements*.txt' '**/poetry.lock' '**/go.mod' '**/go.sum' '**/pom.xml'

# Added or changed direct npm dependencies
git diff --unified=0 origin/main...HEAD -- package.json | grep '^+ '

A lockfile change without a manifest change is a transitive shift. A manifest change is a direct decision. Treat them differently.

Step 2: Confirm each new package exists and is the right one

# npm: latest version, first publish date, source repository
npm view express version
npm view express time.created
npm view express repository.url

# PyPI: 404 means the name does not exist
curl -s -o /dev/null -w '%{http_code}\n' https://pypi.org/pypi/requests/json

# Go: available module versions
go list -m -versions github.com/gorilla/mux

Look for a real package, a first publish date well before this week, and a name and repository that match the library's official docs. If the agent imported foo_client and the docs say foo-sdk, stop. The five minute package vetting checklist goes further.

Step 3: Check the chosen version against advisories

npm audit --audit-level=high                 # npm, fail on high or critical

python -m pip install pip-audit               # Python (PyPA tool)
pip-audit -r requirements.txt

go install golang.org/x/vuln/cmd/govulncheck@latest
govulncheck ./...                             # Go, call-path aware

mvn versions:display-dependency-updates       # Maven, newer versions

# Any ecosystem: query the open OSV database for one package version
curl -s -d '{"package":{"name":"lodash","ecosystem":"npm"},"version":"4.17.15"}' \
  https://api.osv.dev/v1/query | jq '[.vulns[]?.id]'

For a readable answer with CVSS scores and fixed versions, look the package or CVE up on Gold Open Source, free and without login. Note that npm audit fix does not always fix what it reports; see why npm audit fix does not fix vulnerabilities.

Step 4: Run it on every pull request

# .github/workflows/dependency-gate.yml
name: dependency-gate
on:
  pull_request:
    paths:
      - "**/package.json"
      - "**/package-lock.json"
      - "**/requirements*.txt"
      - "**/go.mod"
      - "**/pom.xml"
permissions:
  contents: read
jobs:
  gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      # kev = fail only on known exploited, vuln = any known vulnerability, none = report only
      - uses: safeguard-sh/gold-scan-action@v1
        with:
          fail-on: vuln
      - name: List new direct npm dependencies
        if: hashFiles('package.json') != ''
        run: |
          git diff --unified=0 origin/${{ github.base_ref }}...HEAD -- package.json \
            | grep '^+ ' || echo "No direct dependency changes"

The free safeguard-sh/gold-scan-action can fail on a known exploited vulnerability, on any known vulnerability, or never. Many teams start with kev and tighten to vuln once the backlog is under control. On the Safeguard platform, the same job can run safeguard scan --dir . or safeguard exec, write JSON results under ./output, and let platform policies and gates decide to block, warn, notify or require approval.

Step 5: Install exactly what was reviewed

npm ci                                         # fail if lockfile is out of sync
pip install --require-hashes -r requirements.txt
go mod verify

A manifest change without a matching lockfile change means the reviewer never saw the resolved tree. Treat it as a red flag.

Step 6: Send judgement calls to a human

# .github/CODEOWNERS
package.json        @your-org/platform-deps
requirements*.txt   @your-org/platform-deps
go.mod              @your-org/platform-deps
pom.xml             @your-org/platform-deps

With branch protection requiring code owner review, no agent adds a direct dependency without a named person approving it, while lockfile churn flows on green checks.

Can you give the agent live advisory data so it picks safe versions in the first place?

Yes, and it is the cheapest control because it prevents bad picks instead of catching them.

Layer 1: Write the rules down

Agents read a project instruction file: Claude Code loads CLAUDE.md (Claude Code docs), and many other agents read AGENTS.md. The OpenSSF's Security-Focused Guide for AI Code Assistant Instructions, published 1 August 2025, offers wording such as "Do not add dependencies that may be malicious or hallucinated" (OpenSSF). A block to adapt:

## Dependencies
- Prefer the standard library or an existing dependency before adding a new one.
- Explain every new direct dependency in the PR description.
- Look up known vulnerabilities before choosing a version; use the safe version.
- Never downgrade a dependency to fix a build error. Explain the conflict instead.
- Install only through the package manager and commit the lockfile.
- If a package name is not in the library's official docs, stop and ask.

Instructions are not enforcement. The model still cannot see advisories published after its cutoff.

Layer 2: Give the agent a lookup tool

The Model Context Protocol lets an agent call external tools mid-task. If one answers "what are the known vulnerabilities in this version, and what is fixed", the agent can check before writing the manifest line.

Safeguard's MCP server does this for Claude, ChatGPT and Codex. Add Safeguard from Claude's connector directory on web, desktop or mobile with an OAuth sign-in, or install it from the ChatGPT plugin directory; a custom connector URL is the fallback. Depending on the tools your account enables, the assistant can pull a CVE's full record with its fix version, list the vulnerabilities in your scanned projects, and check package and version data, within the per-tool permissions you set. Make the instruction explicit:

Before adding or changing any dependency, use the Safeguard tools to check the
exact package and version for known vulnerabilities. If any are found, use the
fixed version instead and name the advisories you avoided.

Without MCP, wrap the OSV query from Step 3 in a script and tell the agent to run it before every install.

Layer 3: Check installs before they run

Claude Code supports PreToolUse hooks that run before a tool call; a hook that exits with code 2 blocks the call and its error message is passed back to Claude (hooks documentation). A minimal check a hook could call:

#!/usr/bin/env bash
# check-npm-install.sh <package>: block unknown names and advisories on latest
set -euo pipefail
pkg="$1"
if ! ver=$(npm view "$pkg" version 2>/dev/null); then
  echo "Blocked: '$pkg' is not on the npm registry." >&2; exit 2
fi
n=$(curl -s -d "{\"package\":{\"name\":\"$pkg\",\"ecosystem\":\"npm\"},\"version\":\"$ver\"}" \
  https://api.osv.dev/v1/query | jq '[.vulns[]?.id] | length')
if [ "$n" -gt 0 ]; then
  echo "Blocked: $pkg@$ver has $n known advisories. Pick a fixed version." >&2; exit 2
fi
echo "OK: $pkg@$ver, first published $(npm view "$pkg" time.created)"

It only checks the latest version and says nothing about trustworthiness beyond existing and having no advisories. Treat it as a speed bump; the pull request gate is the real control. Until hooks and gates are in place, keep installs behind approval. In the Safeguard CLI, /permissions offers Ask, Accept edits, Plan, Auto and Strict, and the desktop and phone apps can approve each command a computer running the CLI wants to run.

How the layers compare

What should a policy gate block automatically, and what needs human review?

Be strict where the answer is clear and quiet where it is not, so nobody learns to click past the gate.

Put the fix in the message. "lodash 4.17.15 has CVE-2021-23337, fixed in 4.17.21" lets an agent or human correct the change in one step. "Policy violation" breeds exception requests.

Use reachability to rank, not to excuse. Reachability tells you what to fix first in existing code. It is not a reason to merge a new dependency at a vulnerable version, because the cheapest moment to choose a safe version is before merge.

Gate everyone. Humans picked vulnerable versions 1.64% of the time. One gate improves both groups.

In the Safeguard platform, policies generate findings and gates enforce them with block, warn, notify or require approval, which maps onto this table directly.

How do you measure whether agent pull requests raise or lower your vulnerability count over time?

Copy the study's method: measure what each pull request introduced and fixed, split by author type.

Metrics

1. Net delta per pull request: known vulnerabilities at head minus at base. Sum weekly by author type.
2. Vulnerable selection rate: share of additions and updates that landed on an already advised version. Compare against the study's 2.46% and 1.64% as a rough reference.
3. Avoidable rate: share of those where a fixed version existed. High means lookup tools will help.
4. Major upgrade debt: share of vulnerable dependencies whose fix needs a major jump.
5. Time to remediate from advisory (or introducing merge) to the fixing merge.
6. Gate outcomes by author type. A falling block rate for agent pull requests after you add a lookup tool shows it is working.

Collection

Tag agent pull requests with one convention (bot account, label or commit trailer) so every pull request can be classed. Then scan base and head and record the difference:

git checkout "$BASE_SHA" && npm ci --ignore-scripts >/dev/null
base=$(npm audit --json | jq '[.vulnerabilities[]] | length')
git checkout "$HEAD_SHA" && npm ci --ignore-scripts >/dev/null
head=$(npm audit --json | jq '[.vulnerabilities[]] | length')
echo "delta: $((head - base))"

Use --ignore-scripts because you are measuring, not running. pip-audit -f json or the Safeguard CLI's JSON output work the same way. Store the series somewhere queryable; Safeguard shows which projects contain a given component and version, so you can see whether a version an agent introduced last month has spread. Review monthly. Healthy agent pull requests show a negative net delta, which is achievable because agents are good at the mechanical part of upgrades once they know the safe version.

Common mistakes

• Treating green tests as a security signal. A vulnerable version usually builds and passes.
• Collapsing the lockfile in review. That is where transitive changes hide. Have a tool summarise it.
• Auto mode installs on laptops. A hallucinated name becomes an executed install script before any gate runs. See is it safe to open an untrusted repo with an AI coding agent.
• Blocking everything on day one. A gate failing on every old medium advisory gets disabled within a week. Start with known exploited vulnerabilities and new direct dependencies.
• Treating CLAUDE.md or AGENTS.md as enforcement. Rules shape behaviour; they do not reveal post-cutoff advisories.
• Fixing only the alerting repository. The same agent, prompt or template probably did it elsewhere. Search every project for the component and version.
• Accepting downgrades as build fixes. They can silently undo someone else's security fix.
• Gating only agents. The human rate was lower, not zero.

Fix it without waiting for upstream

The study's most useful finding for defenders is that a safe version existed in most vulnerable picks. When it does, move to it fast across every repository; when it does not, you need a route that does not depend on a maintainer's schedule.

Whatever the vulnerability, most of the risk sits in two gaps. The first is the time between a fix being published and that fix actually running in your production builds. The second, and worse, is the time before any fix exists at all, while a maintainer works out a patch and a release schedule. Waiting through both gaps is a habit, not a requirement. Safeguard gives teams three ways to close them.

1. Autonomous remediation: pull requests, not tickets

When a fixed version exists, Safeguard's Griffin models open a tested, policy gated pull request that moves the affected dependency to the safe version. You can run it on one repository or in bulk across every project that carries the vulnerable version. Remediation covers eight categories, including dependencies, licenses, security posture and findings. Reachability analysis ranks the vulnerable code paths your application actually calls first, so the first pull requests you merge are the ones that matter most. Autonomous remediation ships switched off, with every category set to Manual, so your team decides how much runs on its own and what still needs a human approval.

2. Gold Open Source: the safe version in seconds, free

gold.safeguard.sh is free and needs no login. Search any CVE or package to see affected and fixed versions, CVSS, EPSS, CISA KEV status and a recommended safe version, or use the "Am I affected?" checker. The same data is available through a free read API, README status badges and RSS threat feeds. The safeguard-sh/gold-scan-action GitHub Action can fail a build on a known exploited vulnerability, on any known vulnerability, or never, so a version you have just removed cannot quietly come back in next week's merge.

3. Gold Registry: start from components that are already fixed

registry.safeguard.sh provides hardened, drop in replacements for upstream packages and container images. Every Gold artifact is held continuously to zero exploitable critical or high CVEs, zero malware and a completed license check, with SLSA L3 or higher provenance, a Sigstore signature and a public evidence page you can hand to an auditor. An artifact loses its Gold badge if it regresses. Teams can point their own package directory at the registry so hardened equivalents resolve without changing how developers install anything.

When upstream has no fix yet, or will not ship one, Safeguard first works to get the fix accepted upstream. If the issue is severe or upstream will not move, Safeguard builds a fork, fixes the vulnerability there and publishes a hardened Gold build, so you are not stuck on a maintainer's release schedule. You can also request a hardened, compatibility tested build of a specific package. This fork and patch flow is in early access today, so ask which packages it covers before you plan around it.

One honest limit: when the vulnerable code lives in a vendor appliance, a hosted service or an operating system runtime you do not build yourself, the fix still has to come from that vendor or distribution. The steps above apply to the open source dependencies and container images you build and ship, which is where most teams carry the largest and least tracked share of their exposure.

For the manual route (overrides, patch files, private forks), see how to patch an open source dependency without waiting for upstream.

FAQ

Do AI coding agents introduce more vulnerabilities than human developers?

For dependencies, current evidence says yes: 2.46% of agent dependency picks were known-vulnerable against 1.64% for humans, and agent work increased vulnerabilities on net. The comparison is not fully controlled, so treat it as a strong signal, not a verdict.

Why does an AI agent install an old version of a package?

Its version knowledge comes from training data with a cutoff, so it reproduces versions that were common then. Agents also downgrade to fix build conflicts and avoid breaking major upgrades. Without an advisory lookup, it cannot know a version became vulnerable later.

What is slopsquatting?

Registering a package name AI models tend to invent, so people or agents following the model's suggestion install the attacker's code. Seth Larson of the Python Software Foundation coined the term in April 2025.

How often do AI models hallucinate package names?

USENIX Security 2025 research found average rates of at least 5.2% for commercial and 21.7% for open source models, with 205,474 unique invented names across 16 models. Many recur on repeated prompts. Newer models may do better.

Can I stop Claude Code or Codex from adding vulnerable dependencies?

You can cut it sharply: dependency rules in the instruction file, a vulnerability lookup tool such as an MCP server, approval for package installs, and a CI gate on every pull request. Instructions alone fail because the model cannot see post-cutoff advisories.

Should I block AI agents from editing package.json?

Usually not. Agents handle upgrades well once they know the safe version. Require code owner review for new direct dependencies and downgrades, block known exploited and fixable critical versions, and let routine lockfile updates through.

Does npm audit catch everything an agent adds?

No. It reports known advisories in the resolved tree, but not whether a package is new, unneeded or the wrong library with a similar name. Pair it with existence and history checks and human review of new direct dependencies.

How do I know if agent pull requests are making security worse?

Measure net vulnerabilities per pull request (head minus base) split by author type. If agent pull requests add more than they remove, your controls are not working yet.

See how Safeguard would help

The controls above come down to three moments: when the agent chooses a version, when the change is made, and when you check the result. Each demo shows one. To try it on your own code, download the apps at safeguard.sh/download and read the CLI documentation.

This demo shows connecting Claude to the Safeguard MCP server with OAuth and choosing which tools Claude can use. It walks through the custom connector route; adding Safeguard from Claude's connector directory reaches the same place. Once connected, Claude can check vulnerability data before writing a dependency line, which is Layer 2 above.

Click through it at your own pace, or open it full screen.

This is a real Safeguard CLI session. Asked to fix the lodash CVE in a repository, it reads the finding and advisory, asks before each command, edits package.json to lodash 4.17.21, runs npm install and npm test with approval, prints a summary and pushes nothing. That is the correction you want when the gate blocks an agent pull request: the safe version, applied and tested, with a human approving each step.

Click through it at your own pace, or open it full screen.

This demo sets up the Safeguard runner on your own machine, approves it with its device code, then scans from the terminal. Use the same flow to run the base and head scans from the measurement section, so you can see whether an agent pull request raised or lowered your count before it merges.

Click through it at your own pace, or open it full screen.

Quick check

Sources

• Singla, Çakar, Amusuo, Davis: Understanding Security Risks of AI Agents' Dependency Updates (arXiv 2601.00205, abstract)
• Singla et al.: full paper, revised version (arXiv 2601.00205 v2, HTML)
• Spracklen et al.: We Have a Package for You! (arXiv 2406.10279)
• Zhao et al.: Is Vibe Coding Safe? (ICML 2026, PMLR 306)
• OpenSSF: Security-Focused Guide for AI Code Assistant Instructions
• OWASP GenAI Security Project: LLM09:2025 Misinformation
• Cloud Security Alliance: Slopsquatting, AI Code Hallucinations Fuel Supply Chain Attacks
• Wikipedia: Slopsquatting
• Claude Code documentation: Hooks reference
• Safeguard Gold Open Source

Last reviewed: 22 October 2026. Research on agent dependency behaviour is moving quickly; check the linked papers for revisions.