Safeguard
AI Security

Securing AI coding assistants: governance patterns for to...

AI coding assistants like Claude Code and Cursor now write, install, and execute code with minimal oversight. Here's the governance framework that closes the gap JFrog's artifact scanning leaves open.

James
Principal Security Architect
7 min read

In February 2025, Anthropic shipped Claude Code as a terminal-native agent that can read a repository, write commits, and run shell commands with minimal supervision. Three weeks later, Cursor's parent company Anysphere was raising funding at a $9.9 billion valuation on the strength of "agentic" editing. By mid-2025, engineering teams were routinely granting these tools filesystem access, package-install rights, and even push permissions — without updating a single access policy. Then, in March 2025, security researchers at Pillar Security disclosed the "Rules File Backdoor," showing that hidden Unicode characters in a .cursorrules or .github/copilot-instructions.md file could silently instruct an AI assistant to inject vulnerable code. The tools that write your code are now part of your supply chain, and most governance programs still treat them like a text editor. This post lays out what changed, what broke, and the governance patterns that actually hold up — including where artifact-focused vendors like JFrog stop short.

Why do AI coding assistants change the software supply chain threat model?

Because they collapse three previously separate trust boundaries — code authorship, dependency selection, and execution — into a single non-deterministic process. A human developer choosing to npm install a package makes a discrete, auditable decision. Claude Code or Cursor's Composer mode, given an ambiguous prompt like "add PDF export," will autonomously decide which library to pull, generate the install command, and often run it itself if given shell access. Anysphere's own 2025 usage data put Cursor's install base above 1 million developers, and GitHub reported Copilot users accept roughly 30% of inline suggestions without modification. Multiply that acceptance rate across an agent that can now also execute terminal commands (Claude Code, Cursor's Agent mode, Copilot Workspace), and a single hallucinated dependency name or over-permissioned tool call becomes a production incident, not a code review comment. Traditional SCA tools scan what landed in the lockfile; they were never built to govern the moment before that, when an LLM decided what belonged there.

What actually happened with the March 2025 "Rules File Backdoor"?

Attackers found they could hide adversarial instructions inside the configuration files AI assistants read for project context, using zero-width Unicode and bidirectional text markers invisible in normal editors. Pillar Security's disclosure showed that a poisoned .cursorrules file — the kind teams commonly commit to a shared repo or copy from a public template — could instruct Cursor to quietly insert a backdoor or exfiltration snippet into future AI-generated code, with no warning shown to the developer approving the diff. The same technique applied to GitHub Copilot's instruction files. This mattered because rules files are treated as harmless project metadata: they're rarely code-reviewed with the same scrutiny as a package.json change, and they're increasingly shared across teams and even public template repositories as "prompt engineering" best practice. It was the first widely disclosed proof that AI assistant configuration is itself an attack surface with write access to your codebase — a category no existing SCA or SAST tool was scanning for at the time.

How common is package hallucination in AI-generated code?

Common enough that researchers gave it a name — "slopsquatting" — and measured it at double-digit rates. A 2024 academic study analyzing 576,000 code samples from 16 popular LLMs found that roughly 19.7% of package recommendations referenced dependencies that didn't exist, with open-source models hallucinating packages more often than commercial ones and open-source models hallucinating in some tests over one-third of the time. Attackers have already operationalized this: once a hallucinated package name repeats across enough AI completions, someone registers it on npm or PyPI with a malicious payload, betting that a developer's coding assistant will suggest it again and they'll pip install on autopilot. Combine that with the September 2025 "Shai-Hulud" npm worm, which compromised several hundred packages by harvesting maintainer tokens and republishing trojanized versions of widely used libraries, and the risk compounds: an AI assistant confidently suggesting a plausible-sounding, newly-registered, low-download package is now a realistic initial-access vector, not a hypothetical one.

Why isn't an artifact scanner like JFrog Xray enough on its own?

Because it evaluates the package after the AI assistant has already chosen it, not the decision, permissions, or context that produced that choice. JFrog's strategy — Xray for binary composition analysis, plus the 2024 Qwak acquisition folded into the JFrog AI Catalog for curating vetted models and datasets — is built around the artifact repository as the control point: block a bad package or model before it's pulled into a build. That's necessary, but AI coding assistants introduce risk upstream of the registry pull: an over-permissioned MCP server that gives Claude Code arbitrary filesystem write access, a Cursor session with no audit trail of which suggestions were AI-generated versus human-written, or a CI credential exposed to an agent that can already execute shell commands. Anthropic's own Model Context Protocol, introduced in November 2024, had grown to thousands of community-built MCP servers by mid-2025, the large majority unaudited and installed with a single config-file edit and no permission review. A repository scanner never sees that moment. Governance has to start at the point where the assistant is granted a capability, not the point where its output reaches a registry.

What does real governance for Claude Code and Cursor look like in practice?

It looks like scoped, revocable capability grants rather than blanket trust, applied at four layers: identity, tool permissions, provenance, and review. Concretely: (1) issue AI assistants short-lived, narrowly-scoped credentials instead of reusing a developer's full CI token — Claude Code's --allowedTools and Cursor's permission prompts exist precisely so teams can deny shell or network access by default and grant it per-task; (2) maintain an allowlist of approved MCP servers and rules-file sources, with signed or hash-pinned configuration so a poisoned .cursorrules can't slip in through a template repo; (3) tag AI-generated commits and dependency additions in commit metadata so security review can apply extra scrutiny — one 2023 Stanford study found developers using AI assistants wrote measurably less secure code while reporting more confidence in its safety, which is exactly the population that needs a second check; and (4) require human approval gates before an agent can execute install commands or push to protected branches, logged centrally rather than left in a local terminal history that disappears when the session ends. None of this is exotic — it's the same least-privilege and audit logic SOC 2 already demands of human engineers, applied to a new class of non-human actor that happens to write code at 10x the speed.

How Safeguard Helps

Safeguard treats AI coding assistants as a governed identity class, not a productivity add-on bolted onto CI. Where artifact-centric platforms pick up risk at the registry, Safeguard instruments the point where Claude Code, Cursor, Copilot, and their MCP integrations are actually granted capabilities — giving security teams a live inventory of which agents can execute shell commands, which MCP servers are connected and by whom, and which rules/config files are pinned versus mutable. Every AI-originated commit, dependency addition, and tool invocation is tagged and logged, so a hallucinated package or a suspicious .cursorrules change surfaces in review before it merges, not after an incident retro. Policy enforcement is scoped per-repository and per-agent — block shell execution for a junior team's Cursor instance, allow it with approval gates for a platform team's Claude Code deployment — and every action rolls up into the same SOC 2-aligned audit trail your compliance team already relies on for human access reviews. The goal isn't to slow down agentic coding; it's to make sure the fastest-growing part of your developer workforce is provisioned, monitored, and revocable exactly like every other identity with write access to production code.

Never miss an update

Weekly insights on software supply chain security, delivered to your inbox.