In the space of eighteen months, AI-assisted coding went from a novelty to the default. GitHub reported in 2024 that Copilot alone was writing close to half the code in files where it was enabled, and by 2026 most engineering orgs are running a mix of Copilot, Cursor, Claude Code, and internal chat tools against production repositories every day. That speed has a cost: LLMs invent package names that don't exist, copy license-encumbered snippets without attribution, and generate code that looks reviewed but never touched a human security gate. Sonatype, long the incumbent in software composition analysis, has started bolting AI risk features onto its existing repository-firewall model. That's a reasonable first move, but "AI governance" and "dependency scanning" are not the same problem. This post breaks down what AI and LLM governance actually requires in a software development context, where the market — including Sonatype — is still catching up, and how Safeguard approaches it differently.
What Is AI Governance in Software Development, and Why Is It Suddenly Urgent?
AI governance in software development is the combination of policy, provenance tracking, and automated enforcement that determines which AI tools can touch your codebase, what they're allowed to produce, and how that output is verified before it ships. It's urgent now because the attack surface changed faster than the tooling did: Stanford's 2025 AI Index found that over 70% of professional developers use an AI coding assistant at least weekly, up from roughly a third in 2023, and internal Safeguard customer telemetry from Q1 2026 shows AI-authored commits now account for 30-45% of pull requests in active repositories at mid-size engineering orgs. Most of those orgs still have zero policy governing what those tools can pull in, no audit trail linking a line of code back to the model and prompt that produced it, and no gate that treats AI-suggested dependencies any differently than human-chosen ones. Governance isn't a compliance nicety here — it's the only mechanism that closes the gap between how fast code gets generated and how fast anyone can verify it's safe.
How Does Sonatype Approach AI and LLM Risk Today?
Sonatype's approach to AI risk is an extension of its existing dependency firewall, not a purpose-built governance layer. Sonatype Repository Firewall and Nexus already block known-malicious packages at the point of resolution, and the company's messaging since late 2024 has folded "AI-generated dependency risk" into that same pipeline — flagging packages that match patterns associated with typosquatting or hallucinated names. That's useful, but it's fundamentally reactive and dependency-centric: it catches a bad package after a model suggests it, rather than governing the model interaction itself. Sonatype's own 2024 State of the Software Supply Chain report highlighted a 156% year-over-year rise in malicious open source packages, and its public materials frame AI risk almost entirely as "more bad packages arriving faster," which is true but incomplete. It says little about prompt/output logging, model-provenance attestation, license risk from generated code, or policy differences between an approved internal LLM and an ungoverned public chatbot. For teams that need to answer "which model produced this code, under what policy, and was it reviewed," Sonatype's tooling doesn't have a native answer — because it wasn't built to track models, it was built to track packages.
How Often Do AI Coding Assistants Introduce Insecure or Nonexistent Dependencies?
AI coding assistants hallucinate package names often enough that it now has a name — "slopsquatting" — and researchers have measured it directly. A 2024 study from the University of Texas at San Antonio, Virginia Tech, and the University of Oklahoma tested 16 code-generation models across 576,000 code samples and found that roughly 5.2% of dependencies suggested by commercial models, and up to 21.7% from open-source models, referenced packages that don't exist on npm or PyPI. Attackers don't need to guess at these names — they can pre-register the most commonly hallucinated ones and wait. Separately, a 2023-2024 analysis of GitHub Copilot suggestions by researchers at NYU found that roughly 40% of generated code contained a security weakness mapped to a CWE category when tested across common vulnerability scenarios. Neither of these numbers is a reason to ban AI-assisted coding — it's a reason no organization should let generated dependencies or generated code merge without the exact same (or stricter) automated policy gate applied to human-written code, plus a check that specifically targets hallucinated-package risk, which standard SCA tools weren't designed to catch.
What New Regulations Are Forcing Engineering Teams to Govern AI Output?
Three regulatory tracks are converging on engineering teams in 2026, and none of them are optional for companies selling into regulated industries. The EU AI Act entered into force on August 1, 2024, with obligations for general-purpose AI models applying from August 2, 2025, and the bulk of high-risk AI system requirements — documentation, risk management, human oversight — becoming enforceable on August 2, 2026, just weeks from this writing. In parallel, NIST published its AI Risk Management Framework in January 2023 and followed with the Generative AI Profile (NIST AI 600-1) in July 2024, which is increasingly cited as the de facto baseline U.S. auditors ask about even outside federal contracts. And critically for SaaS vendors, AICPA guidance and SOC 2 auditors have started asking direct questions about AI usage in the software development lifecycle during Trust Services Criteria audits — specifically around change management (CC8.1) and system monitoring (CC7.2) when AI tools have write access to production code or infrastructure. None of these frameworks tell you which specific LLM to use. They all tell you that you need a documented policy, an audit trail, and evidence that generated output goes through the same risk controls as everything else — which is exactly the gap most SCA-first tools leave open.
What Does a Practical AI Governance Policy Actually Require?
A practical AI governance policy needs five concrete controls, not a PDF nobody reads: an approved-tool list (which models/assistants are sanctioned, and under what data-handling terms), provenance tagging (every AI-touched commit or PR labeled with the tool and, ideally, the prompt context), a dependency gate that specifically checks AI-suggested packages against real registry data before merge, a human-review requirement for AI-generated code above a defined risk threshold (e.g., anything touching auth, crypto, or infra-as-code), and an audit log that a SOC 2 or EU AI Act assessor can actually query. Most engineering orgs today have zero of these five in place; Safeguard's own 2025 customer assessments across 40+ mid-market and enterprise codebases found that fewer than 1 in 10 had any automated policy distinguishing AI-generated commits from human ones, and none had a queryable audit trail linking generated code back to model and prompt. Bolting a rule onto an existing SCA scanner gets you maybe one of these five controls — the dependency gate — and even that only if the scanner's threat feed is updated fast enough to catch newly registered hallucinated packages, which is a race most legacy tools are still losing.
How Safeguard Helps
Safeguard was built as a supply chain security platform first, which means AI governance isn't a feature bolted onto a package scanner — it's wired into the same provenance and policy engine that already covers your SBOM, build attestations, and dependency risk. Concretely: Safeguard tags commits and pull requests with the AI tool that produced them wherever that signal is available (Copilot, Cursor, Claude Code, and internal LLM gateways), so every AI-authored change carries a provenance record instead of looking identical to human-written code in your audit trail. Safeguard's dependency policy engine checks AI-suggested packages against live registry and typosquat/hallucination signals at the point of resolution — not just against a static known-bad list — so a model inventing a plausible-but-nonexistent package name gets caught before it reaches a lockfile. Every AI-touched change that meets a configurable risk threshold (auth, secrets, infra-as-code, crypto primitives) is automatically routed into a mandatory human-review gate in CI, enforced as policy-as-code rather than left to reviewer discipline. And because Safeguard already generates SBOMs and compliance evidence for SOC 2 and similar frameworks, AI governance evidence — tool inventory, review logs, policy exceptions — rolls up into the same audit package your compliance team already pulls, instead of becoming a separate spreadsheet someone maintains by hand. For teams evaluating Sonatype's AI messaging against what they'll actually need for an EU AI Act or SOC 2 audit in the next twelve months, the practical question isn't "does it flag bad packages" — it's "does it govern the model, the output, and the evidence trail together." That's the gap Safeguard is built to close.