In November 2022, researchers at Stanford ran a controlled study asking 47 developers to write security-sensitive code — some with access to an AI assistant, some without. The result, published as "Do Users Write More Insecure Code with AI Assistants?" (Perry et al., CCS '23, with Dan Boneh as a co-author), was not subtle: participants using the assistant produced significantly more SQL injection and weak-encryption flaws, and rated their own code as more secure than the control group did. That gap — AI making developers faster and more confident while quietly lowering the security bar — is the central problem every engineering and security-operations team now has to manage, and it's a large part of why ai in cybersecurity has become its own conversation rather than a subset of general AI adoption. It is not a reason to avoid AI tooling; adoption is already too far along for that, and Anthropic's Model Context Protocol (MCP), released November 25, 2024 and since adopted by OpenAI and Google DeepMind, has made it trivial to wire LLMs directly into build systems, ticket queues, and cloud consoles in months rather than years. The question is how to get the productivity gains without handing attackers a new, uncurated set of entry points. This piece walks through the concrete risks — hallucinated packages, over-scoped agent tooling, prompt injection — and what actually mitigates each one.
Why do AI coding assistants make developers write less secure code?
AI coding assistants make developers write less secure code partly because of automation bias — the tendency to trust a fluent, confident output without scrutinizing it — and partly because the models were trained on public code that itself contains vulnerable patterns. In the Stanford study, participants working with the AI assistant were both less secure and more likely to believe their code was safe, a combination the authors called particularly concerning because it suppresses the instinct to double-check. The same study found a mitigating factor: users who engaged critically with the tool — rephrasing prompts, adjusting parameters, pushing back on a first answer instead of accepting it — produced measurably fewer flaws than those who accepted the first suggestion. That is an actionable finding, not just a warning: treating AI-generated code as a first draft requiring the same review, static analysis, and taint tracing as human-written code closes most of the gap. Pattern matching and SAST rules do not care who or what wrote the vulnerable string-concatenated SQL query — a Snyk cybersecurity scan or any other static analysis engine runs the same taint tracing against an AI-generated pull request that it would against a human one.
What is "slopsquatting" and how serious is the risk?
Slopsquatting is the practice of registering a package name on PyPI or npm that an LLM is known to hallucinate, so that any developer who copy-pastes the AI's suggested pip install or npm install command downloads an attacker-controlled package instead of a typo. USENIX Security 2025 research, "We Have a Package for You!", generated 576,000 code samples across 16 different LLMs and found that 19.7% contained at least one hallucinated package reference — open-source models hallucinated at a 21.7% rate versus 5.2% for commercial models. Because these hallucinations are often repeatable (the same prompt tends to produce the same fake name), an attacker only needs to pre-register a handful of frequently hallucinated names to catch a meaningful slice of developers who trust an AI suggestion enough to install it without checking. The mitigation is unglamorous but effective: never let an AI-suggested dependency reach a lockfile without it first passing through the same dependency-confusion and typosquat screening every other new package gets.
What does OWASP's Top 10 for LLM Applications say about the new risk surface?
OWASP's Top 10 for LLM Applications, updated for 2025, ranks Prompt Injection as the #1 risk and Sensitive Information Disclosure as #2 — up from #6 in the prior list — with Supply Chain Vulnerabilities, Data and Model Poisoning, Excessive Agency, and System Prompt Leakage rounding out categories directly relevant to engineering and SecOps tooling. Excessive Agency is the one that matters most once AI moves from autocomplete into agentic workflows: it names the risk of granting an LLM-driven agent more permissions, tools, or autonomy than the task requires, so a single successful prompt injection or hallucinated instruction can cascade into a real action — a merged pull request, a revoked credential, a modified cloud resource — instead of just bad text. Sensitive Information Disclosure's jump up the list reflects how often agents are wired into ticketing systems, code repositories, and internal wikis that contain secrets, customer data, or infrastructure details the model was never meant to surface back to a user or a connected tool.
Why does MCP specifically expand attack surface, and how do you contain it?
MCP expands attack surface because it standardizes exactly the pattern OWASP flags as risky: it gives an LLM client a generic way to discover and invoke arbitrary tools exposed by a server, and the ecosystem of third-party MCP servers grew far faster than any shared standard for vetting, sandboxing, or scoping what those servers are allowed to do. Any cybersecurity SDK that exposes tool-calling to an LLM client inherits this same tradeoff — convenience for the developer wiring it up, and a broader blast radius if the client is ever compromised. A developer who adds an unvetted community MCP server to get one convenience feature has potentially handed that server's authors the same tool-calling surface as any other connector in their client — including, depending on configuration, standing credentials with broad scope. Containing this comes down to the same access-control discipline used for any third-party integration: bearer-token auth scoped to a single tenant rather than shared static credentials, self-hosting sensitive MCP servers inside your own network perimeter rather than depending on a public endpoint, and logging every tool call so unexpected invocations — a tool called in a sequence no legitimate workflow would produce — are visible rather than silent. Safeguard's own MCP server follows this model: it authenticates with a scoped Safeguard API key tied to a tenant, and it can be run self-hosted rather than pointed at the hosted endpoint, so a Claude or Cursor client gets read/action access to vulnerability and SBOM data without anyone handing out a standing cloud credential. The same scoping discipline should apply to any cybersecurity SDK or MCP integration a team wires into an agent, not just ours — a permissive token handed to a well-behaved client today is still a permissive token if that client is compromised tomorrow.
How should teams think about AI models themselves as supply-chain artifacts?
Teams should treat AI models and their weights the same way they treat any other third-party dependency, because the mechanics of compromise are structurally identical: an artifact of uncertain provenance gets loaded into a production system and executes with the permissions of whatever loaded it. This is one more place where ai and cybersecurity concerns overlap directly with traditional supply-chain hygiene, rather than requiring a separate discipline. PyTorch's default pickle serialization format can execute arbitrary code at load time, which means a model file downloaded from an unvetted source is not meaningfully different from an unvetted binary — a model card looking legitimate doesn't guarantee the weights are safe to deserialize. This is also where AI tooling and secrets management intersect directly: a fine-tune trained against internal logs or scripts can bake in an API key or database token that then ships every time the model weights are copied, exported, or shared with a vendor, the same way a .env file accidentally committed to Git leaks a credential. The fix set is the same as for source code — verify provenance and signatures before loading an artifact, scan the artifact contents (weights included) for embedded secrets and unsafe deserialization, and maintain a queryable inventory (an AI-BOM, in CycloneDX ML-BOM or SPDX AI-profile format) so that when a new model-loading CVE or a leaked-secret pattern surfaces, you can answer "are we affected" in minutes.
How Safeguard Helps
Safeguard applies the same reachability-first discipline to AI-introduced risk that it applies to traditional application security — treating ai and cybersecurity as one continuous problem rather than two separate teams passing findings back and forth. Secrets scanning covers model weights and fine-tunes as a first-class surface, not an afterthought, catching tokens baked into a training run the same way it catches an API key committed in a diff — and every issuer-specific finding is verified against a live check before it's raised as critical, so teams aren't triaging noise. The AI-BOM tracks weight provenance, signing status, and fine-tune lineage for every model in your environment, and load-time policy can block an unsigned or unsafely-serialized model before it ever runs, the same way a dependency policy blocks a vulnerable package from merging. For agentic workflows specifically, Safeguard's own MCP server is scoped to a tenant-bound API key and can be run self-hosted instead of against the hosted endpoint, so connecting Claude, ChatGPT, or Cursor to your vulnerability and SBOM data doesn't mean issuing a standing credential to a third-party endpoint — and tool-call logging gives security teams the audit trail to catch an agent doing something a legitimate workflow never would.