AI security is the discipline of protecting AI models, the data pipelines that train and feed them, and the applications built on top of them from attacks that target their unique failure modes — prompt injection, training data poisoning, model theft, and unsafe autonomous tool use, alongside the traditional code and dependency risks that ship inside AI systems. It is not a rebrand of AppSec with "AI" pasted on front. A SQL injection flaw and a prompt injection flaw look nothing alike to a scanner, and a leaked model checkpoint is a different loss event than a leaked API key. The scope has grown fast: OWASP published its first Top 10 for LLM Applications in August 2023 and revised it again in November 2024, adding categories like "excessive agency" that didn't exist in mainstream security vocabulary two years ago. Gartner projected in 2024 that by 2027, over 40% of AI-related data breaches will stem from cross-border GenAI misuse — a category of risk with no equivalent in pre-LLM application security.
What Is AI Security?
AI security is the set of practices and controls used to protect machine learning models, their training data, and the applications that call them from adversarial manipulation, data leakage, and unsafe autonomous behavior. It covers four distinct surfaces: the model itself (weights, architecture, and training process), the data pipeline (training sets, embeddings, and retrieval corpora), the application layer (prompts, system instructions, and output handling), and the agent layer (tool calls, API access, and multi-step autonomous actions). This is broader than "MLSecOps" or "LLM security" alone because it includes supply chain concerns — a compromised model file on Hugging Face is a software supply chain problem, not just a model-behavior problem. In February 2024, JFrog researchers disclosed roughly 100 malicious models hosted on Hugging Face capable of executing arbitrary code on a victim's machine the moment the model was loaded with pickle deserialization, which is functionally identical to a malicious npm package — except the payload was disguised as model weights instead of source code.
Why Is AI Security Different From Traditional Application Security?
AI security is different from traditional AppSec because the attack surface includes the model's behavior itself, not just the code around it — you can't patch a neural network the way you patch a library with a known CVE. A SAST tool can flag a hardcoded credential in source code with high confidence; no equivalent tool can deterministically flag that a 70-billion-parameter model will leak its system prompt when asked the right way, because the vulnerability lives in probabilistic weights, not static syntax. Prompt injection, ranked LLM01 in OWASP's 2025 Top 10 for LLM Applications, has no direct analog in the OWASP Top 10 for web applications. Researchers demonstrated as early as September 2022 that appending adversarial text to a document could hijack an LLM's instructions, and by 2024 real-world incidents followed the pattern — including a widely reported case where a Chevrolet dealership's customer-service chatbot was manipulated into agreeing to sell a 2024 Chevy Tahoe for one dollar. That kind of failure isn't a code defect; it's the model doing exactly what a malicious input told it to do.
What Are the Most Common AI Security Threats Right Now?
The most common AI security threats right now are prompt injection, training data and model poisoning, insecure output handling, and unsafe agentic tool use where an AI system takes real-world actions without adequate guardrails. Prompt injection remains the top-ranked risk in OWASP's LLM Top 10 because it's cheap to execute and hard to fully mitigate — a 2024 study from researchers at ETH Zurich and others showed indirect prompt injection succeeding against production copilots by hiding instructions in emails, web pages, and even code comments the model later ingests. Data poisoning is a supply chain risk one layer removed from the model: in 2023, researchers demonstrated for roughly $60 that they could have purchased expired domains referenced in datasets like LAION-400M and injected poisoned images into a meaningful fraction of a training corpus before anyone rebuilt the dataset snapshot. Insecure output handling shows up when an application trusts LLM output enough to render it as HTML or execute it as code without sanitization, opening the door to classic XSS or command injection through an AI intermediary. And agentic risk is the newest and fastest-growing category — Gartner has flagged agentic AI adoption as one of its top strategic technology trends for 2025, with enterprises granting agents API keys, database credentials, and shell access faster than their access-review processes can keep up.
How Common Are AI-Related Security Incidents?
AI-related security incidents are already common enough to have their own tracking infrastructure: the OECD's AI Incident Monitor and the private AI Incident Database had logged well over 2,500 reported incidents by early 2024, up from a few hundred just three years earlier. IBM's Cost of a Data Breach reporting has separately tracked "shadow AI" — AI tools deployed without security team visibility — as a rising contributor to breach costs, and Cisco's 2024 AI Readiness research found that while the large majority of surveyed organizations were actively using AI, only a minority felt confident they had fully mapped where AI systems touch sensitive data. On the model-supply-chain side specifically, more than 100 malicious models were pulled from Hugging Face after JFrog's February 2024 disclosure, and separate research the same year found thousands of exposed API tokens tied to AI/ML platforms scraped from public code repositories — credentials that, unlike a typical leaked key, often grant direct access to proprietary models and the training data behind them.
What Regulations Govern AI Security?
AI security is governed by a growing patchwork of binding and voluntary frameworks, the most significant being the EU AI Act, the US Executive Order 14110, and NIST's AI Risk Management Framework. The EU AI Act entered into force on August 1, 2024, with prohibitions on the highest-risk AI uses applying from February 2025 and obligations for general-purpose AI model providers phasing in from August 2025 — it's the first comprehensive AI-specific law with fines that can reach up to 7% of global annual turnover for the most serious violations. In the US, President Biden's October 2023 Executive Order 14110 directed NIST to develop guidelines for red-teaming foundation models and required developers of the most powerful models to report training runs and safety test results to the federal government; a January 2025 executive order later revoked it, leaving NIST's AI RMF (published January 2023) and its companion Generative AI Profile as the primary voluntary reference in the US. Separately, ISO/IEC 42001, published in December 2023, gives organizations their first certifiable AI management system standard, filling a role for AI governance similar to what ISO 27001 fills for information security.
How Do You Build an AI Security Program?
You build an AI security program by extending existing AppSec discipline — inventory, threat modeling, testing, and monitoring — to cover models, prompts, and training data as first-class assets alongside code and dependencies. Start with an inventory: most organizations cannot currently answer "which teams are calling which models with what data" without manual effort, which is precisely the shadow AI gap Cisco's research flagged. Next, apply the OWASP LLM Top 10 and MITRE ATLAS (the adversarial ML knowledge base MITRE published in 2021 and has expanded since, modeled after MITRE ATT&CK) as threat-modeling references specific to model-layer risk rather than reusing web-app checklists that don't map cleanly. Red-team the system with adversarial prompts before launch, not after — NIST's AI RMF explicitly recommends this as a "measure" function activity. Finally, treat every model file, dataset, and fine-tuning artifact the way you'd treat a third-party dependency: track its provenance, scan it before loading it, and monitor for drift or tampering after deployment, because an AI-BOM (AI bill of materials) is quickly becoming the model-layer equivalent of the SBOM requirements formalized under Executive Order 14028.
How Safeguard Helps
Safeguard extends software supply chain security practices to the AI layer by generating and ingesting SBOMs that cover model dependencies alongside traditional open-source packages, so security teams get one inventory instead of a blind spot around every model file, weight, and fine-tuning artifact pulled into a build. Reachability analysis applies the same logic to AI risk that it applies to CVEs: instead of flagging every vulnerable model-loading library in a dependency tree, Safeguard surfaces only the ones actually invoked in code paths that process untrusted input, cutting through noise that would otherwise bury a real prompt-injection-adjacent risk under thousands of low-relevance findings. Griffin AI correlates that reachability data with exploit context and business impact to rank AI-related findings the way it ranks any other vulnerability — by real risk, not raw severity score. When a fix exists, whether that's a version bump on a compromised model-serving library or a patched dependency, Safeguard opens an auto-fix pull request so remediation lands in the same PR workflow developers already use, closing the gap between detecting an AI supply chain risk and actually shipping the fix.