The main generative AI security risks are prompt injection, leakage of sensitive data, a compromised supply chain, poisoned data, unsafe handling of model output, and agents with too much authority. NIST and OWASP each publish a list, and they approach the problem from different directions.
What does NIST list?
NIST AI 600-1, the Generative AI Profile of the AI Risk Management Framework, identifies 12 risks unique to or exacerbated by generative AI: CBRN information or capabilities; confabulation; dangerous, violent or hateful content; data privacy; environmental impacts; harmful bias and homogenization; human-AI configuration; information integrity; information security; intellectual property; obscene, degrading and abusive content; and value chain and component integration.
Three matter most to software teams.
Confabulation. NIST defines it as the production of confidently stated but erroneous or false content, colloquially called hallucinations, by which users may be misled. A coding assistant that invents an API or a package name is a security problem when someone installs the invented name.
Information security. NIST describes lowered barriers for offensive cyber capabilities, including automated discovery and exploitation of vulnerabilities.
Value chain and component integration. Non-transparent or untraceable integration of upstream third-party components, improper supplier vetting, and issues that reduce transparency for downstream users. For AI apps, this means models, datasets, plugins and tools you did not make.
NIST also offers a grouping: technical and model risks, misuse by humans, and ecosystem or societal risks.
What does OWASP list?
OWASP's Top 10 for LLM Applications, 2025 edition, ranks:
- LLM01 Prompt Injection
- LLM02 Sensitive Information Disclosure
- LLM03 Supply Chain
- LLM04 Data and Model Poisoning
- LLM05 Improper Output Handling
- LLM06 Excessive Agency
- LLM07 System Prompt Leakage
- LLM08 Vector and Embedding Weaknesses
- LLM09 Misinformation
- LLM10 Unbounded Consumption
OWASP's description of prompt injection is that user prompts alter the model's behavior or output in unintended ways, even through content that is not visible to humans, as long as the model parses it. It adds that retrieval augmented generation and fine-tuning do not fully mitigate it.
How do the two lists relate?
NIST is broad: it covers societal and ethical risk along with security. OWASP is narrow and technical: it lists the weaknesses an engineer can test for in an application. Misinformation in OWASP maps to confabulation and information integrity in NIST. Supply chain maps to value chain and component integration. Sensitive information disclosure maps to data privacy. Use NIST to frame governance and OWASP to write test cases.
A constructed example
A support chatbot reads help articles and customer tickets, then drafts replies and can issue refunds through an internal tool. An attacker submits a ticket containing text that instructs the assistant to refund an order. The model treats the ticket as instructions and calls the refund tool. That one incident touches prompt injection (LLM01), excessive agency (LLM06) and improper output handling (LLM05). The cheapest fix is not a smarter prompt. It is removing the model's ability to issue refunds without human approval.
How to check or apply this
- List every place a model reads untrusted content: tickets, web pages, documents, email, tool output.
- List every action a model can take, and cut it to the minimum. Require human approval for high-impact actions.
- Enforce authorization in downstream systems, not in the prompt.
- Treat model output as untrusted input to the next system. Validate and encode it.
- Track models, datasets, plugins and packages like any dependency, with an owner and a version.
- Keep secrets and personal data out of prompts unless the use case requires them.
- Set limits on usage and cost to avoid unbounded consumption.
- Test with adversarial inputs before release and after every model change.
Safeguard offers an MCP Guardrail for agent tool calls and policies and gates for dependencies, which fit steps 2 and 5.
Quick check
How many risks does NIST list for generative AI?
NIST AI 600-1 lists 12 risks unique to or exacerbated by generative AI.
Which risk should an engineering team address first?
Start with prompt injection combined with excessive agency, since limiting what a model can do reduces the impact of every other failure.
Sources
- NIST AI 600-1, Artificial Intelligence Risk Management Framework: Generative AI Profile, fetched 18 October 2026.
- OWASP Top 10 for LLM Applications, fetched 18 October 2026.
- OWASP LLM01:2025 Prompt Injection.
Last reviewed: 18 October 2026.
