AI helps application security teams read more code, triage faster and draft fixes, but it adds risks: confident wrong answers, data exposure, prompt injection and tools with too much authority. Use it where a human or a deterministic test can check the result.
Where does AI help?
Triage. Scanners produce more findings than people can read. A model can group duplicates, explain a finding in plain terms and point to the code involved.
Code review. It can spot suspicious patterns in a pull request and propose a safer version.
Fix drafting. Turning a vulnerability description into a patch, or a dependency bump into a pull request, saves time. The best case is a fix that a test suite validates.
Coverage. Models can read languages, frameworks and config formats that a hand written rule set does not yet handle.
Documentation and training. Explaining why a finding matters, in terms a developer will read.
What are the risks?
Wrong with confidence. A model can invent a vulnerability, miss a real one or suggest a fix that breaks behavior. NIST's secure software development guidance for generative AI, SP 800-218A, treats this as part of secure development: it adds practices specific to AI model development on top of the Secure Software Development Framework, and it is meant for model producers, producers of AI systems that use models and the buyers of those systems.
Prompt injection. OWASP lists Prompt Injection as LLM01. Code, comments, issues and documentation that a model reads can carry hidden instructions. A reviewer model that reads an untrusted pull request is exposed to this. OWASP's own guidance is that it is unclear whether fool proof prevention exists, so limit what the model can do after reading hostile text.
Excessive agency. OWASP LLM06 describes systems that are given the ability to call functions or interface with other systems and take damaging actions in response to unexpected or manipulated output. An AI assistant that can merge code, change CI settings or read secrets is a high value target.
Data exposure. Sending proprietary code, secrets or customer data to a model service raises privacy and contract questions. Ask where the data goes, how long it is kept and whether it trains anything.
Insecure generated code. Models can reproduce insecure patterns, and developers may trust output more than they should. Review and test AI written code the same way as any other.
Supply chain. Models suggest packages, and a suggestion may name one that does not exist or that an attacker has registered. Check every dependency before adding it.
How should you decide where to use it?
Ask two questions. Can a result be verified cheaply, by a test, a type check, a deterministic scanner or a reviewer? And what is the worst case if it is wrong? Use AI freely where checks are cheap and the worst case is small, such as drafting a description or a unit test. Add gates and approvals where the worst case is large, such as changing production access.
Constructed example: the helpful fix bot
A team lets an assistant open pull requests that bump vulnerable dependencies. The suggestions are mostly good. One bump changes a function signature, and the tests do not cover it. The assistant's pull request goes through review because it looks routine, and a bug reaches production. The team adds required tests and reachability context to each bot pull request, plus a rule that the bot can propose but never merge.
How to check or apply this
- List where AI touches your pipeline: review, triage, fixes, chat, and what each can read and do.
- Remove write and merge permissions from AI tools unless a person approves.
- Treat untrusted text as hostile input to any model that reads it.
- Require tests and a human review for AI generated changes.
- Check data handling terms before sending code or secrets to a service.
- Verify every suggested package exists, is the real one and is not known to be vulnerable.
- Track outcomes: how many AI suggestions were accepted, and how many caused rework.
Safeguard applies this approach with reachability analysis to show whether a vulnerable function is actually called, and with autonomous remediation that opens fix pull requests only when you turn it on, off by default.
Quick check
Should AI be allowed to merge security fixes automatically?
Only with strong tests and clear policy, and many teams keep a human approval. At minimum, start with proposals only.
What is the biggest new risk?
It depends on setup, but giving an AI tool broad permissions while it reads untrusted text is a common and serious combination.
Sources
- OWASP LLM01:2025 Prompt Injection
- OWASP LLM06:2025 Excessive Agency
- NIST SP 800-218A, Secure Software Development Practices for Generative AI and Dual Use Foundation Models
Last reviewed: 18 October 2026.
