The main barriers to adopting AI in cybersecurity are trust in outputs, difficulty measuring risk, opaque behavior, new attack surface on the AI itself, third-party dependency and unclear ownership. None is a reason to refuse AI. Each is a reason to start with narrow, reviewable tasks and to measure results.
Can you trust what the model says?
NIST's Generative AI Profile (AI 600-1) calls this confabulation: generative systems producing confidently stated but erroneous or false content. NIST says it is a natural result of how the models work, since they approximate the statistical distribution of their training data. In a security setting, an invented indicator, a wrong remediation step or a fabricated package name wastes time or does harm.
The practical answer is to treat output as a draft that something checks. Put the model where its answers can be verified cheaply, such as summarizing an alert whose raw data sits next to the summary, and keep a human decision in front of any change that is hard to undo.
Can you measure the risk?
NIST's AI Risk Management Framework is candid here. It lists measurement challenges: third-party software, hardware and data that complicate measurement; the current lack of consensus on reliable and verifiable metrics; risks that differ between early lifecycle stages and real-world operation; inscrutability caused by limited explainability or documentation; and the lack of a human baseline for comparison. It adds that being unable to measure a risk does not mean the risk is high or low.
For security teams, this means "does the tool work" must be answered with your own test. Run it on a labeled sample of past alerts or findings, compare with your analysts' decisions, and track misses as well as hits.
Does AI add attack surface?
Yes, and it is a barrier that is easy to underestimate. NIST names two main information security risks of generative AI: it could lower the barriers to offensive capabilities, and it expands the attack surface because the AI itself is vulnerable to attacks such as prompt injection and data poisoning. OWASP's guidance on excessive agency adds that an agent with too many tools, permissions or autonomy can be steered into damaging actions by manipulated input.
So a security assistant that reads untrusted text, such as phishing emails or logs, and can also act on systems is exactly the kind of tool that needs least privilege and approval steps.
Who owns the risk?
The AI RMF notes that risk perspectives differ between those who develop a model and those who deploy it, and that deployers may not recognize risks their particular use creates. In a vendor product, the supplier may not share how it measured risk. Contracts, documentation and a named internal owner close the gap. The RMF's GOVERN function exists for this reason: policies, roles and accountability across the organization.
What about data and privacy?
Sending logs, tickets or code to an external model service raises data handling questions. NIST lists data privacy among the generative AI risks. Check what is sent, whether it is retained or used for training, and whether a private or self-hosted option is available for sensitive material.
A constructed example
A security operations lead wants an assistant to triage phishing reports. She starts with a pilot: the assistant reads the reported message headers and body, proposes a verdict and explains why, and can only write to a review queue. Analysts compare its verdicts with their own for four weeks. Findings: it is right on obvious cases, weak on targeted messages, and it once invented a sender reputation. The team keeps it for the obvious cases, adds a rule that every verdict cites the header field it relied on, and expands scope only after the miss rate is documented.
How to check or apply this
- Pick a narrow task with a verifiable answer, and write down what success and failure look like.
- Test on your own labeled data before trusting vendor claims, and keep a record of misses.
- Give the tool read-only access first, then the smallest write scope needed, with human approval for irreversible actions.
- Treat everything the model reads as untrusted input, and everything it produces as unverified.
- Review what data leaves your environment, and choose deployment options accordingly.
- Assign an owner and a review date, and decide in advance what result would make you stop.
Safeguard's approach on the product side offers a choice of Full AI, Zero AI or bring your own model, so teams can set how much AI touches their code and findings. Autonomous fix pull requests are off by default.
Quick check
Is the biggest barrier technical or organizational?
Both matter, but NIST's guidance points to measurement and accountability as the hard parts. Without your own test data and a named owner, technical quality is hard to judge.
Should AI be allowed to make changes on its own in security tools?
Start with read-only and review-queue access. Expand only for reversible actions with measured accuracy, and keep human approval for irreversible ones.
Sources
- NIST AI 100-1, AI Risk Management Framework 1.0, section 1.2 on risk measurement challenges
- NIST AI 600-1, Generative AI Profile, sections on confabulation and information security
- OWASP GenAI, LLM06:2025 Excessive Agency
Last reviewed: 18 October 2026.
