How to evaluate AI security claims from vendors: an engineer's checklist

Treat every AI security claim as a hypothesis you can test. Ask what was measured, on what data, against which threats, and what the failure rate was. A claim you cannot reproduce on your own sample, or that has no stated scope, is marketing until shown otherwise.

Why are AI claims harder to verify?

Traditional security tools have decades of test methods. AI features add non-determinism, shifting models and benchmarks the vendor chose. NIST's AI Risk Management Framework asks organizations to map, measure and manage AI risks, and it stresses documenting context and limitations. That gives you a standard to hold a vendor to: do they know and state where the system fails?

What questions expose a weak claim?

Detection rate with no false-positive rate. "Catches 99 percent of threats" is meaningless without the number of false alarms and the test set. Ask for both, and ask who built the test set.

"AI-powered" with no description of the task. What does the model actually decide? Classify, rank, generate, act? Where is a human in the loop?

Benchmarks on public data. Public benchmarks leak into training data. Ask for results on data the model could not have seen, or run your own sample.

Autonomy claims. If a product acts on its own, ask what it is allowed to touch, how that is limited, how you stop it, and what its audit trail shows.

Compliance language. "Aligned with" and "certified" are different statements. Ask for the report, the scope and the date. A framework alignment is not an attestation.

What does a good answer look like?

A constructed example: a vendor says its AI triage cuts alert volume by 60 percent. A strong response names the dataset (your alerts from a pilot, say), the method (analysts reviewed a random sample of dismissed alerts), and the miss rate (missed true positives found in that sample). A weak response cites a percentage from a customer anecdote.

You want the failure case in the first answer, not the fifth. Vendors who volunteer limitations are easier to trust.

What can you test yourself?

Run a bake-off on your data. Hold out a sample with known answers. Include hard cases and a few adversarial ones. Record accuracy, false positives, time saved and the reviewer's effort to check results. Repeat it after a model update to see whether results drift.

Also check data handling: what the vendor sends to model providers, retention, and training use.

What red flags show up in sales material?

Look for superlatives with no method, such as "the most accurate". Look for coverage claims stated as a count of integrations or models with no indication of depth. Watch for before-and-after charts with no axis labels, and for testimonials standing in for measurement. Be wary when every limitation is described as "coming soon".

Also note what is missing. If a vendor sells an autonomous agent but says nothing about its permission model, rollback or audit log, that is the answer to a question you should have asked. Put your questions in writing before demos and score answers afterward, so a polished presenter cannot substitute charm for evidence.

How to check or apply this

1. Write each claim down in your own words and attach a test to it.
2. Ask for scope, data, method, false-positive rate and date for every number.
3. Run a pilot on your own data with a held-out answer set.
4. Ask what the system cannot do and where it fails.
5. Request documentation of data flows, subprocessors and retention.
6. Distinguish alignment claims from audited attestations and ask for the report.
7. Re-test after major model or product changes.
8. Write findings into the procurement record.

We apply the same standard to ourselves: Safeguard's own claims should be checked against the product, and our free scan at gold.safeguard.sh/scan lets you try it on your own dependencies.

Quick check

Sources

• NIST AI Risk Management Framework
• OWASP Top 10 for LLM Applications
• NIST AI 100-2 E2025, Adversarial Machine Learning taxonomy

Last reviewed: 18 October 2026.