The main red flags in a security AI demo are a scripted dataset you cannot change, outputs with no evidence trail, no example of failure, answers that dodge "how does it work", and a feature status that is never stated. Any one is a reason to ask for a hands on trial before you decide.
Red flags and the question each one deserves
| What you see | What to ask |
|---|---|
| Perfect results on sample data | Can we run it on our repository now? |
| Confident output, no reasons | Show me why for this one |
| Never fails | Show me a case where it was wrong |
| "Proprietary AI" and nothing else | Which step uses a model, which is deterministic? |
| Feature appears only on a roadmap slide | Is it generally available today? |
| Rapid clicking past screens | Please slow down and go back |
Demo hygiene you can insist on
Ask the presenter to use a project you provide, or at least one you pick from a list at random. Ask to see the raw output, not only the dashboard. Ask them to repeat a step. If the answer changes, ask why.
Request the feature maturity in plain words: generally available, early access, or planned. Mixed slides often put all three side by side. Regulators have noticed that AI claims can outrun reality. In March 2024 the SEC said two advisers had "AI washing" problems, meaning statements about AI use that were false or misleading. You do not want to repeat a vendor's overstated claim in your own board paper.
Green flags worth noticing
- The vendor volunteers limits before you ask.
- Outputs link to underlying evidence you can open.
- There is a documented way to override or disable a feature.
- A customer or engineer, not only sales, answers technical questions.
- They offer a trial on your data without a long approval chain.
Keep your own notes
During the demo, write down every claim as a sentence. After the demo, mark each as shown, said, or not addressed. Only "shown" earns full credit. Send the list back to the vendor and ask for written confirmation of the "said" items.
Try it yourself
Click through it at your own pace, or open it full screen.
Click through the gate and look for what the screen shows you versus what it merely tells you.
Where Safeguard fits
Safeguard states which capabilities are generally available and offers a free scan at gold.safeguard.sh/scan, so you can see output on your own project rather than a script.
Quick check
Is a polished demo a bad sign?
Not by itself. It is a bad sign only if you cannot go off script.
What if a feature is early access?
That is fine if it is labeled. Price and plan around it as unproven.
Sources
- SEC press release 2024-36, 18 March 2024: https://www.sec.gov/newsroom/press-releases/2024-36
Last reviewed: 19 October 2026.
