To choose an AI detection and response product, check what AI activity it can see, how it decides something is an attack, which response actions it can take, and where your prompts and outputs are stored. Test it against your own AI systems, since results on a vendor demo app do not transfer.
AIDR is a young category with no settled definition. Some products watch model traffic, some watch agent tool calls, some do both. Start by writing down which of your systems you need covered.
Step 1: Map what you run
List the places AI shows up. A customer chatbot. An internal assistant with access to documents. Coding assistants on developer machines. Agents that call tools through Model Context Protocol servers. Scripts that call a hosted model API. Each has different data, different users and different failure modes, and a product that covers one does not cover them all.
Criteria: visibility
Telemetry sources. Ask exactly what is captured: prompts, responses, tool calls, retrieved documents, identity of the caller. If the product only sees the prompt, it cannot tell that an agent just exported a table.
Deployment point. Is it a proxy, an SDK, a log collector or a browser extension? Each has blind spots. A proxy cannot see traffic that goes around it.
Coverage of your providers. Check support for each model provider and framework you actually use.
Criteria: detection
What it looks for. Map claims to the OWASP Top 10 for LLM Applications 2025, which includes prompt injection, sensitive information disclosure, improper output handling and excessive agency. Ask for each item whether the product detects it, prevents it, or does neither.
How detection works. Rules, classifiers, behavioral baselines, or a mix. For classifiers, ask how false positive rates are measured on data like yours.
Transparency. Can you see why something was flagged? A flag with no reason cannot be tuned or defended to the person it blocked.
Criteria: response
Actions. Block a request, redact output, revoke an agent's token, disable a tool, open a ticket. Ask which are automatic and which need approval.
Latency. Anything in the request path adds delay. Ask for measurements in your environment, not a marketing figure.
Failure mode. If the product goes down, does traffic fail open or closed? Choose per system. A public chatbot and a payment-adjacent agent deserve different answers.
Criteria: data and trust
Where do prompts go? Prompts and outputs can contain customer data and secrets. Ask where they are stored, for how long, who at the vendor can read them and whether you can keep them in your own environment.
Attestations. Ask for current security reports and read their scope. A badge on a web page is not evidence.
Model use inside the product. If the vendor uses a model to judge your traffic, ask which one and whether your data trains anything.
Criteria: fit
Identity. Can it tie actions to a specific agent and owner, or only to an API key?
Integration. Check connections to your logging, ticketing and identity systems.
Exit. Ask what you can export, and what happens to stored data at the end.
A trial plan
Pick one real system and one test environment. With the owners' agreement, replay a set of harmless test prompts that mimic documented risks, such as an instruction hidden in a retrieved document, and see what the product notices. Record detections, misses, false positives, latency and the clarity of explanations. Do not publish or reuse any harmful content outside the test.
What should the decision rule look like?
Write it before the trial. For example: the product must detect the risks you ranked highest, add acceptable latency to the systems in scope, explain each flag in terms an engineer can act on, and keep data where your policy requires. If a product misses one of those, record it, even if the demo was polished. A written rule protects you from a persuasive sales conversation and gives the vendor a fair target.
What to do
- List your AI systems and rank them by data sensitivity.
- Map each OWASP LLM risk to the systems where it applies.
- Ask vendors to answer detection and response questions per risk, in writing.
- Run a trial with harmless test prompts on a test copy.
- Confirm data storage, retention and exit terms before signing.
Where Safeguard fits
Safeguard offers an MCP Guardrail proxy and agent identity tracking, both generally available, and an AI Gateway that is in early access. These cover tool access and agent permissions in software delivery. They are not a complete AIDR product, and you should evaluate them against your own list.
Quick check
Do I need AIDR if I only use a hosted chatbot?
Perhaps not a full product. Start with logging, access limits and a review of what data the chatbot can reach. Add tooling when the risk grows.
What is the most overlooked question?
Where your prompts and outputs are stored, since they can hold customer data and secrets.
Sources
Last reviewed: 18 October 2026.
