NIST's Center for AI Standards and Innovation (CAISI) tested agent hijacking and found two lessons for anyone evaluating agents: attacks written for a specific system beat known attacks by a wide margin, and trying each attack many times raised the average success rate. A single clean test run does not show an agent is safe.
What was tested
Agent hijacking is a form of indirect prompt injection. An attacker hides instructions in data the agent reads, such as an email, file or web page, so the agent does a task the user never asked for.
CAISI used AgentDojo, an open-source framework from ETH Zurich with four simulated environments: Workspace, Travel, Slack and Banking. Each scenario pairs a normal user task with a malicious injection task. The agent counts as hijacked if it completes the injection. The target was an upgraded Claude 3.5 Sonnet, and red teaming was done with the UK AI Security Institute. CAISI also added injection tasks for remote code execution, database exfiltration and automated phishing.
What it found
The NIST post reports these results:
- The upgraded model resisted previously known attacks much better than its predecessor.
- New red team attacks did far better. Success rose from 11 percent for the strongest baseline attack to 81 percent for the strongest new attack, on held out user tasks.
- Attacks developed in the Workspace environment also worked in the other three.
- Success varied a lot by injection task, and the average across tasks was 57 percent.
- Repeating each attack 25 times raised that average from 57 percent to 80 percent.
These numbers describe one model and one benchmark at one point in time. Do not read them as a rate for your system. The method is what transfers.
Changes to make in your own tests
Test with adaptive attacks. Public attack lists measure yesterday's attacks. Have someone write attacks against your agent, with knowledge of its tools and prompts.
Repeat attempts. An attacker can often retry for almost nothing. Run each attack many times and report the rate over repeats, not one pass or fail.
Report per task. An average hides impact. A data exfiltration task that rarely succeeds can still be the one that hurts. NIST recommends reporting task specific results next to aggregate numbers.
Keep testing. New model versions and new tools change the picture. Rerun after every material change.
A small test plan
- List the three worst things your agent could be tricked into doing.
- Write at least five injection attempts for each, placed in the places your agent really reads.
- Run each attempt 20 or more times.
- Record the success count per attempt.
- Fix by narrowing tools or adding approval, then rerun.
Where Safeguard fits
Safeguard policies, gates and guardrails give you a measurable control layer to test against.
Quick check
If an agent passes the public benchmark, is it safe?
No. NIST found that newly written attacks did far better than known ones, so passing known attacks says little about adaptive ones.
Why repeat each attack?
Model behavior varies between runs, and attackers can retry cheaply. In NIST's tests, 25 repeats raised average success from 57 to 80 percent.
Sources
Last reviewed: 19 October 2026.
