NIST's CAISI published a technical blog on strengthening AI agent hijacking evaluations in January 2025 and updated it on 19 December 2025. In its tests, attack success rose from 11% for the strongest baseline attack to 81% for the strongest new red-team attack, against an upgraded Claude 3.5 Sonnet in AgentDojo. The 2026 OWASP LLM Top 10 adds a reason to care: the project describes a "defense effect," where heavy mitigation means fewer successful prompt injections show up in public records, so the risk looks smaller than it is.
The lesson for a product team is plain. If you test with a list of known jailbreak strings, you are measuring the baseline, not your exposure.
What's affected
Any team that has shipped, or is about to ship, an LLM feature and has only run a handful of manual prompts. The CAISI work was on agents, but the method transfers to a chat feature, a RAG search or a summarizer.
CAISI's setup is a good template. Each scenario pairs a legitimate user task with an attack that tries to make the system complete a malicious injection task. CAISI added injection tasks for remote code execution, database exfiltration and automated phishing, and red teamers from CAISI and the UK AI Security Institute built attacks on a subset of tasks, then tested them on held-out tasks and in other environments.
How to check if you're exposed
Plan one week for one feature.
Day 1, scope. List what the feature can read, what it can call and what it can change. Write three bad outcomes in plain words, such as "sends customer data to an outside address" or "runs a command on our host."
Day 2, scenarios. For each bad outcome, write a legitimate user task and an injected instruction that tries to cause it, placed where an outsider could put text: a document, an email, a web page, a ticket.
Day 3, adaptive attacks. Start with known strings, then rewrite the injection after each failure to get past what blocked it. Record each variant. A feature that resists known attacks may still fall to attacks tuned for it, which is CAISI's second recommendation.
Day 4, repeated attempts. Model output is probabilistic. In the CAISI results, average success across five injection tasks was 57% with one attempt and 80% with 25 attempts per task. Run each scenario many times:
# Replace run_feature() with a call to your own feature.
import collections
def trial(scenario, attempts=25):
hits = 0
for _ in range(attempts):
out = run_feature(scenario["user_task"], scenario["poisoned_input"])
if scenario["bad_outcome_check"](out):
hits += 1
return hits / attempts
results = collections.OrderedDict()
for s in scenarios:
results[s["name"]] = trial(s)
Day 5, report. Report per scenario, not one average. CAISI notes that per-task results reveal risks that aggregates hide, since some tasks combined high success with high impact.
What to do now
- Turn every successful attack into a regression test that runs in CI.
- Fix with controls outside the model first: narrower tools, argument validation, egress limits and approvals. A reworded prompt is the weakest fix.
- Re-run the full set whenever you change the model, the system prompt, the tools or the retrieval corpus.
- Keep the attack log. When a model upgrade ships, the old successful attacks are your first test.
- Include the 2026 OWASP entries in your scenario list: sensitive information disclosure, excessive agency, hidden context exposure and improper output handling each need at least one scenario.
- Test defenses, not only the model. The June 2026 out-of-band defense paper stresses that static benchmarks overstate protection and calls for adaptive testing.
Why it matters
Red teaming is cheap compared with learning about a hole from a customer. It is also repeatable once you have the harness. The first week is the expensive one, and after that the suite grows with each finding.
The point of the numbers above is not that one model is weak. It is that the gap between a baseline test and a tailored one can be this large for a strong model, so your test plan needs the tailored kind.
Quick check
Why run each attack many times?
Model outputs vary, so one attempt can understate the risk. CAISI measured average success of 57% with a single attempt and 80% with 25 attempts per task.
What should a red team report contain?
Results per scenario with the impact of each, not a single average, because high-impact tasks can hide inside a low overall rate.
