When an AI agent does something it should not, treat it as an ordinary incident with an unusual actor. Contain it by revoking its credentials first, preserve the prompt and tool-call trace, scope what it touched, then fix the permission that allowed the action. Debating why the model did it comes last.
Why does the standard process still apply?
NIST's incident response guidance, SP 800-61 Revision 3 published in April 2025, frames response as part of cybersecurity risk management using the CSF 2.0 functions, not a separate activity. That fits agents well. The preparation, detection, response and recovery work you already have applies. What changes is the evidence you need and how quickly an agent can act.
OWASP describes the root cause pattern behind many agent incidents as excessive agency: too much functionality, too many permissions, or too much autonomy. Triggers include hallucination, prompt injection and compromised peers. Your review should ask which of the three you had.
What do you do in the first hour?
Contain. Revoke the agent's tokens and API keys, or disable the tool it misused. Pausing the agent is not enough if its credentials work elsewhere. If the agent acts as a user, rotate that user's secrets too.
Preserve. Export the conversation, system prompt, tool definitions, tool-call arguments and responses, and the model and version used. Logs rotate, so copy them now. Record who or what triggered the run.
Scope. List every system the agent could reach, then check each for changes during the window. Do not trust the agent's own summary of what it did; treat it as a lead.
How do you assess impact?
Sort effects into three buckets: data exposed, data changed or deleted, and actions taken in the outside world, such as emails, tickets, payments or deployments. Outside actions are the hardest to reverse and need a named owner for each recipient or system. If personal data was involved, bring in the privacy lead early, since notification clocks may be running.
How do you recover?
Restore changed data from backups and verify with a second source. Reverse outside actions where you can and notify affected parties through the normal channel. Return the agent to service only after the permission gap is closed, not before.
What should the review produce?
A useful review ends with changes you can check:
- Permissions cut to what the task needs, with read and write separated.
- Human approval added for irreversible actions.
- Per-user credentials in place of a shared high-privilege account.
- A regression case in your test suite for this failure.
- Logs kept long enough to reconstruct the next one.
How to check or apply this
- Keep a kill switch per agent, tested quarterly, that revokes its credentials.
- Log prompts, tool calls and results to storage the agent cannot edit.
- Name an incident commander for agent events and share the contact list.
- Classify actions as reversible or not in advance.
- After any event, write the permission fix before the model fix.
- Add the case to your guardrail tests.
Safeguard offers agent identity and guardrails that apply policy to agent tool use, which supports steps 1 and 4.
Quick check
Should we ask the agent what happened?
You can, but treat the answer as unverified. Reconstruct events from logs and downstream system records.
Is retraining or prompt editing the first fix?
No. Cut permissions first. Prompts are advice; credentials are enforcement.
Sources
Last reviewed: 18 October 2026.
