What goes in an AI incident response runbook?

An AI incident response runbook is your normal incident process plus four extra questions: what did the model see, what could it do, what did it say, and what changed in the model or its inputs. Write the answers down before an incident, because logs and access are hard to reconstruct afterward.

How does it relate to standard incident response?

NIST published SP 800-61 Revision 3 in April 2025. It frames incident response as part of overall cybersecurity risk management, as a community profile of the NIST Cybersecurity Framework 2.0, and says preparation reduces both the number and impact of incidents. Keep your AI runbook inside that process. Do not build a parallel one, because on-call staff will not know which to open.

NIST's AI Risk Management Framework, released in January 2023, organizes AI risk work into Govern, Map, Measure and Manage. The Manage function is where response lives, but you can only manage what you mapped, so the inventory step below matters.

What should the runbook define before anything goes wrong?

Inventory. A list of every model, agent, prompt, tool and data source in production, with an owner. OWASP's supply chain guidance for LLM applications recommends a current, signed inventory of components, using an AI bill of materials such as CycloneDX as a starting point.

Severity levels. Define in plain words what makes an AI event minor or major. Examples: a harmless wrong answer, a leak of sensitive data, an agent action with real side effects.

Kill switches. For each system, name the way to stop it: disable a tool, revoke a key, route to a fallback model, or switch to read-only mode. Test them.

Logging. Keep prompts, tool calls and outputs where policy allows, with access limits, since they may contain personal data. Decide the retention time in advance.

What are the steps during an incident?

1. Triage. Classify the event: data exposure, harmful or false output, unauthorized agent action, poisoned input, or a compromised model or dependency.
2. Contain. Use the kill switch with the smallest blast radius. Remove tool permissions before you turn off the whole service if you can.
3. Preserve. Snapshot the model version, system prompt, retrieval index state, tool configuration and relevant logs.
4. Investigate. Ask what input started it, which permissions the system held, and whether controls outside the model, such as authorization checks, held. OWASP notes that a system prompt should not be treated as a secret or a security control, so assume an attacker can read it and look for the control that actually failed.
5. Eradicate. Rotate secrets that were in prompts or reachable by tools. Remove poisoned documents from retrieval stores. Replace a suspect model artifact.
6. Recover. Restore service with tighter permissions, and watch for repeat behavior.
7. Review. Record what the detection gap was, and update the runbook.

Which AI-specific causes should you check?

• Credentials or sensitive rules placed in a system prompt, a pattern OWASP lists under system prompt leakage.
• Excess permissions on tools, since OWASP advises enforcing authorization outside the model, with least privilege per agent.
• A third-party model or dataset with weak provenance. OWASP states there are currently no strong provenance assurances in published models.
• Unreviewed changes to prompts, retrieval data or model versions.

Who needs to be in the room?

Security, the owning engineer, a data protection or privacy contact, and whoever speaks to customers. If the system makes decisions about people, add legal early.

What to do

1. Build the inventory of AI systems with owners.
2. Write kill switches for each, and test one this quarter.
3. Decide what you log, for how long, and who can read it.
4. Add the AI event types above to your existing severity scale.
5. Run a tabletop with a leaked secret in a prompt as the scenario.
6. Update the runbook after every drill or real event.

Quick check

Sources

• NIST SP 800-61 Rev. 3, Incident Response Recommendations
• NIST: AI Risk Management Framework
• OWASP: LLM03:2025 Supply Chain
• OWASP: LLM07:2025 System Prompt Leakage

Last reviewed: 18 October 2026.