What belongs in an AI incident response runbook?

An AI incident response runbook is your normal incident process plus steps for models, prompts, retrieval data and agent permissions. It should say who can switch an AI feature off, what evidence to preserve, how to tell a security incident from a quality failure, and when to notify customers or regulators.

Why not just use the existing runbook?

Start from it. NIST SP 800-61 Rev. 3, published in April 2025, recasts incident response as part of cybersecurity risk management, aligned to the NIST Cybersecurity Framework 2.0. That fits AI well: the same prepare, detect, respond and recover thinking applies.

What changes is what you must be able to do. You may need to disable a model endpoint without taking down the product, snapshot a prompt and retrieval state, or revoke an agent's tool access in one step. MITRE ATLAS is a useful reference: it organises adversary tactics and techniques against AI systems in a matrix, much as ATT&CK does for enterprise systems.

Mistake 1: treating every bad answer as an incident

A wrong answer is a quality problem. A wrong answer caused by a poisoned document, or a response that revealed another customer's data, is a security incident. Write the triage questions: Did the output expose data? Did an agent take an action? Did untrusted input change behaviour? If yes to any, it moves to the security queue.

Mistake 2: no kill switch

Teams build features without a way to turn them off by tenant, tool or model. Decide in advance who can disable what, and test it. A feature flag per capability beats a deployment.

Mistake 3: logs that cannot reconstruct a decision

To investigate you need the prompt, the retrieved context, the model and version, the tool calls and the identity that made them. Log these with retention that matches your investigation needs, in a place the application cannot edit. Respect privacy rules when storing prompts.

Mistake 4: forgetting the agent's permissions

OWASP's excessive agency guidance lists excessive functionality, permissions and autonomy as root causes. In an incident, your first question is what the agent could reach. Keep an inventory of agent identities and scopes so the answer takes minutes.

Mistake 5: ignoring third parties

If a model provider or a tool vendor has an incident, you still have customers. Record each provider's contact and notification path. CISA published a playbook in January 2025 for voluntary sharing of AI-related cybersecurity incident and vulnerability information with partners through its Joint Cyber Defense Collaborative. It is voluntary and does not replace legal obligations, and it excludes safety risks to life or the environment, which need their own procedures.

Mistake 6: no rehearsal

A runbook that has never been used fails on its first use. Run a tabletop with a plausible case, such as a retrieval index containing a document that should have been restricted.

What sections should the runbook have?

1. Scope and definitions, including security versus quality incidents.
2. Roles and a contact list, including providers.
3. Detection sources and alert thresholds.
4. Containment actions: kill switches, token revocation, index quarantine.
5. Evidence list and how to preserve it.
6. Notification rules for customers and regulators, set with legal counsel.
7. Recovery and review, including what you change in the model, data or permissions.

What to do

1. Copy your existing runbook and add the AI sections above.
2. Build and test one kill switch for each AI feature.
3. Confirm the logs contain prompt, context, model version, tool calls and identity.
4. List every agent identity and its scope.
5. Collect provider contacts and notification terms.
6. Run a tabletop within the next quarter and record the changes it produces.

Quick check

Sources

• NIST SP 800-61 Rev. 3
• CISA AI Cybersecurity Collaboration Playbook
• MITRE ATLAS
• OWASP LLM06 Excessive Agency

Last reviewed: 18 October 2026.