A good AI incident review has six parts: a short summary, a timeline from logs, what the system did versus what was intended, which controls failed or were missing, the impact, and owned actions with dates. Write it without blame, and mark every claim as confirmed or reported.
Why AI incidents need their own habits
The public cases show three recurring difficulties. The account often comes from one party, as with the July 2025 Replit and SaaStr report, which relied on the customer's posts and had no vendor statement at publication. Agents can describe their own actions wrongly. And the exact cause is often unclear, as in the Samsung case, where the memo reportedly did not say what the leaked code was.
A review should separate what you can prove from what you were told.
The template
1. Summary. Three sentences: what happened, who was affected, current status.
2. Timeline. Times from system logs, not chat transcripts. Include detection, containment and recovery.
3. What the system did. The actions taken, by which identity, with which permissions. Include the prompts and tool calls if you have them.
4. Expected versus actual. What the people involved believed the system would do and why. This is where unclear instructions and missing guardrails show up.
5. Contributing factors. Ask "which control would have stopped this?" for each layer:
- permissions and credentials
- review and approval steps
- input and data handling
- monitoring and alerting
- recovery and backups
6. Impact. Data, money, time, customers. State unknowns openly.
7. Actions. Each with an owner and a date. Prefer controls that enforce over reminders that request.
Confirmed versus reported
Mark each statement. A simple convention helps: "Confirmed by logs," "Reported by a user," "Stated by the vendor," "Unknown." When Google explained its 2024 AI Overviews errors, it said many viral screenshots were faked. Any review that treats every screenshot as fact starts with bad evidence.
What to log before you need it
- who or what ran each action, with a distinct identity for agents
- the tool calls and their arguments
- the model and version in use
- which documents or sources fed an answer
- approvals and who gave them
Without these, the review turns into guesswork. See AI agent blast radius management.
Keep it blameless
Ask how the system allowed the mistake, not who made it. People who fear blame stop reporting early, which makes the next incident worse.
Safeguard's compliance and evidence automation can help keep the records an auditor or reviewer will ask for.
Quick check
Who should write the review?
The people closest to the work, with a neutral facilitator. Share the draft with everyone involved before publishing it.
Should we publish it?
Share internally always. Publish externally only when it helps customers, after legal review.
Sources
- The Register: Replit and SaaStr incident report, 21 July 2025
- CSO Online: Samsung bans staff AI use over data leak concerns
- Google: AI Overviews, about last week, 30 May 2024
Last reviewed: 19 October 2026.