Ask twelve questions in four groups: what the outage did, how you noticed, what you did about it, and what you will change. Include one that teams skip, namely whether the packages pulled during the outage or a fallback were the ones you intended. Keep the tone blameless and finish with owners and dates.
Group 1: Impact
- How long were builds affected, and which pipelines? Use timestamps, not feelings. Note the first failure and the first clean run.
- What did we fail to ship? List delayed releases and hotfixes. This is the number leadership cares about.
- Did anyone ship around the problem? If a team bypassed a gate or built locally to hit a deadline, record it without blame. It shows where the process is too slow.
Group 2: Detection
- Who noticed first, and how? An alert, a developer message, a customer? If the answer is a customer, monitoring needs work. See what to monitor on an internal package mirror.
- How long did it take to know whether it was the registry or us? The method is in is it the registry or your CI network.
- Were the error messages useful? Did logs show the host and status code? See registry errors in CI.
Group 3: Response
- Did the cache or mirror work as planned? Which packages were missing? Each miss is a hidden dependency.
- Did we use a fallback path, and who approved it? If yes, the next question matters most.
- Were the packages we built with the ones we meant to use? Compare the builds made during the outage with the lockfiles and your mirror's contents. A fallback may have bypassed intake rules such as blocked versions or release delays. See should CI fall back to the public registry. If any package differs, treat it as an open item until reviewed.
- Did retries make it worse? Check request volume and 429 counts. See retry storms.
Group 4: Changes
- What would have made this a non-event? Common answers: a mirror in a second location, base images in your own registry, a warm cache, or an offline bundle. Compare the options in vendoring vs cache vs mirror.
- What will we rehearse, and when? Put a drill on the calendar. See the monthly offline build drill.
How to run the meeting
- Hold it within a week, while memory is fresh.
- Keep it blameless. Ask what the system allowed, not who erred.
- Bring the timeline and the data: build failure counts, registry errors, cache hit rates.
- End with a short list of actions, each with an owner and a date. Three real actions beat fifteen vague ones.
- Share a summary with teams who depend on the pipeline. Keep it factual and avoid guessing at causes you have not confirmed.
A simple output format
| Item | Entry |
|---|---|
| Duration and scope | Start, end, pipelines affected |
| Root cause | Confirmed cause, or "unconfirmed" |
| What worked | Cache, mirror, runbook steps |
| What did not | Gaps found |
| Actions | Owner, date |
Where Safeguard fits
Question nine needs data on what each build contained. Safeguard's SBOM and dependency scanning keeps that record per build, and policies and gates can enforce the follow-up rules you agree on.
Quick check
Is a postmortem needed for a short outage?
A brief review still pays off if a fallback was used or a release was delayed. Scale the meeting to the impact.
Who should attend?
The people who ran the response, the mirror owner, and a representative from the teams most affected.
Last reviewed: 19 October 2026.
