Contain first when harm is ongoing or likely, and capture evidence in parallel. Investigate first only when containment would destroy evidence you cannot recover and no active damage is happening. In most supply chain incidents, a short freeze plus a quick evidence snapshot gives you both.
The real tension
Containment stops damage: revoke a token, freeze a pipeline, block a version. Investigation finds scope: what ran, what leaked. Some containment steps erase evidence, such as rebuilding a runner or clearing a cache. The art is choosing steps that stop harm and keep facts.
Decision guide
| Situation | Choice | Why |
|---|---|---|
| Credential in active abuse | Contain now | Each minute adds harm |
| Bad version found in lockfile only | Investigate, then fix | Nothing is running |
| Bad version ran in CI, secrets exposed | Freeze pipeline, snapshot, rotate | Stops new reads, keeps logs |
| Malicious artifact deployed | Contain, then investigate | Customers at risk |
| Unsure whether it ran | Freeze and snapshot first | Cheap, reversible |
Containment steps that keep evidence
- Freeze, do not wipe. Pause pipelines and block the version at the proxy. Do not delete runners or caches yet.
- Snapshot. Export audit logs and copy CI logs, as covered in the evidence guide.
- Revoke, but record. Note the time and the old credential's last use before revoking.
- Isolate rather than reimage. Take an affected machine off the network, not off the evidence list.
- Then clean. After the snapshot, rebuild, purge caches and rotate.
NIST SP 800-61 Rev. 3, from April 2025, treats incident response as part of risk management with detection, response and recovery activities. The practical reading is that these overlap in time. You do not finish one before starting the next.
Who decides
The incident lead decides, and writes the decision and the reason in the log. If there is disagreement between security and engineering, the lead breaks the tie. See incident roles for a small team.
A worked example
Suppose a build runner installed a bad version at 02:10 and a teammate spots the advisory at 09:00. Nothing suggests active abuse, so freeze the pipeline, export the CI logs and audit logs, and block the version. Then rotate publish and cloud keys in order of power, and only after that wipe the runner. The freeze takes minutes and the snapshot takes under an hour. If at 09:20 you see a login to your cloud account from an address nobody knows, the call changes: revoke now, note the time, and keep collecting in parallel.
Quick check
Should we wait for the full investigation before revoking tokens?
No. Revoke exposed credentials now and record their state first.
Is freezing the pipeline too disruptive?
It is usually cheaper than a second bad build. Keep a short break glass path for urgent fixes.
Safeguard's policies and gates can block a risky dependency from entering a build while you investigate. A free scan is at gold.safeguard.sh/scan.
Sources
Last reviewed: 19 October 2026.
