Before shipping, answer seven questions: what is the previous good version, who can trigger the rollback, what signal triggers it, whether data changes block it, whether the rollback has been tried, how long it takes, and how you will confirm it worked. If any answer is "I do not know," fix that first.
Why plan it at all
Google's SRE material notes that small, self contained release artifacts make rollback cheaper and easier, and that a majority of incidents there are triggered by binary or configuration pushes. If change causes most incidents, the ability to undo change is core safety equipment. It matters even more for security regressions, where every hour a flaw stays live is more exposure.
The checklist
1. What is the previous good version, exactly? A specific build identifier, not "the one from last week." Builds should be reproducible and pinned, a point Google's release engineering material makes about hermetic builds.
2. Who may trigger rollback, and can they do it at 3 a.m.? Name a person and a backup. Check that their access works. Release engineering guidance puts access controls on sensitive operations, so make sure the controls do not lock out your own responders.
3. What signal says roll back? Decide in advance. For example: error rate above a threshold for ten minutes, or authentication failures twice the control. Deciding during the incident invites debate.
4. Do data changes block it? If the new version changed a schema or rewrote data, the old version may not run against it. See our post on expand and contract migrations.
5. Has the rollback ever been run? A rollback you have not practiced is a hope. Run it in a staging environment with realistic data, and again after any major pipeline change.
6. How long does it take? Measure it. If rollback takes two hours, a forward fix may be faster, and you should know that now.
7. How will you know it worked? The same metrics that told you to roll back should show recovery.
Config and flags count
Configuration pushes cause incidents as often as code. Treat config changes as releases: versioned, reviewed, and reversible. A feature flag switched off is often the fastest rollback you have.
Things a rollback cannot undo
Be honest about side effects. Emails already sent, messages already published, and calls already made to outside services do not reverse when you redeploy old code. If the bad version leaked data or issued credentials, rolling back stops the bleeding but does not end the incident. You still need to rotate secrets, assess what was exposed, and follow your incident process.
After the rollback
Hold a short blameless review. Record what the signal was, how long detection took and what would have caught it earlier. Our post on blameless reviews covers the format.
Safeguard's autonomous remediation can open fix pull requests for vulnerable dependencies, which is off by default with every category set to Manual, so you decide what merges and when. The free scan is at gold.safeguard.sh/scan.
Quick check
Is a forward fix ever better than a rollback?
Sometimes, particularly when data changes block the old version or rollback is slow. Decide which path applies before you ship, using measured rollback time.
Do config changes need a rollback plan?
Yes. The SRE workbook notes that configuration pushes trigger many incidents, so treat them like code releases.
Sources
Last reviewed: 19 October 2026.
