To roll back a bad dependency upgrade, redeploy the last known good build artifact, confirm health with your smoke test, then revert the lockfile change in source control so the next build matches. Do not hand-edit versions in production. Check first that no data or schema change blocks the old version from running.
Roll back the artifact, then the source
Two things need to return to the old state: what is running, and what the next build will produce. Do them in that order.
- Running state. Redeploy the previous immutable artifact, such as the prior container image digest or release bundle. This is fast because nothing is rebuilt. Rebuilding "the old version" from a loose version range can pull in a different transitive set than before, which defeats the point.
- Source state. Revert the commit that changed the manifest and lockfile. Merge the revert so that tomorrow's build does not reintroduce the problem.
- Prevent re-entry. If a bot opens update pull requests, tell it to skip that version, or pin the package with a comment explaining why, so the same bump does not return next week.
On Kubernetes, a Deployment keeps rollout history by default so you can roll back at any time, and a revision is created only when the pod template changes, for example a new container image. The kubectl rollout undo command reverts to the previous revision, with an option to pick a specific revision and a server-side dry run to preview it. Check how many revisions you keep, because the revision history limit can be changed and old revisions are then gone.
The traps that make rollback fail
| Trap | What happens | Prevention |
|---|---|---|
| Forward-only migration | Old code cannot read the new schema | Expand and contract migrations |
| New data format written by the new version | Old version cannot parse records created during the incident | Write both formats during the canary |
| Cached or queued objects | Messages serialized by the new library break the old consumers | Drain or version queues before the rollout |
| Config changed with the upgrade | Old version ignores or rejects new keys | Ship config backward compatible, or revert it together |
| Rebuild instead of redeploy | A different transitive set resolves | Keep immutable artifacts and lockfiles |
The database case is the most common. Our post on expand and contract migrations shows how to make schema changes that both versions can live with.
Decide when rolling back beats rolling forward
Roll back when the failure is unexplained, widespread or hurting users now. Roll forward, meaning ship a fix, when the cause is clear, small and safe to change, and the rollback itself carries risk such as data written in a new format.
If the upgrade was a security fix and you roll back, you are exposed again. Say so. Record the compensating step, such as a temporary rule at the edge or a feature switched off, and set a date to retry the upgrade. A kill switch built ahead of time helps here, see feature flag kill switch for security incidents.
Practice it once
A rollback you have never run is a guess. During a calm week, roll a staging service back and forward, time it, and fix whatever surprised you. Write the commands in the runbook with the real names of the services.
What to record
Note the versions, the time you decided, the signal that triggered it and the person who verified health afterwards. That record feeds the next attempt and shows an auditor that change control worked.
Safeguard can open the fix pull request and gate it by policy, but the rollback path belongs to your deploy tooling. Make sure it exists before the first upgrade, not after.
Quick check
Should I roll back by reverting the commit and rebuilding?
Prefer redeploying the previous artifact first, since it is quick and exact. Revert the commit afterwards so the source matches what is running.
Can I roll back after a database migration ran?
Only if the migration was backward compatible. Otherwise you need to roll forward or restore data, which is why migrations should be split into expand and contract steps.
Sources
Last reviewed: 19 October 2026.
