Settle these in advance: who can declare an incident, who can roll back or pause a release, which checks may be skipped and by whom, what customers are told and by whom, and when to escalate. Writing the answers down turns hours of waiting for permission into minutes of action.
Why advance decisions help
During an incident, the scarce resource is clear authority. People who know what to do may not know whether they are allowed to do it. NIST SP 800-34 starts its seven step process with a policy statement, which it says gives the authority and guidance needed to develop an effective contingency plan. The idea scales down: a half page of decisions, signed off in calm conditions, serves the same purpose.
The decision list
| Decision | Who decides | What to write down |
|---|---|---|
| Declare an incident | Anyone on call, with a named lead confirming | The threshold, such as customer impact or data exposure |
| Pause deployments | On call engineer | The command or switch, and how to lift the pause |
| Roll back a release | Release owner or on call | Conditions, and what data changes block a rollback |
| Use an emergency change path | Engineering lead plus a second person | Which checks may be shortened, which may not |
| Disable a feature with a kill switch | On call | The flag names and expected effect |
| Rotate or revoke credentials | Security owner | The list, order and who else must be told |
| Notify customers | Named communications owner | Wording templates and approval path |
| Notify regulators or law enforcement | Legal or leadership | Contacts and the question to ask counsel |
| Spend money in an emergency | A named budget holder | A limit, and how to record it |
Do not treat the last two rows as legal advice. They are placeholders for your own counsel to fill in.
What can be skipped and what cannot
The most valuable decision is a short list of checks that stay on in an emergency. For example, you might shorten code review to one reviewer but never remove automated tests, and never push an unsigned artifact. Our post on emergency change paths for critical advisories goes into this, and the one on which checks you can skip for a hotfix lists candidates.
Safeguard's policies and gates can hold the rules you decide in advance, such as blocking a vulnerable version from merging, so enforcement does not depend on someone remembering at 3 a.m.
Getting it signed
- Draft the table with your on call team. They know where the friction is.
- Have the person who carries the risk sign each row.
- Put it where it can be found when systems are down, as described in our post on recovery tools that depend on the failed system.
- Review after every incident and every two quarters.
A worked example: the same incident with and without advance decisions
This example is hypothetical. At 02:10 an on call engineer sees that a recent release is causing errors for a portion of customers. Compare two teams.
| Moment | Team without advance decisions | Team with advance decisions |
|---|---|---|
| 02:10 | Engineer unsure whether to wake anyone | Threshold says customer impact above a set level declares an incident |
| 02:20 | Messages sent asking who can approve a rollback | Release owner on call is paged by rule |
| 02:50 | Waiting for a reply | Rollback decided under written conditions |
| 03:20 | Approval arrives; rollback begins | Rollback completed, deployments paused |
| 04:00 | Customers still affected | Customer update sent from a template by the named owner |
The difference is not skill. The second team had already answered the questions, in daylight, with the people who carry the risk.
Step by step: writing the one page
- Gather the on call team and the owners of the risk, such as the head of engineering and whoever owns customer communication.
- Start from the table in this post and edit it to match your roles.
- For each decision, write who decides, who must be told and where the record goes.
- Add limits. For example, a rollback is permitted if error rates exceed a stated threshold for a stated time, and not permitted if a database migration has already run.
- List the checks that stay on in an emergency, and those that may be shortened with a second person's approval.
- Add an expiry for any exception, so shortcuts do not become habits.
- Have each decision owner sign or acknowledge the page, and store it with your runbooks.
- Test it at the next tabletop: read a scenario and see whether people can find and follow the page.
Handling a decision that is not on the page
No list is complete. Add a catch-all: if a decision is not listed, the incident lead decides, records the reasoning and tells the owner at the first opportunity. After the incident, add the decision to the page. Over several incidents the page grows toward what your team actually meets.
Emergency changes and your gates
A common question is whether to bypass a gate during an urgent fix. A better question is which parts of the gate protect you most in a hurry. Automated tests and vulnerability checks catch mistakes that people miss when tired. Consider keeping them on, and shortening human steps instead, such as accepting one reviewer instead of two. Record the decision and the person who made it. Our post on kill switches and flags in incident response covers a related tool.
Where legal and regulatory questions belong
Some decisions involve obligations to notify customers, regulators or insurers. This page cannot answer them. List the question, name the person who will ask counsel, and note the contact details. Keep the details of any obligation with your legal advisers.
Try it yourself
Open the fix available now demo and look at how a fix is presented with its context. Imagine writing a standing rule for such cases: when a fix is ready and tested, who may apply it and what check stays on?
Click through it at your own pace, or open it full screen.
Quick check
Does pre-authorization mean no oversight?
No. It sets limits in advance and requires a record afterward. Oversight happens at the review.
How long should the document be?
One page is enough to start. If nobody reads it, shorten it.
How do we stop pre-authorized shortcuts from becoming normal?
Attach an expiry and a review to each exception, and report their use at the monthly review. If a shortcut is used often, fix the normal process instead.
Sources
Last reviewed: 19 October 2026.
