A security gate that blocks a build before deployment is not a change failure in DORA terms, because nothing was deployed. A security problem discovered after a deployment that requires immediate intervention, such as a rollback or hotfix, can count. Define the rule once and apply it consistently.
What the definition says
DORA's guide defines change fail rate as the ratio of deployments that require immediate intervention following a deployment, likely resulting in a rollback of the changes or a hotfix to quickly remediate any issues. It defines deployment rework rate as the ratio of deployments that are unplanned but happen as a result of an incident in production. It also describes failed deployment recovery time as the time to recover from a deployment that fails and requires immediate intervention.
The key word is deployment. These measures describe what happens to changes that went out.
Classifying common cases
| Situation | Counts as change failure? | Why |
|---|---|---|
| Gate blocks a pull request | No | No deployment happened |
| Gate blocks a deploy job | No | Change never reached production |
| New advisory published for code already live, patched in the normal cycle | No | Not an immediate intervention caused by the change |
| Deployed change introduced a vulnerability, rolled back urgently | Yes | Immediate intervention after deployment |
| Deployed change shipped with an exposed credential, rotated and hotfixed | Yes | Hotfix after deployment |
| Unplanned deployment to fix an exploited issue | Likely rework | Unplanned deployment due to an incident |
The grey case is a new vulnerability disclosed against a dependency you shipped weeks ago. The change did not fail when it shipped, and the scoring depends on your rule. Most teams keep these out of change fail rate and track them under vulnerability management, with its own measures such as time to remediate.
A practical rule
- Write down your definition of immediate intervention.
- Tag each production rollback or hotfix with a cause category, including security.
- Count a security tagged intervention in change fail rate only if it meets the same bar as any other cause.
- Track security related incidents and exposure time in a separate security view, so they do not disappear if they are not counted as change failures.
Keeping a cause tag lets you answer "how many of our failed changes were security related?" without a debate.
Why not count gate failures
Counting gate blocks as change failures would punish the control that is working. A blocked build is the system catching a problem early. Track gate blocks in your gate outcome metrics instead.
Quick check
Does a vulnerable dependency found after release count?
Only if it led to immediate intervention such as rollback or hotfix under your definition. Otherwise track it as vulnerability backlog.
Who decides the classification?
Agree it between engineering and security up front and review it quarterly, so the number is stable over time.
How Safeguard fits
Safeguard offers dependency and SBOM scanning with policies and gates that catch issues before release, and continuous monitoring for advisories on what has shipped. Try a free scan at gold.safeguard.sh/scan.
Sources
- DORA, DORA's software delivery performance metrics (change fail rate, deployment rework rate, failed deployment recovery time): https://dora.dev/guides/dora-metrics/
Last reviewed: 19 October 2026.
