Safeguard
DevSecOps

The DevSecOps metrics that actually indicate program maturity

CISA's KEV directive now demands 3-day fixes for the riskiest bugs. Here's why raw finding counts are the wrong way to measure a DevSecOps program.

Safeguard Research Team
Research
7 min read

On June 10, 2026, CISA issued Binding Operational Directive 26-04, retiring the blanket "patch every KEV entry within six months" rule that had governed federal agencies since BOD 22-01 in 2021 and replacing it with a risk-scored model: vulnerabilities that are internet-exposed, listed in the Known Exploited Vulnerabilities catalog, automatable, and high-impact must be remediated within three days, with mandatory forensic triage attached. Agencies have until December 7, 2026 to comply. That shift matters far beyond federal contractors, because it codifies something security leaders have suspected for years — a program that tracks "number of open vulnerabilities" or "percentage scanned" is measuring activity, not risk reduction. A team can run scans on 100% of its repositories and still take four months to fix an internet-facing bug under active exploitation. DORA's own 2024 metrics update made a parallel correction, replacing generic MTTR with "failed deployment recovery time" — specifically time to restore service after a change caused the outage, not any outage. This piece walks through the metrics that separate a governance program that looks mature on a slide from one that actually reduces exploitable risk.

Why doesn't a high scan-coverage percentage prove anything on its own?

Scan coverage tells you what fraction of your assets got looked at, not what happened to what was found. A team can report 98% SBOM coverage across its repositories and still carry a six-month-old, internet-facing remote code execution finding that nobody triaged. Coverage is a prerequisite for governance, not evidence of it — you can't manage what you haven't inventoried, but inventory alone doesn't reduce risk. The more meaningful pairing is coverage plus a freshness and completeness check: is the SBOM regenerated on every build, or is it a quarterly snapshot going stale the moment a new dependency is pulled in? A program that reports coverage without a paired remediation-velocity metric is reporting the size of its haystack, not how fast it finds the needle. Mature programs treat coverage as a gate to pass, not a KPI to display — the real signal is what fraction of coverage translates into acted-upon, closed findings within a defined window, split by environment and severity rather than blended into one flattering aggregate number.

Why did DORA replace MTTR with "failed deployment recovery time"?

DORA's research team updated its 2024 metrics set to five indicators, replacing the historically loose "mean time to restore" with a more precise "time to restore service" measured specifically after a change-caused impairment — distinguishing it from an outage triggered by, say, a third-party cloud provider failure. The distinction matters because bundling all incidents together rewards teams that simply have infrequent unrelated outages, not teams that recover well from their own deployments failing. DORA also added a fifth metric, deployment rework rate — the percentage of deployments that turn out to be unplanned fixes for a prior change — which functions as a leading indicator of quality problems before they become incidents. Elite performers, per DORA's research, restore service from a failed change in under one hour. For a DevSecOps governance program, the equivalent security-flavored question isn't "how many findings do we have" but "when a critical, actively-exploited finding lands in production, how long until it's remediated" — the security analogue of change-failure recovery time, and one that should be tracked separately from routine backlog aging.

What does the CISA KEV directive shift teach programs outside federal compliance?

BOD 26-04's move from calendar-based SLAs (14 days, 6 months) to risk-scored SLAs (3 days for the worst-case combination of exposure, exploitation, automatability, and impact) is instructive for any organization, federal or not, because it formalizes risk-weighting a practice many mature commercial teams already do informally. A raw KEV count doesn't tell you whether those entries sit on internet-facing production systems or an air-gapped test environment nobody uses. The metric worth tracking is narrower and harsher: KEV-listed findings currently live in production, a number that should be zero at all times, not a percentage that trends toward zero over a quarter. Tracking it as a binary "are we currently exposed to a known-exploited vulnerability in production" rather than a backlog count changes team behavior — it turns one specific finding into an incident-level escalation instead of another row in a Jira board. This is also the practical reason many governance dashboards now separate "total open findings" from "actively exploited findings in a live environment" as two distinct, non-interchangeable metrics.

How should aging buckets be used instead of a single backlog number?

A single "total open findings" number hides the fact that most risk concentrates in the tail — the findings that have sat unresolved past 30 or 90 days despite passing every SLA checkpoint on paper. Bucketing the backlog into age ranges (under 7 days, 7–30, 30–90, over 90) and tracking how findings move between buckets over time shows whether a program is actually closing old risk or just absorbing new findings at the same rate it resolves them, which keeps the headline total flat while real exposure ages in place. A program with a shrinking under-30-day bucket and a growing over-90-day bucket is quietly accumulating risk debt even if its total count looks stable month over month. Pairing aging buckets with "top packages driving the backlog" and "top affected assets" turns the metric from a vanity chart into a triage input — it tells a security lead exactly where a targeted remediation sprint would move the needle, rather than asking engineering to clear an undifferentiated queue.

What does policy adherence actually measure, and where does it break down?

Policy adherence is often reported as "percentage of deployments that passed the gate," which rewards a program for having lenient policies as much as for good behavior — a gate that blocks nothing passes 100% of the time and proves nothing. The more honest pair of numbers is the exception rate (how often a blocking policy is overridden) and the exception aging (how long "temporary" exceptions stay open). A policy requiring no critical CVEs above CVSS 7.0 that gets overridden on 40% of production deployments isn't a governance control, it's a formality everyone routes around, and that routing pattern is exactly what auditors and, increasingly, CISA-style directives are starting to ask programs to demonstrate they don't have. Tracking exceptions granted, their justification, their expiration date, and whether expired exceptions actually get re-evaluated (rather than silently persisting) is what separates a policy that exists on paper from one enforced in practice.

How Safeguard Helps

Safeguard's Program Overview dashboard reports the metrics this piece argues for directly, rather than leaving teams to assemble them from raw exports: open critical and high findings broken out by environment, KEV-listed findings in production tracked as a should-be-zero indicator rather than a percentage, and rolling 30/90-day time-to-remediate instead of a single lifetime average. The Vulnerability Backlog view adds the aging buckets (under 7 days, 7–30, 30–90, over 90) alongside the top packages and assets driving them, so a remediation sprint can be scoped by what will move the number, not just what's oldest. Every one of these is also exported as an OpenTelemetry metric — safeguard.findings.open{severity, env, team} and safeguard.sbom.coverage_ratio{env} among them — so a governance program can pipe the same figures into its existing observability stack instead of treating security metrics as a separate reporting silo. Policy and gate exception trends surface on the Compliance dashboard alongside framework readiness scores, so an expired exception shows up next to the audit evidence it's supposed to be exempted from, not buried in a separate ticketing system.

Never miss an update

Weekly insights on software supply chain security, delivered to your inbox.