Hold a thirty minute pre-mortem. Ask the people who will be measured how they would hit the number with the least real effort, list those routes, add a counter metric that exposes each one, and agree on a sampling audit. If you cannot name a counter metric, the KPI is not ready.
Why test before launch
Goodhart's law is usually stated as "when a measure becomes a target, it ceases to be a good measure". Campbell's law says that the more a quantitative indicator drives decisions, the more it invites pressure to distort the process it watches. You cannot repeal either. You can discover the weak spots while changing the plan is still free.
DORA's guidance says teams should identify multiple metrics, including some with a healthy amount of tension between them. A pre-mortem is a cheap way to find which tension you need.
The 30 minute pre-mortem
Invite two engineers, one team lead, and someone from security. Do not invite the executive who wants the number.
- State the KPI in one sentence, with its definition, data source, and who sets each input.
- Ask the question: "It is six months from now and this number looks great but nothing is safer. What happened?"
- Write down every answer, including silly ones.
- For each answer, name the signal that would show it. That signal becomes a counter metric or an audit check.
- Decide who can change each input. If the people being measured can change the definition, status, or scope, mark that as high risk.
Example walk through
Take the KPI "percent of critical findings fixed within the window". The group might list these routes:
| Cheap route | Signal that exposes it |
|---|---|
| Downgrade severity | Count of severity changes and who made them |
| Start the clock late by delaying triage | Time from discovery to triage |
| Mark items accepted | Count and age of exceptions |
| Close without deploying the fix | Reopen rate and a sample check in production |
| Exclude hard services from scope | Scan coverage and owner coverage |
Each row becomes a line on the dashboard next to the KPI.
Add an audit step and a kill switch
Choose a small monthly sample, for example ten closed items and five exceptions, and have someone outside the team check them. The goal is not to catch people. It is to confirm that the number still means what you said it meant. Share the results openly.
Write down when you will retire or redefine the KPI, such as when the audit finds the sample no longer matches the number, or when a counter metric moves sharply in the wrong direction. A pre agreed rule makes it easier to change course without blame.
Try it yourself
Click through it at your own pace, or open it full screen.
Pick one number from the dashboard and list two cheap ways to improve it without reducing risk.
Safeguard's vulnerability management and compliance evidence automation can keep the underlying records, such as status changes and exceptions, available for audit sampling. A free scan is available at https://gold.safeguard.sh/scan.
Quick check
Who should be in the pre-mortem?
People who will be measured, a team lead, and security. Leave out whoever will use the number as a target, so the discussion stays honest.
How many counter metrics are enough?
One per realistic gaming route, and no more than the team will actually read. Start with the two or three that are cheapest to bend.
Sources
- Goodhart's law (Wikipedia)
- Campbell's law (Wikipedia)
- DORA, software delivery performance metrics and common pitfalls
Last reviewed: 19 October 2026.
