Measure outcomes that show the tool is used and useful: share of repositories with a recent successful scan, share of blocking findings fixed within the agreed window, dispute resolution time and median scan time. Logins, scans run and alerts raised show activity, not adoption, and are easy to inflate.
Why vanity metrics appeal
They are easy to collect and they go up. A dashboard showing a million scans looks like progress. But a scan nobody reads and a finding nobody fixes cost money and change nothing. Engineering leaders should ask what each number would look like if the tool were ignored.
Four questions, four metrics
| Question | Metric | Why it holds up |
|---|---|---|
| Is it everywhere it should be? | Share of repos with a successful scan in the last 7 days, over all repos | Has a denominator and shows gaps |
| Do people act on it? | Share of blocking findings fixed within the target window | Measures the work, not the alert |
| Is it fair? | Dispute count and median time to resolve | Rising disputes signal bad rules |
| Is it fast enough? | Median and 90th percentile scan time | Slowness drives bypass |
Add a fifth if you can: the share of findings fixed by the authors in the same pull request. High numbers mean findings are reaching developers early and in a usable form.
Show the denominator
A rise from 50 to 80 repos scanned is good news only if the total stayed the same. If repo count doubled, coverage fell. Always show the count and the total, and list the repos that are missing.
Watch for gaming
When a measure becomes a target, people optimize for it. If you reward the number of findings closed, some will be closed as false positives to hit the number. If you reward coverage, a repo may be scanned with a configuration that checks nothing. Check a sample each quarter: read ten closed findings and ten scan configs.
Pair numbers with a short survey
Ask developers two or three questions each quarter, such as whether findings are usually accurate and whether the tool slows them down. Numbers say what happened, and the survey says how it felt. Disagreement between the two is worth investigating.
Report to leadership with a story
For each metric, add one sentence on what changed and why, and one action. A chart without an explanation invites guesses. A single trend line with context is more useful than ten charts.
Quick check
What is a good coverage target?
Set it by tier. Aim for near complete coverage of production code, and accept lower coverage for prototypes if you list them.
How often should adoption be reviewed?
Look at the numbers weekly with the tool owner, and report trends to leadership quarterly.
Last reviewed: 19 October 2026.
