Completion rate, total hours, certificates issued and raw quiz scores are vanity metrics for developer security training: they show activity, not changed behaviour. Keep one for compliance records, and add measures tied to the code your teams ship, such as repeat finding rates and time to fix.
What vanity metrics tell you
A vanity metric is easy to collect and flatters the program. Completion at ninety five percent sounds great. It does not say whether anyone learned anything, because people can click through. Hours logged measure time, not skill.
These numbers have a place. They show coverage, and an auditor may ask for them. The mistake is presenting them as proof of effect.
Metrics to retire or demote
| Metric | Why it misleads |
|---|---|
| Completion rate alone | Measures clicking, not learning |
| Total training hours | Rewards length over quality |
| Raw quiz scores | Test short term recall, and can be gamed |
| Certificates earned | Shows attendance, not practice |
| Number of courses offered | Supply is not demand |
| Satisfaction scores alone | Fun is not the same as effective |
Keep completion for records. Stop leading with it.
Metrics worth tracking instead
Pick a small set tied to behaviour and outcomes.
- Repeat findings. How often does the same category of flaw recur after training on that topic? A falling rate is a sign.
- Time to fix. Do teams close findings faster after training on triage and remediation?
- Findings caught early. Share of issues found in review or at the pull request stage versus later.
- Reports from developers. Do people raise security concerns more often? A rise can be healthy.
- Champion activity. Questions asked, reviews joined and sessions run by non specialists.
NIST SP 800-50 Rev. 1 covers metrics and evaluation methods for improving a learning program, and states that the program should drive behaviour change as part of risk management. That framing favours outcome measures over attendance.
Handle causation with care
Findings can rise after training because developers now look harder, or because a new scanner went live. Note other changes in the same period before claiming credit. Compare trained and not yet trained teams where you can, and be honest about what the data cannot show.
A simple dashboard
| Measure | Frequency | Owner |
|---|---|---|
| Core completion | Monthly | Program lead |
| Repeat finding rate by category | Quarterly | Security lead |
| Median time to fix | Quarterly | Engineering managers |
| Share found at review | Quarterly | Security lead |
Four to six lines is enough. If nobody acts on a number, drop it.
Where Safeguard fits
Outcome metrics need data from your repositories. Safeguard's vulnerability management and continuous monitoring views can supply finding and fix timelines. It does not offer training courses. See what a scan shows with the free scan at gold.safeguard.sh/scan. Related reading: measure whether secure coding training worked.
Quick check
Should we stop reporting completion rates?
No. Keep them for coverage and audit records. Just do not use them to claim the training changed behaviour.
How long before training effects show in data?
Expect at least a couple of quarters for patterns to appear, and watch for other changes that could explain the movement.
Sources
- NIST SP 800-50 Rev. 1: csrc.nist.gov/pubs/sp/800/50/r1/final
Last reviewed: 19 October 2026.
