Raw finding counts say more about how big a codebase is and how well it is scanned than about how secure it is. Before comparing teams, normalise by size, age and scan coverage, and compare each team to its own history. Never rank people by counts alone.
Why counts mislead
Team X has 40 findings and Team Y has 400. Y looks ten times worse. Then you learn that Y owns 30 services with 2,000 dependencies, and X owns one service with 60. Per dependency, X has 0.67 findings and Y has 0.2. The ranking flips.
Other things that distort raw counts:
- Scan depth. A team that enables more scanners and deeper analysis finds more. Counting that as worse punishes the team doing the most work.
- Age of code. Older services have had more time to accumulate issues.
- Language and ecosystem. Some ecosystems publish far more advisories per package than others, regardless of how safe the code is.
- Duplicates. The same flaw in a shared library shows up once per consuming repository.
Pick a denominator that fits the question
- Findings per 100 dependencies: useful for supply chain exposure.
- Findings per thousand lines of code: useful for first party code, with the usual caution that lines are a crude size measure.
- Findings per service or per deployed asset: useful when ownership is by service.
- Open critical findings per team: a workload measure, not a quality one.
State the denominator and the source next to every figure. See percent versus count for why.
Compare teams to themselves
The fairest benchmark is the same team over time. A decline in open critical findings per 100 dependencies across six months says something real. A league table across teams mostly says who inherited what. If you must compare teams, split by severity first to avoid a mix effect like the one in Simpson's paradox.
Watch the incentive
Any number used to rank people gets gamed. If a team is judged by fewer findings, the cheapest response is to scan less. Pair every raw count with a coverage measure, such as the share of repositories scanned in the last week, so reducing visibility cannot improve the score. Our post on Goodhart's law and security metrics shows more examples.
A short checklist before publishing a comparison
- Is the denominator the same kind for every group?
- Are scan settings the same, or at least recorded?
- Have duplicate findings been merged?
- Is the mix of severities shown?
- Does the comparison include a change over time, not just a snapshot?
- Would a team be punished for finding more?
If any answer is no, present the figures as context, not as a ranking.
Quick check
Is findings per thousand lines of code a fair measure?
It is a start for first party code, but lines vary by language and style. Use it for trend within a team more than for comparing teams.
Should teams with more findings be flagged as risky?
Not on counts alone. First check size, scan depth and severity mix. A high count can come from better visibility.
Last reviewed: 19 October 2026.
