Review your testing mix once a year with eight checks: coverage of what you ship, noise level, ownership, input quality, lessons from incidents, known gaps, evidence trail and effort spent. Write the result as a one-page memo with changes, owners and dates. Without a review, a testing stack drifts away from the product.
Why an annual review
Products change. A new language appears, an old service gains customers, a parser gets added. The tools chosen two years ago may not fit. A review is a small, regular check that testing still matches reality.
The eight checks
1. Coverage. List every repository and service. Mark which methods run on each. Unscanned items go to the top of the plan. OWASP's component analysis guidance treats an accurate inventory as the base for everything else, and the same logic applies to testing coverage.
2. Noise. Count findings fixed versus dismissed for each method. If most are dismissed, tune or replace.
3. Ownership. Check each method has a named owner who triages results. Unowned scans should be assigned or stopped.
4. Inputs. Confirm lockfiles are committed, builds work in CI, staging is stable and test data is current.
5. Incidents. List bugs that reached production this year. For each, ask which method should have caught it and why it did not.
6. Gaps. Check what risks are untested: access control logic, parsers, third-party services, infrastructure.
7. Evidence. Check you could show a buyer or auditor dated results from the last quarter without hunting.
8. Effort. Estimate hours spent on triage per method. Compare to value.
Scoring grid
| Check | Green | Amber | Red |
|---|---|---|---|
| Coverage | All repos scanned | A few exceptions with plans | Unknown repo list |
| Noise | Most findings acted on | Mixed | Most dismissed |
| Ownership | Named owner each | Some unowned | Nobody triages |
| Evidence | Retrieved in minutes | Retrieved in days | Cannot produce |
Fix reds first, ambers second.
Outputs
- A one-page memo with decisions
- A dated change list with owners
- An updated coverage map
Share it with engineering leads. The goal is agreement on changes, not a report that sits in a folder.
An example of a finding from the review
A team reviewing its incident list found that two production bugs in a year came from the same file importer. No scan covered it, because it parsed uploaded spreadsheets and nobody had thought of it as an attack surface. The fix to the mix was small: add a fuzz target for that parser and a size limit on uploads. Without the look-back, the gap would have stayed.
Tips for running the review
- Do it at the same time each year so it becomes routine, for example before budget planning.
- Bring numbers, not impressions. Count findings, fixes and hours.
- Limit the output to three changes. A list of fifteen never gets done.
- Check last year's memo first. Did the changes happen?
Where Safeguard fits
Safeguard's asset discovery, continuous monitoring of dependency findings, and compliance evidence automation support the coverage, evidence and noise checks on the dependency side. Static, dynamic and fuzz results come from your other tools. Run the free scan at https://gold.safeguard.sh/scan.
Try it yourself
Click through it at your own pace, or open it full screen. Look at how ongoing results build the trail you use in the evidence check.
Quick check
How long should the review take?
A half day for a small team. Most of the time goes to the coverage list and the incident look-back.
Who should lead it?
The security owner or a senior engineer, with each method's owner contributing their numbers.
Sources
Last reviewed: 19 October 2026.
