Neither wins everywhere. Precision is the share of flagged items that are real problems. Recall is the share of real problems the tool flags. Favour precision where findings block developers, and recall where a human triages the output and a miss would be expensive.
The two definitions, with numbers
Say a test set holds 20 real vulnerabilities. A reviewer flags 25 items. Of those, 15 are real and 10 are noise.
- Precision is 15 of 25, or 60 percent.
- Recall is 15 of 20, or 75 percent.
The five missed vulnerabilities are false negatives. The ten wrong flags are false positives. If you want a refresher on the four outcomes, see true positive, false positive and false negative explained.
The two numbers pull against each other. A reviewer that flags everything has perfect recall and terrible precision. One that flags almost nothing has high precision and almost no recall.
Where the reviewer sits changes the answer
| Position in the flow | Favour | Reason |
|---|---|---|
| Blocks a merge | Precision | Wrong blocks stop work and teach people to bypass the gate |
| Comments on a pull request, advisory only | Balance | Cheap to ignore, but noise trains people to ignore |
| Runs in the editor as you type | Precision | Interrupts focus, so every alert must earn it |
| Nightly sweep reviewed by a security engineer | Recall | A person filters, and a late catch beats no catch |
| Checks code touching authentication or payments | Recall | The cost of a miss is high |
This is why one tool can be right for one stage and wrong for another. When a vendor quotes a single accuracy figure, ask which stage it was measured for.
Ask for the counts, not the percentage
Percentages hide the base rate. Most changes in a normal week contain no vulnerability at all. If one change in 200 is vulnerable, a reviewer with 95 percent precision on your test set may still bury you on real traffic, because the many clean changes each carry a small chance of a wrong flag.
So measure on a sample of real recent pull requests as well as on seeded cases. Count the wrong flags per 100 pull requests, not just a ratio. A number such as "four wrong comments per 100 pull requests" is something an engineer can picture.
Cost per error
Put rough costs on each kind of error, in minutes. A wrong flag costs a developer perhaps five to fifteen minutes to read and dismiss. A miss costs whatever the bug costs later. You do not need exact figures. Even a rough ratio tells you which direction to lean, and it makes the trade-off discussable with people who do not live in the tool.
Do not rely on the tool's own confidence score
Some tools print a confidence number beside each finding. Check it against your labelled cases. If findings marked high confidence turn out wrong as often as the low ones, the score is decoration. If it does track reality, you can set a threshold per stage: strict for the merge gate, loose for the nightly sweep.
A simple decision rule
- List the stages where you would use the tool.
- For each, decide whether a person or a gate acts on the output.
- Set a target for wrong flags per 100 pull requests where a gate acts, and a target for catches on high-risk cases where a person does.
- Run the tool on your own data and see which stages it fits.
Quick check
Is a high recall number enough on its own?
No. A tool that flags everything scores perfect recall and is useless. Always read recall next to precision.
Can I improve both at once?
Yes, with better context, such as whether the vulnerable code is actually reachable. That removes wrong flags without hiding real ones.
Safeguard includes reachability analysis, which checks whether a flagged dependency issue is actually reachable from your code, a common way to raise precision without losing recall. You can try a scan at gold.safeguard.sh/scan.
Last reviewed: 19 October 2026.
