Record every security job outcome with a reason code that separates "the scanner ran and found something" from "the scanner could not do its job". Four or five codes are enough. Without them, tool failures inflate your findings count and real risk gets lost.
The problem with a pass or fail bit
Most pipelines store one bit per job: green or red. A red scan job could mean a critical vulnerability, a corrupted lockfile, an expired API token or a runner that ran out of disk. Dashboards built on that bit show a rising failure rate that nobody can interpret.
A minimal code set
- clean: the scan completed and nothing crossed the threshold.
- findings: the scan completed and results crossed the threshold.
- policy: the scan completed and a non-vulnerability rule failed, such as a license rule.
- incomplete: the scan ran but could not cover everything, for example a manifest it could not parse.
- error: the scan did not complete.
The incomplete code matters most and is the one teams forget. A job that exits clean after skipping half the repository is worse than a failure, because it looks like safety. If your scanner can report skipped inputs, record them.
Where to record the code
Pick the place your team already queries.
- A line in the job log with a fixed prefix, such as SECURITY_RESULT followed by the code. Easy to grep.
- A job artifact holding a small JSON summary with the code, the counts and the scanner version. Easy to collect later.
- A custom field on a pipeline or check run, if your CI system supports it.
Include the scanner version and the data feed date in the same record. When results shift, the first question is often "did the tool change?"
Reading results from the SARIF format
If your scanner writes SARIF, the standard gives you some of this vocabulary already. A result can carry a kind of pass, fail, open, review, informational or notApplicable, and a level of error, warning, note or none. Use them as inputs, but do not stop there. SARIF describes results, not whether the job itself ran correctly, so the incomplete and error codes still come from your wrapper.
Reviewing the data
Once a week, look at four ratios: findings over completed scans, error over all scans, incomplete over all scans, and policy over completed scans. Keep the denominators in view, since a handful of builds in a quiet week can swing any ratio.
If error and incomplete together exceed findings, your problem is the pipeline, not the code. Fix that first, because every other metric is built on completed scans.
When you fix a recurring error, write down what changed and the date, so you can see the effect on the chart.
Quick check
Should incomplete scans fail the build?
On protected branches, usually yes or at least warn loudly, since the code was not fully checked. On feature branches, many teams warn and fix in a follow-up.
Do I need SARIF for this?
No. SARIF helps with portability, but a plain JSON summary with the five codes works fine.
How Safeguard fits
Safeguard's dependency and SBOM scanning feeds policies and gates, so you can tie a failed gate to a named finding or rule rather than a bare red build. A free scan is available at gold.safeguard.sh/scan.
Sources
- OASIS, Static Analysis Results Interchange Format (SARIF) Version 2.1.0, result kind and level properties: https://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html
Last reviewed: 19 October 2026.
