A security check fails a CI build for one of five reasons: it found something over your threshold, a policy rule was broken, the tool itself errored, the job hit a time limit, or the environment let it down. Only the first two are real security signal. Count them separately.
Why one "failed builds" number misleads
If you report "builds failed because of security" as a single figure, a team that fixes slow runners looks the same as a team that ignores critical findings. The number goes up and down, and nobody knows what to do about it. A taxonomy fixes that by giving every failed security job exactly one cause label.
The five causes
| Cause | What it means | Who usually fixes it |
|---|---|---|
| Finding over threshold | A vulnerability or weakness crossed your severity or policy line | The developer |
| Policy violation | A rule such as a license or banned package was broken | The developer or policy owner |
| Tool error | The scanner crashed, could not parse a file or lost its data feed | Platform team |
| Timeout or limit | The job ran past its time budget and was terminated | Platform team |
| Environment failure | No runner, network error, registry outage, missing credential | Platform team |
The first two are what a security gate exists to catch. The last three are reliability problems that happen to show up as red builds. They erode trust, because a developer who sees a red build for a reason unrelated to their change learns to re-run and ignore.
Platforms do enforce limits. GitHub documents, for example, that a job reaching its execution time limit is terminated and fails, and that a job waiting too long in the queue is cancelled. Those are timeout and environment outcomes, not findings.
How to label failures
You do not need a new tool to start. Three steps are enough.
- Make the scanner step write one line to the job log or a small JSON file with a reason code, such as finding, policy, tool_error, timeout or environment.
- Map exit codes to those codes in a short wrapper script. Treat "scanner could not run" as different from "scanner ran and found problems".
- Ship the reason code with each pipeline run record so you can group by it later.
If your scanner uses one exit code for everything, wrap it. The wrapper can check whether a results file exists and parsed correctly before deciding which label to apply.
What to do with the counts
Look at the mix, not the total. A healthy picture is mostly finding and policy failures, with the other three small and shrinking. If tool errors or timeouts dominate, your gate is training developers to distrust it, and the fix belongs with the platform team. If environment failures cluster at certain hours, look at runner capacity.
Review the taxonomy monthly. When a new kind of failure shows up that does not fit, add a label rather than forcing it into "other". If "other" is more than a small slice, your taxonomy is too coarse.
Try it yourself
Click through it at your own pace, or open it full screen. Watch which outcomes come from a finding and which would come from the tool or environment.
Quick check
Should a tool error block a merge?
That is a policy choice. Many teams block on tool errors for protected branches and warn on feature branches. Whatever you choose, label it as a tool error so it is never counted as a finding.
Is a timeout a security failure?
It is a reliability failure with security consequences, because the code was not checked. Count it, label it, and fix the time budget.
How Safeguard fits
Safeguard provides policies and gates for dependency and SBOM scanning, so a failed gate points to a specific finding or policy rule you can act on. You can try a free scan at gold.safeguard.sh/scan.
Sources
- GitHub Docs, Actions limits (job execution time and job queue time behaviour): https://docs.github.com/en/actions/reference/limits
Last reviewed: 19 October 2026.
