A threshold can be set and still do nothing. Warn-only switches override it, some flags change exit codes but not output, and defaults may be stricter or looser than you think. Test your gate with a known bad dependency to confirm it fails.
Ways a threshold goes quiet
| Cause | What happens | Evidence |
|---|---|---|
| Warn-only enabled | Run succeeds, findings logged as warnings | The GitHub dependency review action says warn-only overrides fail-on-severity |
| Wrong exit code handling | Scanner fails, pipeline step continues | A shell step ignores non zero results |
| Threshold affects output, not exit | Findings print, build passes | npm audit docs say its audit level does not filter output |
| Check not required | Gate fails, merge allowed | Branch protection missing the check |
| Scan skipped | Nothing ran | Conditional or path filter |
The check that proves it
Do a seeded test, at least once per repository and again after any pipeline change.
- Create a branch that adds a dependency with a known Critical advisory.
- Open a pull request.
- Confirm the gate reports failure.
- Confirm the merge button is disabled.
- Remove the dependency, and confirm the gate passes.
Write down the date and result. It takes ten minutes and tells you more than any dashboard.
Read the defaults
The dependency review action lists a default minimum severity of low, which means almost anything fails. That is strict and may be a nasty surprise at rollout. The npm audit docs say that by default it exits non zero if any vulnerability is found, and that the exit code depends on the audit level once set, so a run can exit 0 even with findings below that level.
Read your own tool's documentation for three facts: the default severity, how it maps findings to an exit code, and whether warn modes override thresholds.
Common misconfigurations
- A pipeline step ends with a pipe to another command, which hides the scanner's exit code.
- The threshold lives in a config file that is not read in CI.
- A staging pipeline has the gate, the production one does not.
- The gate was set to warn during rollout and never switched back.
Add an alert for silent gates
Track the share of pull requests where the gate ran and where it failed. A gate that never fails across hundreds of changes either guards a very clean codebase or does nothing. Know which.
Quick check
How often should I run the seeded test?
After every pipeline change, and at least once a quarter.
Why not just trust the green check?
A green check can mean pass or can mean the scan never ran. Only a seeded failure shows the gate works.
Safeguard's policies and gates show each decision with the rule that caused it, which makes a silent gate easier to spot. The free scan is at gold.safeguard.sh/scan.
Sources
Last reviewed: 19 October 2026.
