A policy and gate feature decides which code ships, so evaluate it for clarity, testability, exceptions and failure behaviour before you enforce anything. A good test is to run the proposed rules in report mode against recent history and see what would have been blocked.
What a policy engine should let you say
At minimum, you should be able to write rules like: block a release if a known exploited vulnerability is present in a reachable dependency; warn on a disallowed licence; require a signed build for production branches. Rules should be readable by a human who did not write them, and ordered in a way you can explain. If two rules conflict, you want to know which wins and why.
The evaluation checklist
- Rule clarity. Can a non-expert read the rule and predict the outcome?
- Inputs. What data can a rule use: severity, exploit signal, reachability, licence, package age, source?
- Dry run. Can you run a rule in report mode and see impact on past builds?
- Scope. Can rules apply by team, repository or branch?
- Exceptions. Can an exception be time limited, require an approver, and appear in a log?
- Failure mode. If the policy service is unreachable, does the build fail open or fail closed? Which do you want?
- Messages. Does a blocked developer see what rule fired and what to do?
- Audit history. Can you show who changed a rule and when?
Do a replay test
Take the last two months of merged pull requests. Apply the draft policy in report mode. Count how many would have been blocked, how many of those were legitimate catches, and how many were false alarms. If more than a small share of blocks are noise, adjust before enforcing.
Roll out in stages
Start with warn, move to block for the narrowest high confidence rule, and widen from there. Tell developers what is coming and where to ask for exceptions. Surprise blocks are the quickest way to lose goodwill.
Trade-offs
Tight rules reduce risk but add friction. Loose rules protect velocity but weaken the point of the control. Match strictness to the system: a payments service deserves stricter rules than an internal prototype.
Try it yourself
Open the exploitability check demo and see how an exploit signal changes the decision for a finding. Imagine which rule you would write from it.
Click through it at your own pace, or open it full screen.
Where Safeguard fits: policies, gates and guardrails are generally available, along with SBOM and dependency scanning; the free scan at https://gold.safeguard.sh/scan is an easy first look.
Quick check
Should gates fail open or closed?
It depends on the system. Many teams fail closed for production releases and open for early branches, but decide on purpose.
How long should an exception last?
Short and renewable. Open-ended exceptions tend to become permanent.
Last reviewed: 19 October 2026.
