Document every gate the old scanner enforces, rebuild each on the new scanner in report only mode, compare what each would have blocked against what the old gate did, and switch to enforcing one gate at a time. Never turn the old gate off before the new one is proven.
Start with the current behavior
Write down what the old gates actually do, not what the policy says. For each gate:
- Where it runs: pull request, merge to main, release pipeline.
- What it blocks on: severity, specific advisories, license, secrets.
- Who can override it and how.
- What the developer sees when it fails.
You will often find undocumented exceptions in the pipeline files.
Recreate in report only mode
Add the new scanner's check to the pipeline so it runs and reports but never fails the build. GitHub's documentation describes how code scanning results appear as a separate check per analysis tool in a pull request, which is a convenient way to see both side by side. Whatever your CI system, keep the new check visibly labeled as non-blocking.
Compare outcomes
For a few weeks of real pull requests, collect:
| Question | What to record |
|---|---|
| Would the new gate have blocked the same builds? | Matches and differences |
| Did it block anything the old one passed? | Review each for validity |
| Did it pass anything the old one blocked? | Find the cause: coverage or threshold |
| How long did it add to the pipeline? | Compare with your deploy window |
Differences are expected. Each one is either a threshold to adjust or a coverage point to fix. See mapping severity between scanners.
Enforce one gate at a time
Start with the least disruptive gate, for example blocking on newly introduced critical issues only. Wait a release cycle, then add the next. Our guide to block, warn or observe helps pick the right mode for each.
Keep an override path
A gate with no override path creates workarounds. Define who can approve an exception and require a reason and expiry. See setting up a security exception process.
Try it yourself
Click through it at your own pace, or open it full screen.
Click through the pull request gate and note the difference between a gate that warns and one that blocks.
Sources
Quick check
How long should report only mode last?
Long enough to cover a representative set of pull requests and at least one release. Often a few weeks.
Should the old and new gate both block during overlap?
Keep the old one blocking and the new one reporting. Two blocking gates double the friction for developers.
Safeguard covers SBOM and dependency scanning, reachability analysis, policies and gates, and compliance evidence automation. You can try a free scan at gold.safeguard.sh/scan.
Last reviewed: 19 October 2026.
