Start in report only mode, tune the noisiest rules with developers in the room, name a champion in each team, and gate only new findings in the last week. A rollout that respects developer time and shows its reasoning is far more likely to stick than a switch flipped overnight.
Why rollouts fail
The usual failure is predictable. A security team enables a scanner, blocks merges on day one, and a wave of findings lands on people who did not ask for them. Many findings are false alarms. Trust drops, overrides rise and the tool gets switched off. OWASP notes that static code analysis tools often produce false positives, and that analysts treat them as aids that point at security relevant code, not as machines that find flaws automatically. A rollout plan should assume that and build in time to tune.
The four weeks at a glance
| Week | Goal | Mode | Output |
|---|---|---|---|
| 1 | Learn the baseline | Report only, no comments on pull requests | Baseline scan, list of noisy rules |
| 2 | Tune with developers | Report only, comments on a pilot team | Tuned rule set, first champion notes |
| 3 | Expand and train | Comments on all teams, still not blocking | Training done, override process live |
| 4 | Gate new findings | Block only new high confidence findings | Gate on, baseline frozen |
Week one: baseline and listening
Run the tool on your main repositories without posting anything to developers. Save the results in an exportable form with the commit hashes. Group findings by rule and by weakness type, and count how many rules produce most of the output. Usually a small number of rules create most of the noise.
Meanwhile, talk to two or three developers. Ask what frustrates them about past tools. Common answers include slow scans, unclear messages and findings in code they did not write. Write the answers down, since they will shape weeks two and three.
Week two: tune with a pilot team
Choose a team that is willing, with a mix of seniority. Turn on pull request comments for that team only. For each finding the team disputes, decide together whether it is a real issue, a rule to tune, or a missing definition such as an unrecognized helper function. Keep a shared log of decisions.
A worked example of what you may see: a rule for building queries from strings fires on a helper that actually escapes values. The team adds the helper as a recognized sanitizer if the tool allows, or disables that rule for that file with a reason and a review date. Next scan, the finding is gone, and the developers see that feedback changes the tool.
Week three: expand and train
Roll comments out to all teams, still without blocking. Run a thirty minute session, once per team, with three parts: what the tool checks, how to read a finding (source, sink, path), and how to request an override. Use real findings from the team's code, one real and one false alarm.
Name a champion in each team. The champion is not a gatekeeper. They answer questions, nominate noisy rules for tuning and help colleagues read findings. Give them a direct channel to the security team.
Set up the override path before gating. A short decision guide on who can override a gate helps you define approvers and expiry.
Week four: gate new findings only
Freeze the baseline from week one. Configure the gate to block only new findings that are high severity and high confidence. Existing findings stay visible but do not block. This is sometimes called a ratchet, and it lets the team improve without having to clear the backlog first.
Announce the date a week ahead. On the day, watch the override and block counts. If the numbers surprise you, pause and tune before blaming people.
What to measure in the first month
| Measure | Why it matters |
|---|---|
| Share of findings marked false positive, by rule | Points to rules needing tuning |
| Scan time on pull requests | Slow scans erode goodwill |
| Number of override requests and reasons | Shows friction and abuse |
| Findings fixed per week | Shows real progress |
| Developer survey, five short questions | Gives a view the numbers miss |
Avoid vanity measures such as total findings. A growing count can reflect better coverage and says little about quality.
Common mistakes to avoid
- Gating before tuning. Block only after the noise is under control.
- Ignoring scan time. If pull requests wait many minutes, developers will resent the tool.
- No owner for rules. Someone must own tuning requests, or they will sit forever.
- Treating the baseline as a debt to pay immediately. Plan a gradual burn down alongside normal work.
- Skipping communication. A short note explaining what is changing and why prevents many arguments.
After day thirty
Review the plan with the champions. Decide which repositories to add next, which rules to promote to blocking, and which to retire. Keep a quarterly rule review on the calendar. A tool that is tuned regularly stays useful, and one that is left alone slowly turns into noise.
Try it yourself
Click through it at your own pace, or open it full screen. Look at what a developer sees on a pull request when a check fails, since that moment decides whether the rollout feels fair.
Quick check
Should the first scan comment on pull requests?
No. Start in report only mode so you can tune noisy rules before developers see anything.
What should block a merge in week four?
Only new findings that are high severity and high confidence. Existing findings remain visible but do not block.
Who should own rule tuning?
A named person in security or platform engineering, with champions in each team who send requests. Without an owner, tuning stalls.
Safeguard offers policies, gates and guardrails for pull requests, SBOM and dependency scanning, and a free scan at gold.safeguard.sh/scan.
Sources
Last reviewed: 19 October 2026.
