Ask ten things: how it groups, how it schedules, what it automerges and when, how it handles monorepos and private registries, how it treats security fixes, which ecosystems and indirect dependencies it covers, what it runs, how it limits noise, and how you will measure it. Test each in a pilot, not a demo.
The ten questions
| # | Question | What a good answer includes |
|---|---|---|
| 1 | How does grouping work? | Rules by name, type and level, a documented order, and one pull request per group |
| 2 | How is scheduling controlled? | Timezone setting, cron or intervals, per package schedules, and whether security fixes bypass them |
| 3 | What is the automerge policy? | Per update type, required checks, queue support, and what happens with no tests |
| 4 | How are monorepos handled? | Multiple directories in one config, grouping across them, path exclusion |
| 5 | How are private registries supported? | Declared hosts, read-only credentials, OIDC or tokens, private network reach |
| 6 | How are security fixes treated? | Separate limits, labels, no delay from cooldowns, indirect coverage |
| 7 | Which ecosystems and dependency types? | A written list, including indirect packages and lockfile updates |
| 8 | What does the job execute? | Whether manifest code runs, where, with which credentials |
| 9 | How is noise limited? | Open and hourly caps, approval for majors, a dashboard, rebase control |
| 10 | How do we measure success? | Merge rate, time to merge, lag, and closed without merge counts |
How to test each one
Questions one to four are about shape, and a pilot answers them quickly. Pick one monorepo and one ordinary service, run the tool for two weeks, and read every pull request title. Do the groups match your intent? Did schedules fire when expected? Did automerge work only where you allowed it?
Question five is the most common cause of failed pilots. Use a real private registry and a real lockfile update. Our post on giving an update tool private registry access safely lists what to watch.
Question six needs a known vulnerable dependency in a test repository. See what the tool opens, how fast, and how it is labeled. Read whether security pull requests count against the open limit first.
What the documentation shows you can ask for
Update tools differ on exactly these points, as their public documentation shows. One has a dedicated grouping system where a dependency joins the first matching group. One offers a hosted mode and a self-run mode. One documents that automerge handles one pull request per target branch per run. One disables external code execution by default when registries are configured. A vendor who cannot point you to documentation for the answer to a question probably does not have a stable answer.
Red flags
- No way to cap pull requests or rate limit creation.
- No documentation on how security fixes bypass schedules and caps.
- Credentials requested with write access when read would do.
- No path to require approval for major updates.
- Claims of fixing everything with no mention of tests or review.
Beyond the tool
A tool opens pull requests. It does not tell you which vulnerabilities your code can actually reach, and it does not enforce policy across teams. Decide up front whether you need another layer for prioritization, evidence and gates, and how it fits with the bot. Our post on when automated updates are enough helps you judge.
Try it yourself
Click through it at your own pace, or open it full screen.
Click through the findings and notice how triage and ownership sit above the pull request, which is the layer an update tool alone does not provide.
Quick check
How long should a pilot last?
Two update cycles at least, so you see schedules, rebases, automerge and at least one security fix.
Should the pilot include a monorepo?
Yes. Monorepos expose grouping, directory and lockfile behavior that small repositories hide.
Safeguard does not replace an update bot. It builds an SBOM from manifests and lockfiles, uses reachability analysis to show which flagged dependencies your code calls, and applies policies and gates to pull requests. Fix pull requests are off by default, with every category on Manual. Try a free scan.
Sources
- Code host documentation: update bot options reference
- Update bot documentation: configuration options
- Update bot documentation: noise reduction
Last reviewed: 19 October 2026.
