Require that the tool can be automated end to end, fits your identity setup, produces open formats, runs fast, fails safely, and lets you leave with your data. Test each point in a trial with your own repos. A polished dashboard matters less than these operational basics.
The checklist
| Area | Ask | How to test |
|---|---|---|
| Automation | Is everything doable by API or CLI, including onboarding and policy? | Onboard ten repos by script |
| Pipeline fit | Does it run in your CI systems and as a reusable step? | Add to the shared template |
| Identity | Single sign on, role based access, group sync | Set up a team scoped role |
| Policy | Org wide defaults, per project overrides, exception expiry | Create a tiered policy |
| Formats | SARIF and SBOM in standard formats, export of findings | Export and re read |
| Speed | Median and slow scan time on your largest repo | Time it, with caching |
| Failure modes | What happens on timeout, outage or rate limit? | Break the network on purpose |
| Data handling | Where code and metadata go, retention, local run option | Read the data terms, test local mode |
| Noise | Reachability or exploitability context to cut false positives | Compare findings to known truth |
| Reporting | Coverage and trend via API | Build your coverage report |
| Exit | Can you export history and move off? | Ask for the export before you sign |
Run a two week trial
- Pick five repos: a monorepo, a small service, a legacy app, an infra repo and one container image.
- Define success before you start: coverage, scan time, noise rate, developer rating.
- Integrate through the shared pipeline, not by hand.
- Seed known findings to see what the tool catches and misses.
- Ask developers who did not choose the tool to rate it.
- Write down what was hard. Those costs continue after the contract.
Questions about the vendor
Ask for security documentation and the status of any attestations in plain terms. Do not accept vague claims about certifications. Ask how product changes are communicated and how breaking API changes are handled. See how to run a two week trial for a similar method.
Beware of lock in by default
Some tools make exit painful through proprietary formats. Prefer open formats for results and inventories. Even if you never switch, the ability to do so keeps the relationship honest.
Quick check
Is a free trial enough to decide?
Only if it runs on your real repos through your real pipeline. A demo on sample code tells you little about fit.
Who should be in the evaluation?
Platform, security, at least two developers from different teams, and whoever owns compliance evidence.
Safeguard offers a CLI, SBOM and dependency scanning, policies and gates, and a free scan at gold.safeguard.sh/scan you can try on your own repo.
Last reviewed: 19 October 2026.
