Ask the vendor to scan a repository you choose, live, and to show ten specific things: setup time, a real finding with its data flow, a suppression, a pull request comment, the output format, scan duration, rule customisation, history after a rescan, access controls, and what happens on failure. A canned demo shows none of that.
Why a script helps
Demos are rehearsed. The sample app is picked because the tool does well on it. A written script gives every vendor the same test and gives you notes you can compare a week later, when the memories have blurred together.
Send the script a few days ahead so they can prepare, but do not send the repository until the call. If they ask to see your code first, offer a public project in your main language instead.
The ten requests
- Start from nothing. Ask them to connect a fresh repository and time it. Note whether the build needed changes.
- Open one finding. Ask them to show the path from user input to the dangerous call, not just a file and line.
- Explain a rating. Pick one high severity result and ask why it is rated that way and what would lower it.
- Dismiss a finding. Show how to mark one as not applicable, where the reason is stored, and who can see it.
- Show the pull request view. Open a branch with a deliberate flaw and watch where the result appears.
- Export the results. Ask for the raw output. Check whether it is a standard format such as SARIF.
- Time a full scan and an incremental one. Ask what they mean by incremental.
- Write or edit a rule. Even a trivial one tells you how much work customisation is.
- Rescan after a change. Fix one issue and rescan. Does the old finding close, and does the history survive?
- Break something. Ask what a developer sees if the scan service is down or times out.
What good answers look like
For request 2, a useful answer names the source, each step the data takes, and the sink. A tool that only highlights a line has told you little about whether the flaw is reachable by an attacker.
For request 4, look for a reason field and an audit trail. Dismissals with no record turn into a place where risk hides.
For request 6, OWASP's list of selection criteria for source code analysis tools includes interoperability of output and names SARIF as the standard. If the answer is a proprietary report only, ask what export you would have if you left.
For request 10, the right answer is a policy you control. A scan outage should not silently pass or silently block your releases.
Scoring
Give each request a 0, 1 or 2 and write one sentence of evidence. Keep the sheet to one page. If you plan a full trial afterwards, the demo score decides who gets one, and the trial decides who gets bought.
Be wary of answers that start with "that is on the roadmap". Record them as no, with a note, and move on.
Try it yourself
Click through it at your own pace, or open it full screen. Follow one finding from detection to fix, and compare how much context it gives you with what the vendors show you.
Quick check
Should I let the vendor pick the repository?
No. Let them pick for a first look if you like, then switch to your own code. Their sample will be chosen to flatter the tool.
How long should the demo take?
Ninety minutes is enough for ten requests if the vendor prepares. Anything they cannot show live goes on the follow up list.
Safeguard's pull request gates and policies, reachability analysis, and SBOM and dependency scanning are generally available, and you can try the dependency side free at gold.safeguard.sh/scan.
Sources
Last reviewed: 19 October 2026.
