A CISO approving a new application security tool gets the most from ten questions that go past the demo: what problem it solves, how that is measured, where data goes, how it fails, who runs it, and how to leave. If the team cannot answer them in plain words, the purchase is not ready.
Problem and measure
- Which specific risk does this reduce? "Better visibility" is not an answer. Name the incident type or audit finding you expect to lessen.
- What number will change, and by when? Time to remediate, share of internet facing services scanned, or findings with an owner are all measurable.
- What will we stop doing if this works? A tool that adds work without replacing any is a warning sign.
Evidence and trust
- How do we know the findings are right? Ask for a labelled sample from your own code and a false positive rate you can check.
- What can we show an auditor or board from this tool? Look for dated history, not only current state.
- What does the vendor do when its own product has a vulnerability? Ask for the disclosure and patch process.
Data and access
- What data leaves our environment, and who at the vendor can see it? Ask for the data flow in writing.
- What happens when the service is down? Does the pipeline fail open or closed, and who decides?
People and exit
- Who operates this day to day, and do they have time? An unowned tool decays within a quarter.
- How do we leave? Open format exports, deletion on request and a clear notice period.
How to use the list
Give each question a score of green, amber or red, and write one sentence next to it. The point is not a number. The point is that the reasoning is on file when someone asks later why the tool was chosen.
When the answer is to wait
Sometimes the honest outcome is "not yet". If the team has no owner, no baseline and no agreed measure, a pilot will produce opinions instead of evidence. Fix those first and the later decision will be faster.
Reporting back
After three months, report the one number you named in question two. If it did not move, say so and decide whether to adjust the setup or end the contract.
Where Safeguard fits: it provides SBOM and dependency scanning, policies and gates, reachability analysis, and compliance and evidence automation, and the free scan at https://gold.safeguard.sh/scan is a low commitment way to see output.
Quick check
Do these questions suit small teams?
Yes. A small team can answer them in a short document, and the exercise often shows that a simpler option is enough.
What if the vendor will not answer in writing?
Treat that as an answer. Unclear data handling is a reason to pause.
Last reviewed: 19 October 2026.
