Ask about setup effort, noise, failures, support and what the customer would change. References are chosen by the seller, so lead with concrete questions about weeks, counts and incidents, not with whether they are happy. Specifics reveal more than praise.
Before the call
Ask the vendor for references that match your size, languages and pipeline. Ask for at least two, and if possible one that has used the product for more than a year. Also look for peers you find yourself, through your network or community, and treat their views as a separate sample.
Write your own test results down first so you can compare answers with what you saw.
Setup and rollout
- How long did it take from contract to first useful result?
- Which teams adopted it first, and which resisted?
- What did you have to change in your pipeline?
Noise and accuracy
- How many findings did you get in the first month, and how many did your team act on?
- How did you tune it, and who owns that tuning now?
- Have you found a real issue the tool missed?
Reliability and support
- Has a scan ever failed silently or returned an incomplete result?
- How long did support take to answer your last serious question?
- How were you told about changes that affected your results?
Fit and cost of ownership
- Which part of the product do you use daily, and which do you ignore?
- What does it take, in people hours per month, to run?
- What surprised you after the contract was signed?
Exit and honesty
- What would make you leave?
- What would you tell a team like yours to test before buying?
- If you could change one thing, what would it be?
How to weigh the answers
Look for concrete numbers and dates. Notice what a reference cannot answer. Notice hesitation on question 13. Compare notes across calls and with your own test. If every reference says exactly the same words, ask whether they were briefed.
Remember that a reference is one customer's experience in one environment. It is a lead for your test, not a replacement. See how to run your own bake-off.
Running the call well
Send three questions ahead of time so the reference can prepare numbers. Take notes in a shared document so a second person can check them. Ask the same questions of every reference, so answers can be compared. End by asking who else you should speak to, since a reference who suggests another customer is usually relaxed about the product.
Where Safeguard fits
Safeguard offers SBOM and dependency scanning, reachability analysis and policy gates. You can form a first view without any call by running the free scan at gold.safeguard.sh/scan.
Quick check
Are reference calls worthless because they are curated?
No. Specific questions still surface real detail. Just add peers you find yourself and weigh the call below your own test.
Last reviewed: 19 October 2026.
