Score a security tool trial with criteria and weights fixed before the trial, a simple 0 to 3 scale with written definitions, a link to evidence for every score, and tie breakers agreed in advance. Add a rule that time already spent cannot count as a reason to buy.
Fix the scorecard before the trial
Scoring after you have seen results invites you to weight what the tool does well. Write the criteria and weights at kickoff, have the decision maker sign off, and do not change them mid trial.
A scale people can use
Five point scales invite arguments between a 3 and a 4. Use four levels with definitions.
| Score | Meaning |
|---|---|
| 0 | Failed, or not demonstrated |
| 1 | Partly met, with significant gaps |
| 2 | Met with minor gaps |
| 3 | Fully met, with evidence |
A score with no evidence link counts as 0. This one rule keeps the exercise honest.
Example criteria and weights
Adjust to your needs. The numbers are an illustration, not a recommendation.
| Criterion | Weight | Evidence source |
|---|---|---|
| Finding accuracy | 25 | Triage sample |
| Fit with developer workflow | 20 | Developer notes, build timings |
| Integrations | 15 | Integration tests |
| Setup and ongoing effort | 15 | Hours log |
| Data handling and security | 10 | Kickoff answers, documents |
| Support | 10 | Ticket response record |
| Reporting and evidence export | 5 | Export test |
Total weight is 100. Multiply score by weight, divide by 3 times the total and you get a percentage. Treat small differences, a few points, as a tie, because the inputs are not that precise.
Add gates
Some criteria should be pass or fail, whatever the total. For example, if data handling terms are unacceptable or the tool cannot scan a core language you use, it fails regardless of score. Define gates before the trial.
Tie breakers
When two tools are within a few points, apply these in order:
- Which one did developers prefer, in writing?
- Which one needed less vendor labour to set up?
- Which one has fewer open risks on the list?
- Which one makes leaving easier, through open formats and exports?
Guard against sunk cost
Weeks of effort make people want a positive outcome. Counter it with three habits:
- Score each criterion separately, ideally by different people, before discussing.
- Ask, "If we were starting today with only this evidence, would we choose it?"
- Record dissent. If one reviewer scored low, write down why.
A no is a good result. It cost a few weeks instead of a year of shelfware.
Make the decision meeting short
Circulate the scorecard 24 hours before. In the meeting, review gates first, then the scores, then open risks, then decide. See a go or no go meeting for a longer format.
Keep the record
Store the scorecard, the evidence log and the decision in one place. If you revisit the choice in a year, you will know what you knew then. Safeguard's free scan can contribute a data point on dependency findings for the accuracy row.
Quick check
Why use four levels instead of five?
Four levels have no middle, which forces a decision, and clear definitions reduce disagreement between reviewers.
What if the scores are very close?
Treat a gap of a few points as a tie and use the tie breakers you agreed in advance.
Last reviewed: 19 October 2026.
