A weighted scoring rubric for an application security tool decision

A weighted rubric turns a vague preference into a number you can defend. Pick five to seven categories, assign weights that sum to 100, score each vendor from 0 to 4 against written anchors, multiply, and add. The weights should come from your problem, not from a template.

Step 1: choose categories from your pain

Start with why you are buying. If your biggest problem is alert fatigue, accuracy deserves a heavy weight. If developers ignored the last tool, workflow fit matters most. A sample set:

• Detection accuracy: 25
• Developer workflow fit: 25
• Coverage of your languages and build systems: 15
• Data handling and deployment: 15
• Integration and reporting: 10
• Commercial terms and support: 10

These numbers are an example. A regulated team might move 10 points from workflow to data handling. Write down the reason for each weight, because stakeholders will ask.

Step 2: score 0 to 4 with anchors

Do not let reviewers score on a feeling. Define what each number means for each category. For accuracy:

• Score 0: no usable results on the test repositories
• Score 1: finds some known issues, noisy, many false positives
• Score 2: finds most known issues, noise needs frequent suppression
• Score 3: finds nearly all known issues, noise manageable
• Score 4: finds all known issues with few false positives and clear explanations

Write anchors for the other categories the same way. See writing score anchors for more.

Step 3: calculate

Multiply the score by the weight, divide by 4, and add across categories. A worked example for one vendor:

• Accuracy: score 3, weight 25, gives 18.75
• Workflow: score 4, weight 25, gives 25.0
• Coverage: score 2, weight 15, gives 7.5
• Data handling: score 3, weight 15, gives 11.25
• Integration: score 3, weight 10, gives 7.5
• Commercial: score 2, weight 10, gives 5.0

Total: 75.0 out of 100. Another vendor with a stronger accuracy score but weaker workflow might land at 71. The gap is small, so treat both as finalists and use the trial to break the tie, not the spreadsheet.

Step 4: add gates

Some requirements should not be traded against points. A vendor that fails a data residency rule is out, even at 90 points. Keep a separate pass or fail column and apply it first.

Step 5: protect against gaming

• Lock the weights before demos. Changing them afterwards is how a favorite wins.
• Have at least three scorers work independently, then compare. Large disagreements point to unclear anchors.
• Score evidence, not promises. Give credit only for what was demonstrated or documented.
• Record the evidence next to each score so you can explain it later.

When the numbers lie

A rubric hides risk that does not fit a category, like a vendor's financial health or a single missing integration your team cannot work around. Add a short "concerns" column and read it aloud in the decision meeting.

Run a sensitivity check: change each weight by 5 points and see if the winner changes. If it does, the decision is close and deserves more evidence, not more arithmetic.

Quick check

Last reviewed: 19 October 2026.