Pick five to eight criteria, set the weights before you look at the candidates, score each library from evidence on a one to five scale, and multiply. Treat license or security stops as vetoes, not low scores. The total should support a decision, not replace judgment.
Why use a matrix
When two libraries both seem fine, people choose by habit or by whoever argues loudest. A matrix makes the trade-offs visible and gives reviewers something to challenge. It also shows when a choice is really a tie, which is useful: if scores are close, pick the one that is easier to remove.
Step 1: set criteria and weights first
Decide weights before you evaluate, so the numbers do not drift toward a favorite. A sample set that works for many teams:
| Criterion | Weight | Evidence to use |
|---|---|---|
| Fit for the job | 30 | Your spike results on real inputs |
| Maintenance health | 20 | Release history, open issue handling, archive status |
| Security posture | 20 | Security policy, advisory history, signed releases |
| Footprint | 10 | Transitive package count |
| Documentation and API stability | 10 | Docs quality, versioning habits |
| Ease of exit | 10 | How well it can be wrapped |
Weights should add to 100. Adjust for your context: a library on a payment path might weight security higher.
Step 2: add vetoes
Some findings should end the comparison no matter what the score says:
- No license, or terms your counsel rejects.
- Archived repository, when you need future fixes.
- Unfixed high severity advisories with no maintainer response.
Write vetoes as a separate column labelled pass or stop. A cheap, fast library with a license you cannot use is not a bargain.
Step 3: score from evidence
Give each criterion a score from one to five and write the evidence beside it. A score without a link or a note is an opinion.
| Criterion | Weight | Library A | Library B |
|---|---|---|---|
| Fit | 30 | 4 | 5 |
| Maintenance | 20 | 5 | 3 |
| Security | 20 | 4 | 4 |
| Footprint | 10 | 3 | 5 |
| Docs and stability | 10 | 4 | 3 |
| Exit | 10 | 4 | 4 |
Weighted total is the sum of weight times score, divided by 5. Here A is 82 and B is 80. The numbers are made up to show the method; that gap is a tie in practice.
Step 4: read the result sensibly
- A tie goes to the option that is easier to replace.
- A large gap on one criterion deserves a conversation, not just arithmetic.
- If the winner scores low on security or maintenance, plan mitigation: a wrapper, an exit plan, extra tests.
The OpenSSF Scorecard documentation is a useful source of evidence, though its own text says low results are prompts to investigate, not verdicts. See also measuring project health.
Avoid these traps
- Adjusting weights after seeing the results.
- Scoring a library you know well higher out of familiarity.
- Comparing a mature library with a brand-new one on maintenance without noting age. Scorecard's maintenance check will not assess projects under 90 days old.
Where Safeguard fits
Safeguard's SBOM and dependency scanning, policies and continuous monitoring provide ongoing evidence after the decision. Try the free scan at gold.safeguard.sh/scan.
Quick check
How many criteria should a matrix have?
Five to eight. More than that dilutes the weights and slows the decision.
Who should set the weights?
The team that will own the library, with security input for risky domains. Set them before evaluating.
Sources
Last reviewed: 19 October 2026.
