Write your scorecard before you open any analyst report. List what you need, give each item a share of 100 points, and agree the weights with security, engineering and procurement. Then compare the report to your scorecard instead of letting it write one for you.
Why do it in this order
Once you have seen a chart, it is hard to un-see. Requirements start to resemble the capabilities the report praises. A scorecard written first keeps your priorities tied to your own incidents and constraints.
Forrester's public method describes analysts weighting criteria by importance to produce the graphic. That is a reasonable thing for an analyst to do for a broad audience. It is also exactly why you should do your own.
A 100-point method
- Pick six to ten criteria. More than ten dilutes everything.
- Spend 100 points across them. Force trade offs. If everything is 10, you have not decided anything.
- Define a 0 to 3 scale for each. Write what a 0 and a 3 look like in plain words, so two scorers agree.
- Have each stakeholder score weights alone, then average and discuss large gaps.
- Freeze the card before the first demo.
Criteria that often matter for supply chain tools
Pick from this list and add your own.
- Coverage of the package ecosystems you actually use.
- SBOM export in the format you need. CycloneDX and SPDX are the two common open standards; CycloneDX is published as ECMA-424 and SPDX is recognised under ISO/IEC 5962.
- Prioritization that uses more than severity scores, for example exploit signals. CISA says organizations should use its Known Exploited Vulnerabilities catalog as an input to prioritization, and FIRST's EPSS estimates the probability a CVE will be exploited in the next 30 days.
- Policy enforcement in pull requests and pipelines.
- Developer experience, measured by time to first useful result.
- Evidence and audit support.
- Deployment model and data handling.
- Integration with your ticketing and chat tools.
Handle the hard to score items
Some criteria resist numbers, such as vendor viability. Use a short written note and score only after reading it. For adoption risk, ask two developers to try each tool for an hour and write down where they got stuck.
Compare the card to the report
After you freeze the card, open the report. Where does it value what you value? Where does it ignore something you rated high? The second list is your set of extra questions to ask vendors.
Safeguard covers SBOM and dependency scanning, policy gates and reachability analysis, and a free scan at gold.safeguard.sh/scan is available to test against your own code.
Try it yourself
Open the demo and see how exploit signals reorder a findings list. Decide how much weight prioritization deserves on your scorecard.
Click through it at your own pace, or open it full screen.
Quick check
Should price be a criterion?
Often yes, as commercial terms, but handle it separately from the capability score so attractive terms cannot hide a poor fit.
Who should own the scorecard?
Security usually owns the content, with engineering and procurement as reviewers. Shared ownership improves adoption.
Sources
- Forrester Wave methodology
- CISA Known Exploited Vulnerabilities catalog
- FIRST EPSS
- CycloneDX specification overview
- SPDX about
Last reviewed: 19 October 2026.
