How do you evaluate an SBOM management tool in a trial?

Evaluate an SBOM management tool by feeding it a repository you already understand, then scoring the output. Check that it finds the components you know are there, includes the NTIA minimum data fields, exports SPDX and CycloneDX, regenerates on every build, and helps you act on findings rather than just storing files.

What should you decide before the trial?

Write down the job. Is the SBOM for a customer who asked for one, for your own vulnerability response, for license review, or for a regulator? The answer changes which tests matter most. A tool that is excellent at storing files may be weak at telling you what to fix.

Pick two repositories: one simple, and one messy, with a monorepo, a container build or several languages. Choose ones where you can list a dozen dependencies from memory, including a few transitive ones.

What does an SBOM need to contain?

The NTIA's minimum elements report lists seven baseline data fields: supplier, component name, version of the component, other unique identifiers, dependency relationship, author of SBOM data and timestamp. It also lists practices and processes: frequency, depth, known unknowns, distribution and delivery, access control and accommodation of mistakes.

That list is a good scoring sheet. Ask the tool to produce an SBOM for your test repo and check each field. Missing identifiers or flat lists with no dependency relationships are common weaknesses.

The trial scorecard

Score each line from 0 to 3.

1. Coverage. Does it find the dozen components you know about? Does it find transitive ones? Does it handle every language and package manager in the repo?
2. Depth. NTIA says an SBOM should contain primary components with their transitive dependencies. Check how deep the tree goes.
3. Known unknowns. When the tool cannot resolve something, does it say so or silently leave it out? An honest "unknown" is better than a confident gap.
4. Fields. Are all seven baseline fields present, with usable identifiers such as package URLs?
5. Formats. Can it export both SPDX and CycloneDX? NTIA recognizes both, along with SWID tags. Validate the files with an independent parser, not only the vendor's viewer.
6. Frequency. NTIA says a new build or release should produce a new SBOM. Can the tool generate on every build in CI and keep versions side by side?
7. Accuracy of findings. For three known vulnerable packages, does it match them correctly? For three safe ones, does it stay quiet?
8. Noise reduction. Does it help you decide what matters, for example with reachability analysis or exploit signals, or does it hand you a flat list?
9. Sharing and access. Can you control who sees an SBOM and send it to a customer in a standard format?
10. Fixing. Does it show a fixed version, and help you open the change?
11. Exit. Can you export everything, including history, in open formats?

How should the two weeks run?

1. Day 1: connect the two repos and record setup time.
2. Day 2: generate SBOMs and compare against your known list.
3. Day 3 to 4: validate exports in a second tool.
4. Week 1: wire into CI and run on every pull request. Note build time added.
5. Week 2: introduce a known vulnerable dependency and measure the time from commit to alert.
6. End of week 2: have an engineer and a compliance person each answer a real customer question using the tool.
7. Tally scores and write the limits you found.

What are honest limits to expect?

SBOM generators work from manifests, lockfiles and sometimes binaries. They do not see everything, and different tools produce different component counts for the same repo. Do not treat the count as a quality score. Treat disagreement between tools as a prompt to look, because it often reveals a build step the first tool missed.

Safeguard generates SBOMs in the CLI and web app and supports reachability analysis, and the CLI runs a local runner mode where scanning happens on your machine. If you trial it, run the same scorecard. A vendor that welcomes a scorecard is a good sign.

Quick check

Sources

• NTIA: The Minimum Elements for a Software Bill of Materials
• CycloneDX specification overview
• SPDX specifications

Last reviewed: 18 October 2026.