How do you evaluate a container scanning tool in a trial?

Evaluate a container scanner by running it on three images you already ship, then scoring layer coverage, false positives, pipeline gates, speed and fix guidance against a list you wrote beforehand. A demo image proves nothing. Your own base images, language packages and odd build habits do.

What does NIST say a container tool must do?

NIST SP 800-190 is the standing reference for container security. In its countermeasures for image vulnerabilities, it says traditional vulnerability management tools make assumptions about host durability and update patterns that do not fit containers, and that such tools are often unable to detect vulnerabilities inside containers, leading to a false sense of safety.

It lists what effective tools look like:

1. Integration with the whole image lifecycle, from the build, through registries, to runtime.
2. Visibility into vulnerabilities at all layers, including application frameworks and custom software, not only the base layer.
3. Policy-driven enforcement, with quality gates that stop images over a chosen threshold from progressing.

Use those three as the spine of your trial.

How should you set up the trial?

Pick three images. One should be a small, well-kept service. One should be old, with a stale base. One should include several language ecosystems, such as a Python app with a Node build step. Write down the expected findings for the old image before you scan, using what your team already knows.

Run every candidate on the same three images, on the same day, with the same policy. Save the raw output.

What should you score?

Coverage. Does the tool find packages at all layers, including language dependencies inside the image, not just operating system packages? Does it handle images without a package manager database? Compare against your expected list.

False positives. Count findings your team can show are wrong, such as a vulnerable function the image never runs. Ask whether the tool offers a way to record an exploitability statement. CISA's VEX minimum requirements define statuses like not affected, with justifications such as vulnerable code not in the execute path, which is the vocabulary a good tool should be able to import or export.

Pipeline fit. Can it run in your CI, return a pass or fail exit code, and enforce a threshold? Can it also scan from a registry and at admission, as NIST suggests? Measure scan time on your biggest image.

Beyond vulnerabilities. NIST also names configuration defects, embedded secrets, embedded malware and untrusted images as image risks. Check whether the tool covers any of these, and be honest about which you need now.

Fix guidance. Does a finding say which base image or package version removes it? A finding with no next step is a ticket someone else must research.

Evidence and export. Can you export results in a standard format, and does it produce an SBOM? You will want this for audits.

Data handling. Where do images and metadata go? Some teams need scanning to run in their own environment.

What are the honest limits?

A scanner reports known vulnerabilities in what it can identify. It does not catch every flaw, and it cannot tell you how your configuration changes risk unless it is built to. Scores differ between tools because databases and matching rules differ. Do not read a gap between two tools as proof one is wrong; look at the specific findings.

Also, a trial is a snapshot. Ask how the vendor handles new advisories after you have deployed, and how fast.

Safeguard's CLI can scan an image reference with safeguard scan --image plus the image name, alongside directory, repository and SBOM inputs. It is one option among many categories, so test it against the same list as anything else.

What to do

1. Write your scoring list before you talk to any vendor.
2. Choose three of your own images, including one old one.
3. Record your expected findings first.
4. Run every candidate on the same images and policy.
5. Compare raw output, not dashboards.
6. Test the pipeline gate with a deliberately failing image.
7. Ask each vendor how a false positive gets recorded and shared.
8. Decide with the scorecard, then check pricing last.

Quick check

Sources

• NIST SP 800-190, Application Container Security Guide
• CISA: Minimum Requirements for Vulnerability Exploitability eXchange (VEX)
• Safeguard CLI docs

Last reviewed: 18 October 2026.