Safeguard
Container Security

Scanning container images in CI/CD pipelines

Where to put container image scanning in your CI/CD pipeline, what it actually catches, and how to stop CVE floods from blocking every build.

Priya Mehta
DevSecOps Engineer
7 min read

In February 2024, researchers disclosed CVE-2024-3094, a backdoor planted inside the xz-utils compression library that ships in most Linux base images, including Debian and Fedora derivatives commonly used as container bases. Any team that had baked a vulnerable xz-utils version into an image weeks earlier and pushed it to production had no way to know without re-scanning. That is the core problem container image scanning in CI/CD solves: catching known vulnerabilities, exposed secrets, and misconfigurations in the exact artifact that will run in production, before it ships, not after an auditor or an incident responder finds it. A typical Node.js or Python service image today pulls in 400-800 OS and language-level packages between the base layer and application dependencies, and each one is a potential entry point. Scanning that image at build time turns an unknown, sprawling attack surface into a defined, ranked list of issues a developer can act on in minutes.

What Does Container Image Scanning Actually Check For?

Container image scanning checks three distinct things: known CVEs in OS packages and application dependencies, hardcoded secrets or credentials baked into layers, and misconfigurations in the Dockerfile or image metadata. Scanners unpack each image layer, build a manifest of installed packages (via APK, APT, RPM, or language-specific lockfiles like package-lock.json or requirements.txt), and match versions against vulnerability databases such as the NVD, OSV, and vendor-specific advisories like Red Hat's OVAL feeds or Alpine's secdb. A single python:3.11-slim base image pulled fresh from Docker Hub in mid-2024 commonly carried 20-40 known CVEs in OS-level packages alone, most of them low-severity and unexploitable in the specific runtime context, but a handful worth checking every time. Secret scanning separately greps layer history for patterns matching AWS access keys, private key headers (-----BEGIN RSA PRIVATE KEY-----), and API token formats, because a COPY .env . line three Dockerfile edits ago doesn't disappear just because a later RUN rm command deleted the file — Docker layers are additive, and the secret persists in the layer history even after deletion.

Where in the Pipeline Should Scanning Actually Happen?

Scanning should happen at three points: on every pull request build, immediately after the final image push to a registry, and on a recurring schedule against images already deployed. Scanning only at PR time misses vulnerabilities disclosed after the image was built and merged — CVE-2021-44228 (Log4Shell) was disclosed on December 9, 2021, and any image built and deployed on December 8 passed its build-time scan clean, then became critically exposed overnight without a single code change. That's why registry-triggered rescans matter: when a new CVE lands in the NVD feed, a scanner watching your registry can flag every existing image tag that contains the affected package within hours, not at the next rebuild. Teams running weekly base image refreshes plus PR-time scanning typically catch the fast-moving supply chain issues; teams that scan once at build and never again are running images that were "clean" the day they were built and increasingly stale afterward, sometimes for the 90-180 days a service goes without a rebuild.

Why Do Teams Get CVE Floods That Nobody Actually Fixes?

CVE floods happen because scanners report every match against a version string without checking whether the vulnerable code path is reachable or even loaded at runtime. A default Alpine or Debian base image scan routinely surfaces 50-150 findings, and when a security team mandates "fix all Highs and Criticals before merge," developers face a backlog that grows faster than they can triage it. The 2021 Log4Shell aftermath is the textbook case at industry scale: Log4j was present transitively in thousands of Java applications, but a large share of those instances never called the vulnerable JndiLookup class at all, meaning the CVE was present in the artifact but not exploitable in that specific application's execution path. Teams that triaged by CVSS score alone spent weeks patching builds that were never actually at risk, while teams that could confirm reachability of the vulnerable function narrowed remediation to the subset that mattered. Without that context, "critical" severity becomes a label attached to almost everything, and developers rationally start ignoring the scanner.

What's the Difference Between Scanning an Image and Generating an SBOM for It?

Scanning produces a point-in-time list of vulnerabilities; an SBOM produces a structured, machine-readable inventory of every component in the image, independent of what's vulnerable today. A CycloneDX or SPDX SBOM generated at build time lists exact package names, versions, licenses, and hashes for every layer — typically hundreds of entries for a moderately complex service image — and that inventory becomes searchable retroactively. This distinction mattered directly during the xz-utils backdoor in March 2024: organizations with SBOMs already generated for their images could query "which images contain xz-utils 5.6.0 or 5.6.1" in minutes across their entire fleet. Organizations without SBOMs had to re-pull and re-scan every image in every registry, a process that took some teams days. U.S. Executive Order 14028 and the resulting NTIA minimum elements guidance have pushed SBOM generation from a nice-to-have into a procurement requirement for federal software vendors since 2021, and scanning pipelines that generate an SBOM as a build artifact (rather than as a bolt-on report) satisfy that requirement without extra tooling.

Should a Failed Scan Block the Build, and on What Criteria?

A failed scan should block the build only when the finding is both above an agreed severity threshold and confirmed relevant to the deployed application, otherwise gating on raw CVE count trains developers to seek exceptions rather than fix issues. A common, workable policy is: block on Critical or High severity findings with a known, non-theoretical exploit (CISA's Known Exploited Vulnerabilities catalog, which listed over 1,200 CVEs as of mid-2024, is a good filter for "known exploited" rather than "theoretically scorable"), allow Medium and Low findings to log without blocking, and require a time-boxed exception (commonly 14-30 days) for anything allowed through. Teams that instead block on every Critical/High regardless of exploitability or reachability see PR merge times stretch and eventually see the security gate bypassed via admin override, which defeats the purpose of having a gate. The goal of a build-blocking policy is to stop the small number of issues that are actually dangerous, not to maximize the number of red X's in a CI dashboard.

How Safeguard Helps

Safeguard scans container images at every stage described above — PR, registry push, and scheduled rescan — and layers reachability analysis on top of raw CVE matches so a finding is flagged as blocking only when the vulnerable function is actually called from your application's code paths, cutting through the flood of theoretical matches that make scanners easy to ignore. Griffin AI, Safeguard's analysis engine, correlates each vulnerability with call-graph and dependency-tree context to explain why a given CVE matters in your specific image, not just that it matched a version string. Every scan also generates a CycloneDX SBOM automatically (and ingests existing SBOMs from upstream vendors), so when the next xz-utils-style event happens, answering "are we affected" is a query, not a fire drill. For findings that are confirmed exploitable, Safeguard opens an auto-fix pull request with the minimum version bump needed to resolve the issue, so remediation is a review-and-merge action rather than a manual dependency hunt.

Never miss an update

Weekly insights on software supply chain security, delivered to your inbox.