Multi-Stage Docker Builds: The Security Implications Nobody Talks About

Multi-stage Docker builds are universally recommended as a best practice. They reduce image size by separating build-time dependencies from runtime dependencies. A Go application's build image might be 800 MB, but the final production image can be under 20 MB. That is a massive reduction in attack surface.

But multi-stage builds are not a security silver bullet. They introduce their own class of problems that most tutorials gloss over.

How Multi-Stage Builds Work

A multi-stage Dockerfile defines multiple FROM statements. Each FROM starts a new build stage. You can copy artifacts from earlier stages into later ones using COPY --from=. Only the final stage becomes the output image.

FROM golang:1.22 AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o server ./cmd/server

FROM alpine:3.19
RUN apk add --no-cache ca-certificates
COPY --from=builder /app/server /usr/local/bin/server
CMD ["server"]

The builder stage has the full Go toolchain, source code, and all dependencies. The final stage has only Alpine, CA certificates, and the compiled binary. An attacker who compromises the running container cannot access the Go source code or build tools.

The Build Secret Problem

Build stages often need secrets: private SSH keys for git clones, API tokens for private package registries, credentials for downloading proprietary dependencies. The standard approach of using environment variables or COPY commands bakes these secrets into intermediate layers.

Even though intermediate stage layers are not included in the final image, they still exist in the build cache. Anyone with access to the build machine or the Docker layer cache can extract them.

The Right Way: BuildKit Secrets

Docker BuildKit introduced --mount=type=secret, which makes secrets available during the build without persisting them in any layer:

FROM golang:1.22 AS builder
RUN --mount=type=secret,id=github_token \
    GITHUB_TOKEN=$(cat /run/secrets/github_token) \
    go mod download

The secret is available during the RUN command but is never written to the image filesystem or cached in a layer. This is the only secure way to use secrets during Docker builds.

Common Mistakes

ARG for secrets: Build arguments are visible in image history and layer metadata. docker history reveals every ARG value used during the build.

COPY then delete: Copying a credentials file, using it, then removing it still persists the file in the layer where it was copied. Docker layers are additive; deletion in a later layer does not remove the file from earlier layers.

Multi-stage as a security boundary: Assuming that secrets in a builder stage cannot leak to the final image is mostly correct, but if you accidentally COPY --from=builder / or copy a directory that contains cached credentials, those secrets end up in production.

Layer Caching Risks

Docker caches each layer based on the instruction and the files it depends on. This caching is essential for build performance but can create security issues.

Stale Dependency Caches

A common pattern separates dependency installation from source code copying to maximize cache hits:

COPY go.mod go.sum ./
RUN go mod download
COPY . .

If go.mod and go.sum have not changed, Docker reuses the cached dependency layer. This means you might be building against dependencies that were downloaded weeks ago, potentially with known vulnerabilities that have since been patched.

The fix is to periodically invalidate the cache or use --no-cache for security-critical builds. Some teams run nightly builds with --no-cache to ensure fresh dependencies, while development builds use caching for speed.

Cache Poisoning

If an attacker can modify the Docker build cache, they can inject malicious layers that will be used in subsequent builds. This is a concern in shared CI/CD environments where multiple teams or projects share a Docker daemon or build cache.

Use BuildKit's cache isolation features and avoid sharing Docker sockets or build caches across trust boundaries.

What Gets Copied Between Stages

The COPY --from= instruction is the bridge between stages. It determines what crosses from the build environment into the production image. This is where careful attention pays off.

Copy Files, Not Directories

Instead of copying an entire directory from the builder stage, copy only the specific files you need:

# Risky - copies everything in /app
COPY --from=builder /app/ /app/

# Precise - copies only the binary
COPY --from=builder /app/server /usr/local/bin/server

Broad directory copies can inadvertently include configuration files, cached credentials, source code, or test data that should not be in production.

Verify What You Copied

After building your image, inspect its contents. Run the image and list the filesystem, or use docker export to examine the layers. Verify that only intended files made it into the final image.

Supply Chain Implications

Build Image Provenance

Your builder stage image is part of your supply chain. If golang:1.22 is compromised, your build is compromised, even if the final image is based on Alpine. The build stage has access to your source code and produces the binary that runs in production.

Pin builder images to specific digests, not floating tags. Verify the provenance of your builder images with the same rigor you apply to your runtime base images.

Dependency Downloads During Build

When the builder stage runs go mod download, pip install, or npm install, it fetches dependencies from public registries. These downloads are subject to the same supply chain risks as any dependency installation: typosquatting, dependency confusion, compromised packages.

Use checksums, lock files, and private registry mirrors to verify dependencies during the build. The builder stage is where supply chain attacks enter your images.

Build Reproducibility

Multi-stage builds should be reproducible. Given the same source code and the same pinned dependencies, you should get the same binary output. Non-reproducible builds make it impossible to verify that a production binary matches the reviewed source code.

Pin everything: base images to digests, dependencies to checksums, build tools to specific versions. Use --build-arg BUILDKIT_INLINE_CACHE=1 to enable cache metadata that helps with reproducibility verification.

Best Practices for Secure Multi-Stage Builds

1. Use BuildKit secrets for any credentials needed during the build
2. Pin all base images to SHA256 digests, not tags
3. Copy specific files between stages, not entire directories
4. Invalidate caches periodically to pick up security updates
5. Scan both stages: scan the builder image for supply chain risks and the final image for runtime vulnerabilities
6. Run as non-root in the final stage
7. Minimize the final stage to only what the application needs at runtime

How Safeguard.sh Helps

Safeguard.sh analyzes the complete container build pipeline, including multi-stage builds. It scans both builder and runtime images for vulnerabilities, verifies that build dependencies are free of known supply chain compromises, and generates SBOMs for the final production image. When a vulnerability is discovered in a build tool or dependency, Safeguard.sh identifies which images were built with the affected component and need to be rebuilt.