Architecture

Safeguard Gold Build Pipeline: How It Works

A walkthrough of the Gold Build pipeline that produces reproducible, attested, policy-verified container images and binaries for Safeguard customers.

Shadab Khan
Security Engineer
7 min read

A Gold Build in Safeguard is an artifact that has been produced in a controlled pipeline, is bit-for-bit reproducible, carries in-toto and SLSA attestations, and has been policy-cleared for a specific tenant and environment. We ship Gold Builds for the base images, language runtimes, and third-party dependencies that our customers need to use but cannot afford to trust blindly. This post explains the pipeline architecture, the design choices we made, and the places where we deliberately chose operational simplicity over theoretical purity.

What problem does the Gold Build pipeline actually solve?

Most enterprise software teams do not build their base images from source. They pull from Docker Hub, GitHub Container Registry, or a language-specific registry like PyPI. These images are produced under conditions the consumer does not control, often with limited transparency into what went in. The result is a software supply chain whose root is effectively a shrug emoji. Gold Builds replace those roots with artifacts built inside a pipeline the consumer can audit end-to-end.

Secondarily, Gold Builds solve the policy-at-the-root problem. A tenant might have a policy that disallows GPL-3 components, mandates FIPS-validated cryptographic providers, or requires every binary to be signed by a specific key chain. Applying those policies only at deployment time is too late — an image is already in the registry and engineering may already be using it. Applying them at build time, before any downstream process can see the artifact, keeps the footprint clean.

What does the pipeline look like?

The pipeline has six stages: source ingest, build isolation, reproducibility check, attestation, policy gate, and publish. Each stage produces artifacts that downstream stages consume, and every intermediate artifact is content-addressed.

 (Source Ingest) ──▶ (Build Isolation) ──▶ (Reproducibility Check)
       │                    │                         │
       │                    │                         │
   pinned sources     hermetic builder          two independent
   signed by maintainer  no network            builds must match
       │                    │                         │
       ▼                    ▼                         ▼
 (Attestation)  ──▶  (Policy Gate)  ──▶  (Publish)
       │                    │                     │
  in-toto + SLSA      tenant policy       OCI registry
   provenance          evaluated            + Rekor log

Every stage is a separate container that runs in an isolated namespace, and stages communicate through a signed manifest that names the inputs and outputs. This means any stage can be independently audited, replaced, or temporarily extended for a specific build without rewriting the whole pipeline.

How do you achieve reproducibility?

Reproducibility is the foundation of the whole system. Without it, the other attestations are just claims. We achieve it through three disciplines. First, pinned everything — every source commit, every toolchain version, every base image digest, and every build-time dependency is pinned to a content hash. There are no floating tags in the inputs. Second, hermetic builds — the builder runs with no network access. All fetches happen in the source ingest stage, after which the builder sees a sealed filesystem. Third, timestamp and ordering controlSOURCE_DATE_EPOCH is pinned, file ordering in archives is sorted, and any build tool that introduces non-determinism (for example, Go's build ID by default) is patched or configured to produce deterministic output.

To verify reproducibility, we run every build twice, on two independent builder nodes, using two different base container images built from the same sources. If the output digests do not match, the build is rejected and we file a reproducibility-debug ticket automatically. We currently ship around 98 percent of our Gold Builds on the first attempt, with the remainder requiring manual intervention — usually a build tool that introduces non-determinism we have not seen before.

What goes into the attestation bundle?

Every Gold Build ships with an attestation bundle that contains in-toto link metadata for each pipeline step, a SLSA v1 provenance predicate, a CycloneDX SBOM, and a signed manifest binding them all together. The bundle is signed by a keyless signing flow using Sigstore and the signature is published to Rekor so anyone can verify a given artifact was produced by our pipeline.

A simplified slice of the SLSA predicate looks like this:

{
  "_type": "https://in-toto.io/Statement/v1",
  "subject": [{
    "name": "ghcr.io/safeguard/python-3.12-gold",
    "digest": {
      "sha256": "a1b2c3..."
    }
  }],
  "predicateType": "https://slsa.dev/provenance/v1",
  "predicate": {
    "buildDefinition": {
      "buildType": "https://build.safeguard.sh/gold/v1",
      "externalParameters": {
        "source": {
          "uri": "git+https://github.com/python/cpython@v3.12.1",
          "digest": { "gitCommit": "a8a4..." }
        }
      },
      "resolvedDependencies": [
        { "uri": "pkg:oci/alpine@sha256:...", "digest": { "sha256": "..." } }
      ]
    },
    "runDetails": {
      "builder": {
        "id": "https://build.safeguard.sh/builders/hermetic-v3"
      },
      "metadata": {
        "invocationId": "sgb-2026-01-27-14a89",
        "startedOn": "2026-01-27T09:12:04Z",
        "finishedOn": "2026-01-27T09:47:22Z"
      },
      "byproducts": [
        { "name": "reproducibility.diff", "digest": { "sha256": "0000..." } }
      ]
    }
  }
}

Downstream consumers can verify this end-to-end with cosign verify-attestation and a policy document. We ship a reference cosign policy for every published image, and customers can layer their own on top.

How does the policy gate decide what ships?

The policy gate is the same evaluation engine that runs in our admission controller and in per-finding triage — we deliberately reuse one engine so decisions are consistent. It takes the attestation bundle, the generated SBOM, and a tenant-specific policy file, and emits either an allow verdict or a structured reject with the failing rule set.

Common rules customers enable at this stage include: no component with an unresolved KEV entry, no component below a minimum maintenance score, cryptographic providers must be FIPS-validated, license must be on the approved list, and all in-toto steps must be signed by known key identities. Policies are written in Rego on top of our knowledge graph projections, and the gate caches the graph slice needed for a build so evaluation is fast.

If a policy fails, the build is not published. Instead the pipeline produces a Gold Build candidate record with the rejection reason and notifies the maintenance team. A maintainer can remediate (patch the source, pin a different dependency, update a license exception) and re-run. Nothing reaches the publish stage without a clean gate.

How do Gold Builds get updated when a CVE drops?

This is the operational question that matters most to customers. When a new CVE lands against a component inside a Gold Build, our advisory ingester flags every Gold Build that transitively contains the affected version. For each flagged build, we run an automated remediation attempt: find a fix version, update the pin, rebuild, verify reproducibility, regenerate attestations, re-run the policy gate. For around 70 percent of advisories, the automated path completes in under an hour.

For the remaining 30 percent, a human maintainer picks it up. The reasons vary — sometimes the fix is backported and we need to apply a patch rather than bump a version; sometimes a fix introduces an API break and downstream Gold Builds need updates too; sometimes there is no fix yet and we document mitigation guidance. We publish SLAs per severity class and track them internally. For critical KEV entries the target is four hours from advisory publication to new Gold Build availability.

Customers consuming Gold Builds can opt into either an auto-update channel that tracks the latest passing build or a pinned channel where they take explicit updates. Most production workloads use the pinned channel with a policy that forces updates after a configurable grace period.

How Safeguard.sh Helps

The Gold Build pipeline gives Safeguard customers a trustworthy root for their software supply chains — hermetic, reproducible, attested, and policy-verified before any downstream system ever sees the artifact. By applying policy at build time rather than at deploy time, we prevent bad artifacts from entering the registry rather than chasing them down after the fact. The pipeline is operationally simple because every stage is a separately-auditable unit with signed inputs and outputs. If your organization is trying to build its own secure build platform, Safeguard's Gold Build program lets you consume the output of ours directly or run the pipeline on your own infrastructure with our tooling.

Never miss an update

Weekly insights on software supply chain security, delivered to your inbox.