Container Security

OCI Artifact Signing Rollout Program

A program plan for getting OCI artifact signing across an organisation: trust roots, key custody, build integrations, registry policy, and the inevitable cleanup of unsigned legacy content.

Shadab Khan
Security Engineer
7 min read

OCI artifact signing has the unusual property of being conceptually simple and operationally invasive. Generating a signature is a one-line command. Getting every image, chart, and policy bundle in the organisation signed by the right identity, verified at the right gates, and audited on a schedule is a multi-quarter program. The conceptual simplicity tempts teams to underestimate the work. The work is real and worth doing well.

This post is the program plan we used. It covers the trust root design, the signer identities, the build pipeline integration, the registry policy, and the cleanup of unsigned legacy artifacts that always ends up larger than expected.

The Trust Root Question

The first decision is the trust root. Sigstore's keyless model anchors trust in OIDC identities and the public Fulcio CA. A self-managed model anchors trust in keys held in an internal HSM and a custom CA. Most organisations want a hybrid.

The model we settled on. Public artifacts intended for external consumption are signed keylessly with Sigstore, anchored in the public Rekor transparency log, with the OIDC identity tied to a build-system service account. Internal artifacts are signed with keys backed by an internal HSM, with a private transparency log that we operate. Both flows produce a signature attached as an OCI referrer to the artifact.

The choice between keyless and keyful is less about security and more about operational fit. Keyless removes long-lived secrets at the cost of dependency on the public Sigstore stack. Keyful removes the external dependency at the cost of running and rotating keys yourself. Pick based on which dependency your organisation tolerates better, not based on which model is fashionable.

Signer Identities

The signer identity is what verification ultimately matches against. Getting it wrong is the most common cause of rollout pain.

A signer identity should be a build system role, not a human and not a long-lived service account. The role is assumable only by a specific CI workflow, on a specific branch, in a specific repository. When the workflow runs, it gets an ephemeral OIDC token that proves it is that role, and the signing operation uses that token.

The implication is that signing cannot happen on a developer laptop. That sounds restrictive and turns out to be the right answer. Laptops are stolen, compromised, and lent. Signing keys held on laptops have a lifecycle measured in incidents.

For artifacts that genuinely cannot be produced in CI, we use a hardware token held by a designated release engineer, with the token requiring a second factor at signing time and the signing event logged to the transparency log with a release ticket reference. That flow is rare, awkward, and intentional. It should not be the common path.

Build Integration

Integrating signing into the build pipeline has three concrete parts. Generating the signature, attaching it as an OCI referrer to the artifact in the registry, and producing an in-toto attestation of the build steps that sits alongside the signature.

The generation part is well-handled by tools like cosign, notation, and the build-system-native signing actions. The attachment part has improved dramatically as registries have adopted the OCI 1.1 referrers API. The attestation part is where most teams underinvest.

A signature without a provenance attestation says only that an approved identity blessed the artifact. A signature with a provenance attestation says that an approved identity blessed an artifact that was built from a specific source at a specific commit using a specific build process. The second statement is what supply chain verification actually wants. Generate both at build time, attach both at push time, and the rest of the chain has something to verify.

Registry Policy

The registry is where signing policy gets enforced or quietly bypassed. Three policy clauses pay off.

Push of an unsigned artifact is rejected. The registry rejects the push if the artifact does not arrive with a signature in the same transaction or in a closely-coupled follow-up. This forces signing into the build pipeline rather than letting it become an afterthought.

Tag mutation is restricted. Once an artifact has been signed and tagged, the tag cannot be moved to point at a different digest without an explicit administrative action that gets logged. Tag mutation is one of the easier supply chain attacks; restricting it removes a class of confusion and abuse.

Old unsigned content is quarantined. Existing unsigned artifacts are tagged read-only and moved to a quarantine namespace. They remain pullable for a defined deprecation period, after which they are removed. This is the cleanup phase that always ends up larger than the original program plan.

The Legacy Cleanup

The cleanup phase deserves its own discussion. Every organisation has artifacts in its registries that predate the signing program. Some are in active use. Some are residual from cancelled projects. Some are being pulled by automation that no one remembers configuring.

The cleanup process we ran. First, an inventory. Every namespace, every repository, every tag, every digest, with last-pull timestamps from the registry access log. Second, a categorisation. In active use, in occasional use, dormant, and orphaned. Third, a remediation plan per category. In-active-use artifacts get rebuilt and re-signed. Occasional-use artifacts get a deadline, after which they are quarantined unless rebuilt. Dormant artifacts get quarantined immediately. Orphaned artifacts get deleted after a notification window with no claimants.

The hardest cases are not technical. They are organisational. An artifact pulled twice a year by a build system that no longer has a clear owner. An image referenced in a runbook for a service that was migrated away from. We kept a documented "no claimant after thirty days" rule and stuck to it. Three months in, the registry was meaningfully cleaner.

Verification Surfaces

Signing is half the program. Verification is the other half. A signature that is never checked is operationally equivalent to no signature.

The verification surfaces we run. Admission control on every cluster, blocking pods whose images do not have a verifiable signature. CI policy on every build that pulls a base image, refusing to proceed if the base is unsigned. Mirror sync verification, which checks signatures at sync time and quarantines anything that fails. And a periodic offline audit that walks every artifact in production registries and verifies its signature, catching any drift in the chain.

The audit catches things the live verifications miss. Signatures whose key has been revoked since signing time. Transparency log entries that have been tampered with. Artifacts whose attestations have been mismatched at the registry level. None of these come up often, and all of them matter when they do.

How Safeguard Helps

Safeguard runs the program rather than just the tooling. The trust root configuration, including the keyless OIDC settings and the keyful HSM settings, lives in a single policy store. Signer identities are managed alongside the build system integrations, with an explicit list of which build workflows are allowed to assume which identities. The registry integration enforces unsigned-push rejection, tag mutation restrictions, and quarantine tagging. The verification stack runs admission, CI gating, mirror sync verification, and the periodic offline audit, with a unified report that shows where the chain is solid and where it has drifted. The cleanup phase has a dedicated workflow that drives inventory, categorisation, and deadline tracking against legacy artifacts. The result is a signing rollout that gets to coverage, holds coverage, and keeps a verifiable audit trail across the whole chain.

Never miss an update

Weekly insights on software supply chain security, delivered to your inbox.