Safeguard
Developer Tools•

Will running pods survive a container registry outage?

Running containers keep running if the registry goes down, but restarts and new nodes may not. See how image pull policy and digests change what survives.

Safeguard Security Team
7 min read

Running containers keep running if their registry becomes unreachable, because the image is already on the node. What breaks is anything that needs a new pull: a rescheduled pod, a new node or a scale up. Pull policy, pinned digests and a nearby cache decide how far the problem spreads.

What the Kubernetes docs say

The Kubernetes documentation on images explains that by default, when a pull policy is not set, the policy is IfNotPresent, which makes the kubelet skip a pull if the image already exists on the node. The policy Always means the kubelet asks the container runtime to pull on every container launch. The runtime contacts the registry, resolves the tag to a digest, and downloads any layers not cached locally. If all layers are present, the cached image is used. The policy Never means the kubelet does not try to fetch the image and starts the container if it is already present.

The documentation adds that the caching behavior makes even Always efficient, as long as the registry is reliably accessible. Read that last clause closely. It describes a dependency: with Always, a launch needs the registry to resolve the tag, even if it will not download layers.

Which situations need the registry

SituationNeeds a pull?
Pods already running, nothing changesNo
A pod restarts on the same node, policy IfNotPresentNo, image is cached
A pod restarts, policy AlwaysYes, to resolve the tag
A pod moves to a node that lacks the imageYes
A new node joins during a scale upYes
A new release is rolled outYes

This is why a registry problem often looks calm at first and then grows as nodes churn.

Tags versus digests

The documentation says tags can be moved to point to different images, while digests are hashes of the image content and are immutable. If you reference an image by digest, the runtime uses that digest when pulling, and the same content is what you tested and scanned. It also advises avoiding the latest tag in production, because it is harder to track which version is running.

For continuity, digests help in two ways. A cache or mirror can serve the same content without ambiguity, and a node that already holds the digest can start it without checking anything.

Making it more resilient

  1. Pin production images by digest, and keep the human readable tag in a comment or label.
  2. Choose pull policy on purpose. IfNotPresent reduces registry calls on restart, at the cost of relying on pinned digests so a stale tag is not reused.
  3. Use a pull-through cache or mirror close to your cluster for public base images. Our post on a pull-through cache as a security control covers trade-offs.
  4. Pre-pull critical images onto nodes, or bake them into node images, so a new node can start key workloads without the registry.
  5. Keep an emergency copy of the last known good images for your most important services in a second registry, and test a deploy from it.
  6. Watch for image pull errors in your cluster events and alert on a rise.

Do not lose the scan

If you change where images come from, keep scanning them. A mirror or copy should be checked for known vulnerabilities and the check should reference the digest, so you know the scan result applies to the bytes you run.

A worked example: a node replacement during an outage

This example is hypothetical. A cluster runs a payment service with six pods on three nodes. The pods reference images by tag, with the default pull policy for a tagged image. At 14:00 the registry becomes unreachable.

EventNeeds a pull?Result
14:00 Registry goes downNoAll six pods keep running
14:20 A pod crashes and restarts on the same nodeNo, with IfNotPresentRestarts from the cached image
14:40 A node is drained for maintenanceYes, if the new node lacks the imagePod stays pending on that node
15:00 Traffic spike triggers a scale upYes on nodes without the imageNew pods cannot start there
15:30 Release pipeline tries to deployYesDeployment fails

The service keeps working for current load. The trouble grows as events require new pulls. The team's actions after the incident are practical: pre-pull the payment image on every node, hold a little headroom so a spike does not need new pods, and pause non-urgent maintenance during a registry problem.

Step by step: a registry resilience review

  1. List the images each critical workload runs, including sidecars and init containers. A missing sidecar image blocks the whole pod.
  2. For each image, record the registry, the tag or digest and the pull policy. Look for the mutable latest tag and replace it.
  3. Check which nodes have each image cached. A simple node level listing shows which would fail a new start.
  4. Choose how to keep critical images available: pre-pull through a daemon set, bake into node images, or mirror to a second registry inside your network.
  5. Add pull errors to your alerts, such as image pull back off events, so a problem is visible before a user reports it.
  6. Practice a node replacement while the registry is blocked in a test cluster.
  7. Document the emergency procedure, including how to point workloads at the second registry and who may do it.

Supply chain angle

Registry resilience and supply chain security meet at the digest. If you copy images into a second registry, copy by digest and verify that the digest matches. Scan the image at the digest you run, and keep the record. If you use signatures or attestations, ensure the copy keeps them or that verification still works against the second location. A mirror that silently serves different content defeats the purpose of pinning.

Base images deserve a note. Public images pulled at build time add a dependency on an external registry whenever you build. Mirroring base images into your own registry reduces that dependency and gives you a place to scan and approve them.

Keep proportion

A registry outage is an ordinary kind of dependency failure, and Kubernetes behavior in these cases is well documented. Plan for it with modest steps, and check them. You do not need to hold every image on every node, just those tied to your most important services.

Try it yourself

Open the container image scan demo and look at how a result ties to a specific image. Notice what information you would need to carry across to a second registry so that the scan still applies.

Click through it at your own pace, or open it full screen.

Quick check

Do running pods stop if the registry is down?

No. The image is already on the node. Trouble starts when something needs a new pull.

Is IfNotPresent always the safer choice for outages?

It avoids registry calls on restart, but a moved tag could leave old content on a node. Pin by digest so the choice is explicit.

Should I set the pull policy to Never?

Rarely. It prevents any registry call, but it also means a node missing the image cannot start the pod. Use it where you control exactly what is on each node, and test the failure.

Sources

Last reviewed: 19 October 2026.

Never miss an update

Weekly insights on software supply chain security, delivered to your inbox.

Self-healing security runs on Safeguard.

Your first fix PR is minutes away.

No sales call required, even your agent can complete the purchase over MCP.