Safeguard
Container Security

Kubernetes Secrets

Kubernetes Secrets are base64, not encrypted, by default. Here is how they actually leak, why scanners like Aqua fall short, and how to fix it.

Karan Patel
Cloud Security Engineer
8 min read

A Kubernetes Secret sounds like a vault. In practice, it's often a base64 string sitting in etcd, mounted into a pod's filesystem, and readable by anyone with get permission on the namespace. Security teams evaluating tools like Aqua Security's Trivy or KSPM modules frequently discover that "we scan for secrets" and "we manage secrets" are very different claims. Aqua's platform is strong at flagging hardcoded credentials inside container images and IaC templates at scan time, but Kubernetes secrets sprawl across git history, CI/CD pipelines, registries, and live clusters — and a leaked API key in a 2022 commit does not care that your latest image passed a scan. This post breaks down how Kubernetes Secrets actually work, where they leak in the real world, why point-in-time scanning falls short, and what a complete kubernetes secrets management program looks like in 2026.

What Is Kubernetes Secrets Management, and Why Isn't Base64 Encryption?

Kubernetes Secrets management is the practice of creating, storing, distributing, rotating, and revoking sensitive values — API keys, database credentials, TLS certificates, service account tokens — used by workloads running in a cluster, and by default Kubernetes does almost none of this securely. When you create a kind: Secret object, Kubernetes stores the value as base64 in etcd, its backing key-value store. Base64 is an encoding, not encryption: anyone with kubectl get secret -o jsonpath access or read access to the etcd data store can decode it in one command. Unless a cluster operator explicitly configures an EncryptionConfiguration resource to enable encryption-at-rest (a manual step introduced as beta functionality years ago and still not the default), secrets sit in etcd in a form that is trivially reversible. Etcd snapshots, backups, and disaster-recovery copies inherit the same exposure, which means a single misconfigured backup bucket can leak every credential in a cluster's history.

How Do Kubernetes Secrets Actually Leak in Production?

Kubernetes secrets leak primarily through four paths: source code commits, CI/CD logs, container images, and overly broad RBAC — not through etcd being hacked directly. GitGuardian's 2024 State of Secrets Sprawl report found that 12.8 million new secrets were exposed on public GitHub commits in 2023, a 28% increase year over year, and internal enterprise repositories showed even higher density per developer. Kubernetes YAML manifests are a repeat offender: teams paste a Secret object into a values file "temporarily" for a demo cluster, commit it, and it stays in git history indefinitely — deleting the file later does not remove it from prior commits. CI/CD is the second major path. The January 2023 CircleCI incident, in which an engineer's laptop was compromised and used to exfiltrate customer environment variables and secrets, forced thousands of organizations to rotate every credential that had ever passed through the pipeline. On the RBAC side, the default Kubernetes get/list/watch verbs on the secrets resource are namespace-wide by design — a role intended to let a monitoring service read one ConfigMap frequently ends up scoped to "all secrets in this namespace," and Kubernetes has no native mechanism to scope access to a single named secret without a resourceNames field that most teams never set.

Why Isn't Aqua Security's Approach to Kubernetes Secrets Enough on Its Own?

Aqua Security's tooling is built primarily around image and workload scanning, so it catches secrets baked into container layers or IaC files but has limited visibility into the earlier and later stages of the secret's life. Trivy's secret-scanning module (added in 2021) does a genuinely good job of regex-matching known credential patterns inside a filesystem, image layer, or repository at the moment you run the scan. Aqua strengthened this further with its 2021 acquisition of Argon Security, adding software-supply-chain and CI/CD pipeline coverage. But the core model is still scan-triggered: a developer or a CI job has to invoke it, and it evaluates a snapshot. It does not continuously watch git history as new branches are rebased, it does not correlate a secret found in a Dockerfile with the same value still live inside a running cluster's etcd three months later, and it doesn't track whether a flagged secret was ever actually rotated. Aqua's own 2023 Cloud Native Threat Report noted that misconfigured secrets and overly permissive access remain among the top cloud-native attack vectors it observes — evidence that scanning coverage and remediation follow-through are two different problems. A kubernetes secrets management program needs both: detection at every stage, and closed-loop proof that a detected secret was rotated and invalidated, not just flagged in a dashboard.

What Does Good Kubernetes Secrets Management Look Like in Practice?

Good Kubernetes secrets management replaces native Secret objects with an external secrets store, enforces encryption at rest, and treats every credential as short-lived. Concretely, that means five things. First, enable etcd encryption at rest via EncryptionConfiguration with a KMS provider (AWS KMS, GCP Cloud KMS, or HashiCorp Vault's Transit engine) rather than the default identity provider, which performs no encryption at all. Second, stop storing long-lived secrets as native Secret objects and instead use an external secrets operator — the CNCF-adjacent External Secrets Operator or HashiCorp Vault's Kubernetes integration — so the source of truth lives outside etcd and can be centrally rotated and audited. Third, scope RBAC with resourceNames so a role grants access to one named secret, not every secret in a namespace, and run kubectl auth can-i --list audits quarterly. Fourth, shorten credential lifetimes: Kubernetes 1.24 (released May 2022) stopped auto-generating long-lived service account token Secrets and moved to time-bound, audience-scoped tokens via the TokenRequest API — a change many clusters still haven't fully adopted three years later because legacy workloads assume a mounted, non-expiring token. Fifth, use admission controllers (OPA Gatekeeper or Kyverno policies) to block pods that mount secrets as environment variables, since env vars are far more likely to leak into logs, crash dumps, and error-tracking tools than a mounted file.

How Do You Detect a Leaked Kubernetes Secret Before an Attacker Does?

You detect leaked Kubernetes secrets by scanning continuously across every stage a credential passes through — git history, CI logs, registries, and live clusters — and correlating findings, not by running a single scan at build time. A secret's lifecycle typically spans a developer's laptop, a feature branch, a pull request diff, a CI job's environment, a built image layer, a registry, and finally a running pod — six to eight handoff points, any one of which can leak the value. Point-in-time scanners miss the temporal dimension: a secret committed and then deleted three commits later is still permanently readable via git log -p, and a CVE like CVE-2021-25742 (an ingress-nginx configuration-snippet vulnerability disclosed in October 2021) showed how a workload-level misconfiguration could be used to read arbitrary files, including mounted secret volumes, well after the original image had already passed scanning. Effective detection means diffing every commit against a secrets-pattern and entropy-based detector before it merges, re-scanning CI logs (not just artifacts) for accidental echod credentials, and reconciling what's actually live in etcd and in running pod specs against what's been flagged — so a security team can answer "is this specific leaked key still active in production right now" in minutes, not after the next quarterly audit.

How Safeguard Helps

Safeguard was built around the idea that a leaked credential isn't a code-scanning problem or a runtime problem in isolation — it's a supply-chain problem that spans the whole path from a developer's keystroke to a running container. Rather than treating secrets detection as one more rule in an image scanner, Safeguard continuously monitors git commits, pull requests, CI/CD job logs, container registries, and live Kubernetes clusters as one connected graph, so a credential flagged in a Dockerfile is automatically correlated with the same value if it later shows up in an etcd snapshot, an env var, or a running pod spec. That closes the exact gap that point-in-time, scan-triggered tools like Trivy leave open: Safeguard doesn't just tell you a secret exists, it tells you where else that same secret is currently live, whether it's still active, and walks your team through rotation and revocation with verification — not a checkbox. Safeguard also ships policy templates that map directly to the practices above: enforcing etcd encryption-at-rest configuration, flagging Secret objects that should be external-secrets-managed, catching RBAC roles with unscoped secrets access, and blocking admission of pods that mount credentials as environment variables. For teams comparing tools, the practical difference is this: Aqua's strength is deep image and IaC scanning at the moment of build; Safeguard's is continuous, cross-lifecycle correlation that turns "we found a secret" into "we confirmed it's dead everywhere." For a control surface as leaky as Kubernetes secrets, that continuous, closed-loop model is what actually reduces breach risk — not another dashboard of unresolved findings.

Never miss an update

Weekly insights on software supply chain security, delivered to your inbox.