On March 12, 2024, GitGuardian published its annual State of Secrets Sprawl report with a number that should stop any engineering leader mid-scroll: 12.8 million new secrets were exposed on public GitHub in 2023 alone, a 28% increase over the prior year, found across 1.1 billion scanned commits — meaning roughly 8 million commits, or about 7 out of every 1,000, leaked at least one live credential. GitGuardian also reported that more than 90% of those secrets were still valid five days after exposure, which means the gap between "a developer commits an API key" and "someone revokes it" is where almost every one of these incidents actually turns into a breach. GitHub's own response reflects how seriously the industry takes this: push protection, which blocks known-pattern secrets before they ever leave a developer's machine, reached general availability for GitHub Advanced Security customers on May 9, 2023, and was made free for all public repositories the same day. This post walks through how secrets get into repos in the first place, the three detection techniques (pattern matching, entropy analysis, and git history scanning) that catch them, and the remediation workflow that turns a finding into a closed incident instead of an open one.
How do hardcoded secrets actually end up in a repo?
Almost always through convenience, not carelessness in isolation. A developer wiring up a new integration drops an API key directly into a config file or a .env -like constant to get something working locally, intending to move it to an environment variable or a vault before committing — and then commits anyway, often under deadline pressure or because the change felt too small to warrant the extra step. From there the common vectors multiply: credentials copy-pasted from vendor "quickstart" example code, tokens baked into CI/CD pipeline YAML so a build can authenticate, API keys embedded in Jupyter notebook cells that get checked in wholesale, secrets baked into Docker image layers during a RUN step, and credentials that leak through build logs when a script echoes an environment variable for debugging. A private repo forked or later made public is a particularly sharp version of this problem — a secret that was "safe" behind private-repo access control is suddenly public the moment visibility changes, and nothing about the commit itself changed at all.
Why doesn't deleting the secret from the file actually fix it?
Because Git is a content-addressable history, not a live document — removing a line from the current version of a file does not remove the blob that contained it from any earlier commit. A git rm or an edit that deletes the secret only changes what HEAD points to; the original commit, and the object it references, still exists in the repository's .git directory and is retrievable with git log -p, git show <commit>, or by walking the reflog, indefinitely, unless someone deliberately rewrites history. This is precisely why GitGuardian's methodology scans the full commit history of a repository rather than just its current state, and why dedicated tools like TruffleHog and Gitleaks exist specifically to walk every commit a repository has ever had rather than trusting what's visible in the latest checkout. A secret that was committed once and "fixed" in the next commit is not fixed from a security standpoint — it is exposed forever unless it is revoked at the issuer and purged from history, and until that happens, anyone who clones the repo (or already has) can extract it.
What does pattern and entropy-based detection actually look for?
Detection tools generally run two complementary layers. The first is issuer-specific pattern matching: a curated set of regular expressions and structural signatures for known credential formats, such as AWS access keys starting with AKIA , GitHub personal access tokens with their ghp_ prefix, Stripe live keys, Slack bot tokens, and JWTs, private keys, and database connection strings that follow generic, recognizable shapes. These patterns are precise and produce few false positives, but they can only catch what someone has already cataloged. The second layer is entropy analysis: a Shannon-entropy calculation flags any string with unusually high randomness — the statistical signature of a generated key or token — even when it matches no known vendor format, which is essential for catching custom internal tokens and homegrown auth schemes. The tradeoff is noise: hashes, UUIDs, and base64-encoded non-secrets also score as high-entropy, so a usable entropy scanner needs a classification step layered on top to keep the false-positive rate from burying real findings.
How do you tell a finding from a real, exploitable secret?
By verifying it against the service that issued it, not by trusting the pattern match alone. A regex hit or high-entropy string only tells you something looks like a credential; calling a low-privilege endpoint on the issuing service tells you whether it is one. In practice this means making an authenticated call like AWS's sts:GetCallerIdentity , GitHub's /user endpoint, or Stripe's read-only /v1/balance using the discovered value, and checking whether the issuer accepts it. Safeguard's secrets-scanning documentation describes exactly this layer: verified findings against issuers such as AWS, GitHub, Stripe, Slack, and OpenAI are marked as live and prioritized as Critical or High depending on the credential's privilege level, while unverified pattern matches are still surfaced but ranked lower, since a huge share of what a naive scanner flags turns out to be expired keys, sandbox credentials, or values that were rotated the same day they were committed.
What does a real remediation workflow look like once a secret is found?
A workable remediation workflow runs in a fixed order because doing the steps out of sequence either leaves the exposure live or makes cleanup harder. First, revoke the credential directly at the issuer — this is the only step that immediately stops an attacker from using it, and it should happen before anything else, including before you've finished investigating scope. Second, rotate the credential through a secrets manager such as HashiCorp Vault, AWS Secrets Manager, GCP Secret Manager, or Azure Key Vault, so the replacement is generated and distributed through a system built for it rather than pasted into Slack. Third, purge the secret from Git history using git-filter-repo or BFG Repo-Cleaner — a step that requires a force-push and coordinated cleanup across every clone, which is why teams often treat it as optional compared to revocation, but it matters because the exposed value remains discoverable in history even after rotation. Fourth, notify the owning team and any downstream service that depended on the credential so nothing breaks silently when it stops working.
How do you stop the same secret from being committed again?
By pushing detection as early in the workflow as possible and gating merges on it, rather than relying on a periodic scan to catch what already shipped. A pre-commit or pre-push git hook that runs the same pattern and entropy checks locally catches a secret before it ever reaches a shared branch — Safeguard's own tooling installs this with a single safeguard install-hook --hook pre-push command, scanning new commits with zero added latency when nothing is flagged. Backing that up with a CI policy gate — for example, blocking any pull request where a verified secret is present — closes the gap for contributors who haven't installed the hook or who bypass it. Allowlisting matters here too: documentation and test fixtures legitimately contain example keys that look real, and a workable program needs inline, path-based, and issuer-based allowlist scopes (tracked in an audit log) so real findings don't get lost in a wall of accepted false positives.
How Safeguard helps
Safeguard's secrets scanning runs all three detection layers — issuer-specific patterns for 200+ providers, generic patterns for keys and tokens, and entropy-based detection — across source code, Git history, container image layers, packaged artifacts, build logs, and vendor SBOMs, and verifies every issuer-pattern finding live against the provider before raising an alert, so a Critical severity means the credential is confirmed live and exploitable right now. When a secret is found, Safeguard runs the same revoke-rotate-purge-notify playbook described above, with each step gated so a team can choose to run revocation headlessly or require approval, and it integrates that playbook with your policy engine so a rule like blocking any commit with a verified secret becomes a one-line YAML condition rather than a manual review step. The Secrets dashboard tracks time-to-revoke and a Git history heatmap of which branches and authors carry the most historical leaks, giving a security team the same before-and-after visibility GitGuardian's industry report gives the whole ecosystem — but scoped to their own repositories.