In September 2022, Toyota disclosed that a subcontractor had accidentally published part of its source code to a public GitHub repository — with a hardcoded access key left inside. The key sat exposed for nearly five years, from December 2017 until the leak was caught, and gave anyone who found it access to a database containing email addresses and customer management numbers for roughly 296,000 people. No exotic exploit was involved. A credential was typed into a config file, committed, pushed, and never rotated. This is the pattern behind most secrets-related breaches: not a zero-day, but a string of characters sitting in plaintext where a scanner should have caught it and didn't. Most teams already run some form of secrets detection tool in CI, often as one module inside broader source code scanners that also check for vulnerable dependencies and misconfigurations. Yet leaks like this keep happening, because the dominant approach — regex pattern matching against known credential formats — was never built to catch everything a real codebase can hide.
What Makes Regex-Based Scanners Miss Real Secrets?
Regex scanners miss secrets because they can only recognize what someone already anticipated the shape of. A rule written to match AKIA[0-9A-Z]{16} will reliably catch a well-formed AWS access key ID, but it says nothing about a database password assigned to a variable called dbPass, a bearer token pasted into a YAML block with unusual indentation, or an internal API key your platform team invented last quarter that follows no public vendor format at all. Industry research from secrets-scanning vendors has repeatedly found that generic, non-vendor-specific secrets — internally generated tokens, custom auth strings, service-to-service credentials — make up close to half of all secrets found in public repositories, and these are exactly the category that fixed-pattern rule sets are weakest against. Regex is precise but brittle: it trades recall for specificity, and every credential format that doesn't exist in the ruleset yet is invisible by definition.
This is compounded by how secrets actually appear in real source trees. They show up split across concatenated string literals ("AKIA" + "IOSFODNN7"), base64- or hex-encoded inside config blobs, embedded in Jupyter notebook cell outputs, buried in Terraform state files, or reconstructed at runtime from environment variable fragments. A line-by-line regex pass over a file's current text simply does not see any of this. It sees the literal bytes on disk, not the value the program will actually use in production.
How Many Secrets Are Actually Leaking Into Public Repositories?
The honest answer is: millions per year, and the number keeps climbing. Public monitoring of GitHub found over 12.8 million secrets exposed in public commits in 2023 alone, a 28% year-over-year increase, with subsequent annual tallies climbing past 23 million. Those numbers only cover secrets that were public and got caught — private repositories, internal GitLab instances, and self-hosted Bitbucket servers aren't in that count at all, and neither are the secrets that regex-based tools simply didn't flag. Independent studies of leaked credentials have found that the large majority remain live and unrevoked weeks after being reported, meaning most organizations don't have a fast path from "detected" to "revoked." A secrets detection tool that only fires an alert but never confirms whether the credential is still valid is solving half the problem at best.
Why Do High-Entropy Checks Create So Much Alert Fatigue?
High-entropy checks create fatigue because randomness alone doesn't mean "secret" — it just means "looks random," and plenty of harmless strings look random. Git commit hashes, UUIDs, minified JavaScript bundle identifiers, base64-encoded images, and cryptographic test fixtures all score high on Shannon entropy calculations, the same math most scanners use to flag "this string looks like a credential." Security teams running early-generation scanners across large monorepos routinely report false-positive rates well above 50% on entropy-only rules, to the point where developers start reflexively dismissing scanner output in pull requests. Once that habit forms, the one true positive that shows up in week 12 gets dismissed along with the noise. A detection tool that can't distinguish a live AWS secret access key from a randomly-generated test UUID isn't just wasting review time — it's training your engineers to stop looking.
Can Secrets Hidden in Git History Survive After Being "Removed"?
Yes — deleting a secret from the current file version does not remove it from the repository, because git preserves every prior commit by design. A developer who accidentally commits a .env file, notices the mistake, and deletes it in the next commit has not fixed anything from a security standpoint: the credential is still fully retrievable with git log -p or git show, and it will remain there indefinitely unless someone performs a history rewrite (git filter-repo or the BFG Repo-Cleaner) and force-pushes, coordinating that across every clone and fork. Many regex-based secrets detection tools only scan the current HEAD of a branch during a CI run, which means a secret introduced and "deleted" between two commits on the same pull request, or one buried thousands of commits deep in a repo's history, never gets flagged at all. This is precisely the mechanism behind the Toyota incident and dozens of similar disclosures: the exposure window is measured not from when the secret was noticed, but from when it was first committed, and history-blind scanning cannot close that gap.
Where Do Platforms Like Endor Labs Still Fall Short on Secrets?
Endor Labs and similar software composition analysis platforms still fall short because secrets detection is layered on as an adjunct to their core dependency and reachability analysis, not built as a first-class detection engine. Endor Labs has done genuinely useful work reducing noise in vulnerability management through call-graph-based reachability analysis, but that same reachability lens doesn't extend to secrets scanning, which fundamentally needs pattern coverage breadth, entropy tuning, and live-credential validation rather than dependency-graph context. In practice this shows up as SCA-centric platforms treating secrets as one more finding type in a broader risk feed: useful for a checklist, less useful when the question is "is this specific AWS key still active, and which of our 40 repositories also contain it?" Coverage gaps tend to concentrate in exactly the areas regex struggles with everywhere — custom internal token formats, multi-line certificates and private keys with irregular line wrapping, and secrets sitting in commit history rather than the current branch tip. Teams evaluating any secrets detection tool, including Endor Labs, should ask directly how deep the historical scan goes, what percentage of findings get automatically validated against the live provider API, and how the tool handles credential formats that were invented in-house rather than published by a vendor. This is also a useful question to ask of source code scanners generally: bundling secrets detection into a broader SAST or SCA product doesn't guarantee the secrets module got the same investment as the rest of the platform.
How Safeguard Helps
Safeguard treats secrets detection as a core capability, not a bolt-on to dependency scanning. Detection combines pattern libraries covering hundreds of vendor-specific credential formats with contextual and structural analysis that catches split literals, encoded blobs, and irregular multi-line formats that single-line regex passes skip. Where entropy analysis is used, it's weighted against surrounding code context — variable names, file type, import statements, and known noise sources like test fixtures and generated IDs — to cut false positives instead of just flagging every high-randomness string in the tree.
Just as important, Safeguard scans full repository history by default, not just the current branch tip, so a credential committed three years ago and "deleted" the next day still surfaces in results. When a secret is found, Safeguard performs live validation against the relevant provider (confirming, for example, whether an AWS key, Slack token, or database connection string is actually active) so triage starts with real risk instead of a flat list of maybe-secrets. Findings are correlated across every connected repository, so a leaked internal API token that got copy-pasted into five different services is treated as one incident with five blast points, not five unrelated tickets.
The result is a secrets detection tool built for how secrets actually leak in practice: buried in history, disguised in custom formats, and often already live long before anyone notices. Regex is part of that picture, but only part — closing the gap Toyota, and thousands of smaller incidents like it, fell into requires validation, history depth, and context that pattern matching alone was never designed to provide.