On March 24, 2023, GitHub published a blog post with a characteristically understated title: "We updated our RSA SSH host key." Behind that calm headline was an incident that affected every developer and CI/CD system that connects to GitHub over SSH — which is to say, most of the software industry.
GitHub had accidentally exposed its private RSA SSH host key in a public repository. The key that cryptographically proves "you are talking to GitHub, not an impersonator" was sitting in the open for anyone to copy.
What Happened
GitHub detected that its RSA SSH private key had been "briefly exposed in a public GitHub repository." They didn't specify which repository, how long the exposure lasted, or how the key ended up there. What they did say was clear: they rotated the key immediately.
The rotation happened at approximately 05:00 UTC on March 24, 2023. At that moment, every SSH connection to GitHub that relied on the RSA host key started failing with a host key mismatch warning.
For individual developers, this meant updating their ~/.ssh/known_hosts file. Annoying, but straightforward. For organizations with hundreds or thousands of automated systems connecting to GitHub, it was significantly more disruptive.
The Scale of Impact
GitHub is the world's largest code hosting platform. Over 100 million developers use it, and a significant portion connect via SSH. The impact rippled through:
CI/CD Pipelines
Every Jenkins, GitHub Actions, GitLab CI, CircleCI, and custom CI/CD system that clones repositories from GitHub over SSH needed its known_hosts updated. For organizations running hundreds of pipelines, this meant:
- Failed builds across the entire organization
- Manual intervention on systems that couldn't be updated remotely
- Temporary workarounds that sometimes introduced security risks
Automated Deployment Systems
Many deployment systems pull code or configurations from GitHub as part of the deployment process. Failed SSH connections meant failed deployments until the keys were updated.
Developer Workstations
Every developer who uses SSH to interact with GitHub saw the host key warning on their next git operation. While this is a simple fix, it created confusion — many developers initially thought their own SSH keys had been compromised.
Container Images
Docker images and other container images that have GitHub's SSH host key baked in needed to be rebuilt. Images that used ssh-keyscan github.com during build time would automatically pick up the new key, but images with hardcoded host keys needed manual updates.
What Could Have Gone Wrong
GitHub stated they found no evidence that the exposed key was exploited. But had an attacker obtained the key before the rotation, the potential consequences were severe:
Man-in-the-Middle Attacks
With the private host key, an attacker could set up a server that perfectly impersonates GitHub. Any developer or CI/CD system connecting to the attacker's server instead of GitHub would have no way to detect the deception through SSH host key verification.
Code Injection
An attacker performing a man-in-the-middle attack could modify code as it's being pushed to or pulled from GitHub. A developer thinks they're pushing clean code, but the attacker intercepts it and injects a backdoor before it reaches GitHub. Or worse — a CI/CD system pulls code for a build, but the attacker serves a modified version containing malicious code.
Credential Theft
SSH connections to GitHub include authentication — either SSH key-based or token-based. A man-in-the-middle attacker could capture these authentication credentials and use them to access repositories directly.
The Response Was Right, But the Lessons Go Deeper
GitHub's response was textbook: detect, rotate, communicate. They rotated the key within hours of detection and published clear instructions for updating known_hosts files. But the incident reveals deeper issues.
Secret Scanning Gaps
GitHub has robust secret scanning capabilities that detect when users commit API keys, passwords, and tokens to repositories. Yet their own private SSH host key ended up in a public repository. This raises questions about:
- Whether GitHub's internal repositories are subject to the same secret scanning policies as customer repositories
- Whether the secret scanning rules covered SSH host key patterns
- How the key made it into a commit in the first place
Key Management Practices
A host key of this criticality should have extraordinary controls around it:
- Access should be limited to the absolute minimum number of systems and people
- The key should never exist in a location where it could be accidentally committed to a repository
- There should be automated monitoring for any appearance of the key outside its expected locations
The known_hosts Problem
The incident highlighted how fragile the SSH host key trust model is. Millions of systems had GitHub's RSA host key fingerprint hardcoded, and updating all of them was a manual, error-prone process. There's no automated mechanism for SSH host key rotation that's widely adopted.
Better Practices Going Forward
Use Ed25519 Keys Instead of RSA
GitHub noted that this incident only affected their RSA host key, not their Ed25519 or ECDSA keys. Ed25519 is the recommended algorithm for SSH keys today — it's faster, more secure, and produces shorter keys that are less prone to accidental exposure.
Automate known_hosts Updates
Rather than hardcoding host key fingerprints, use automated tools to maintain known_hosts files. GitHub publishes their SSH key fingerprints via their API (api.github.com/meta), allowing automated verification and updates.
Prefer HTTPS Over SSH
For CI/CD systems, HTTPS with token authentication is often more practical than SSH. Tokens can be scoped, rotated easily, and don't depend on host key verification.
Monitor for Key Exposure
Implement monitoring that alerts when any cryptographic key material appears where it shouldn't — in repositories, logs, configuration files, or error messages.
Have a Key Rotation Playbook
Document the process for rotating critical keys, including all downstream systems that need updating. Test the process before you need it in an emergency.
How Safeguard.sh Helps
Safeguard.sh provides the supply chain visibility needed to manage incidents like this:
- Configuration Scanning: Safeguard.sh scans your infrastructure configurations for insecure SSH settings, hardcoded host keys, and disabled host key checking that could leave you vulnerable.
- Artifact Integrity Verification: Independent of transport security, Safeguard.sh verifies that the code you build matches what was committed, catching any modifications that might occur during a man-in-the-middle attack.
- SBOM Consistency Checks: By comparing SBOMs across builds, Safeguard.sh can detect unexpected changes that might indicate source code was modified in transit.
- Supply Chain Monitoring: Safeguard.sh tracks security incidents affecting your toolchain, including critical infrastructure like GitHub, alerting you when you need to take action.
The GitHub key exposure was caught quickly and handled well. But it was a reminder that even the most critical infrastructure can have security failures, and every organization needs to be prepared for when it happens.