SSH is the primary access mechanism for servers, network devices, cloud instances, and Git repositories. In most organizations, SSH key management is decentralized: each developer generates their own keys, adds them to servers manually, and never rotates them. This creates an unmanaged credential sprawl that grows with every new hire and every new server.
The average enterprise has tens of thousands of SSH keys. Many of these keys are orphaned (belonging to former employees), over-permissioned (granting root access when read-only would suffice), or shared (the same key used by multiple people or systems).
The Problem With Manual Key Management
Manual SSH key management breaks down at organizational scale. The authorized_keys file on each server accumulates entries over years. Nobody removes keys when employees leave. Nobody audits which keys grant access to which systems. Nobody enforces key rotation.
The result is an invisible network of trust relationships that nobody fully understands. An attacker who obtains a single SSH private key may have access to dozens or hundreds of systems, and the organization has no way to determine the blast radius without manually checking every authorized_keys file.
SSH Certificate Authority
The most effective approach for organizations is SSH certificate authentication. Instead of distributing public keys to individual servers, you operate a Certificate Authority (CA) that signs developer keys. Servers trust the CA, and any key signed by the CA is accepted.
Benefits:
- Centralized revocation. Revoke a certificate and the user loses access everywhere, immediately. No need to remove keys from individual servers.
- Automatic expiration. Certificates have validity periods. A certificate signed for 8 hours expires automatically, eliminating the problem of orphaned keys.
- Auditable access. The CA signs certificates with identity information (username, authorized principals), creating an audit trail.
- Reduced server configuration. Servers need only the CA public key, not a list of individual authorized keys.
Implementation
HashiCorp Vault provides SSH certificate signing through its SSH secrets engine. Developers authenticate to Vault, request a signed certificate, and use it for SSH access. The certificate expires after a configurable period (typically hours, not months).
Smallstep provides an open-source SSH CA with similar capabilities. Facebook has published their approach to SSH CA at scale.
For lighter-weight setups, OpenSSH natively supports certificate authentication with ssh-keygen -s for signing.
Key Rotation Strategies
Short-lived certificates. The best rotation strategy is to make keys short-lived enough that rotation is unnecessary. An 8-hour certificate expires before an attacker can use a stolen key.
Automated rotation. For long-lived keys (service accounts, CI/CD, automated systems), implement automated rotation on a regular schedule. Store keys in a secrets manager (Vault, AWS Secrets Manager) that handles rotation.
Forced rotation on incidents. If any compromise is suspected, force rotation of all SSH keys. This is operationally expensive but necessary when the scope of compromise is uncertain.
Access Control and Auditing
Principle of least privilege. SSH keys should grant the minimum access needed. Use command= restrictions in authorized_keys to limit what a key can execute. Use separate keys for separate purposes (one for Git, another for server access).
Bastion hosts / jump boxes. Route all SSH access through a bastion host that logs connections and enforces additional authentication. Direct SSH access from the internet to production servers should be prohibited.
Session recording. Record SSH sessions for audit purposes. Tools like Teleport, Gravitational, and OpenSSH session logging provide this capability.
Automated scanning. Regularly scan your infrastructure for unauthorized SSH keys, unencrypted private keys on servers, and keys with no passphrase protection.
Cloud-Specific Considerations
AWS. Use EC2 Instance Connect or SSM Session Manager instead of traditional SSH keys. These provide temporary access, logging, and IAM-based authorization.
Google Cloud. OS Login integrates SSH access with Google identity, providing centralized access management and audit logging.
Azure. Azure Bastion provides managed SSH access without exposing VMs to the public internet.
Each of these approaches eliminates the need for managing SSH keys on individual instances, replacing them with identity-based access that integrates with your existing IAM infrastructure.
How Safeguard.sh Helps
Safeguard.sh extends supply chain security to your infrastructure access layer. Our platform monitors for exposed SSH keys in source repositories, detects unauthorized access patterns, and integrates with your CI/CD pipeline to ensure that SSH credentials used for deployment are properly managed. Combined with SBOM generation and vulnerability tracking, Safeguard.sh provides comprehensive security visibility from code to infrastructure.