How to rotate SSH keys safely

Every SSH key you've ever generated for a server, a CI pipeline, or a contractor's laptop is a standing credential — and most organizations have no idea how many of them still work. If you've never had to rotate SSH keys before, the first attempt usually happens under pressure: an employee offboarding, an auditor's finding, or worse, a leaked private key discovered in a public repo. This guide walks through how to rotate SSH keys safely and deliberately, without locking yourself out of production or breaking automation that depends on key-based auth. By the end, you'll have a repeatable process — inventory, generate, deploy, cut over, revoke, verify — that you can turn into an actual ssh key management policy instead of a one-time fire drill.

Why SSH Key Rotation Matters (and When to Do It)

SSH keys don't expire on their own. Unlike passwords, there's no built-in mechanism forcing a change after 90 days, which is exactly why they accumulate silently across servers, jump boxes, and deployment scripts for years. A key generated by an employee who left eighteen months ago may still be sitting in an authorized_keys file granting root access.

Solid ssh key rotation best practices call for rotating keys on a schedule (commonly every 6–12 months for human users, shorter for high-privilege service accounts) and immediately whenever:

• An employee or contractor with key access leaves or changes roles
• A private key may have been exposed (committed to git, copied to a laptop that was lost, shared over Slack)
• You're consolidating from ad hoc individual keys to a centralized identity provider
• A compliance framework (SOC 2, ISO 27001, PCI DSS) requires documented credential rotation

Write these triggers down. A one-page ssh key management policy that defines rotation cadence, key algorithm standards (ed25519 over RSA-2048 at this point), and an offboarding checklist turns rotation from a reactive scramble into a scheduled maintenance task.

Step 1: Inventory Every Key Currently in Use

You can't rotate what you can't see. Before touching anything, build a complete map of:

• Every human user's public key and which servers/repos it can access
• Every service account, CI runner, and automation tool using SSH for deploys or git operations
• Every authorized_keys file across your fleet, including forgotten staging boxes and legacy bastion hosts

A quick way to audit a single host:

# List all keys authorized on this server
cat ~/.ssh/authorized_keys

# Find every authorized_keys file on the box
sudo find / -name "authorized_keys" 2>/dev/null

# Check the SSH daemon config for key-based auth settings
grep -E "PubkeyAuthentication|AuthorizedKeysFile" /etc/ssh/sshd_config

For fleets larger than a handful of machines, script this across your inventory (Ansible, a config management tool, or a fleet-wide pssh run) and dump the results into a spreadsheet or ticket. This inventory is also your baseline for verifying rotation completed successfully later.

Step 2: Generate a New SSH Key Pair

With the inventory in hand, generate a new ssh key pair for each user or service account being rotated. Use ed25519 unless you have a specific legacy requirement for RSA:

# Generate a new ed25519 key pair with a passphrase
ssh-keygen -t ed25519 -C "jane.doe@safeguard.sh-2026-07" -f ~/.ssh/id_ed25519_new

# If a system still requires RSA, use at least 4096 bits
ssh-keygen -t rsa -b 4096 -C "ci-deploy-key-2026-07" -f ~/.ssh/id_rsa_new

Two habits worth keeping here: label the comment field (-C) with a date or purpose so future audits don't require guessing which key is which, and always set a passphrase on human-held keys. Store the private key in your password manager or secrets vault immediately — never leave a copy sitting in Downloads.

For service accounts and CI systems, generate the pair in your secrets manager (Vault, AWS Secrets Manager, or your CI provider's encrypted secrets store) rather than on a laptop, so the private key never touches a human's filesystem at all.

Step 3: Deploy the New Public Key Everywhere It's Needed

Add the new public key alongside the old one — don't remove the old key yet. This overlap period is what lets you rotate SSH keys without an outage.

# Append the new public key to a server's authorized_keys
cat ~/.ssh/id_ed25519_new.pub | ssh user@server "cat >> ~/.ssh/authorized_keys"

# Or push via config management (Ansible example)
ansible-playbook -i inventory deploy-ssh-key.yml --extra-vars "pubkey_file=id_ed25519_new.pub"

Also update:

• Git hosting providers (GitHub, GitLab, Bitbucket) — add the new public key under the user's or deploy key's settings
• CI/CD secret stores that inject a private key for deploys
• Any bastion/jump host and its downstream authorized_keys entries
• Third-party integrations or partner systems that whitelist your keys

Step 4: Test the New Key Before Cutting Over

Verify the new key actually works from a fresh session before you touch the old one:

# Test with the new key explicitly, skipping ssh-agent's cached identity
ssh -i ~/.ssh/id_ed25519_new -o IdentitiesOnly=yes user@server "echo connected"

# For CI, trigger a manual pipeline run against a non-production branch

If this fails, debug it now while the old key is still a working fallback — troubleshooting after removal is far more stressful.

Step 5: Rotate SSH Keys Across Production and Retire the Old One

Once every dependent system authenticates cleanly with the new key, remove the old public key from authorized_keys, git providers, and secret stores:

# Remove a specific old key line from authorized_keys
sed -i '/old-key-comment-or-fingerprint/d' ~/.ssh/authorized_keys

# Confirm the old key's fingerprint before deleting, to avoid removing the wrong line
ssh-keygen -lf ~/.ssh/authorized_keys

Then shred the old private key material wherever it was stored:

shred -u ~/.ssh/id_ed25519_old

Do this for every server and service in your Step 1 inventory — an incomplete rotation that leaves the old key valid on even one forgotten host defeats the purpose.

Step 6: Update Your SSH Key Management Policy and Log the Change

Record what was rotated, when, and by whom in your access log or ticketing system. This is the step teams skip and later regret during an audit. At minimum, capture: the key owner, old and new fingerprints, systems updated, and the date of full retirement of the old key. Feed this back into your ssh key management policy so the next rotation cycle — whether scheduled or triggered by an offboarding — has a documented template to follow instead of starting from scratch.

Verification and Troubleshooting

After rotation, confirm nothing was missed:

• "Permission denied (publickey)" after removing the old key: The new key wasn't deployed to that specific host, or IdentitiesOnly isn't set and SSH is offering the wrong key from your agent. Run ssh -v for verbose output showing which key was offered and rejected.
• CI pipeline suddenly failing to clone or deploy: The deploy key in your CI provider's secrets wasn't updated — check pipeline logs for the exact host and key fingerprint being used.
• A host was missed entirely: Re-run your Step 1 inventory scan after rotation and diff it against your original list; any authorized_keys file still containing the old key's fingerprint needs manual cleanup.
• Old key still works after "removal": Check for multiple authorized_keys files (some systems reference more than one path) or cached credentials in ssh-agent — run ssh-add -l and ssh-add -D to clear stale identities.
• Unsure if a key is actually gone everywhere: Grep your entire configuration management repo, secrets manager, and CI variable stores for the old key's fingerprint (ssh-keygen -lf) rather than trusting memory.

How Safeguard Helps

Manual SSH key rotation works for a handful of servers; it breaks down fast across a real fleet, multiple cloud accounts, and dozens of service identities. Safeguard gives security and platform teams continuous visibility into every SSH key in your software supply chain — where it lives, what it can access, how old it is, and whether it maps to an active human or service identity. Instead of running ad hoc inventory scripts before every rotation, Safeguard maintains that inventory automatically and flags keys that are stale, orphaned from an offboarded user, or overdue against your defined rotation cadence.

When it's time to act, Safeguard's policy engine lets you codify your ssh key management policy once — rotation interval, approved key algorithms, required overlap windows — and enforces it across environments, alerting your team before a key becomes a liability instead of after an incident forces the question. That turns "how to rotate SSH keys safely" from a quarterly scramble into a continuously enforced baseline.