How to harden a Linux server

Every internet-facing Linux server starts life more exposed than it needs to be. Default SSH configs, unnecessary open ports, unpatched packages, and permissive firewall rules hand attackers a wide surface to probe — and automated scanners find that surface within minutes of a host going live. Learning how to harden Linux server security isn't a one-time checklist you run before launch and forget; it's a repeatable process that closes the gap between "it works" and "it's defensible in an audit." This guide walks through a practical linux server hardening checklist you can apply to any Debian, Ubuntu, or RHEL-family host, from locking down SSH to aligning your configuration with the CIS benchmark for Linux. By the end, you'll have a server with a minimal attack surface, verified logging, and controls you can point to when a customer or auditor asks how you secure production infrastructure.

Step 1: Establish a Hardened Linux Server Security Baseline

Hardening starts before you touch a single config file — it starts with what's installed. Every extra package, service, or compiler on a production box is a future CVE waiting to be scanned for.

# Update and patch first, always
sudo apt update && sudo apt upgrade -y        # Debian/Ubuntu
sudo dnf update -y                            # RHEL/CentOS/Alma/Rocky

# Enable unattended security updates
sudo apt install unattended-upgrades -y
sudo dpkg-reconfigure --priority=low unattended-upgrades

# Remove packages you don't need
sudo apt purge telnet rsh-client xinetd -y
sudo apt autoremove -y

Pin this to a schedule with cron, systemd timers, or your patch management tool — stale patches are consistently one of the top root causes in post-incident reviews. Also disable any bundled demo services, sample web apps, or default accounts that ship with your base image; they're rarely audited and often forgotten.

Step 2: Follow an SSH Hardening Guide Before You Do Anything Else

SSH is the single most attacked service on any public Linux host. If you only do one thing from this article, do this. Edit /etc/ssh/sshd_config:

PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
AllowUsers deploy ops
Protocol 2
X11Forwarding no
MaxAuthTries 3
ClientAliveInterval 300
ClientAliveCountMax 2

Generate and deploy key-based auth before disabling passwords, or you will lock yourself out:

ssh-keygen -t ed25519 -C "deploy@yourorg"
ssh-copy-id -i ~/.ssh/id_ed25519.pub deploy@server-ip

# Validate config syntax before restarting
sudo sshd -t
sudo systemctl restart sshd

Consider moving SSH off port 22 to cut down on noise from mass scanners, and layer on fail2ban to auto-block repeated failed logins:

sudo apt install fail2ban -y
sudo systemctl enable --now fail2ban

Keep a second terminal session open while you make these changes so you can revert quickly if something breaks.

Step 3: Enforce a Default-Deny Firewall

A server should only accept traffic it explicitly expects. Start from deny-all and open exceptions one at a time.

sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow OpenSSH
sudo ufw allow 443/tcp
sudo ufw enable
sudo ufw status verbose

On RHEL-family systems, use firewalld with zones instead:

sudo firewall-cmd --set-default-zone=drop
sudo firewall-cmd --permanent --add-service=ssh
sudo firewall-cmd --permanent --add-service=https
sudo firewall-cmd --reload

If the box sits behind a cloud provider, mirror these rules in the security group or NSG too — host-level and network-level firewalls should agree, not contradict each other.

Step 4: Apply Least Privilege to Accounts and Sudo

Attackers who get a foothold almost always try to escalate privileges next. Limit what any single compromised account can do.

# Lock or remove unused accounts
sudo passwd -l olduser
sudo usermod -s /usr/sbin/nologin serviceacct

# Require sudo group membership explicitly
sudo visudo   # ensure %sudo or %wheel is the only group with NOPASSWD-free access

# Set a sane password policy
sudo apt install libpam-pwquality -y

In /etc/login.defs, tighten password aging:

PASS_MAX_DAYS   90
PASS_MIN_DAYS   1
PASS_WARN_AGE   14

Audit /etc/passwd and /etc/group periodically for accounts with UID 0 that aren't root, and for empty password fields — both are common findings in penetration tests.

Step 5: Align Kernel and Filesystem Settings With the CIS Benchmark for Linux

This is where the linux server hardening checklist stops being ad hoc and starts being auditable. The CIS Benchmark for Linux defines specific, testable controls for kernel parameters, filesystem mounts, and service configuration. A few of the highest-impact ones:

# /etc/sysctl.d/99-hardening.conf
net.ipv4.ip_forward = 0
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.icmp_echo_ignore_broadcasts = 1
kernel.randomize_va_space = 2

sudo sysctl -p /etc/sysctl.d/99-hardening.conf

Disable unused filesystem modules commonly flagged in CIS scans:

echo "install cramfs /bin/true" | sudo tee -a /etc/modprobe.d/blacklist-rare-fs.conf
echo "install freevxfs /bin/true" | sudo tee -a /etc/modprobe.d/blacklist-rare-fs.conf

Rather than hand-checking every control, run an automated scan against the benchmark and treat the report as your remediation backlog:

sudo apt install lynis -y
sudo lynis audit system

Lynis and OpenSCAP both map findings back to CIS control IDs, which makes it far easier to track remediation over time and hand evidence to an auditor.

Step 6: Turn On Logging, Auditing, and File Integrity Monitoring

Hardening reduces the odds of compromise; logging is what lets you detect and investigate the one that gets through anyway.

sudo apt install auditd audispd-plugins -y
sudo systemctl enable --now auditd

# Watch key files for changes
sudo auditctl -w /etc/passwd -p wa -k identity
sudo auditctl -w /etc/ssh/sshd_config -p wa -k sshd_config

Add file integrity monitoring with AIDE, and ship logs off-box to a central collector so an attacker who gains root can't quietly delete the evidence:

sudo apt install aide -y
sudo aideinit

Forward /var/log/auth.log and auditd output to a SIEM or log aggregator. If you only keep logs locally, you're one rm -rf away from losing your entire incident timeline.

Verifying the Hardening and Troubleshooting Common Issues

Once the changes are live, verify each control rather than assuming it took effect:

• Confirm SSH changes didn't lock you out. Before closing your original session, open a fresh connection in a new terminal and confirm you can still log in with your key. If it fails, your original session is your rollback path — fix sshd_config and run sudo sshd -t to catch syntax errors before restarting.
• Re-check firewall state after reboot. ufw status or firewall-cmd --list-all should reflect your intended rules; a reboot occasionally reverts an unsaved iptables change if it wasn't persisted through the proper service.
• Watch for fail2ban over-blocking. If a teammate reports being locked out, check sudo fail2ban-client status sshd and unban with fail2ban-client set sshd unbanip <ip> — this is far more common than an actual attack once the jail is tuned aggressively.
• Validate sysctl settings persisted. Run sudo sysctl -a | grep <param> after a reboot; settings applied only via sysctl -w at the command line don't survive a restart unless they're also written to a .conf file.
• Re-run Lynis or OpenSCAP after each change batch. Track your hardening index score over time so regressions — a new package reopening a port, a config drift from a hotfix — surface immediately instead of at the next audit.
• Test from outside the host, not just from localhost. A firewall rule that looks correct in ufw status can still be undermined by a permissive cloud security group; scan the public IP with nmap from an external machine to confirm only intended ports respond.

How Safeguard Helps

Manually running through a linux server hardening checklist across dozens or hundreds of hosts doesn't scale, and configuration drift creeps back in the moment someone ships a hotfix under deadline pressure. Safeguard continuously monitors your infrastructure and software supply chain for exactly the gaps this guide addresses — exposed services, weak SSH configurations, unpatched packages, and drift away from CIS benchmark baselines — and surfaces them as prioritized, actionable findings instead of a static report that goes stale the day it's generated.

Because Safeguard ties infrastructure posture to your broader supply chain security program, hardening evidence isn't siloed from your build pipeline, dependency, and provenance data — it's part of the same audit trail. That means when a customer or SOC 2 assessor asks how you harden Linux server security across your fleet, you have continuous, verifiable evidence rather than a point-in-time checklist someone ran six months ago. Teams using Safeguard catch configuration regressions before they reach production and spend less time re-proving controls they already implemented.

If you're maintaining hardening standards across a growing number of Linux hosts, talk to Safeguard about automating the verification layer on top of the checklist you just built.