The SolarWinds attack didn't exploit a bug. It exploited trust. Organizations installed a backdoored update because it came through the official, trusted update channel and was signed with SolarWinds' legitimate code signing certificate. The update mechanism itself became the attack vector.
Software updates are inherently trusted. When an application says "an update is available," users install it without a second thought. When a server automatically applies a security patch, no human verifies the contents. This trust makes the update mechanism one of the most valuable targets for supply chain attackers.
How Update Mechanisms Get Compromised
Build System Compromise
The attacker compromises the vendor's build infrastructure and injects malicious code into the update before it's signed and distributed. The SolarWinds attack used this approach -- the SUNBURST backdoor was injected during the build process, so the malicious update was signed with the legitimate certificate.
Signing Key Theft
If an attacker steals the code signing private key, they can create and sign malicious updates that pass verification. The signing key is often the single point of failure for the entire update trust model.
Update Server Compromise
Attackers compromise the server that hosts updates and replace legitimate files with malicious ones. If updates aren't signed (or signature verification is optional), this is sufficient to distribute malware.
Man-in-the-Middle
Without proper TLS and certificate verification, an attacker positioned on the network can intercept update checks and serve malicious updates. Even with TLS, DNS or BGP hijacking can redirect traffic.
Rollback Attacks
An attacker forces the update mechanism to install an older version that contains known vulnerabilities. The old version is legitimately signed, so signature verification passes. The attacker then exploits the known vulnerability.
Freeze Attacks
The attacker prevents the update mechanism from seeing new updates. The client continues running an outdated, vulnerable version while believing it's up to date.
Metadata Manipulation
Update mechanisms use metadata to determine what's available, what's current, and what to install. Manipulating this metadata can cause the client to install wrong versions, skip updates, or download from attacker-controlled locations.
Principles of Secure Updates
The Update Framework (TUF)
TUF is a framework specifically designed to secure software update systems against all known attacks. Developed by researchers at NYU, it's been adopted by major projects including Docker Content Trust, Python's PEP 458, and Uptane (for automotive OTA updates).
TUF provides:
Role separation: Different keys for different operations. The key that signs timestamps is different from the key that signs the target files. Compromising one key doesn't compromise the entire system.
Threshold signatures: Critical operations require multiple keys. Even if an attacker obtains one key, they can't sign malicious updates without the others.
Expiration: Metadata expires, preventing freeze and replay attacks. If the client can't obtain fresh metadata, it refuses to install updates.
Consistent snapshots: The client can verify that the complete set of metadata and targets is consistent, preventing mix-and-match attacks.
Delegation: Maintainers can delegate trust for specific packages to different keys, limiting the blast radius of any single key compromise.
Code Signing Best Practices
HSM-protected keys: Store signing keys in hardware security modules. The key never leaves the HSM, making theft via software exploitation impossible.
Separate signing environments: Sign updates in an isolated environment, not on the build server. This way, build system compromise doesn't automatically compromise signing.
Timestamping: Use RFC 3161 timestamping so signatures remain verifiable even after the signing certificate expires.
Certificate transparency for code signing: Log all code signing certificates and signatures to transparency logs, enabling detection of unauthorized signatures.
Short-lived certificates: Use short-lived signing certificates (e.g., Sigstore's Fulcio) to reduce the window of opportunity for stolen key abuse.
Transport Security
Mandatory TLS: Updates should only be downloaded over TLS with proper certificate verification. No fallback to HTTP.
Certificate pinning: Pin the expected certificates for update servers to prevent CA compromise from enabling MitM attacks.
Multiple download sources: Allow clients to download updates from multiple mirrors and verify consistency.
Client-Side Verification
Verify before install: Always verify signatures before applying updates. Never assume that a downloaded file is legitimate because it came from the right server.
Rollback protection: Maintain a record of the current version and refuse to install older versions. Use monotonic version numbers or counters that can't be replayed.
Integrity verification: Check file hashes against signed metadata. Even if the download is intercepted, hash verification catches modifications.
Automatic verification: Don't make verification optional. If it can be skipped, it will be -- either by user choice or by attacker manipulation.
Common Mistakes
Relying Solely on HTTPS
HTTPS ensures the connection is encrypted and authenticated, but it doesn't verify the content. A compromised server serves malicious files over a perfectly valid HTTPS connection.
Single Key Architecture
Using one key for everything (signing updates, signing metadata, authenticating to the update server) means that compromising one operation compromises everything.
Optional Signature Verification
If signature verification is optional or can be disabled through configuration, attackers will disable it. Verification should be mandatory and enforced.
Trusting the Update Server
The update server should be treated as potentially compromised. Clients should verify everything they receive from the server, not trust it implicitly.
No Metadata Expiration
Without expiration, old metadata remains valid forever. An attacker who captures today's metadata can replay it indefinitely, freezing the client on a vulnerable version.
Insufficient Logging
Update mechanisms often lack adequate logging. When an incident occurs, there's no record of what was installed, when, or from where.
Industry Progress
The adoption of frameworks like TUF and Sigstore is improving the state of update security:
Sigstore provides free, transparent code signing infrastructure. Projects using Sigstore benefit from keyless signing (no long-lived keys to protect), transparency logs (all signatures are publicly auditable), and automated verification.
Uptane, built on TUF, is becoming the standard for automotive over-the-air updates. It addresses the unique challenges of updating embedded systems in vehicles.
Python's PEP 458 brings TUF to PyPI, protecting pip installations against multiple attack types.
Docker Content Trust uses TUF-based Notary to sign and verify container images.
These implementations show that secure update mechanisms are achievable and practical.
How Safeguard.sh Helps
Safeguard.sh complements secure update mechanisms by providing independent verification of software components after they're installed. Through SBOM generation and continuous analysis, Safeguard.sh verifies that the components in your deployed software match their expected versions and haven't been tampered with. When a software update introduces new or modified components, Safeguard.sh's scanning validates them against known vulnerability databases and integrity records, catching compromised updates that might pass code signing checks if the signing infrastructure itself was compromised. The platform's policy gates provide an additional verification checkpoint, ensuring that every update meets your organization's security requirements before reaching production.