Supply Chain Security

XZ Utils Backdoor (CVE-2024-3094): The Most Sophisticated Supply Chain Attack Ever Discovered

A multi-year social engineering campaign planted a backdoor in XZ Utils that would have compromised SSH on most Linux distributions. Technical deep dive into what happened.

Shadab Khan
Senior Security Engineer
6 min read

On March 29, 2024, Andres Freund, a Microsoft engineer and PostgreSQL developer, posted a message to the oss-security mailing list that would shake the foundations of open source trust. He had discovered a backdoor in XZ Utils versions 5.6.0 and 5.6.1, carefully hidden in the build system of one of the most widely used compression libraries in the Linux ecosystem.

CVE-2024-3094 was assigned a CVSS score of 10.0, the maximum severity rating. But the score barely captures the significance of what was found. This was not a typical vulnerability. It was a deliberate, meticulously planned supply chain attack that had been years in the making, and it came within weeks of shipping in every major stable Linux distribution.

What XZ Utils Is and Why It Matters

XZ Utils provides the LZMA/LZMA2 compression algorithm and is installed on virtually every Linux system. The liblzma library it provides is linked by a staggering number of packages. Critically, on many Linux distributions, liblzma is indirectly loaded by OpenSSH's sshd daemon through a dependency chain involving systemd and its libsystemd library.

This means that backdooring liblzma effectively backdoors the SSH daemon, the primary remote access mechanism for nearly every Linux server on the internet.

The Technical Mechanism

The backdoor was not present in the source code repository in a directly readable form. Instead, it was hidden in binary test files committed to the repository (tests/files/bad-3-corrupt_lzma2.xz and tests/files/good-large_compressed.lzma) and activated through a series of obfuscated steps in the build system.

Here is how it worked:

Stage 1: Build system injection. The m4/build-to-host.m4 script, part of the autotools build system, contained obfuscated code that would extract a script from one of the test files during the build process. This script was only activated when building from a release tarball (not from a git clone), which meant it would not trigger during casual code review or CI builds that worked from the git repository.

Stage 2: Object file extraction. The extracted script decoded and decompressed a prebuilt shared object file from the binary test data. This object file contained the actual backdoor payload.

Stage 3: Linker manipulation. The build process was modified to link this malicious object file into liblzma.so, using GNU indirect function (IFUNC) resolvers to hook specific functions.

Stage 4: SSH authentication bypass. The backdoor hooked the RSA_public_decrypt function in OpenSSH (via the liblzma -> libsystemd -> sshd chain). When sshd performed RSA signature verification during authentication, the backdoor would check incoming public keys against an attacker-controlled key embedded in the payload. If the key matched, the backdoor would execute a command payload hidden in the certificate data, effectively giving the attacker unauthenticated remote code execution as root.

The sophistication was remarkable. The backdoor:

  • Only activated on x86-64 Linux systems using glibc and systemd
  • Checked for specific environment variables and would disable itself if it detected a debugger
  • Used a custom steganographic encoding to hide the payload in binary test files
  • Employed multiple layers of XOR-based obfuscation
  • Modified its behavior based on the process name, only activating when loaded by sshd

The Social Engineering Timeline

The technical sophistication was matched by the social engineering campaign that enabled it. The attack began in 2021, when an account using the name "Jia Tan" (GitHub user JiaT75) began making legitimate contributions to the XZ Utils project.

2021-2022: Jia Tan submitted patches and engaged constructively with the project. Simultaneously, other accounts (possibly sock puppets) began pressuring the sole maintainer, Lasse Collin, to add another maintainer, citing slow response times and the maintainer's personal health issues.

2022-2023: Lasse Collin, overwhelmed and under pressure, granted Jia Tan co-maintainer status. Jia Tan gradually took over more maintenance responsibilities, building trust through legitimate contributions.

Late 2023: Jia Tan began introducing the components of the backdoor: the malicious test files, the modified build scripts, and changes to the build system that would activate the payload.

February 2024: XZ Utils 5.6.0 was released with the backdoor included. The timing was calculated. Jia Tan also began pressuring distribution maintainers to include the new version in upcoming stable releases.

March 2024: XZ Utils 5.6.1 was released with additional refinements to the backdoor. Fedora 40, Debian testing/unstable, and openSUSE Tumbleweed picked up the compromised versions.

How Close Was the Disaster

The backdoor made it into Fedora 40 beta, Debian sid (unstable), and several rolling-release distributions. It was weeks away from landing in stable releases of Fedora, Debian, Ubuntu, and other major distributions.

If it had made it into stable releases, the attacker would have had a skeleton key to SSH on a significant fraction of all Linux servers on the internet. The scope of potential compromise would have been unprecedented.

The Detection

Andres Freund noticed the backdoor because he was benchmarking PostgreSQL and observed that SSH logins were taking approximately 500 milliseconds longer than expected. The backdoor's code path added measurable latency to the SSH authentication process.

He investigated, found the CPU was spending unexpected time in liblzma during SSH operations, and traced it back to the malicious code. His detailed analysis and responsible disclosure gave distributions the information they needed to respond immediately.

It is worth pausing on this. The most sophisticated supply chain attack ever attempted against the open source ecosystem was caught because one engineer noticed a half-second performance regression and was curious enough to investigate.

Impact and Response

CISA issued an alert the same day. All major Linux distributions issued advisories and patches reverting to XZ Utils 5.4.x. GitHub suspended the JiaT75 account and temporarily restricted access to the XZ Utils repository.

The community response was swift, but the incident exposed fundamental structural problems with open source maintenance that patches and reverts cannot fix.

How Safeguard.sh Helps

Safeguard.sh is built for exactly this class of threat. Our SBOM generation and continuous monitoring capabilities track every dependency in your software supply chain, including transitive dependencies like liblzma. When CVE-2024-3094 was published, Safeguard.sh users could immediately identify every system and container image in their inventory that included the compromised XZ Utils versions. Our policy gates can enforce version pinning and block deployments that include known-compromised packages, giving organizations a systematic way to respond to supply chain attacks rather than relying on manual audits and emergency change windows.

Never miss an update

Weekly insights on software supply chain security, delivered to your inbox.