Background#

On March 29, Developer Andres Freund discovered a backdoor in a commonly used Linux compression library, XZ Utils. Within a few days, many companies including Red Hat and SUSE have rushed to address the issue, with Red Hat releasing an urgent security alert for Fedora 40 and Fedora Rawhide. The harmful backdoor has been identified in versions 5.6.0 and 5.6.1 of the xz package. Consequently, a number of distributions, namely development builds of Debian and Fedora as well as Kali Linux and openSUSE Tumbleweed, incorporated these vulnerable versions. If you’re running an affected version of the xz package, remember to downgrade the package to unaffected builds.

What is XZ Utils?#

XZ Utils is a collection of lossless data compression utilities for managing archives. XZ compression offers a higher compression ratio than other compression methods like gzip or bzip2, making it used commonly in distributing software packages, archiving data, or compressing release tarbells. If you’ve ever tried to download packages on a linux machine, you might’ve seen files with the file extension .tar.xz.

Additionally, the project is developed open source with its repository hosted on github. Sadly, the repo has since been disabled by Github..

Understanding the malicious script#

As stated in the title of the article, the vulnerability has been assigned CVE-2024-3094 by Red Hat. Let’s look into the method of attack and how the package became compromised.

The first thing to note is that malicious code is present in the release tarbells but *not in the upstream source. The version of build-to-host.m4 in the release tarbells differs wildly from the upstream source. Furthermore within the tests folder, we get some two very suspicious compressed test files. Essentially, the test files contain the payload, and the m4 script unpacks the malicious payload, using it to modify the build process. Here are some snippets of the malicious script that gets unpacked by the modified m4 script.

if ! (echo "$build" | grep -Eq "^x86_64" > /dev/null 2>&1) && (echo "$build" | grep -Eq "linux-gnu$" > /dev/null 2>&1);then

if test -f "$srcdir/debian/rules" || test "x$RPM_ARCH" = "xx86_64";then

Based on this, we can tell that the attack is targeted at specific architectures and distributions, more specifically amd64 systems running either Debian or Red Hat distributions like Fedora or OpenSUSE.

Payload Injection#

Now we get to the actual backdoor itself. Once all the requirements have been fulfilled, the payload is then injected into the source tree where it then loads directly into sshd, the Secure Shell Daemon. Afterwards it replaces the RSA_public_decrypt function with a malicious implementation. While the exact specifics of the backdoor is not yet known, this would essentially allow a malicious attacker to break sshd authentication and gain unauthorized access to the system.

Understanding OSS Supply Chain attacks#

As mentioned before, XZ Utils is an open source project where anyone submit issues and contribute to the code. The security of Open Source projects have been widely debated for years. Some claim that open source projects provide more transparency, allowing anyone to inspect the code of vulnerabilities, and thus allowing issues to be identified and fixed faster. Others claim that Open Source Software (OSS) might be more vulnerable as anyone can inject malicious code and smaller projects might not have the time or ability to fix these issues.

When it comes to OSS, the security of the package heavily depends on the community itself. Another point to note is that OSS often rely on other open source libraries, creating a long and complex dependency chain. Such is the case here. While sshd doesnt use xz utils directly, liblzma is depended on by other parts of libsystemd. As such, an OSS supply chain attack is one that focuses on open source libraries and packages, which are often depended on and used by other packages and libraries. Introducing malicious code to the upstream package would result in the downstream packages being vulnerable as well. As such, in order to protect ourselves from such vulnerabilities, we need to employ better dependency management and be more aware of all the dependencies, direct or indirect, that our software depends on.

Identifying the root cause#

Another aspect of this incident that fascinated me was that the perpertrator was not some random user who decided to commit malicious code, which got merged accidentally. The author of the malicious commits and packager of the faulty tarbells was Jia Tan, one of the two Maintainers of the package itself. After doing some digging, we can see that Jia Tan (likely an alias) started contributing regularly and became a maintainer around June 2022. It was also during this time that the original maintainer Lasse Collin started feeling burnout from being the sole maintainer of the project

Beyond the technical details of the exploit, I think there is a deeper issue here that needs to be addressed, and that is the problem that many Open Source developers face. When you work on an Open Source project, it is often out of passion and love for the project itself. However maintaining such a project is often times a thankless job. Moreso when the project you’re maintaining is such a crucial part of the Open Source ecosystem, it can be very easy to get burnt out or lose interest. While the technical parts of the exploit is interesting, I think the main exploit here is a Social Engineering Attack. Pretending to be a developer passionate in the project, helping out with issues and releases in an attempt to build trust, and exploiting the mental fatigue that many Open Source Developers face, ultimately Jia Tan pulled off a successful Social Engineering attack and built up a large amount of trust in the community before exploiting that trust to sneak in his malicious code.

Ending remarks and credits#

I decided to write this blog as a consolidation of information from here and the original report by Andres Freund. While many news sites have already covered the vulnerability, I think it’s interesting to go slightly deeper in depth to see how exactly the payload was delivered and the situations that lead to this scenario. It also serves a stark reminder that behind all the Open Source Projects and Software that we use on a daily basis, there’s a burnt out maintainer tirelessly working to maintain a library that a thousand other packages depend on.