Service Vulnerabilities: Shared Hosting Symlink Security Issue Still Widely Exploited on Unpatched Servers

The Wordfence site cleaning team helps numerous customers recover from malware infections and site intrusions. While doing so, Wordfence Security Analysts perform a detailed forensic investigation in order to determine how the site was compromised by attackers. In a set of recent cases, we were able to identify a service vulnerability allowing malicious attackers to use symlinks to compromise numerous sites on the tsoHost Managed cPanel VPS platform. In our investigation, we validated the vulnerability by creating a proof of concept and reached out to tsoHost, who promptly secured their systems against further attacks.

These service vulnerabilities are not unique to tsoHost’s Managed cPanel VPS platform, and successful exploitation by an attacker could result in extensive damage and risk to numerous sites.

As such, we are sharing details of what we have found so that managed hosts, resellers of hosting services, and site owners managing their own servers can also take steps to protect their accounts against these types of vulnerabilities.

A service vulnerability is any issue with a technology service that represents an exploitable security risk for its users.

Attackers using SymLinks to Spread Infection

A symbolic link, typically referred to as a symlink, is a way to create a shortcut or alias to a file on a Linux system and is commonly used to create a more intuitive or application-specific directory tree without having to redesign system functions and utilities.

Symlinks are also useful for malicious actors as they can be used to hide infected files. These suspicious symlinks typically link to files outside of the site’s path, but the symlinks themselves are not malware since they contain no actual code.

In the cases discovered on tsoHost’s Managed cPanel VPS systems, attackers used a malware kit known as “AnonymousFox” to quickly pivot from a single infected WordPress site on one cPanel user account to infect numerous other cPanel user’s sites across the same tsoHost Managed cPanel VPS. Normally we expect separate users in shared hosting environments to be isolated from one another in order to mitigate the spread of malware, so it was surprising to see the effectiveness of this attack. The AnonymousFox malware kit was used by a malicious actor to infect numerous users on separate virtual hosts on the same server within 90 minutes.

While this vulnerability was discovered within tsoHost’s systems, we found that it was not unique to tsoHost’s Managed cPanel VPS. Exploitation can happen on any cPanel, Plesk, or other shared hosting platforms running Apache or NGINX where proper security configurations and patches are not in place.

Attempts by malicious attackers using “AnonymousFox” and other malware kits appear to be growing. Therefore, we strongly recommend that all managed hosting providers, resellers, and site owners operating their own shared hosting servers take action to ensure they are not a victim of these attacks. Unlike malware attacks that target a single site and are contained, these attacks aim to cripple the larger hosting environment as a whole and can affect numerous sites and the data contained within those sites, including the personally identifiable information (PII) of customers making purchases on e-commerce sites.

Site Cleaning Investigation

In early April 2021, our site cleaning team performed a site cleaning for six sites on a server instance running on the tsoHost Managed cPanel VPS platform.These sites were hosted on six different virtual host user accounts. The site users we conducted a detailed investigation on represented 20-30% of all the virtual hosts on the cPanel VPS instance. Other sites on the server were also infected. Our forensic research on a limited set of the sites allowed the server manager to remove malware on the rest.

Initially, these attacks on their own appeared to be a series of WordPress administrator credential compromises that could have been attributed to password reuse. However, our investigation uncovered a more advanced compromise rooted in the Apache symlink vulnerability.

A vulnerability allowing attackers to read sensitive files across users in a shared hosting environment can happen for a variety of reasons and be exploited with a variety of methods. This particular attack exploited a security vulnerability that allows read access to files in other user accounts through use of a symlink followed by download over HTTP.

Details of the Attack

The initial account (Site User #1) was compromised due to insecure credentials, a common intrusion vector for a single site infection. Once logged in the attacker uses their privileges to install a plugin as a backdoor.

We determined the five additional site hosting accounts investigated were infected methodically, in alphabetical order, over the next 90 minutes. Server logs showed clear evidence that the other sites on the server had sensitive files exfiltrated and sites compromised in the same methodical order.

The subsequent site user intrusions looked like WordPress administrator credentials were compromised. However, each login was linked in the server logs as a referrer from the malware script on Site User #1.

The referrer in the logs provided an important clue that these additional infections were not a routine credentials compromise. Upon further investigation, Wordfence Security Analysts discovered that the malware plugin was obfuscated with several levels of code, such as eval (gzinflate(base64_decode($text))). Eventually the core functionality was revealed, exposing a combination of automation and human moderation allowing the attacker to pivot to other sites under different user accounts.

Starting with the symlink vulnerability, the attacker was able to download a wp-config.php file owned by another user on the same server by creating a symlink to the file path of that wp-config.php file. The wp-config.php file is the one WordPress file containing sensitive information including database connection credentials. The symlink could then be fetched through http, downloading the file as plain text. Though WordPress wp-config.php files were a focus of the attack in this case, we frequently see symlinks to a variety of common web application configuration, panel platform configuration, and other server configuration files in attempts to exfiltrate sensitive details.

Here is a log sample where the symlink downloaded through the SITE USER #1 site was used to retrieve the confidential credentials from Site User #2 that were stored in their wp-config.php file:

87.247.240.75 - - [03/Apr/2021:07:32:50 +0100] "GET /wp-content/plugins/seoo/home/SITECPANELUSERNAME2-WORDPRESS.txt HTTP/2.0" 200 2840 "-" "Mozilla/5.0 (Windows NT 6.1; rv:32.0) Gecko/20100101 Firefox/32.0"

The symlink in the Site User #1 (/home/sitecpaneluser1/) account looked like this, linked to the private file in Site User #2:

/home/sitecpaneluser1/public_html/SITECPANELUSERNAME2-WORDPRESS.txt -> /home/sitecpaneluser2/public_html/wp-config.php

With database credentials exposed and local database access achieved through the infected Site User #1, the attacker then used the capabilities of the malware to cycle through the process of infecting more sites on the same server. This process followed these steps:

• Log into the database server of site #2 using credentials from the site’s wp-config.php
• Deactivate all plugins in the WordPress database _options table
• Reset the WordPress administrator credentials to the attacker’s own password in the database _users table
• Send an email to multiple addresses with information about the compromised account and server:
  Malicious email accounts identified:
  superstar0882@gmail.com, superstar0882@hotmail.com
  Third-party URLs holding malicious email addresses to mail to:
  https://pastebin[.]com/raw/5M63g44m
https://pastebin[.]com/raw/Cz0dLKL3
https://raw.githubusercontent[.]com/devildrinker/mail/master/mail.txt
• The attacker logs in to the newly compromised site as a WordPress administrator and uses that access to infect it with malware. The read vulnerability pivot to write access is complete.

Once each site had been compromised, the attacker moved on to the next site on the server to repeat the cycle. What initially looked like a series of WordPress administrator credential compromises were actually a more advanced compromise rooted in the Apache symlink vulnerability.

In a properly secured shared hosting environment, if one user account becomes infected the other users are safe from the initially infected account.

Attack Pivot Part 1: The symlink web server vulnerability allows an attacker to read files in other user accounts. A symlink is created to application configuration files like the WordPress wp-config.php, enabling the linked files to be downloaded through the initially infected web host.

Attack Pivot Part 2: With database credentials in hand, the attacker can now log into the databases of other user accounts through the localhost access of the initially infected site. Databases store users and passwords for applications like WordPress. The attacker sets their own administrator password and takes other action to infect the database or control the application.

Mass infection is completed when the attacker logs into WordPress through the front door with the stolen administrator access. Malware is then installed into each additional site compromised.

Remediation for Site Owners

A site owner may be able to protect against the symlink read vulnerability by making sure sensitive files such as wp-config.php have restricted permissions instead of the typical default 644 permissions mode that allows other users and groups to have access. On most platforms restricted permissions mode of 600, restricting permissions of other users and groups on the Linux server, is sufficient to protect the file so only the account user owning that file can read it. However, this may not be sufficient in all cases, since it is possible that permissions could get automatically changed back by the hosting platform or during routine site maintenance.

In addition, we recommend that site owners verify with their shared hosting provider that their platform is properly patched against the symlink vulnerability specifically, as well as secured against read and write access from other user accounts on the same server.

As a precaution you should always keep backups of your site that are stored outside of your production shared hosting platform.

It is important to note that with shared hosting platforms there is an increased potential risk to exposure of private data hosted on your site. If your site runs e-commerce storing customer data such as name, email, address or other Personally Identifiable Information (PII) or collects sensitive information about customers or site visitors, moving your site onto its own virtual private server (VPS) is worth considering and highly recommended to adhere to requirements of the Payment Card Industry Data Security Standard (PCI-DSS).

Remediation for Service Providers

Shared hosting platform providers should take action to ensure their environment is properly secured against this symlink vulnerability as well as other common permissions-based cross-account read vulnerabilities.

Reference articles related to patching for the symlink vulnerability are available on a variety of platforms including: KernelCare, cPanel, and Plesk. Patches and platform specific recommendations to deal with the symlink vulnerability have been published and available for 1-3 years or longer.

Make sure that if a user creates a symlink to another user’s web site file that you are patched against the symlink vulnerability that exposes read access to files owned by other users when downloaded over HTTP(S).

Proof of Concept

Warning: Testing these vulnerabilities on shared hosting systems should not be done by site owners or developers. Doing so could break Terms of Service with your hosting provider or result in legal repercussions.

The following is a proof of concept that shared hosting providers and website owners, with the appropriate authorization, can use to test for this vulnerability and validate that the appropriate security remediation is in place.

• Create a symlink to private file in another user account:
  /home/exampleuser1/public_html/ $ ln -s /home/exampleuser2/public_html/wp-config.php exampleuser2-private-file.txt
/home/exampleuser1/public_html/ $ ls -la exampleuser2-private-file.txt
lrwxrwxrwx  1 exampleuser1 exampleuser1   56 May 01 14:03 exampleuser2-private-file.txt -> /home/exampleuser2/public_html/wp-config.php
• FollowSymLinks in Apache and symlinks in nginx are standard settings that can typically be turned off or on in site configuration files. In most cases the setting is on by default. The attacker that compromised exampleuser1 would likely be able to turn the setting on in the compromised site, if it wasn’t on by default.
• Try downloading the file through the symlink: http://exampleuser1[.]com/exampleuser2-private-file.txt

Review your hosting configuration for permissions-based cross-account read vulnerabilities:

• Make sure a user cannot connect via SSH/FTP and read files in another user’s website file paths.
  • Proof of Concept: Run a simple test of being able to read a file between user accounts:
    /home/exampleuser1/ $ cat /home/exampleuser2/public_html/wp-config.php
  • Configuration Remediation: In most cases, configuring SSH, SFTP and/or FTP sessions to be chroot or jailed secures this issue
  • Permissions Remediation: Some hosting platforms will restrict the web site home path (e.g., /home/exampleuser1/public_html/) to 750 permissions mode, limiting directory traversal capabilities. This setting may require other web server configuration changes.

Investigation and Disclosure Timeline with tsoHost

April 15, 2021 – We contract our own tsoHost Managed cPanel VPS to create a proof of concept.
April 20, 2021 – We confirm proof of concept on the default tsoHost Managed cPanel VPS we were provided.
April 28 – April 30, 2021 – We coordinate sending full disclosure to tsoHost. Our disclosure includes notifying tsoHost of two read-based vulnerabilities:

• A cross-user symlink read vulnerability being actively exploited.
• A cross-user read-based vulnerability through SSH that is present, but not actively exploited.

May 3, 2021 – tsoHost changes configuration to chroot SSH sessions. We confirm that the SSH-based read vulnerability is patched.
May 11, 2021 – tsoHost applies Free KernelCare Symlink Protection patch. We confirm that the Symlink read vulnerability is patched.
May 17, 2021 – tsoHost confirms both the SSH configuration change and the symlink security patch are applied across all servers in their Managed cPanel VPS platform.

Conclusion

Wordfence is committed to ensuring that WordPress sites are secured. This often leads to assisting businesses in the WordPress ecosystem identify and patch security issues. As such, service vulnerability disclosures are a necessary part of our Vulnerability Disclosure Policy.

Hosting provider tsoHost was responsive to our disclosure and promptly applied the confirmed fix across their servers in their Managed cPanel VPS platform.

Research into this attack also allowed us to gain a better perspective on the capabilities and methods used to pivot from a single infected user account on a shared hosting platform to other user accounts, through a read-based vulnerability that allowed unauthorized database access.

We hope that this research is useful to shared hosting service managers, resellers, and site owners alike.

Special Thanks to my Wordfence team members Ram Gall and Chloe Chamberland for their mentoring in threat research and Kathy Zant for help with editing this article. We also want to extend kudos to tsoHost for quickly responding to and resolving the security issues on their platform.