Mass Hosting && PCI: A Case Study

Just as a quick follow up to my earlier post, here is a prime example of a "PCI Compliant" company who is probably not.

I attempted responsible disclosure several weeks ago and received no response from this company, a New York based web hosting and development company. I'm not trying to get them in trouble, so I redacted the screenshots and won't refer to them by name, but I have no doubt that anybody who cared could find them quickly enough.

As part of my research, I made a phone call to this company. I inquired about the PCI compliance of their servers, and the sales associate assured me that their servers are scanned regularly and are compliant. He did not tell me who their ASV is, and while I have my theories, I won't speculate publicly.



Regardless, this company has a login portal on their main web site. I know one of their customers, so I can tell you that the credentials used to log into this portal are used for customer cPanel accounts, billing system, and webmail. Those credentials are probably pretty valuable, right?



How did this site ever pass any kind of PCI testing? It's only a theory, but I suspect this is because it's on a mass-hosted server. When you hit the server directly by IP, it brings up the default cPanel page.



But this company was, by their claim, 100% PCI compliant. They told me that some customers also get their sites tested and are in turn PCI compliant.

Is Mass Hosting PCI Compliant? (Biting the hand that feeds me)

If anybody actually pays attention to what I have to say, there's a good chance that this post is going to make a lot of people awfully mad. It needs to be said.

Most web sites built by my company are mass-hosted on cPanel servers. We're not the only company around that does this- there are many more. On the internet, there are hundreds of thousands of mass hosting servers, with hundreds of users on each server. On our systems, about 1/3 of those accounts have some kind of ecommerce software. I don't know whether that fits with industry averages, but it paints a pretty interesting picture.

The internet has a massive number of ecommerce web sites, all processing credit card transactions, hosted on servers with at least a few hundred other users. Let's presume one thing for the sake of this post: breach of one account can lead to compromising another account through local attacks (world-readable configuration files, local file inclusion, and a whole host of others that I haven't publicly disclosed yet) as well as web-based attacks (see my previous post on server and client attacks via mod_userdir). I'll be posting a lot more ways to attack shared hosting in the future, but humor me and take it for granted if you must.

I've seen some debate as to whether PCI standards apply to ecommerce apps that outsource the payment processing to a third party like PayPal or Authorize.net, but I honestly don't think it should matter. First off, any site that handles CC data, even if they are not stored on the system, can be abused. If I can break into your Zen Cart installation and 'customize' your payment module to email those numbers to me, it doesn't matter who does the payment processing. If you have a payment form, the PCI standards should apply to you.

But here's the thing: every one of my company's servers gets scanned for PCI compliance. Every one of them passes, and I've got the certificate to prove it. When you hit a cPanel server directly (via IP address), only one website will come up, and it is usually a default cPanel page, which is pretty minimal. Each of these servers has hundreds more accounts that don't even get touched by McAfee's web app scanner.

Finding XSS holes on at least one of those accounts is practically guaranteed. In fact, when I first started in my current position, I was able to gain shell access on any given server in less than 5 minutes. I've since been able to make incredible progress with mass patchings and my own vulnerability research, but there are enough unknown issues and custom code that I'll never be able to fix them all.

So why are these servers being certified as PCI compliant?

Now, my company don't actually advertise that our shared servers are PCI compliant, but we do test them as part of our own merchant agreements. In addition, we've greatly improved our processes and sales practices to make sure that new code is reviewed and apps with sensitive data go on VPSes, but that does nothing for all the old clients, and even less for the millions of other sites out there.

The main point here, of course, is that these servers are consistently tested as PCI compliant. This isn't really a attack on McAfee. Many of our clients use other ASVs for their own sites, and none have a problem with the fact that these sites are mass-hosted, nor the fact that other sites on the same server are not PCI compliant (though if they tried to scan the other sites on the server, I'd put a stop to it pretty quickly, for obvious reasons).

Can somebody with more background in PCI weigh in on this? Is this just a massive oversight on the part of every ASV out there? When my company gets big enough that they need a PCI QSA audit rather than the ineffective ASV scans, are we going to be have to rebuild all of our customers' websites? Are we going to be completely screwed compliance-wise?

cPanel Root CSRF: Round 3

Readers who have been with me from the beginning (as well as the guy that read my entire site yesterday) will recall that last fall, I posted several XSS and CSRF vulnerabilities with the cPanel WHM interface. Essentially, these vulnerabilities would allow me, as an attacker, to root a cPanel server simply by having an authenticated administrator browse my website. At the time, the vendor did not respond to my vulnerability reports, and the problems went unfixed until I resubmitted them to Secunia in February.

cPanel finally took action, fixing the XSS hole that I had found. However, the CSRF issues (which were much more critical in this case) were ignored. Their response to these issues was that CSRF can be prevented using the "XSRF Protection" feature in the WHM administration interface. I double checked and sure enough, there was such a feature. It was obscure, disabled by default, nowhere to be found on the cPanel documentation pages and FAQ (which ironically has its own XSS holes), and had verbage next to it that would discourage most administrators from enabling the feature. Essentially, cPanel treats these critical CSRF holes as a feature:



I didn't have all that much time to look into it until today, when I decided to actually test this feature. Digging into the code, the first thing I noticed was that all the "XSRF Protection" does is check the referrer that your browser sent- if it doesn't match one of the known domains for the server, the requested action is not allowed to take place.

The first way around this was obvious. If your browser does not send any referrer, the XSRF protection does not kick in. Referrers can be suppressed several ways, but the first one that comes to mind for me is SSL. Many browsers will not send referrers when coming from an SSL page. This morning, I wrote a proof of concept that bounces the user off an SSL-encrypted open redirect (there's plenty of them out there). A few tests with the latest version of cPanel confirmed that Firefox 3 does not send a referrer, and the XSRF Protection feature had no effect.

Assuming that the browser did send referrers, there are still plenty of ways to bypass it. The next thing that comes to mind is simply using a domain on the whitelist to execute the attack. With a few hundred users on most cPanel servers, finding an open redirect, file upload, XSS, or cross-site framing hole is usually trivial. If you combine that with the mod_userdir attacks I posted last week, you can increase the effectiveness of this attack even further.

The next idea I had was to add my domain to the list of allowed domains. It requires access to a local account on the server (legitimately or not), but any user can add extra domains and subdomains to his site- subdomains which are considered "allowed" by cPanel. This provides yet another effective way to avoid the XSRF protection feature.

The back end code for the cPanel interface is an absolute mess- it's amazing that it works as well as it does. Not just the WHM interface, but the entire package is vulnerable to CSRF and XSS attacks at nearly every level. This software manages hundreds of thousands of websites.

This is bad.

But preventing CSRF would break your third party billing software, so we should probably allow it by default. That's a great idea.

Edit: cPanel did, in fact, mention the XSRF Protection feature on their blog about a year ago. This post calls the CSRF issues "security issues, which range in severity from trivial to medium-critical"

cPanel, mod_userdir, and Shared Hosting

I know I've made my views on shared hosting clear in the past, but I keep coming up with more reasons to not use it.

To begin with, let me start by saying that yes, I know that mod_userdir isn't recommended by its creators, cPanel, or Apache for secure applications. I may be grabbing at low-hanging fruit here, but that doesn't change the fact that its use is hugely popular for mass-hosted webservers. It is a convenient way to get a lot of sites onto a server, to get valid SSL certificates for each, and for accessing/managing content on all of them.

With that out of the way...

For those that aren't in the know, mod_userdir is an Apache module that allows you to create a separate website for each user on a server. These sites can all be accessed by going to http://servername.com/~username. The module is installed on most Apache setups by default, but isn't necessarily always enabled. Most mass-hosted cPanel servers make extensive use of it for a variety of reasons.

I should note that in the following examples, 'victim' is the username and 'victim.com' the domain of the targeted web site. 'attacker' and 'attacker.com', respectively, are used to compromise victim, but do not have to be run by the attacker- they can simply be accounts with poor log management, PHP or XSS holes, depending on the exploit. With hundreds of sites on a server, finding such a site is usually trivial.

cPanel's default Apache configuration has a handful of flaws in it that make mod_userdir suprisingly useful to the wily attacker. To begin with, the attacker can attack one site, but have the logs deposited in another's log directory. To do this, hit the server through the domain of the site that you want to logs to show up on, but use the username of the target site:

http://attacker.com/~victim/admin.php

With this url format, we can scan, probe, and bruteforce logins without the target site hearing a peep from his apache logs. Even if he does suspect foul play, he will have a hard time getting the logs, as (theoretically, at least) both the server's administrator and the attacker's site will be rather resistant to turning over their own site's logs without a court order.

If you can't find a decent user account on the server, you can always use the server's IP address instead of a domain and run the scripts as apache rather than attacker. This way, only root will be able to access the logs, and surprisingly few administrators ever review the global apache logs.

The one caveat here is that cPanel does allow you to set up a "default" account which may end up catching the logs, but if you go directly to the server's address and see something like this, you have a winner.

Due to poor default Apache configuration, directory listings on /cgi-bin are enabled when you access it in similar fashion:

http://victim.com/~victim/cgi-bin

This alone has been enough for me to crack accounts in past penetration tests. While it has been fixed recently, past versions of cPanel would also print out cgi scripts' contents rather than executing them. Still, included files without a .cgi, .pl, .plx, .ppl or .perl extension can be easily located and downloaded with this method.

By going to http://attacker.com/~victim, we can execute PHP scripts from the user "victim", with the server thinking that we're "attacker". This can be handy for a few things. First off, printing out $_SERVER['DOCUMENT_ROOT'] tells me that I'm running in /usr/home/attacker/. By contrast, the server knows that $_SCRIPT['FILENAME'] is /usr/home/victim/public_html/script.php. The script is executed with victim's privileges, so you can't read or write anything that the user doesn't normally have access to, but if victim is including files or locating resources based on the $_SERVER['DOCUMENT_ROOT'] variable, we can manipulate it, forcing the application to load your own PHP code.

Let's say you don't even want to bother with PHP. If you can find a single XSS hole on any user's site on the server (and you can, I promise), you just have to get people to hit http://victim.com/~attacker/XSSedPage.php.

Let me repeat that- You can XSS every domain on an entire server. Many of these servers contain hundreds of users. In cases where an entire server is owned by one web development company, that server's php applications are usually homogenous. The implications of this are huge. While many are certified as such, no site on a cPanel server should be considered PCI compliant. Many people downplay the importance of XSS exploits, because they have to be individually found and have custom exploit code written. With mod_userdir and a bit of fancy coding, an attacker can compromise hundreds of accounts, all performing hundreds of ecommerce transactions per day.

That's just good economics.