[TriLUG] Possibly compromised Apache binary on my webserver

[Brian]

When setting up a server I recommend looking at programs like OSSEC which
monitor the integrity of your system binaries and notify you if they change.


To find out how the binary got on your system, be sure to check your failed
logon logs to see if someone was trying to break into your system via ssh.
Also, check the current Wordpress vulnerabilities and try to assess if you
could have been victimized by one of them. If you have any plugins that
allow users to upload stuff to your server, look at what shell commands have
been run recently- I once saw a server hacked because someone uploaded a
malicious .php file with shell commands and managed to call it from their
browser. The attacker didn't clean up after themselves very well, so we were
able to see what they did.

-Brian


On Thu, Jun 11, 2009 at 10:33 PM, Mark Turner <jmarkturner at gmail.com> wrote:

> Hi folks! It looks like I found a compromised Apache binary on my
> Xen-hosted webserver tonight. Thought I'd share my info in case anyone else
> runs into this.
>
> I began getting suspicious about my Wordpress's xmlrpc.php install when I
> saw so many requests for it in the Apache logfiles. I upgraded to WP 2.8
> today and figured I was safe. And the WP folks seemed pretty confident that
> xmlrpc.php was fine.
>
> This afternoon I decided to delete xmlrpc.php anyway, as I wanted to take
> no chances. That's when I noticed that Apache was still quite happy to serve
> up xmlrpc.php - even though IT NO LONGER EXISTED!!! I cleared my browser
> cache multiple times to make sure I wasn't seeing things but that's what
> happened.
>
> My dodgy Apache even served up xmlrpc.php from a vanilla install of WP 2.8
> in /var/www/wordpress (having deleted the xmlrpc.php from that dir). All I
> could figure was that Apache was somehow compromised.
>
> A reinstall of Apache magically fixed everything. Suddenly I was no longer
> getting a ghost xmlrpc.php. Unfortunately, I no longer have the compromised
> binary for studying (it's in a backup I'm sure).
>
> I'm thinking that bogus Apache looked at the source of the xmlrpc.php
> request and depending on the IP or user agent, the contents of the returned
> xmlrpc.php was either the good file or a haXOR file. And I've always
> wondered why a few Apache binaries sometimes refused to shut down when I ran
> the init script.
>
> I suggest that if you want to test your own servers:
>
> 1. Delete or rename your xmlrpc.php.
> 2. Clear your browser cache
> 3. See if Apache still sends xmlrpc.php to you. ("wget
> http://www.somesite.com/xmlrpc.php" should suffice)
>
> Now the question is how that binary got there. It could mean the Xen server
> I'm using is compromised. That's my guess at this point and I don't have a
> lot of trust in it. I have a solution for that but no time to implement it
> just yet.
>
> In addition, I'm going back to Red Hat / CentOS for my servers as the rpm
> -qV command is far too handy not to have in a case like this. dpkg should
> have a way of testing a package contents versus what's installed but it does
> not. Pity.
>
> --
> Mark Turner
> www.markturner.net
> --
> TriLUG mailing list        : http://www.trilug.org/mailman/listinfo/trilug
> TriLUG FAQ  : http://www.trilug.org/wiki/Frequently_Asked_Questions
>