Since we had a break-in on pilot recently, I thought I would bring up a
couple of points.
(1) WHAT HAPPENED
First of all, it appears that what happened to pilot was that a
vulnerability in "RoundCube", a fancy web mail package, was exploited by
a script that installs a "bot" (part of a botnet).
As far as I can tell, no files or emails were damaged. Everything
appeared to be intact. It looks to me like it was just talking to a lot
of other machines via an IRC channel (and that's how we noticed it).
The bot was running as user 'www-data'. So technically, we were not
'rooted'... we were 'apache-ed'.
(2) KEYS AND PASSPHRASES
But since we have had a break-in, it makes me think of what damage
could have been done.
Personally, I was thinking about my SSH keys. On any semi-public
machine like pilot, I encrypt my SSH keys with a passphrase (see
"ssh-keygen -p"). So if someone were able to read my private key in
~porter/.ssh/id_rsa, all they would get was a load of DES-encrypted
bits. In this case, it's doubtful that they could have read the file,
since it has 700 perms.
But if an attacker had read these files, and if my key were not
encrypted, then now would be a good time to go onto all my other
accounts and make sure that my TriLUG SSH key was not listed in the
~/.ssh/authorized_keys files. This would keep one break-in from leading
to a series of break-ins. This is left as an exercise for the paranoid
As it stands, it looks like your SSH keys were never at risk. At least
not from the bot... remember that myself and the other sysadmins can
read these files. So the extreme paranoid users (you know who you are)
might want to look into SSH key passphrases.
I also wanted to make a note here that our unofficial policy on backups
is that users are responsible for backing up their home directories (and
now that your mail is stored in ~/Maildir, that means email, too). We
currently do not back up /home. Remember, we're a group of volunteers,
and we're doing "best effort" service. We try, but we're not
I am VERY happy that we did not lose anything in this latest incident.
In the meantime, I am making daily backups of everything EXCEPT /home.
And I am also entertaining the idea of putting a larger disk on dargo so
we can back up /home, too. Donations are gladly accepted.