Well, it turns out that there was really something to this:
Yesterday I kept looking and I found a connection from our system to the IRC server deathy mentioned in a seperate email. The process making the connection was a bash process, and this is all I could tell from the output of ps. I killed it, the connection went away. This was fairly late in the day so I more or less left it at that.
I checked again this morning and found the process back again. I killed it again, but this time I checked back every few minutes. It took only a few minutes and it reappeared. Huh. crontab -l (for root) showed why. I commented out the crontab entry and killed the process again. As of now it has not reappeared. I left the commented out crontab entry and it points to stuff in /usr/local/games/ which I have left for now. I'm examining it and I welcome examination from others. At the moment it looks like this stuff was installed around Oct 31st, but that's just a quick guess. I don't know much yet, I don't even know if there is other stuff we should be looking for.
Relatedly, when we had trouble with the server in October we temporarily set a rather easy root password. I meant to change it and let everyone relevant know, I thought I did. But I can't find a record of doing so. Can anyone confirm that we changed it or not? In any case perhaps we should change it again.
Ken
On Tue, 2009-02-03 at 09:42 +0100, Marcus Denker wrote:
Hi,
There is a complaint from undernet about our server.
-------- Original-Nachricht -------- Betreff: [REF#: 1257]: To whom it may concern Datum: Mon, 02 Feb 2009 19:59:03 +0000 Von: deathy@undernet.org Antwort an: deathy@undernet.org An: abuse@hetzner.de
Security coordinators,
I found these suspicious looking connections on the Undernet IRC Chat Network connecting from a netblock you control. The originating ip(s) and undernet server(s) each one was connected to is listed below. The destination port they were using is most likely port 6667. Other possible ports are included between 6000-9999 (a full list of our servers can be found at www.undernet.org/servers.php ).
box2!~box@box2.squeakfoundation.org [85.10.195.197] - DIEMEN.NL.EU
Please check for a compromise, possible hidden process running and an altered process listing. Run the updates for your system to close possible exploit holes, and send any unusual programs found to info@cyberabuse.org for investigation.
We strive to eliminate these abusive connections from our network, but simply banning them can only be a temporary solution. We hope to work with authorities to achieve our aim of reducing abuse on our network, as well as the general internet community.
If you are not familiar with it, IRC is a text based chat communication medium, details at:
and our webpage:
www.undernet.org
Time of capture for the affected IP(s) is: Mon, 02 Feb 2009 19:44:05 +0000
We have assigned an internal reference number 1257 to this report and it is included in the subject line of this e-mail message. We would appreciate your including it in the subject line of future correspondence about this report. We would really appreciate your cooperation in looking into this matter.
Please take into account that most bots used these days are either GTbots (used on Windows and which can be found by searching for a file named mirc.ini which is normally required to run these bots) or emechs (used on linux/unix which can be generally found easily by doing a: find . -exec grep -l "undernet.org" {} + )
Thank you for your cooperation.
Regards,
Caesar Stoica
Undernet Irc Operator www.undernet.org
-- Marcus Denker -- denker@iam.unibe.ch http://www.iam.unibe.ch/~denker