[Box-Admins] About the server downtime today

[Levente Uzonyi]

On Fri, 21 Oct 2011, Ken Causey wrote:

> What exactly was not working?  The only reference I can find is to a

None of squeak.org, source.squeak.org, map.squeak.org was accessible. The 
apache restart solved the issue, the stalled connections disappeared. 
Graceful restart didn't help.

> problem with source.squeak.org and it's unlikely that apache would
> need to be restarted in that case, simply the squeak process.  But
> then I don't know the details.  I found bugs.squeak.org working when I
> got on and in that case I just killed the php processes (running under
> only the mantis account) which allowed daemontools to restart it and
> all was good.

You mean bugs.squeak.org _not_ working?

>
> Ken
>
> P. S. I would appreciated it if we could improve the communication
> within this team.  Frankly I freaked out when I saw multiple root
> logins on the server.  I hope that is Levente...

It's me, I'm using screen which shows up as multiple logins.


Levente

>
> On Fri, Oct 21, 2011 at 10:13 AM, Levente Uzonyi <leves at elte.hu> wrote:
>> Hi,
>>
>> the webserver stopped responding today, while ping was working fine. I
>> contacted Bert on IRC who gave me access to the box. As he suggested I tried
>> restarting apache, which temporarily solved the problem, but in 10-20
>> minutes the server went down again. At this point there were several
>> connections to port 80 in SYN_RECV state, which is a sign of a possible SYN
>> flood attack. I did a few more apache restarts and I installed tcpdump in
>> order to find out the cause, but after checking quite a few (non-suspicious)
>> connections, I gave up. I enabled tcp_syncookies on the server which solved
>> the problem without further apache restarts.
>>
>> Cheers,
>> Levente
>>
>>
>
>
>
> -- 
> http://kencausey.com/
> Custom Software Solutions and Consulting
> Leveraging the Power and Flexibility of Open Source and Free Software
>