[Webteam] Re: [Box-Admins] RE: [FWD: ** PROBLEM Service Alert: squeak box2/Squeak website is CRITICAL **]

[Ken Causey]

OK, so let's examine the possibility this was the result of a DOS of
some kind (intentional or unintentional).  

First let's work out the timeline a bit:

The Nagios notice was sent out at "Sat Feb 26 11:24:15 CET 2011" or
10:24:15 GMT Saturday.

I'm a little less certain when I restarted the image.  I forwarded the
error notice and commented on the state of things at about 10:39.  I
replied that I had restarted the image at about 10:46.

Looking at the apache logs for www.squeak.org:

207.46.195.239 - - [26/Feb/2011:10:20:53 +0000] "GET /robots.txt
HTTP/1.1" 404 149 "-" "Mozilla/5.0 (compatible; bingbot/2.0;
+http://www.bing.com/bingbot.htm)"

80.81.242.100 - - [26/Feb/2011:10:21:49 +0000] "GET
/stats.html?view=main&year=1702&month=8 HTTP/1.1" 502 399 "-"
"Mozilla/5.0
 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

at 10:20:53 We have Bing requesting a robots.txt which we don't have. 
The next line is the next event:  At 10:21:49 we have Google (it
appears) surfing the stats.html page and the request failing.  After
this it is all 502 responses until

173.192.238.44 - - [26/Feb/2011:10:43:30 +0000] "GET / HTTP/1.1" 502 232
"-" "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9
.0.19; aggregator:Spinn3r (Spinn3r 3.1); http://spinn3r.com/robot)
Gecko/2010040121 Firefox/3.0.19"

87.59.72.91 - - [26/Feb/2011:10:43:41 +0000] "GET
/Documentation/Installation/ HTTP/1.1" 200 13209
"http://www.seaside.st/down
load" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; en-us)
AppleWebKit/533.19.4 (KHTML, like Gecko) Version/5.0.3 Safari/
533.19.4"

The second at 10:43:41 being the first successful response after I
restarted the image.  

I estimate there were about 50 hits between these two times, all
failures of course.  At a glance, before 10:20:53 there is no large
number of hits logged; in general several seconds to several minutes
goes by between each logged event.

One suspicious event shortly before:

84.227.142.227 - - [26/Feb/2011:10:20:22 +0000] "POST / HTTP/1.1" 413
329 "-" "Apache-HttpClient/4.1 (java 1.5)"
84.227.142.227 - - [26/Feb/2011:10:20:23 +0000] "POST / HTTP/1.1" 413
329 "-" "Apache-HttpClient/4.1 (java 1.5)"
84.227.142.227 - - [26/Feb/2011:10:20:23 +0000] "POST / HTTP/1.1" 413
329 "-" "Apache-HttpClient/4.1 (java 1.5)"
84.227.142.227 - - [26/Feb/2011:10:20:23 +0000] "POST / HTTP/1.1" 413
329 "-" "Apache-HttpClient/4.1 (java 1.5)"

After this there is

123.125.71.52 - - [26/Feb/2011:10:20:39 +0000] "GET
/Documentation/Installation/ HTTP/1.1" 200 13209 "-"
"Baiduspider+(+http:/
/www.baidu.com/search/spider.htm)"
207.46.195.239 - - [26/Feb/2011:10:20:53 +0000] "GET /robots.txt
HTTP/1.1" 404 149 "-" "Mozilla/5.0 (compatible; bingbot/2.0;
+http://www.bing.com/bingbot.htm)"

The second of these two being the same robots.txt request mentioned
before.

Other than this, again at a glance, nothing appears suspicious.  The
majority of the traffic is search bots but the requests do not come in
at any significant speed.

Ken

P. S. Casey: perhaps you could look into adding a basic robots.txt which
tells the bots to avoid the stats and other administrative stuff.

> -------- Original Message --------
> Subject: [Webteam] Re: [Box-Admins] RE: [FWD: ** PROBLEM Service Alert:
> squeak box2/Squeak website is CRITICAL **]
> From: Janko Mivšek <janko.mivsek at eranova.si>
> Date: Sat, February 26, 2011 12:08 pm
> To: Squeak Hosting Support <box-admins at lists.squeakfoundation.org>
> Cc: Squeak Webteam <webteam at lists.squeakfoundation.org>
> 
> 
> Hi guys,
> 
> This fast growing image problem could be cause because of Dos attack, So
> Sean, go looking there if there you'll see some enormous amount of
> requests from our site and specially, are they coming from there same
> IP. Knowing that IP we can narrower the culpit closer.
> 
> Past two image crashes were caused by image not snapshoting every hour.
> We switched snapshoting off a time ago and forgot to switch on, ok, now
> it is on again.
> 
> Best regards
> Janko