[Box-Admins] [FWD: ** PROBLEM Service Alert: squeak box2/Squeak website is CRITICAL **]

[Janko Mivšek]

Hi Ken,

I checked too and see no website process active, so I restarted it and
now I'm connected with VNC to the image.

Today seems that image crashed, but snapshoted correctly at 9pm GMT last
time.

Can we see some vm dump somewhere?

Best regards
Janko

On 28. 02. 2011 23:47, Ken Causey wrote:
> After I received this notice I checked and the website process had the
> CPU pegged with normal memory usage.  I tried to connect with VNC and
> got connected but the image was locked up.  There was a debugger open
> and I took a screenshot which can be found at
> 
> http://users.squeak.org/~kencausey/website_locked.png
> 
> I chatted in the IRC channel as I was fiddling with it:
> 
> 2011-02-28 16:24:21     kencausey       JankoMivsek: website process is
> flipping out again
> 2011-02-28 16:26:14     kencausey       the memory usage is normal this
> time, it just has the CPU pegged
> 2011-02-28 16:26:41     kencausey       looking at the logs, the last
> successful hit was the nagios check oddly enough, 2 hits before google
> hit the stats page again
> 2011-02-28 16:26:56     kencausey       I don't see anything suspicious
> like the last time
> 2011-02-28 16:29:41     kencausey       there is a debugger open on a
> send of #bottomContext to UndefinedObject
> 2011-02-28 16:29:50     kencausey       I can't interact with it
> 2011-02-28 16:30:54     kencausey       it's in a call to
> Process>>terminate
> 2011-02-28 16:32:38     kencausey       restarting it now
> 2011-02-28 16:34:09     kencausey       website is back up
> 
>>From the apache logs:
> 
> this is when it went down:
> 
> 80.81.242.100 - - [28/Feb/2011:21:41:03 +0000] "GET
> /stats.html?view=main&year=1684&month=8 HTTP/1.1" 200 24288 "-"
> "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.
> google.com/bot.html)"
> 173.255.225.4 - - [28/Feb/2011:21:42:13 +0000] "GET /favicon.ico
> HTTP/1.1" 200 1406 "-" "Safari/6533.19.4 CFNetwork/454.11.5
> Darwin/10.5.0 (i386) (MacBook3%2C1)"
> 89.212.16.244 - - [28/Feb/2011:21:42:18 +0000] "GET /ping.html HTTP/1.1"
> 200 - "-" "check_http/v1.4.14 (nagios-plugins 1.4.14)"
> 38.99.97.225 - - [28/Feb/2011:21:42:49 +0000] "GET /Smalltalk/ HTTP/1.1"
> 502 399 "-" "Mozilla/5.0 (compatible; ScoutJet;
> +http://www.scoutjet.com/)"
> 67.195.112.235 - - [28/Feb/2011:21:43:15 +0000] "GET
> /Merchandise/?version=3 HTTP/1.0" 502 403 "-" "Mozilla/5.0 (compatible;
> Yahoo! Slurp; http://help.yahoo.com/help/us/
> ysearch/slurp)"
> 
> We have a googlebot hit to the stats page (relevant?), an irrelevant
> favicon request, a successful nagios ping which is I assume Janko's and
> not relevant, then hits start failing.  Before that I see nothing
> suspicious and no flood of requests.
> 
> Ken
> 
>> -------- Original Message --------
>> Subject: ** PROBLEM Service Alert: squeak box2/Squeak website is
>> CRITICAL **
>> From: nagios at mivsek.eranova.si (User for Nagios)
>> Date: Mon, February 28, 2011 3:49 pm
>> To: ken at kencausey.com
>>
>>
>> ***** Nagios *****
>>
>> Notification Type: PROBLEM
>>
>> Service: Squeak website
>> Host: squeak box2
>> Address: 85.10.195.197
>> State: CRITICAL
>>
>> Date/Time: Mon Feb 28 22:49:47 CET 2011
>>
>> Additional Info:
>>
>> CRITICAL - Socket timeout after 10 seconds
> 
> 

-- 
Janko Mivšek
Svetovalec za informatiko
Eranova d.o.o.
Ljubljana, Slovenija
www.eranova.si
tel:  01 514 22 55
faks: 01 514 22 56
gsm: 031 674 565