[Box-Admins] squeaksource.com is down (EOM)

[David T. Lewis]

On Thu, Feb 16, 2017 at 07:22:22PM -0500, David T. Lewis wrote:
> Thanks Tobias,
> 
> There is definitely something wrong with the image. It is growing in
> memory and currently using a lot of CPU. I will attempt an off-line
> fix, but brief outages are possible over the next hour or two.
> 
> The image normally saves itself every hour, and it looks like this last
> happened successfully three days ago:
> 
> squeaksourcecom at dan:~/SqueakSource$ ls -lt squeaksource.6.*
> -rw-r--r-- 1 squeaksourcecom www-data  16317608 Feb 16 14:13 squeaksource.6.changes
> -rw-r--r-- 1 squeaksourcecom www-data 814463464 Feb 14 08:33 squeaksource.6.image
> 
> That means that today's restart would have begun with the Feb 14 copy of
> the image, so there may be some data loss related to this (I'll check it
> later).
> 


I fixed the image off line and restarted it. Memory and CPU are back to normal.
Notes added to dan.box.squeak.org:~squeaksourcecom/README:


-----------------
Fri Feb 17 00:55:29 UTC 2017 dtl
Now running squeaksource.7.image from the run script. There is some sort of problem
that is occasionally running the image out of memory. It appears to be an error
handler in Seaside that tries to send an email via smtp to mail.squeak.org, but is
unable to connect to smtp for some reason. The memory usage is all related to that
single process. Although the image runs out of memory on this server, I am able
to download and run it locally on another machine, terminate the process, and save
the image for upload back to dan.box.squeak.org.

The download-and-fix procedure is needed because of a separate and unrelated issue.
The VNC server in image is not working when running on dan.box.squeak.org. I am
not sure, but this may be related to name server lookup of localhost on current
Linux distros, which is a bug that was fixed in less ancient Squeak images. It is
probably time to update the image a bit, at least to Squeak 4.6. Note, we should
move to 64-bit Spur but I would prefer to wait for source.squeak.org to do that
first.

Summary - healthy for now, but we need to check about once a week or so to make
sure the memory size (size of squeaksource.7.image on disk) is not getting out
of hand.
-----------------

Dave