> -------- Original Message --------
> Subject: Re: [Box-Admins] Did www.squeak.org crash?
> From: Levente Uzonyi <leves(a)elte.hu>
> Date: Fri, September 13, 2013 6:22 pm
> To: Squeak Hosting Support <box-admins(a)lists.squeakfoundation.org>
> Cc: "squeak-dev(a)lists.squeakfoundation.org developers list"
> <squeak-dev(a)lists.squeakfoundation.org>
>
>
> My guess is that the server was restarted, or it ran out of memory a few
> times, because all images seem to be down.
> There used to be some sort of monitoring system (nagios maybe). I'm not
> sure if it's still working or not, but if it's not, then we can try using
> Hetzner's simple system monitoring sevice:
> http://wiki.hetzner.de/index.php/System_Monitor_%28SysMon%29/en
>
>
> Levente
>
> On Fri, 13 Sep 2013, Bert Freudenberg wrote:
>
> >
> > - Bert -
No, actually I rebooted the system which would inevitably cause
everything to stop working for a period of time.
I decided to take this step after having www.squeak.org go down again.
But, perhaps unsurprisingly, that didn't actually help. It turns out
that I in fact made a mistake and copied the files from the backup as
root (sudo bash, old-fashionedness and typing laziness) and failed to
change the ownership to the website account/group. Once I fixed this
and restarted it is up again, I believe everything is up. Of course
that may change so I will continue to check periodically for a while.
Ken
> -------- Original Message --------
> Subject: RE: [Box-Admins] Did www.squeak.org crash?
> From: "Ken Causey" <ken(a)kencausey.com>
> Date: Fri, September 13, 2013 3:45 pm
> To: "Squeak Hosting Support" <box-admins(a)lists.squeakfoundation.org>
> Cc: webteam(a)lists.squeakfoundation.org,
> "squeak-dev(a)lists.squeakfoundation.org developers list"
> <squeak-dev(a)lists.squeakfoundation.org>
>
>
> > -------- Original Message --------
> > Subject: [Box-Admins] Did www.squeak.org crash?
> > From: Bert Freudenberg <bert(a)freudenbergs.de>
> > Date: Fri, September 13, 2013 2:16 pm
> > To: Squeak Hosting Support <box-admins(a)lists.squeakfoundation.org>
> > Cc: "squeak-dev(a)lists.squeakfoundation.org developers list"
> > <squeak-dev(a)lists.squeakfoundation.org>
> >
> >
> > - Bert -
>
> Something like that. I tried to simply restart it but that didn't work.
> I've tried restoring a backup from a day ago and that doesn't seem to
> be working. I will continue to try b ackups but at a glance all the
> image files look larger than they should be which in the past has not
> been a good sign in terms of a working image. This may require
> assistance from someone with knowledge of the image/AIDA. But I will
> report back after I have tried all the backups or it is up again.
>
> Ken
I finally succeeded on the 4th or 5th attempt. But frankly I am
confused.
-rw-r--r-- 1 website website 10085010 Sep 13 09:00
squeaksite-badq.changes
-rw-r--r-- 1 website website 100529268 Sep 13 09:00
squeaksite-badq.image
These are copies of the image/changes (i/c for short) that I found
initially when the site was not responding.
-rw-r--r-- 1 website website 10085010 Sep 13 09:00
/var/cache/rsnapshot/daily.0/localhost/home/website/website/squeaksite.changes
-rw-r--r-- 1 website website 100529268 Sep 13 09:00
/var/cache/rsnapshot/daily.0/localhost/home/website/website/squeaksite.image
The above are the most recent backup i/c. I never even tried them since
they appeared to be direct copies of what obviously wasn't working.
-rw-r--r-- 1 website website 10081707 Sep 12 17:00
/var/cache/rsnapshot/daily.1/localhost/home/website/website/squeaksite.changes
-rw-r--r-- 6 website website 93703960 Aug 13 05:00
/var/cache/rsnapshot/daily.1/localhost/home/website/website/squeaksite.image
-rw-r--r-- 1 website website 10076943 Sep 11 16:00
/var/cache/rsnapshot/daily.2/localhost/home/website/website/squeaksite.changes
-rw-r--r-- 6 website website 93703960 Aug 13 05:00
/var/cache/rsnapshot/daily.2/localhost/home/website/website/squeaksite.image
-rw-r--r-- 1 website website 10072081 Sep 10 15:00
/var/cache/rsnapshot/daily.3/localhost/home/website/website/squeaksite.changes
-rw-r--r-- 6 website website 93703960 Aug 13 05:00
/var/cache/rsnapshot/daily.3/localhost/home/website/website/squeaksite.image
-rw-r--r-- 1 website website 10067622 Sep 9 15:00
/var/cache/rsnapshot/daily.4/localhost/home/website/website/squeaksite.changes
-rw-r--r-- 6 website website 93703960 Aug 13 05:00
/var/cache/rsnapshot/daily.4/localhost/home/website/website/squeaksite.image
-rw-r--r-- 1 website website 10062810 Sep 8 15:00
/var/cache/rsnapshot/daily.5/localhost/home/website/website/squeaksite.changes
-rw-r--r-- 6 website website 93703960 Aug 13 05:00
/var/cache/rsnapshot/daily.5/localhost/home/website/website/squeaksite.image
-rw-r--r-- 1 website website 10058287 Sep 7 15:00
/var/cache/rsnapshot/daily.6/localhost/home/website/website/squeaksite.changes
-rw-r--r-- 6 website website 93703960 Aug 13 05:00
/var/cache/rsnapshot/daily.6/localhost/home/website/website/squeaksite.image
And above are the other local backups we have, most recent to oldest. I
tried each of these in turn starting from daily.1. I waited at least 2+
minutes after restarting the image in each case. The one where it
finally worked and I stopped is daily.5. Now I didn't notice this until
after it was working and I looked more closely but the images in
daily.1-daily.6 all have the same size and date, in fact that date is a
month ago. I don't know how the image save works in the squeak.org
image, perhaps it was intelligent enough to note that there had been no
changes and so there had in fact been no need to save since Aug 13?
Why did the 5th one work? I don't know. I don't believe the image has
any external dependencies. Could it somehow depend on the changes file?
Any other explanation I can come up with suggests at minimum it is time
to reboot the system as a whole or that we have other more serious
problems. Everything else seems to be working for now, even
www.squeak.org is working. I'm just going to leave it alone for the
time being.
Ken
P.S. Note the number 6 in the second column of each of those
daily.1-daily.5 .image backups? That number indicates the links to the
file data, the number of paths in the directory table that all point to
the same file data. These 6 files are in fact actually just 6 directory
entries pointing to the same data (they daily.6 one is the 'original').
This is exactly how our backup system (rsnapshot/rsync) is supposed to
work. If a file has not changed from one backup to the next a hard link
is created to the file rather than copy the file. This is actually done
using the venerable cp command (see the -l option in the man page or
other documentation). Anyway the point is that when the each backup
took place, it was rsync's opinion that compared to the previous backup,
the image file was unchanged, in which case it did not copy the file
over the previously created hard link.
> -------- Original Message --------
> Subject: [Box-Admins] Did www.squeak.org crash?
> From: Bert Freudenberg <bert(a)freudenbergs.de>
> Date: Fri, September 13, 2013 2:16 pm
> To: Squeak Hosting Support <box-admins(a)lists.squeakfoundation.org>
> Cc: "squeak-dev(a)lists.squeakfoundation.org developers list"
> <squeak-dev(a)lists.squeakfoundation.org>
>
>
> - Bert -
Something like that. I tried to simply restart it but that didn't work.
I've tried restoring a backup from a day ago and that doesn't seem to
be working. I will continue to try b ackups but at a glance all the
image files look larger than they should be which in the past has not
been a good sign in terms of a working image. This may require
assistance from someone with knowledge of the image/AIDA. But I will
report back after I have tried all the backups or it is up again.
Ken
