[Box-Admins] [Board] root cause of squeaksource.com outage

[Chris Muller]

Given that we've already restored squeaksource.com to the month-old
domain, we now have a fork in the domain.  Unless you're planning to
try to merge them, I think attempting to recover the one from
squeaksource.5 might be moot at this point..

On Sun, Nov 6, 2016 at 9:32 PM, David T. Lewis <lewis at mail.msen.com> wrote:
> On Sun, Nov 06, 2016 at 07:43:23PM -0600, Chris Muller wrote:
>> I think I just stumbled onto the root cause of the image growth.  I
>> had downloaded the new production squeaksource.6.image (copy of the
>> Oct-5 image) to my laptop to get a "data.obj" file exported.  This
>> export process runs in the background and takes a long time, so when I
>> came back to check on it, I found two debuggers with the message "Port
>> not available."
>>
>> What's happening is when the image is saved, it gets most of the way
>> through SmalltalkImage>>snapshot:andQuit:embedded: so that the file is
>> saved on disk, however, at the end it calls #processStartupList:
>> whereupon SSRepository class>>#startUp: tells SSKom to start up.
>> SSKom produces a debugger if you tell it to start when its already
>> started.
>
> I think I have seen that debugger notification before. That sort of thing
> may happen occasionally, but not in any way that would accumulate into
> an out of memory condition.
>
> Whatever happened yesterday is different from the usual operation that
> we have seen on squeaksource.com and source.squeak.org in recent years.
> It is not just an error notification, it is some kind of failure that
> improperly leads to an out of memory condition.
>
> It would be helpful if we could get an export of the 'data.obj' from
> squeaksource.5, which is the one that would have up-to-date account
> information for the repository. The squeaksource.6 and squeaksource.4
> images are one month out of date, and are not helpful in that regard.
>
> Unfortunately squeaksource.5 is broken, and I don't know how to rescue
> data.obj from that image.  Maybe Tim can get at it from a Cog simulator
> image.
>
> Dave
>
>
>>
>> So the debuggers from the prior image save are slowly piling up within
>> each subsequent image save, a slowly growing until it reached the
>> limit.
>>
>> I guess the bug should still be there; I guess I'm puzzled why this
>> didn't happen on box2..
>>
>> I think we should work on moving squeaksource to the new image ASAP.