[squeak-dev] Re: Build.squeak.org and squeaksource.com in danger (was Re: [Box-Admins] Disk space usage on box3)

Frank Shearar frank.shearar at gmail.com
Mon Dec 9 10:22:36 UTC 2013


On 8 December 2013 22:20, Chris Muller <ma.chris.m at gmail.com> wrote:
> On Sun, Dec 8, 2013 at 3:02 PM, Frank Shearar <frank.shearar at gmail.com> wrote:
>> On 8 December 2013 20:57, Chris Muller <ma.chris.m at gmail.com> wrote:
>>>> That's very cool, but it's not the zips we're talking about. We're
>>>> talking about the artifacts produced by Jenkins, which are Squeak
>>>> images. I wrote the release process to produce a series of artifacts
>>>> with different names. Because nothing gets replaced/overwritten, disk
>>>> usage is unbound.
>>>
>>> The rate of untamed growth is too fast.
>>
>> I have no idea what you mean by that. I don't consider ~30MB of new
>> data to be "untamed growth", nor do I consider the accumulation of a
>> mere 6 GB over, what, 600 builds? to be "too fast".
>
> Ken should not have to keep reminding us about space.  We don't need
> to keep 600 builds, and I respect Ken's concern about proactive
> management.

No, Ken absolutely should _not_ have to keep reminding us about
anything. But you are not suggesting a solution. The solution is this:
more people should be doing more to keep an eye on the box.

>>>>> The result is that the redundant Magma-backed copy of
>>>>> source.squeak.org consumes less than 1/4th the space of the original
>>>>> File based version.  A Magma-backed copy of squeaksource.com would
>>>>> about 12GB of space.
>>>>>
>>>>> For now, though, Frank has the ability and responsibility to trim up
>>>>> the jenkins stuff.  Thanks Frank.
>>>>
>>>> I nuked the ReleaseSqueakTrunk target directory, freeing up 5.9 GB of
>>>> space. (We should always consider the target/ directory of every job
>>>> as being evanescent. Feel free to nuke these at any time. Every job
>>>> should be written such that it can reconstitute its working
>>>> environment in target/.
>>>
>>> Automating something (builds) shouldn't introduce a new manual
>>> process.  There should be something to keep the size reasonable so we
>>> don't have to remember do it manually every month..
>>
>> Er. I explicitly decided to produce versioned artifacts that a human
>> would assess for quality. I stand by that decision. You _should_ be
>> able to easily go and say "Well, version N is crap, but N - 2 is just
>> fine. Let's just go with that."
>
> Of course, so may we discuss something somewhere _in-between_ 600 and
> none?  Are the last 10 builds enough?  50?

I am more than happy to hear suggestions on how to do that. One
suggestion is this: someone periodically takes the latest
ReleaseSqueakTrunk artefact, tries it out, blesses it by pushing it to
ftp.squeak.org, and then tells us. Some kind soul throws away the old
potential release candidates.

>> What's crazy is that we don't have graphs that we can use to easily
>> track disk usage, nor alerting mechanisms other than Ken to warn of
>> impending doom. Munin and Icinga would do fine, with Apache/nginx to
>> serve up status pages. If only someone with sufficient time had the
>> energy to put in the graft necessary to set up these standard sysadmin
>> tools.
>
> Space monitoring and alerts are required for any 24/7 system
> irregardless of the level of sustainability of processes running on
> the machine.  They're both important, but since we don't have alerts,
> sustainability is all the more important.

We don't have a 24/7 system, because those require operational staff.

Seriously, the big problem here is not that ReleaseSqueakTrunk did
what I told it to. The big problem here is that no one except Ken
cares enough about box3 to monitor it.

frank

> You're now sharing box3 with SqueakSource.  The best approach is for
> everyone with access to the box to keep their resource impact
> reasonable enough that it doesn't affect service or require alert
> notices from others.
>
> Thanks again.


More information about the Squeak-dev mailing list