On 1 April 2013 20:44, Eliot Miranda <eliot.miranda at gmail.com> wrote:
>
>
> On Sat, Mar 30, 2013 at 4:52 AM, Frank Shearar <frank.shearar at gmail.com>
> wrote:
>>
>> On 25 February 2013 18:51, David T. Lewis <lewis at mail.msen.com> wrote:
>> > On Mon, Feb 25, 2013 at 04:57:54PM +0000, Frank Shearar wrote:
>> >> On 25 February 2013 16:37, Bert Freudenberg <bert at freudenbergs.de>
>> >> wrote:
>> >> > On 2013-02-25, at 17:07, Frank Shearar <frank.shearar at gmail.com>
>> >> > wrote:
>> >> >
>> >> >> On 25 February 2013 15:35, David T. Lewis <lewis at mail.msen.com>
>> >> >> wrote:
>> >> >>> On Mon, Feb 25, 2013 at 03:01:18PM +0000, Frank Shearar wrote:
>> >> >>>> On 25 February 2013 14:49, Bert Freudenberg <bert at freudenbergs.de>
>> >> >>>> wrote:
>> >> >>>>>
>> >> >>>>> On 2013-02-25, at 15:33, Frank Shearar <frank.shearar at gmail.com>
>> >> >>>>> wrote:
>> >> >>>>>
>> >> >>>>>> On 14 February 2013 16:50, Frank Shearar
>> >> >>>>>> <frank.shearar at gmail.com> wrote:
>> >> >>>>>>> No, this takes the Trunk image as is and runs the job. No
>> >> >>>>>>> stripping
>> >> >>>>>>> has occurred. Noone's actually said "oh, that's a sane idea
>> >> >>>>>>> someone
>> >> >>>>>>> should do that" yet (except possibly for you, by virtue of not
>> >> >>>>>>> saying
>> >> >>>>>>> "are you crazy?" :) ), so I haven't done it yet.
>> >> >>>>>>>
>> >> >>>>>>> So the installation is essentially a no-op.
>> >> >>>>>>>
>> >> >>>>>>> With green lights on these builds, we can now verify that
>> >> >>>>>>> removing
>> >> >>>>>>> these packages won't break anything. At least, to the limit of
>> >> >>>>>>> the
>> >> >>>>>>> tests, which isn't much at all. So if we _do_ actually strip
>> >> >>>>>>> these
>> >> >>>>>>> guys out of Trunk, the builds should stay green. If they don't,
>> >> >>>>>>> we'll
>> >> >>>>>>> need to roll back the change and investigate.
>> >> >>>>>>>
>> >> >>>>>>> But I don't actually know how to strip them out of Trunk. I
>> >> >>>>>>> _suspect_
>> >> >>>>>>> it'll be via a Monticello config map of some description.
>> >> >>>>>>
>> >> >>>>>> Anyone? Bert? How do we strip packages out of Trunk?
>> >> >>>>>
>> >> >>>>> Manually once, when you prepare the Core image. Otherwise, if we
>> >> >>>>> put it in the trunk update stream, the packages would be unloaded from
>> >> >>>>> everyone's working images when updating.
>> >> >>>>
>> >> >>>> Given that we're early in 4.5's release cycle, if we do this kind
>> >> >>>> of
>> >> >>>> thing - stripping unnecessary things out of trunk - we ought to do
>> >> >>>> it
>> >> >>>> now rather than later. And announce loudly what we're doing, why,
>> >> >>>> and
>> >> >>>> how people can load in their favourite not-Core packages.
>> >> >>>
>> >> >>> NO.
>> >> >>>
>> >> >>> If Squeak developers work with images that contain just "their
>> >> >>> favourite
>> >> >>> non-Core packages" then none of the non-Core packages get
>> >> >>> maintained.
>> >> >>
>> >> >> Clearly we disagree quite a bit on this, but I don't think anyone
>> >> >> should actually _work_ on a Core image. _Noone_ should be able to
>> >> >> work
>> >> >> effectively in a minimal image because such a thing wouldn't even
>> >> >> have
>> >> >> a Browser!
>> >> >
>> >> > We don't disagree on this at all. People should be working in a full
>> >> > image.
>> >>
>> >> OK, cool. Finding out points of agreement helps in finding out where
>> >> we disagree :)
>> >>
>> >> >>> It is important to work on making packages loadable and unloadable,
>> >> >>> but
>> >> >>> you do not need to break the trunk image in order to accomplish
>> >> >>> this.
>> >> >>
>> >> >> Who's breaking trunk? I want to remove packages that are already
>> >> >> unloadable,
>> >> >
>> >> > Yes, but not from *user's* images, from the *core* image that is only
>> >> > used by the CI to build a full image.
>> >> >
>> >> >> and have CI jobs that run every time Trunk updates that
>> >> >> verifies that the packages can (a) reload and (b) run as well as we
>> >> >> already know now (which is not very). To be precise: Universes (for
>> >> >> example) might be smashed beyond recognition, and there is no way to
>> >> >> know, because it has something like 23% method coverage.
>> >> >
>> >> > Right.
>> >> >
>> >> >> Removing these packages from Trunk _will_ affect people updating
>> >> >> from
>> >> >> Trunk, I agree. How else do we make Trunk simpler, if not by
>> >> >> removing
>> >> >> stuff?
>> >> >
>> >> > I think we may be talking past each other. There are two types of
>> >> > packages we want to "remove from trunk":
>> >> >
>> >> > full = core packages + unloadable packages + deprecated packages
>> >> >
>> >> > So there are unloadable packages which should still exist in a full
>> >> > image that everyone is using. And there are deprecated packages which should
>> >> > not exist in a full image.
>> >> >
>> >> > Both core packages and unloadable packages are in Trunk. The only
>> >> > packages that we actually want to remove for good are the deprecated ones.
>> >> > The mechanics of unloading a deprecated package is by committing an empty
>> >> > version (which will remove all the classes and methods from user's images
>> >> > when updating), and eventually removing the now empty package from the
>> >> > update map (and bumping the squeak-version package to keep update numbers
>> >> > continuous).
>> >> >
>> >> > Maybe that's what you were asking about?
>> >> >
>> >> >> (It's also simple enough to reinstall your favourite packages:
>> >> >> Installer squeakmap update; install: 'WhateverItIs (someVersion)'.)
>> >> >
>> >> > Sure, that's how you would re-install a deprecated package.
>> >> >
>> >> >>> Also, please recognize that not everybody wants to do all their
>> >> >>> work in
>> >> >>> a clean image built by a robot. I for example do most of my work in
>> >> >>> a
>> >> >>> Squeak3.11alpha image that has been continuously updated from the
>> >> >>> update
>> >> >>> stream for quite a while. I also periodically check things against
>> >> >>> a
>> >> >>> fresh 4.4 image, but my usual pattern is to use one image over a
>> >> >>> long
>> >> >>> period of time with the trunk update stream, and most of my updates
>> >> >>> to
>> >> >>> trunk come from that old image.
>> >> >>
>> >> >> OK, but then we can't have a minimal core without an entire reboot.
>> >> >> I
>> >> >> mean, seriously, making a minimal core by unloading packages is an
>> >> >> order of magnitude harder when you cannot ever actually unload the
>> >> >> packages!
>> >> >>
>> >> >> So, given the resistance to my slow, gradual unwinding of the
>> >> >> tangle,
>> >> >> what's your counterproposal?
>> >> >>
>> >> >> frank
>> >> >
>> >> >
>> >> > I still fail to see the problem. For purposes of the build server,
>> >> > you once make a Core image by unloading all possible packages. This becomes
>> >> > the new "master image" from which all subsequent releases are built. Then
>> >> > you use that as the base to run tests, build full images, etc.
>> >> >
>> >> > Does that sound reasonable?
>> >>
>> >> Ah! OK, right. So, to see if I understand correctly:
>> >> * We _don't_ issue an empty version for the packages we're interested
>> >> in unloading. (Just for argument's sake, I'll talk about Universes.)
>> >> * If at some point in the future we decide to deprecate Universes,
>> >> _then_ we commit the empty version, and update the map. That decision
>> >> may be deferred for an arbitrary period after unloading it from Core.
>> >> * We take a clean Trunk image from CI, unload all unloadable packages,
>> >> and store that as our new master 4.5 image. (This will likely be a
>> >> manual step, and the clean image committed to the squeak-ci script
>> >> repository.)
>> >> * The stripped master remains stripped, and the existing SqueakTrunk
>> >> job publishes updated copies of the core image as per normal.
>> >> * We adjust the ReleaseSqueakTrunk CI job to, via
>> >> ReleaseBuilderForSqueak4dot5, loads the unloadable packages from
>> >> SqueakTrunk's output to create a "full fat" version.
>> >>
>> >> Dave, sound good?
>> >>
>> >
>> > Yes :)
>>
>> OK, so I thought I'd start the ball rolling with inspiration from
>> SmalltalkImage >> #unloadAllKnownPackages -
>>
>> #('Nebraska' 'Universes' 'XML-Parser') do: [:pkgName|
>>     (MCPackage named: pkgName) unload.
>>     MCMcmUpdater disableUpdatesOfPackage: pkgName].
>>
>> I've stored this as Squeak4.5.image in the squeak-ci repository. This
>> is the new base that we update from trunk. ReleaseSqueakTrunk then
>> loads these packages back into the image.
>>
>> That's the theory. In practice, the above is insufficient because it
>> leaves obsolete classes lying around. So I added
>>
>> MCWorkingCopy flushObsoletePackageInfos.
>> MCFileBasedRepository flushAllCaches.
>> MCDefinition clearInstances.
>> Behavior flushObsoleteSubclasses.
>> ChangeSet current clear.
>> ChangeSet current name: 'Unnamed1'.
>> Smalltalk flushClassNameCache.
>>
>> This too is insufficient: The SqueakCI-Benchmarking package happens to
>> depend on XML-Parser, so I reload the latest version of that
>> package... and things break because there are AnObsoleteXMLWriter
>> instances.
>>
>> What have I missed?
>
>
> This is when the tedium starts.  Use PointerFinder to track down the
> reference path to the obsolete XMLWriter, then determine (e.g. find or
> invent) the initialization(s) that eliminate that path (either by deleting
> the thing because it shouldn't be referred to, or evaluating something that
> replaces it).

Right. That tells me there's an Alias still pointing to the XMLWriter
class. So my half-baked theory is that something's missing in the
Environment work and how it integrates with Monticello?

frank

>> frank
>>
>> > Dave
>> >
>> >> frank
>> >>
>> >> > - Bert -
>> >> >
>> >> >
>> >
>>
>
>
>
> --
> best,
> Eliot
>
>
>