On Sat, Mar 30, 2013 at 4:52 AM, Frank Shearar <frank.shearar at gmail.com>wrote:

> On 25 February 2013 18:51, David T. Lewis <lewis at mail.msen.com> wrote:
> > On Mon, Feb 25, 2013 at 04:57:54PM +0000, Frank Shearar wrote:
> >> On 25 February 2013 16:37, Bert Freudenberg <bert at freudenbergs.de>
> wrote:
> >> > On 2013-02-25, at 17:07, Frank Shearar <frank.shearar at gmail.com>
> wrote:
> >> >
> >> >> On 25 February 2013 15:35, David T. Lewis <lewis at mail.msen.com>
> wrote:
> >> >>> On Mon, Feb 25, 2013 at 03:01:18PM +0000, Frank Shearar wrote:
> >> >>>> On 25 February 2013 14:49, Bert Freudenberg <bert at freudenbergs.de>
> wrote:
> >> >>>>>
> >> >>>>> On 2013-02-25, at 15:33, Frank Shearar <frank.shearar at gmail.com>
> wrote:
> >> >>>>>
> >> >>>>>> On 14 February 2013 16:50, Frank Shearar <
> frank.shearar at gmail.com> wrote:
> >> >>>>>>> No, this takes the Trunk image as is and runs the job. No
> stripping
> >> >>>>>>> has occurred. Noone's actually said "oh, that's a sane idea
> someone
> >> >>>>>>> should do that" yet (except possibly for you, by virtue of not
> saying
> >> >>>>>>> "are you crazy?" :) ), so I haven't done it yet.
> >> >>>>>>>
> >> >>>>>>> So the installation is essentially a no-op.
> >> >>>>>>>
> >> >>>>>>> With green lights on these builds, we can now verify that
> removing
> >> >>>>>>> these packages won't break anything. At least, to the limit of
> the
> >> >>>>>>> tests, which isn't much at all. So if we _do_ actually strip
> these
> >> >>>>>>> guys out of Trunk, the builds should stay green. If they don't,
> we'll
> >> >>>>>>> need to roll back the change and investigate.
> >> >>>>>>>
> >> >>>>>>> But I don't actually know how to strip them out of Trunk. I
> _suspect_
> >> >>>>>>> it'll be via a Monticello config map of some description.
> >> >>>>>>
> >> >>>>>> Anyone? Bert? How do we strip packages out of Trunk?
> >> >>>>>
> >> >>>>> Manually once, when you prepare the Core image. Otherwise, if we
> put it in the trunk update stream, the packages would be unloaded from
> everyone's working images when updating.
> >> >>>>
> >> >>>> Given that we're early in 4.5's release cycle, if we do this kind
> of
> >> >>>> thing - stripping unnecessary things out of trunk - we ought to do
> it
> >> >>>> now rather than later. And announce loudly what we're doing, why,
> and
> >> >>>> how people can load in their favourite not-Core packages.
> >> >>>
> >> >>> NO.
> >> >>>
> >> >>> If Squeak developers work with images that contain just "their
> favourite
> >> >>> non-Core packages" then none of the non-Core packages get
> maintained.
> >> >>
> >> >> Clearly we disagree quite a bit on this, but I don't think anyone
> >> >> should actually _work_ on a Core image. _Noone_ should be able to
> work
> >> >> effectively in a minimal image because such a thing wouldn't even
> have
> >> >> a Browser!
> >> >
> >> > We don't disagree on this at all. People should be working in a full
> image.
> >>
> >> OK, cool. Finding out points of agreement helps in finding out where
> >> we disagree :)
> >>
> >> >>> It is important to work on making packages loadable and unloadable,
> but
> >> >>> you do not need to break the trunk image in order to accomplish
> this.
> >> >>
> >> >> Who's breaking trunk? I want to remove packages that are already
> >> >> unloadable,
> >> >
> >> > Yes, but not from *user's* images, from the *core* image that is only
> used by the CI to build a full image.
> >> >
> >> >> and have CI jobs that run every time Trunk updates that
> >> >> verifies that the packages can (a) reload and (b) run as well as we
> >> >> already know now (which is not very). To be precise: Universes (for
> >> >> example) might be smashed beyond recognition, and there is no way to
> >> >> know, because it has something like 23% method coverage.
> >> >
> >> > Right.
> >> >
> >> >> Removing these packages from Trunk _will_ affect people updating from
> >> >> Trunk, I agree. How else do we make Trunk simpler, if not by removing
> >> >> stuff?
> >> >
> >> > I think we may be talking past each other. There are two types of
> packages we want to "remove from trunk":
> >> >
> >> > full = core packages + unloadable packages + deprecated packages
> >> >
> >> > So there are unloadable packages which should still exist in a full
> image that everyone is using. And there are deprecated packages which
> should not exist in a full image.
> >> >
> >> > Both core packages and unloadable packages are in Trunk. The only
> packages that we actually want to remove for good are the deprecated ones.
> The mechanics of unloading a deprecated package is by committing an empty
> version (which will remove all the classes and methods from user's images
> when updating), and eventually removing the now empty package from the
> update map (and bumping the squeak-version package to keep update numbers
> continuous).
> >> >
> >> > Maybe that's what you were asking about?
> >> >
> >> >> (It's also simple enough to reinstall your favourite packages:
> >> >> Installer squeakmap update; install: 'WhateverItIs (someVersion)'.)
> >> >
> >> > Sure, that's how you would re-install a deprecated package.
> >> >
> >> >>> Also, please recognize that not everybody wants to do all their
> work in
> >> >>> a clean image built by a robot. I for example do most of my work in
> a
> >> >>> Squeak3.11alpha image that has been continuously updated from the
> update
> >> >>> stream for quite a while. I also periodically check things against a
> >> >>> fresh 4.4 image, but my usual pattern is to use one image over a
> long
> >> >>> period of time with the trunk update stream, and most of my updates
> to
> >> >>> trunk come from that old image.
> >> >>
> >> >> OK, but then we can't have a minimal core without an entire reboot. I
> >> >> mean, seriously, making a minimal core by unloading packages is an
> >> >> order of magnitude harder when you cannot ever actually unload the
> >> >> packages!
> >> >>
> >> >> So, given the resistance to my slow, gradual unwinding of the tangle,
> >> >> what's your counterproposal?
> >> >>
> >> >> frank
> >> >
> >> >
> >> > I still fail to see the problem. For purposes of the build server,
> you once make a Core image by unloading all possible packages. This becomes
> the new "master image" from which all subsequent releases are built. Then
> you use that as the base to run tests, build full images, etc.
> >> >
> >> > Does that sound reasonable?
> >>
> >> Ah! OK, right. So, to see if I understand correctly:
> >> * We _don't_ issue an empty version for the packages we're interested
> >> in unloading. (Just for argument's sake, I'll talk about Universes.)
> >> * If at some point in the future we decide to deprecate Universes,
> >> _then_ we commit the empty version, and update the map. That decision
> >> may be deferred for an arbitrary period after unloading it from Core.
> >> * We take a clean Trunk image from CI, unload all unloadable packages,
> >> and store that as our new master 4.5 image. (This will likely be a
> >> manual step, and the clean image committed to the squeak-ci script
> >> repository.)
> >> * The stripped master remains stripped, and the existing SqueakTrunk
> >> job publishes updated copies of the core image as per normal.
> >> * We adjust the ReleaseSqueakTrunk CI job to, via
> >> ReleaseBuilderForSqueak4dot5, loads the unloadable packages from
> >> SqueakTrunk's output to create a "full fat" version.
> >>
> >> Dave, sound good?
> >>
> >
> > Yes :)
>
> OK, so I thought I'd start the ball rolling with inspiration from
> SmalltalkImage >> #unloadAllKnownPackages -
>
> #('Nebraska' 'Universes' 'XML-Parser') do: [:pkgName|
>     (MCPackage named: pkgName) unload.
>     MCMcmUpdater disableUpdatesOfPackage: pkgName].
>
> I've stored this as Squeak4.5.image in the squeak-ci repository. This
> is the new base that we update from trunk. ReleaseSqueakTrunk then
> loads these packages back into the image.
>
> That's the theory. In practice, the above is insufficient because it
> leaves obsolete classes lying around. So I added
>
> MCWorkingCopy flushObsoletePackageInfos.
> MCFileBasedRepository flushAllCaches.
> MCDefinition clearInstances.
> Behavior flushObsoleteSubclasses.
> ChangeSet current clear.
> ChangeSet current name: 'Unnamed1'.
> Smalltalk flushClassNameCache.
>
> This too is insufficient: The SqueakCI-Benchmarking package happens to
> depend on XML-Parser, so I reload the latest version of that
> package... and things break because there are AnObsoleteXMLWriter
> instances.
>
> What have I missed?
>

This is when the tedium starts.  Use PointerFinder to track down the
reference path to the obsolete XMLWriter, then determine (e.g. find or
invent) the initialization(s) that eliminate that path (either by deleting
the thing because it shouldn't be referred to, or evaluating something that
replaces it).


> frank
>
> > Dave
> >
> >> frank
> >>
> >> > - Bert -
> >> >
> >> >
> >
>
>


-- 
best,
Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20130401/6f08866b/attachment.htm