Hi Doug and all!

(I was actually typing on a reply when you reminded me Doug! :-))

Doug Way <dway at riskmetrics.com> wrote:
> (Sorry, I've been out of commission for the last couple of days because 
> of the eastern-U.S. blackout.)
> 
> Argh, this is not turning out to be a simple decision.  I guess I'd 
> like at least a little input from the Guides and community before 
> deciding, although we need to do something soon.  Say, in the next 
> couple of days.
> 
> There are roughly 3 options to how we prepare for the 3.6 release:
> 
> 1. The original (?) plan:  Keep the 9 removed packages out of the 
> update stream.  When we move to gamma, the current update stream image 
> becomes the "Basic" release, and then the 9 packages are added to this 
> release to form the "Full" release.  The Full release is the one 
> provided on squeak.org, although the Basic release is also available on 
> the ftp site.

The only problem being that if you download Basic image and install the
9 removed packages through SM they may have changed since 3.6 was
finalized. The classic problem until SM2 arrives.

It could be solved by simply not changing those 9 packages for a while.
:) Sure, a hack because we don't have SM2 yet. Another hack is to
duplicate those 9 packages with names like "XXX 3.6 release". Also a
hack.


> 2. The plan I described in my previous message:  Fork the update stream 
> now into a Full stream in addition to the regular (Basic) stream.  The 
> 9 packages are added to the Full stream shortly by loading the "upgrade 
> to full" package from SqueakMap.  When we move to gamma, a Full release 
> image (candidate) is created from the Full update stream, and a Basic 
> release image is created from the regular stream.  The Full release is 
> provided on squeak.org, and the Basic release is available on the ftp 
> site (as with option #1).

Disadvantages being that the fork is... a fork. :-) But we did that with
3.2 too.
Another disadvantage is that people aren't really "letting go" of the
image. I know that SM2 isn't here yet, but it is still a bit of "not
letting go".
 
> 3. Daniel's suggestion:  Keep just one update stream.  Add the 9 
> packages to the update stream now.  This becomes the Full release, 
> there is no separate Basic release, although there are "removal" 
> packages which should work to get you down to Basic.  At the beginning 
> of 3.7alpha, the 9 packages are removed again.

Personally I must say this sounds like the worst solution. :) Generally
I tend to agree with Daniel in most things, but this sounds backwards to
me.
 
> Any thoughts on these?  My thoughts:
> 
> The main problem with option #1 is that no one is really testing the 9 
> external packages right now to make sure that they work, so there will 
> probably be problems when the Full release is put together for 3.6.  
> But otherwise it's simple.

I am not sure I agree with the "no one is really testing" bit. If people
are performing testwork specifically for the release (anyone?) then hey,
they should simply load these packages.

And if they are not and simply going about heir business as usual, then
they only test the stuff they actually use anyway - no difference if it
is on SM or in the image. I assume.

So in short - I personally don't think it affects the testing bit.

> Option #2 is a bit more complicated and splits the folks testing into 
> two groups, but is otherwise okay IMHO.

Something like that, yes. And of course, doing a fork always confuses
people - the 3.2.1 fork did IIRC.

> Option #3 might work, although I'm not sure if everyone wants to go 
> without a Basic release.  Also, I'm not sure if we have working removal 
> packages for all 9 extra packages right now if someone wants to strip 
> down to a Basic release.  (And this does sort of take us back to 
> stripping/shrinking to get a smaller image.  Although Daniel has a 
> point that the important thing is that the refactorings have been done 
> so that the removals should be simple.)

Don't like this one. :-)
 
> At some point quite awhile ago I think Goran suggested just providing a 
> Basic release which has the 9 extra packages bundled, and a button or 
> some simple mechanism in the image to upgrade to the Full release.  
> This is sort of a variation of option #1.  Although I tend to think 
> that we should just provide a Full image as part of the release instead 
> (in addition to a Basic image if needed).

You might have been thinking about what we can do with SM2. Or I might
have said that. :-) Anyway, it is just another "hack" to get around the
fact that we don't have releases. And the other two hacks mentioned
above seems simpler.
 
> Also, one other thing to think about is how things might be different 
> when we have SqueakMap 1.1, and we get to release 3.7final.  SM 1.1 
> will support versions and dependencies/configurations on top of that, 
> which solves some important problems, but I'm not sure if it resolves 
> the issue we're talking about here... at least not while we're still 
> chopping away at the image.  We'd still like a way for a largish number 
> of people to test a beta Full image/configuration before it goes final. 

Well. First of all SM2 will most importantly support releases of
packages and have a proper cache. Those two things solves the problem.
Package releases means we can release a Basic image and then publish one
or more loadscripts that installs *specific versions* of packages. These
loadscripts will not change over time when the packages change because
they refer to package *releases* and not just packages.

The cache will help us in delivering Squeak. We simply zip up the Basic
image together with the "sm" directory. We can still have different
downloads and the difference will be how much stuff is inside that "sm"
dir - how much stuff is in the cache.

A "Full" 3.7 release would include an "sm" dir with at least the removed
packages that we consider as official packages. (Not PWS for example).
Or more specifically, for those packages it would hold the release that
we consider to be the one for Squeak 3.7.

Another download could be a super download with a full "sm" cache
including all the latest package releases for 3.7.

And a mega-I-have-tons-of-bandwidth-download that simply includes the
full sm cache including all releases of all packages registered.

Ok, secondly, a clarification (Doug knows this but perhaps other people
don't) - SM2 will *not* include an implementation of dependencies. But
it *will* include the mechanisms making it very easy to add (map
embedded resources and package releases). If anyone is willing to start
implementing the planned dependency system (or *any* such system) please
tell me! I have a Grand Plan for it but it is important that we
parallellize this effort. I have enough to do with SM2.

And finally - down to the question at hand: testing.

The problem, as understood by me, is that people track the update stream
and when we remove stuff out into packages those packages don't get
tested. This assumes people aren't installing those packages from SM.
The question is - is it really more likely to get tested just because it
is in the image?

I don't think so - but just for the hell of it I will assume the answer
is yes. Ok, so how do we keep up with the effort at hand while still
maintaining the "feel" of a full glorious image that people can explore
at their whim (proper english?)?

Well, follow my reasoning:

1. People need a fully packed amazing image to really test stuff. Ok,
let's give them that then:
	1. Finish SM2. Working on it hard - a non-complete alpha coming up on
the web this week.
	2. Change the format of the download so that it contains ONE rather
small image (Basic to start with) and a nicely preloaded package cache
and SM catalog.
	3. Write a nice loadscript that produces the "Full" image by installing
the correct packages.
	4. Add a Big Fat Button on the Squeak desktop properly labelled "Take
me to the candy store NOW!". :-)
	5. The load sripts installs all packages needed (and there is no
network involved at all here) and then asks to save this mega image with
a new name.

Now, many people will push the button and then live in the candy store.
Fine. Things will hopefully get tested.

>   Hmm, haven't thought about that enough yet.  But whatever we do for 
> 3.6, we should think about whether we'll be doing something different 
> for 3.7.
> 
> (Enough rambling for now...)

:-) It is funny how these seemingly simple problems turn complicated in
the head, right? I don't claim to have the answers here - I just hope
that SM2 will solve many of the issues involved.
 
> - Doug

regards, Göran