On Tuesday, August 10, 2004, at 03:10 AM, Andreas Raab wrote:

> Hi Doug,
>
> Some thoughts:
>
>> 1. Start up the separate "internal" update stream as Michael 
>> suggested.
>
> Please be clear about what you're trying to achieve with this step. 
> All I'm
> reading under point 1. is techno-babble about the mechanics but no 
> reason
> *why* we want to have that update stream[*]. And making an extra update
> stream for the update stream's sake will only increase perceived 
> "latency"
> of getting fixes out. If there isn't a reason for having an extra 
> update
> stream then please don't do it for bureaucratic reasons only.
>
> [*] Understand that what I'm asking for here is some public 
> acknowledgement
> that "we are introducing the extra update stream because ... and the 
> extra
> update stream addresses this by ..." (for whatever values of ... you'd 
> like)

Yeah, right before I sent out that message I realized I hadn't given 
much explanation for why we should make these changes, but it was 
getting late and I wanted to get the discussion started.

Daniel summarized the "why" pretty well in his message, but I'll add 
some more.

With point 1, the main idea is that this would enable points 2 and 3.  
Well, mostly point 3 actually.  If a small group of people (perhaps 
called "committers", since they're not necessarily "harvesting" other 
people's changes)... if these committers have the ability to post their 
own updates directly without any review from anyone else, this will 
probably make the update stream more unstable, with major bugs 
appearing occasionally.

The idea is that adding an additional "unstable" update stream will 
allow these major bugs to be fixed before they appear in the more 
widely used regular/"testing" update stream.  It gives some insulation 
for people updating their images from having an untested update wreak 
havoc.  This is why Squeak Central had a separate internal and external 
update stream, I assume.

Now, having read the rest of this thread and thinking about it a bit 
more, I'm not totally convinced that the separate update stream is 
absolutely necessary, even if we do #2 and #3.  If we enforce (via the 
broadcast mechanism) that the ConflictChecker is run, and especially if 
SUnit tests must pass, before an update can be broadcasted, we could 
arguably avoid "major" bugs from occurring in a single update stream.  
I think SUnit tests take a minute or two to run right now... that's 
probably reasonable overhead when broadcasting an update.  (And if 
several updates are broadcasted at once, SUnit tests still only need to 
be run once for all of them.)

Hm, then again, one problem with having a single update stream and 
various committers broadcasting updates at various times, is that 
updates will not appear in "bunches", with an [updates] message on 
squeak-dev, with an obvious point for Bruce to create a new alpha image 
to upload to the ftp site, etc.  Instead, updates may trickle in 
constantly.  Maybe this isn't a big problem, though.

(Eh, "committers" is perhaps not the best term either, as it implies 
committing code to a source code repository (e.g. CVS or Monticello).)

>> 2. (This one is my suggestion.)  If we have this unstable update
>> stream, posting updates to this stream doesn't have to be as
>> coordinated as with the regular stream.  So, we could allow harvesters
>> who [approve] an item in the BFAV to also immediately "broadcast" that
>> item as an update to the unstable stream.  (The broadcasting mechanism
>> to do this is also already there from the SqC days, but needs some
>> fixing.)
>
> In which way do we expect this to address what issue(s)?

This reduces the time between an item being approved and its appearance 
in the update stream.  It's typically 1-2 weeks right now, and this 
would be reduced to 0.

Also, it distributes my workload better, so that I don't act as a 
bottleneck for updates to come through.  (It's probably not sustainable 
for one person to keep doing this for too long anyway...)

> [...some more technicalities deleted...]
>
>> 3. Let some harvesters/guides have direct "commit"/broadcast ability,
>> without going through the BFAV approval process, as some have
>> suggested.  However, I think this should be limited to only certain
>> types of updates, probably just fixes and smallish 
>> refactorings/cleanups.
>> New enhancements or major redesigns would still need to go through
>> the approval process.
>
> To what extend do the above restrictions model what is currently 
> practice in
> any of the "external" packages? ("external" in quotes simply because 
> they
> are "official" and as such I believe pretty much the same rules apply 
> to
> them as to the updates) As far as I can tell, there have never been any
> discussion about decisions made by any of the "official" package
> maintainers. Is the goal here to restrict (and more conservatively 
> manage)
> changes that are "in-image" compared to changes that are "ex-package"? 
> (this
> is NOT a trick question btw)

Well, the above restrictions model is a very tentative idea, it 
probably need more thought/discussion.  Basically, I thought we'd want 
something in between the current "extremely restrictive" model (can't 
self-approve anything except really important/trivial fixes, and 
comments), and "anything goes".

True, external/official packages are sort of "anything goes", in other 
words, the package owner can make whatever changes he/she wants.  But I 
think the base image is different, mainly because it hasn't been 
divided up into ownership areas (yet).  So I think "anything goes" 
would be a bit much.  We wouldn't someone to arbitrarily decide "hey, I 
don't like the way the window titlebars look, I'll redesign them", and 
broadcast the change to the update stream.  Unless they're "in charge" 
of that area, perhaps, but for now, the base image is not divided up 
yet.

But on the other hand, I think it would be good to be less restrictive 
than we are now, e.g. allow someone like Ned to post fixes without 
waiting for review, for a more rapid/xp-like development pace, with 
faster feedback.  This is the core idea of point #3.

Perhaps we do just need to move ahead with dividing up responsibility 
inside the image as much as possible (i.e. Goran's TNFR project), but 
still there will be some areas such as the kernel which will be 
maintained by committee.  And then sometimes fixes/enhancements will 
span across several areas of the image.  Whew!  Not really a simple 
problem.

>> So... how do these ideas sound?
>
> Somewhat half-baked, if you ask me ;-) It depends on where we want to 
> steer.
> The above proposal(s) seem not directly to address any of the issues 
> that
> have been raised (or if they do I couldn't find the relation).

Well, my idea of the degree of restrictions of what committers should 
be able to commit directly is pretty half-baked, I'll admit.  Also, my 
original idea of "2-4 weeks" between moving updates from the unstable 
to the testing stream (if we use two streams) was pulled out of thin 
air.

But I think the main ideas of #2 and #3 are reasonably sound.

>> #2 and #3 would hopefully remove me as the bottleneck at the update
>> stream, which would be a bit of a relief at this point... ;)
>
> Well, if that's your reason to propose them why not put it forward that
> way - I think everyone will agree that it's useful to ease your (really
> good) work.

True.  This was not originally the main reason, but it turns out to be 
an important side benefit.  The main reason is still speeding up the 
development cycle/testing feedback loop.

(Leaving the BFAV/Mantis and Monticello discussions for the other 
threads)

- Doug