New subject: DeltaStreams status (Re: Let's push it)

30 Jun 2009


      Folks -
I truly love this discussion and I think it has generated enough insight 
that we can try pushing a few things. As a first step, I would like to 
propose a slightly different development model for Squeak, because I 
think that is one of the fundamental bits that is wrong. Here we go:
1) Agree on MC as the core development tool/model. This is painful and 
consequently the decision has to be made consciously. However, from my 
perspective there is really no suitable alternative [1].
2) Decide on the canonical repositories. I would suggest that 
http://source.squeakfoundation.org/trunk is a fine starting point [2].
3) Decide on who gets commit rights. I would suggest that the board 
decides that (equally for eventual access removals). The "masters" list 
from Squeak-people could act as a starting point.
4) Implement an update model for MC. I would suggest to adopt a similar 
model to what we've been using for Tweak in the past: Use MC config maps 
to mark points in the update process where order is important and 
otherwise update the packages in a well-defined order[3].
These four steps are intended to achieve the following: I can propose to 
the board that Eliot should be given access to the kernel repository so 
that he can commit any fixes to the compiler or other changes required 
for Cog directly without the need for an intermediate. I can propose to 
the board that Nicolas gets commit rights to the kernel repositories so 
that he can commit any fixes for numeric stuff he has or comes up with 
in the future. If the board agrees, they can do that, period. If they 
screw up or abuse their powers the board can retract commit rights.
The main difference to what we've done in the past is simply that we 
have an inclusive list of committers instead of a select few and that we 
do understand the image as truly being "work in progress" inbetween the 
releases. In other words, be more active, change things faster, make it 
possible to report, devise, submit, and deploy a change in a matter of 
minutes.
For people who don't have commit rights, there is both Mantis or 
commit-by-proxy which reinstates the SqC model of having a champion for 
your change. I.e., to commit something you can ask someone else to 
commit your change but that person implicitly takes responsibility for 
your change and will be the person who gets blamed for it and is 
expected to take care of it (and they may loose their commit rights if 
the change they commit is complete crap). To me, that is a useful 
intermediate state because the proxy acts as a mentor to teach the 
original author about how to provide changes, make sure it's okay etc.
For Mantis, Keith just added a bunch of wonderful scripts that allow us 
to pull changes from it and build images (and hopefully soon run tests 
like the integration tests that I've been working on earlier) which 
means that we can build images with those changes included, test them, 
and if they pass those tests (and possibly other review), commit them to 
trunk. Again, I think having a champion who does the commit would be a 
good model. Obviously there is still a manual element involved here but 
the idea is that the regular committers would be able to go straight to 
the repository without requiring further approval.
For users of Squeak there would be an update process which (although 
much slower than the update stream of the days of yore) would allow 
people to keep track of the ongoing changes and (for forks) merge those 
changes as desired along the way using Monticello.
What do people think about this as a straw-man? Obviously, I want to 
enable people to contribute in the most straightforward way possible. 
Allow development to be more fluid, make mistakes faster, fix them 
faster. It does not address the issue of controversial / strategic 
changes but I think we're mostly loosing out on the most basic level and 
if we want to fix the processes we must make simple things simple and 
complex things possible, not the other way around.
Cheers,
   - Andreas
More detail on some of the issues above:
[1] Why I think update streams don't cut it: When you do distributed 
development it is necessary to know if there are conflicts with what you 
are working on. If you only get to spend two hours a week on development 
you need to be able to quickly find out if anything changed in an area 
that you've been working on since last week. If you want to provide 
backwards compatibility you need to be able to reflect about what has 
changed, etc. Monticello is pretty bad but it's still the best we got.
[2] http://source.squeakfoundation.org/trunk: The idea is to use the 
version specific directories (Squeak39, Squeak310, Squeak311 etc) as 
branches for released versions. The "current" version is always on 
trunk, the specific ones provide updates after a version was released.
[3] MC based updates: The basic idea is that the config maps define load 
order and intermediate stages where required. There is an 
update-<version>.mcm in every repository that is consulted by the update 
loader process. The update process loads the update.mcms in order and 
uses the order in the last loaded update.mcm to update all the remaining 
packages. In  other words, as long as the packages load with the last 
defined order, we change nothing. If Bob finds a problem we issue a 
(series of) configurations that get past that update bump. Yes, there is 
a certain element of risk to it but if Bob runs test builds of the image 
for us we can simply update to the last version which Bob was able to 
build for himself (and Bob can notify us that there's a problem).