A process proposal for 3.10

brad fowlow fowlow at pacbell.net
Tue Oct 17 18:28:05 UTC 2006


The main idea in the kind of process being suggested here is
the value of decoupling the release process from the development  
process.
You don't define the release in terms of what's in it,
but rather in terms of how things get into it.

How do you do that, when one feeds the other?

The same way an airline decouples  its flight schedules from
the passenger's process of getting their bags packed and getting their
butts to the airport.

You set up the release process as a small set of gating dates,
and a set of criteria for what can pass through that "boarding" gate.

I.e. declare that there's a release on April 2.
To be in it, the development must be complete on December 2,
and it must have integrated and passed tests by February 2.     (Just  
examples here.)

All you know about that release now is that it might be different  
from the current one,
if anything makes the dates, and that it -will- be released on April 2.

Meanwhile, as a developer, if you want to contribute,
you know exactly what needs to be done:
the release process tells you how to get stuff into a release.
That information is just those same gating dates and criteria.
The criteria are usually about submission format,   quality,
and who gets to issue 'tickets' for things you'd like to submit;
usually there are gatekeepers for the broad functional areas,
and they work in the usual consensus ways.

So as a developer you know what you need to do, by when,
and you know that if you do it, then you'll get that change into the  
release.

If you want to make something that can't be ready for this release,
or if you just don't make it, then no problem.  You try for the next  
one.

The 'gating' criteria need to be clear and workable for both bug fixes
and major surgery, so you usually express them in terms of stability;
the usual form is that you'll take things if they don't cause  
regressions,
and if the relevant gatekeeper(s)
(whose allegiance is to a functional component, not the release process)
recommends the change.

This kind of process is usually called date-driven, or 'train';
it's different than a content-driven approach in that dates determine
content, rather than the other way around.

A lot of organizations have found it a good way to untangle the knots
when managing large releases that have a lot of separately-developed  
moving parts;
you let the subsystems (sometimes just one or two guys) run their own  
processes.
If they pack your bags on time, and make it through security,
they get on the flight.

There is art in it.  You have to manage the inflow and date staging
so you don't flood the gating processes.   But the focus there is on  
understanding
the integration and testing and other "boarding" processes, not on  
predefining the content.

Anyway, I just thought a rundown from that pure-process point of view  
might be helpful.

It's not there isn't a place for setting vision and goals;
it's just that those aren't part of this kind of a release process.
They're part of the development processes, which are separate.

- brad



> I basically agree.
> So in my proposal I was trying to see on what I want to work (based  
> on the artefacts already there and also what could
> be vision) and see if other people are willing too.
> Just making the list of things that should be integrated and  
> integrate them is a lot of time.
> In general I would love to stop harvest fixes and produce more fun  
> stuff (but may be I'm a housekeeper that like to
> clean stuff), may be I should stop because my job is different now.
>
> Stef
>
>> Hi Giovanni -
>>
>> I fully agree with the meta goals of your proposal, namely to  
>> enable people to do their own development and to limit the amount  
>> of time spend on each individual part of the release.
>>
>> That said, I'm in almost complete disagreement with the rest of  
>> your proposal ;-) The fundamental flaw in it is that you are  
>> trying to commit other people's time. Starting from "the board  
>> appoints a release team" (phase 0) over "the teams announce the  
>> development plans" (phase 1) and "code will get dropped" (phase 2)  
>> up until "the VMMaker team will want to..." (the example) you're  
>> happy committing other people's time. In my experience this has  
>> *always* been a fatal error in a volunteer open source project.
>>
>> The *only* time you can commit is your own. And given that, how  
>> much of your proposal works if the only time that you can count on  
>> is your own? Even if we want to change the release process we  
>> cannot change some fundamentals about the environment we're in.  
>> And one of those fundamentals is that (on a scale where it  
>> matters) you can't tell people what to do. They're doing it  
>> voluntarily, they're doing it for their own reasons and trying to  
>> change that simply won't work.  To give a more concrete example,  
>> the maintenance work I do for FFI, Graphics, Balloon, and  
>> Compression packages is purely a volunteer effort. If you ask me  
>> about my "development plans" for the next six months, all I can  
>> say is "uhm ... ". Most importantly, because I have a fulltime job  
>> in a startup I for example couldn't possibly commit to a six  
>> months active development plan. It's simply not doable and even if  
>> it were I'd reject it on the grounds that unless you pay me for it  
>> I'm not going to treat this serious enough to make six month  
>> plans ;-)
>>
>> Both Stef and Markus understand that basic point. They both  
>> understand that in reality, all they can count on is their own  
>> time and they have reacted by investing more and more of their own  
>> time and effort in the process. And I would like us to change that  
>> process from one that leaves people burned out into one that can  
>> be done in a reasonable amount of time with a sustainable amount  
>> of energy.
>>
>> That change starts by admitting that we're all volunteers. Whether  
>> something will happen depends on so many factors that it's  
>> basically useless to try to do serious upfront planning for it.  
>> And again this is meant on scale - for each individual team it may  
>> make perfect sense but on scale I think you really can't.
>>
>> To emphasize that point even more I'll give you two recent  
>> examples: First, we have been talking for *years* about changing  
>> the source pointer in the methods. That it happens *now* is (from  
>> my point of view) exclusively due to the fact that we got a new  
>> sources file in 3.9 and lost the entire previous history for the  
>> single reason of the source limitations. Could anyone really have  
>> planned for that? I don't think so. Second, the SSL implementation  
>> we got recently. A while ago we started looking at SSL ourselves  
>> and decided to go with OpenSSL - we talked about Crypto team and  
>> their work but honestly, I would have called anyone a nut who  
>> would have told me that this stuff will be ready by now (I  
>> probably would have just laughed them out of the room). At least  
>> for me, this project came out of the blue, completely unforeseen  
>> and again nothing you could have possibly planned for.
>>
>> My point here is (in case you haven't guessed it by now) that in a  
>> volunteer community "planning" on the scale you are looking at is  
>> in many ways a futile exercise. Things happen for their own  
>> reasons at their own speed and not because you want them. And  
>> unless you want to put in endless amounts of energy by developing  
>> all the stuff that "your release" needs, you better find a way of  
>> dealing with that.
>>
>> Which brings me to the proposal I made earlier. The main goal of  
>> that proposal is really to get away from the upfront feature  
>> planning process and deal with the chaotic reality of open source  
>> development by *embracing* it. So instead of feature list of  
>> things that we SHOULD do, I'd like us to discuss things we HAVE  
>> done. Instead of wishful thinking I'd like us to talk about all  
>> the cool stuff that is actually happening!
>>
>> I think that this may be the only way to make sure we can survive  
>> the release process in a volunteer community like ours and it's  
>> surprisingly similar to what we did at SqC - the main difference  
>> is that SqC (instead of the community) picked which of the  
>> contributions got adopted and which ones didn't. At SqC (or any  
>> other commercial entity that I have been involved in) we have  
>> always been aware of error 33 and therefore never relied on the  
>> open source effort to produce specific results in a specific time  
>> frame. That's what companies do, that's why they take money to do  
>> it. However, anything that happened was always welcome and often a  
>> surprise. And I can say for sure that during those times nobody  
>> got burned out by the process.  And I think that process can be  
>> recreated.
>>
>> Cheers,
>>   - Andreas
>>
>> Giovanni Corriga wrote:
>>> Hi all,
>>> since we've started discussing 3.10/4.0, I'd like to relay some  
>>> thoughts
>>> that I've expressed some times ago on #squeak about a possible
>>> development process for Squeak 3.10.
>>> This proposal is based on two points:
>>> - decentralization of development
>>> - fixed duration of iterations ("timeboxing")
>>> "Decentralization of development" means moving most of the
>>> responsibility of development from the release team to the single
>>> package mantainers (stewards). The release team should just be an
>>> integrator of the code blessed by the stewards, and should  
>>> facilitate
>>> coordination between the stewards of different packages.
>>> "Timeboxing" means having a fixed length release cycle, at the  
>>> end of
>>> which a new stable version is released. This means that if there are
>>> some problems with a new feature, the deadline doesn't get  
>>> postponed;
>>> instead it's the feature that gets dropped from the current  
>>> development
>>> release and rescheduled for the next release.
>>> How would this work? Here's a more detailed description.
>>> -------------------------------------------------------------------- 
>>> ---
>>> PHASE 0: RELEASE TEAM FORMATION
>>> The SqF board will appoint a release team for 3.10. This team  
>>> will have
>>> 3-7 members; 5 would be the sweet spot.
>>> The first task for the release team will be to find a mantainer  
>>> for each
>>> one of the packages currently in the image. If they can't find a
>>> mantainer for a package, than said package _will get dropped from  
>>> the
>>> image_ without any further discussion.
>>> PHASE 1: RELEASE PLAN
>>> After having found mantainers for every package, there's a package
>>> selection phase, during which the team and the maintainers:
>>> - decide which packages currently in the image should get removed  
>>> from
>>> the image and turned into SqueakMap-loadable packages.
>>> - decide which new packages can be added to the image or replace  
>>> other
>>> packages currently in the image.
>>> Input from the community and the stakeholders will be taken into
>>> account.
>>> After deciding which packages will be in the 3.10 image, the release
>>> team, the mantainers and the community will work on creating the  
>>> release
>>> plan.
>>> Each package mantainer will announce what it plans to do for  
>>> 3.10. The
>>> release team will integrate these proposals into a coherent release
>>> plan, solving possible conflicts and coordination problems. The
>>> community will be able to make comments and/or requests.
>>> After all these steps, which shouldn't take more than a month, the
>>> development of 3.10 can start.
>>> PHASE 2: DEVELOPMENT
>>> Development will have three stages:
>>> - a 3 months alpha stage
>>> - a 2 months beta stage
>>> - a 1 month gamma/RC stage.
>>> The deadline for each stage will be decided at the beginning of the
>>> iteration.
>>> Every week during each stage the release team will integrate the new
>>> releases of each package blessed by the package mantainers.
>>> Package mantainers will be the main responsible for development  
>>> of their
>>> package and they'll be free to manage development as they like;  
>>> ideally
>>> they should create a development team.
>>> If at the end of the alpha stage some of the work items of the plan
>>> aren't near completion, _these items and the corresponding code  
>>> will get
>>> dropped_. If this is not possible, the code will be kept in the  
>>> image,
>>> but it will have not to break the other code in the image.
>>> During the beta and gamma stages, no new features will get added  
>>> to the
>>> image; only bug fixes can go in. During the gamma stage, only
>>> showstopper bugs will be fixed; other bug fixes (eg. cosmetical  
>>> fixes)
>>> will be scheduled for 3.10.1.
>>> During the gamma stage of 3.10, the release team for 3.11 (or 4.1 or
>>> whatever) will get appointed and will work with the package  
>>> mantainers
>>> to form the release plan for the next version.
>>> EXAMPLE
>>> Here's a an example of how things could possibly work.
>>> Release plan:
>>> During phase 1, the various package mantainers propose work  
>>> items. So
>>> let's pretend that...
>>>   - VMMaker team will want to:
>>>     * integrate the async event patches useful for SqueakGtk.
>>>     * integrate the Ffenestri code in all VMs, including the Unix  
>>> one.
>>>     * work with the Kernel team/maintainer on CompiledMethod  
>>> cleanup.
>>>   - Nebraska team will work on removing it from the image.
>>>   - Pavel will want to work with various mantainers to integrate his
>>> patches for his KernelImage system.
>>>   - Tools team will want to:
>>>     * better integrate AST and the Refactoring services.
>>>     * make Omnibrowser the default browser.
>>> Now let's pretend that at the end of the alpha stage, the Tools team
>>> still hasn't produced an Omnibrowser based debugger. Since it's  
>>> the end
>>> of the alpha stage, the OBDebugger will be punted to the next  
>>> release,
>>> and 3.10 will have a class browser based on OB but still the old
>>> debugger.
>>> -------------------------------------------------------------------- 
>>> ---
>>> I believe a process such as the one I'm proposing can work and help
>>> produce good quality releases.
>>> I'd like to hear comments on this, even if they just say that my
>>> proposal sucks.
>>> 	Ciao,
>>> 		Giovanni
>>
>>
>
>




More information about the Squeak-dev mailing list