Keith Hodges wrote:
> Joshua Gargus wrote:
>   
>>>  If you improve something its for us in our branch
>>> that's what we want, after all we are moving forward and that is the
>>> most important thing right?
>>>   
>>>       
>> Right!  So we're in agreement... what's the problem?
>> Just kidding... a few remarks:
>>
>> Moving forward is not the only important thing, but it probably is the
>> most important (if it's not, what is?).  We shouldn't lose sight of that.
>>     
> Bear in mind that the Board put forward Spoon as Squeak 5.0, and they
> cancelled 3.11. Therefore moving forward is not as important as you
> might suppose. If it was then they would never have conceived of
> cancelling the 3.11 effort, which they did.
>
>   

I'm not intimately familiar with those decisions, but I'll pretend that
I am :-)

> They also told me that too much progress on 3.11 would be a problem
> since the re-licencing would have more to catch up on, and the
> relicencing was more important.
>
> (no one has sacked Craigs ideas because he hasn't delivered anything,
> not even an email in a year, is he still alive?)
>   

Don't worry, he's still alive, I saw him the other day.  I don't think
that a public sacking is in order (maybe a comedy roast?).  However, I
don't think that the plan is really still in effect... I think that if
someone proposed a realistic Squeak 4.1 plan, and Craig piped up with
"hey guys, what about my 5.0 plan", then he would have a very tough time
convincing anybody to hold off on 4.1 because of 5.0.

Do you believe otherwise? 

> Andreas would be better off working on spoon to move us all forward.
>   

He stated from the beginning that he does not want to do all the work
himself.  His election platform was to create a process allowing
community contributions to flow more smoothly.  He is doing some work
himself now to kick-start the process, to show others that it's a viable
way to contribute.

> So moving forward is not the most important thing at all.
>   

This is a very odd conclusion to draw.  At the time, there was reason to
believe that big progress (i.e "moving forward") was right around the
corner, and that things would be simplified if we waited for the big
step to occur.  In hindsight this may have been a bad decision, but it
does not support your conclusion.

> I believe that having a process that supports open contributions that
> are not in the control of one person/release team is. Until we have that
> process, any work "moving forward" is actually making the problem worse.
> Someone will have to sift through all of the work that is being done in
> trunk and do it all again. Just as I had to sift through Edgar's
> SqueakLightII in order to try and capture the knowledge in a usable form.
>   
>> Throughout this thread, you have repeatedly stated that we will thrash
>> about the repository "without thinking in advance about it".  Could
>> you clarify what you mean by "it"?  Clearly, you don't mean
>> "anything", because you don't think that we're all complete idiots. 
>> Does "it" mean "backward/cross-compatibility"?  Some other
>> software-engineering concern? 
>>     
> You have established a scenario, where only one person can make big
> plans. That person is (currently) Andreas, and he doesn't publish a
> project proposal, document an API, etc etc. He just sets to work on the
> trunk repository. No body knows what he is going to do in advance.
> Everyone is waiting for him, and is limited to throwing in a couple of
> fixes here and there.
>   

No, you're just arguing against your straw-man again.  Actually, you
seem to have at least two, the one where chaos ensues because everyone
is making changes willy-nilly, and the one where Andreas is the
bottleneck (apparently an even worse bottleneck than in the old
process).  The two are not logically consistent, and lead me to believe
that you are more interested in winning an argument than coming to a
common understanding.  This may not be conscious behaviour on your part,
or I may be mistaken, but that's how it appears to me.

(I now see that below you wrote "chaotic free for all, for those that
are in the inner circle, a bottleneck for everyone else".  I'll accept
this.)

You elaborate on this point further below; I'll respond there.

> If you want to make a contribution to trunk, you have to review all of
> the changes made before you first, since it is a moving target. 

That's not generally true.  If I need to make a change to a part of the
system that I haven't been keeping track of, then all I need to do is
look at the current code, since that's what I'll be changing.  In other
words, if I wasn't mentally tracking that part of the system before,
there's no need to gain an understanding of the entire evolution up to
that point before starting to work (just like if I'm working in 3.10.2,
I don't need to understand all of the changes since 3.2, I just need to
understand the code I'm looking at in the browser).

It's true that if that part of the system is in flux, then I need to be
aware of the ongoing changes.  I can notice this in several ways.  The
first one that I do is to glance at the last modification date of the
relevant methods.  If the last modification was months ago, then I go
ahead and make my changes; if there is a collision with someone else's
efforts, it's not hard to fix.  If the last modification was yesterday,
then I need to talk to somebody about what's going on.

> You have
> to guess what other people are up to and fit around them. This is by
> default a situation in which it is virtually impossible to plan more
> than one step forward at a time, but no one knows what other people are
> planning as their next step.
>   

I think that this is a valid concern, but I also think that it's OK.

This works fine for my company with 10 people very actively working on a
code-base, and I think that it will work for Squeak too (the amount of
explicit communication required to avoid stepping on toes is minimal). 
If toes do get stepped on, you just say "hey man, you just stepped on my
toes", they say "sorry", and you figure out what to do about it.

Most of the time, however, you simply are not stepping on toes.

> How is someone supposed to announce, "I am going to replace morphic" and
> do it in trunk? It wont happen.

It does not have to.  There are two possible ways to proceed.  The first
is to do the big change against the last stable version.  Once it is
done, you tell the world about the great thing you've done, and then
decide whether to port forward to the moving trunk, or to wait for the
next stable version before porting forward (the right choice will depend
on the situation).

The other option is to do track changes to the trunk as you develop your
big change in a private repository.  Again, the right choice will depend
on the situation.  The important point is that you can still do things
the old way; no options are taken away.

>  Not only that but the person who wants
> to make a radical proposal has no place to contribute it now. 

There's still Mantis (but I understand the point you make later about
eventually getting the change into trunk... see below).

> Anyone
> else who has big plans, has to get his ideas through Andreas, because he
> is defacto in charge of the trunk repository. If you have a big plan,
> and start putting it into practice in the repository, you will break a
> few things, and that will not work with the updates mechanism.
>   

The process assumes that developers will be responsible.  If you're
making changes that are likely to have bugs that can disrupt others'
work, then you should be available to fix those bugs.  Otherwise, don't
commit the potentially-disruptive code until you will be available to
deal with the fallout. 

If you have what you believe to be innocuous changes, then the standard
is lower.  If your wrong, and someone else has to fix your mess, then
you should learn a lesson.  If you keep causing people problems, your
commit access will be revoked.

> So in summary, the trunk is defacto in control of Andreas, only he knows
> what he is going to do next, and this then limits everyone else. They
> are not idiots, but they have no means to plan "interventions", because
> they cant read Andreas' mind, and neither can Andreas read their mind.
>   

I see your point, although I don't think that it will play out that way.

Let me ask a question: do you believe Andreas when he says that this
isn't his goal, or do you think that his public statements mask his true
intentions of silently taking over?  I know, because I know him, that
he's sincere when he says that he doesn't want the process to depend on
him.  His ego doesn't need the boost, and he doesn't want to spend the
time over the long-haul.  If he sees that the process is dependent on
him, then he'll look for a way to fix it.  He's a very smart guy (not
just at writing code), and a good listener.  I think that we should give
him the benefit of the doubt that he'll be able to make the right
adjustments.  Of course, we can't grant him the benefit of the doubt if
we don't think that his heart's in the right place.  Which it is :-)

> Squeak as yet again hitched its wagon to one person's view of the way
> forward. However the fact that forks exist indicate that this isn't the
> way to go, and we need a process that fits the forking/forked world better.
>
> Trunk is a linear development model where only one thing can be
> integrated into the image at a time, and that thing has to be totally ready.
>   

I understand what you're saying, but it's generally true in practice. 
In our code-base, people often make changes to smooth the way for
subsequent changes.  The initial changes don't add any new functionality
or fix any bugs, but they prepare the way.  The intention of the changes
is captured in the Monticello comments. 

For large additions, the communication problem is a bit bigger for an
open-source project than for a company, but for most smaller changes
there will be no difference.

>> (BTW, I hink that many people find this condescending... you are quite
>> literally saying that only those who agree with you are doing any
>> thinking;
>>     
> Not at all, I am saying that the situation does not allow the kind of
> thinking that is required, and anyone who does the thinking offline,
> will not be able to apply it, and get their ideas accepted for several
> reasons.
>   

OK, thanks for the clarification.

> Firstly they will have difficulty deploying their ideas into trunk,
> because trunk is moving continuously. To get an idea accepted they need
> to gain mindshare, the best way of doing that is to show it working they
> need to integrate it into the image. To integrate it into the image they
> need to gain mindshare.
>
> The only way of having a big idea integrated is is the trunk guys stop
> work on their stuff moving their ideas forward for a bit, and say right
> we are going to spend time integrating X's contribution. But X's
> contribution has to be completely ready.
>
> So big progress is now limited to linear one integration of a completed
> thing at a time, at Andreas' behest.
>   
>> only by reading between the lines can we infer that "it" must refer to
>> something more specific.  Not exactly conducive to a constructive
>> dialogue).
>>
>>     
>>> The old old process had a significant barrier to contribution. That
>>> barrier was "the release team". This process has a barrier too, its call
>>> the "release team a la Andreas". Conceptually nothing has changed.
>>>   
>>>       
>> I don't buy that.  
>>     
> Well thats the way it was, and now apparently still is.
>   
>> Either it's a chaotic free-for-all, or we're substituting one
>> bottleneck for another; it can't be both.
>>
>> Forgive my presumption, but for the rest of my response I'm going to
>> assume that you'll pick "chaotic free-for-all", because that seems
>> more like the gist of what you've been writing.
>>     
> Its a chaotic free for all, for those that are in the inner circle, a
> bottleneck for everyone else.
>   
>>> it cant be refined over time,
>>>   
>>>       
>> What?!?  Isn't "incremental" synonymous with "refined over time"?
>>
>>     
> Incremental means here is your new method and it must work, the
> increment is not for refinement, every increment is a release.
>
> No one can post a non-working work in progress to the update stream that
> is for the community to refine.
>   

OK, I see your distinction, but you're wrong that the new process
doesn't support refinement over time  You can keep your work-in-progress
in a private repository, periodically merging changes from the trunk as
you go.  This is how I work every day, it works great!

>   
>>> it has to work.
>>>   
>>>       
>> Hopefully most of the time, yes.
>>     
>
>   
>>> My model is a pull model. You pull what you want, bob pulls what some
>>> think would make a great release, those in forks can pull what they feel
>>> would best contribute to their goals, 
>>>       
>> OK so far...
>>
>>     
>>> and those things get planned,
>>> implemented and refined over time. Those things can be small fixes or
>>> revolutionary things like new gui's.
>>>   
>>>       
>> ... but now I disagree.  You seem to be arguing against a strawman.
>>
>> Revolutionary things like GUIs are actually easier to develop when you
>> are continually integrating changes during development.  Of course I
>> don't inflict the horrible brokenness upon people while I develop it;
>> I do it in my private Monticello repository.  So, there's no
>> disadvantage compared to your approach.  On the other hand, there are
>> significant advantages (please correct any misconceptions about your
>> approach).  By continually merging as I develop, I am able to take
>> advantage of the latest features and bug-fixes, and avoid having to do
>> it all at once just before posting it on Mantis (or wherever).  I
>> could approximate this incremental development pattern with Bob by
>> repeatedly generating new images, and porting my work-in-progress to
>> the new image.  However, this is inconvenient because I build up a lot
>> of working context in my images (workspaces full of code snippets,
>> to-do lists, long-lived inspectors on objects, etc.), and it is
>> inconvenient to recreate this context.
>>     
> You work in your image, you pick other peoples fixes from mantis as they
> come along and load them into your image, and you publish to your MC
> repository  bob builds test images from your MC repository, and the
> minimum set of fixes that you nominate are necessary.
>   

So I continue working in my image, and use the Bob-built images only to
test that things aren't breaking?  OK, I now understand the workflow
that you propose; it wasn't at all clear to me before.  Is this how you
work?

Nevertheless, you still have to keep track of what fixes are on Mantis,
so that you know what you want to pull in.  Plus, Mantis may have
multiple conflicting fixes to the same part of the system, so I may have
to massage the fixes to work together.  I'd rather trust the people who
are making those fixes to integrate them in the trunk; I have better
things to do with my time, like actual development.  I'm aware that this
is a fundamentally optimistic view of our community, but I've seen an
existence proof that this can work, so am willing to give it a try here.

> Your new gui needs to be published as a unit, together with a minimum
> set of fixes that are necessary for it to work. Then other forks can use it.
>   
>> I don't see how one process supports things that "get planned,
>> implemented and refined over time", but the other doesn't.  Can you
>> explain to me why it is impossible to do this in the new process?
>>
>>     
> Because the plan isn't written in code. The only way for you to know
> what the plan is is to read others minds.
>   

There is nothing in the new process that precludes planning, whether via
Mantis or some other means.

>>>> 4) if there is a demand for lean, Monticello-less images, then Bob can
>>>> easily build them
>>>>
>>>> You seem to take it as a philosophical necessity (axiomatic) that MC
>>>> must be external, when in reality the question is one of pragmatics. 
>>>>     
>>>>         
>>> Your "pragmatics" leads to every version of squeak having different
>>> tools, and no way of updating them to what is current. Your pragmatics
>>> leads to 4 different diverging forks of the MC tool set. Your pragmatics
>>> leads to versions of squeak being released with significant parts of MC
>>> tools not even working correctly (3.10 Monticello Configurations)
>>>   
>>>       
>> No, it does not.  Not if we care about backward-compatibility.
>>     
> I am telling you what has happened historically and is happening before
> my very eyes now.
>   
>> Say that the community has been progressing the trunk for 9 months,
>> starting at 3.10.2.  Now, it has been decided (somehow) that it's
>> about time to make the next stable release X.x.x (the version numbers
>> aren't important).  Bob will try out the latest versions of various
>> packages (let's say Monticello) in 3.7/8/9-lpf, and may find out that
>> stuff is broken.  If so, it probably won't be hard to fix, either by
>> extending LPF on the older platforms, or by rewriting some of the
>> changes in a less disruptive fashion.  The process can even happen
>> earlier, if someone notices changes that will cause problems, or if
>> the developer solicits opinions.  Why do you think that this can't work?
>>     
> Because fixes written for integration into 3.11 have to be
> simultaneously loadable into 3.10. Any 3.10 user with an existing image,
> must be able to apply fixes in order to maintain some level of API
> parity, and thus this facilitates him moving his code to the new image.
>   

You are taking a very radical philosophical position: you are saying
that literally every change to the system must be verified to work in
all stable versions of the system before it can be accepted.  Can you
point to another successful project, ever, that has worked this way?  I
don't think that there is one.

Someone using 3.10 shouldn't expect to be able to load from 3.11-alpha
trunk.  When 3.11-stable is released, they can load from that.  If
someone (whether human or Bob) verifies an intermediate version as OK,
then it can be "tagged" (meaning copied to another repository) as OK for
other versions to load. 

> All fixes that are included in 3.11 must also be published as individual
> ad ons for users in 3.10. Infact the build process builds 3.10+fixes
> first, and then applies the 3.11 generating script.
>
> There is no place for an intermediate "some image that some folks have
> worked on and made incompatible with previous releases 

Sure there is.  Because it allows development to go faster. 
Cross-version-compatible versions can still be verified and published at
the same frequency as before.  The only difference is that there will be
more unverified/potentially-incompatible changes in the middle.

> because they felt
> like it"
>   

"Because they felt like it" is loaded language... I already said what I
think of "SmalltalkImage current blah", so you know what I think of
gratuitous incompatibility.  I'm confident that such changes will be
discouraged.

>> Granted, this assumes that people care to make the effort.  But if
>> they don't care, then your process can't magically fix that.
>>
>> BTW, I think that we should try not to break compatibility
>> gratuitously.  My (least) favorite example is typing "SmalltalkImage
>> current blah" instead of "Smalltalk blah".  Why?  There is only one
>> SmalltalkImage (Hydra was not conceived until years later).  So, for
>> no benefit other than some ill-defined notion of "cleanliness", we
>> have more to type and less compatibility.  The good news it that I
>> think that we've grown past that (or at least, we can wage
>> Wikipedia-style undo/redo battles against such inanities :-)  ).   But
>> I digress...
>>
>>     
>>> My pragmatics results in 3.7-lpf 3.8-lpf 3.9-lpf croquet-lpf Pharo-lpf
>>> 3.1.2-lpf and 3.11 all having the latest MC tools all of the time.
>>>   
>>>       
>> Yes, as a result of the hard work of one prima donna,
>>     
> Actually I merged the work of several contributors, and others have
> contributed. Merging forks is not being a pre-madonna, its clearing up
> other people's messes.
>   

I thought my point was clear.  I wasn't implying that you're a prima
donna, but instead that the term is just as applicable to you as to
Andreas (i.e. not very).

>> the latest MC is available across multiple Squeak versions.  Sorry, I
>> couldn't resist.  But, there is an element of truth in the jest... how
>> does that constitute evidence that your process works, when similar
>> efforts by Andreas constitute evidence that his doesn't?
>>     
> If Andreas was to take the contributions of every version of MC that had
> ever existed and was to merge them all and then move things forward, I
> would be all for it. But I doubt he has time.
>
> That's what I did.
>   

We know.  And, sincerely, thank you for your work. 

>   
>>>> Borrowing Edgar's "Devil's Advocate" hat for a second... why does
>>>> Squeak-3.10 come with the Network package installed?  Surely this can
>>>> be loaded manually by anybody that wants it, right? 
>>>>     
>>>>         
>>> Yes, why indeed, and this has been done with the Kernel Image.
>>>   
>>>       
>> Sorry, I didn't make myself clear, because that wasn't my point. 
>> 3.10.2 does contain the network code.  Why?  Perhaps I should have
>> gone with the example that I was originally going to use, the
>> Compiler.  Just as easily as for the Network code, we could create an
>> image without the Compiler package.  But we don't.  Why?  For the same
>> reason I describe below.  For the *desired purpose*, an image without
>> a compiler is not as useful as one that has it. 
>>
>> Similarly, for an image whose *purpose* is to support the process that
>> Andreas introduced, it is more useful to put it in than leave it out.
>>
>>     
>>>> The answer is that the convenience outweighs any the negatives for the
>>>> Squeak community as a whole.  Someone running embedded Squeak on a
>>>> submarine-robot might have to strip out the network code to make it
>>>> fit, but nevertheless the network code "belongs" in the base image
>>>> simply because of the result of this cost-benefit analysis.
>>>>     
>>>>         
>>> But if the Network code is unloadable and replacable, then it doesnt
>>> matter if you release an image with it or without it. If you want it it
>>> is there, if you don't it is unloadable and replacable.
>>>   
>>>       
>> Maybe I should have used the term "unload" instead of "strip out"... I
>> apparently gave you the wrong idea.  Of course, I don't mean to imply
>> that Monticello should have tendrils deep into the system such that it
>> is difficult to unload.
>>     
> The release should simply include the latest version of every external
> package. The release/trunk should not patch or fix any external package.
> If Andreas wants to work on MC he should join the MC team and work in
> the MC repository.
>   

Ah, I see where you're coming from.

> Yes compiler can be an external package, as can Network, MC is, SUnit is.
>   
>>> If you think about the problem in terms of modularity, 
>>>       
>> I do...
>>
>>     
>>> then we can move
>>> forward. If you think about things in terms of an image that someone
>>> owns, then you lock people in.
>>>   
>>>       
>> ... and I don't (think in terms of an image that someone owns).
>>
>> Of course, development in Squeak cannot happen without an image.  As
>> an open-source project, Squeak thrives (or fails to) with the
>> contributions of its community.  Therefore, when someone visits
>> squeak.org, the first image that they find to download should be one
>> that promotes contributions.  For an experienced developer, they can
>> contributed immediately.  For a newbie, they gain visibility into the
>> process.
>>
>> What do you mean by "lock people in"?  I cannot see how what I
>> describe above constitutes lock-in, by any stretch of the imagination.
>>     
> Lock in occurs when you pick an image to base your project on. You are
> then locked in to the choices that that fork's release team makes on
> your behalf.
>   

Only if you can't migrate to another image.

>>>> If Monticello is the way that the development trunk is maintained,
>>>> then I want it in the same image.  To keep it outside is to generate
>>>> unnecessary friction in the process, both to contributors and to those
>>>> who want to simply want to keep up-to-date.
>>>>     
>>>>         
>>> Its got nothing to do with whether a release image has it loaded or not.
>>> It has to do with how you treat it, and maintain it.
>>>   
>>>       
>> OK...
>>     
>>> Keeping MC outside the image,
>>>       
>> ... or in it, since you just said it doesn't matter.
>>
>>     
>>>  means that MC is developed and maintained
>>> as a service to the whole squeak community, by a benevolent (and
>>> hopefully appreciated) team working for the good of everyone. This then
>>> means that the MC team has the freedom to implement new features, like
>>> atomic loading and binary packages for the good of all.
>>>   
>>>       
>> This can happen in the new process, no problem!  It just needs some
>> benevolent (and hopefully appreciated) people working on it.  You
>> apparently care about cross-version MC compatibility more than anyone
>> (thank you!).  And I would venture to say that nobody is opposed to it.  
>>     
> So why dont they see using it as even a low priority.
>   
>> So, if you bring the same energy to the new process as you did in the
>> old one, then you will continue to be successful, and people will
>> continue to enjoy the fruits of your labor.  If some dumbass
>> inadvertently screws things up, then you can help them understand what
>> they did wrong so they can fix it.  By sharing your knowledge and
>> wisdom, the community will more quickly learn the ups-and-downs of
>> cross-version development.  In the unfortunate case where someone
>> obstinately insists on making technically-indefensible changes, the
>> SOB reserves the right to revoke commit privileges.
>>     
> Because this process is unworkable, it doesn't provide any slot into
> which I can contribute.
>   

I don't see it that way.  See below... can't Bob be used to detect
incompatibilities introduced during on-going development?

>> On the other hand, consider some other package (say Balloon3D,
>> although it's probably not the best example).  It hasn't been actively
>> developed in a while, but all of a sudden someone feels a burst of
>> enthusiasm.  But, they don't care (or maybe don't have the time) to
>> make their changes backward-compatible.  Your process doesn't have a
>> magic solution for this; it still takes someone who cares to put in
>> the effort.  It's unfortunate that Squeak 3.7 users will be stuck with
>> the same crufty old B3D, but at least the new process makes
>> contributions easier, and makes it easier for the enthusiasm of the
>> moment to drag in other users to help (which would be very unlikely
>> with a Mantis-based process).
>>     
>>> Maintaining it as part of a release image limits it to serving the users
>>> of a not yet released image only, which represents the minimum of MC
>>> users out there.
>>>   
>>>       
>> Again (because advertisers know that repetition works ;-) )... there
>> is nothing in the new process that forces incompatibility,
>>     
> And no way of knowing either way.
>   

Why can't Bob be used to test intermediate versions from the trunk?  If
incompatibility is detected, it can be addressed sooner rather than
later?  Maybe I misunderstand what it's good for.

>> and nothing in your process that ensures compatibility.
>>     
> Yes there is. Developing the new release based upon the fixed point of
> the old release, does ensure some compatibility. Any incompatibilities
> found can be rectified on the next build. ALL fixes are by definition
> backportable because they are developed for the old image not the new one.
>   

Are we being honest here?  I'm admitting that your approach makes it
easier to achieve compatibility, but with the cost of increased friction
to contributions; this seems accurate to me.  (I hear that) you are
saying that your approach *ensures* that there will be compatibility,
where as the new process *ensures* that there will not be.  This doesn't
seem accurate.

Cheers,
Josh

> Any big packges for integration in the new release can also be
> backported to the previous release if necessary, because the fixes
> needed are backported be definition.
>   
>> Thanks for the explanation, I feel like I understand where you're
>> coming from a bit better.  Do you feel the same?
>>
>> Cheers,
>> Josh
>>     
> its a pleasure
>
> Keith
>
>   

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20090716/0a9e1928/attachment.htm