On Jan 23, 2010, at 5:43 PM, keith wrote:

> 
>> Wow, you're really not willing to concede a single point, ever, are you?
> 
> Hi Josh,
> 
> You are really not understanding my point. So I didn't concede your point, because your point seemed to me to be being scored against the wrong target. 


That's completely counter-productive.  If you think that my point is off-target, just tell me.


> You appear to be attempting to point score on the relative technical merits and possible uses of individual scm tools. Whereas I was attempting some abstract thinking about the process here, and the actual tools are not that relevant.


OK, I believe you.  But it seemed like you were focusing on the tools.    I understood that you were saying that Monticello wasn't the right tool because it didn't support working in a distributed mode.  You apparently confirmed that when, in response to me arguing that Monticello *is* substantially like git/mercurial/bazaar, you didn't say that I was missing the point.  Instead, you elaborated (incorrectly) about how Monticello doesn't support this mode of working, and prompted me to argue in more detail that "yes, I can and do commit to a local repository on the train, and then copy the results to a different repository once I'm connected to the network".

BTW, now that we've agreed that it doesn't matter, do you concede the point that Monticello is substantially like git/mercurial/bazaar from a technical perspective, even if we're not using it in the way that the Mercurial folks advocate?  ;-)


> 
> You appear to repeat the same mistake when you have a go at "bob", bob is only a hacked together second attempt at a concrete tool for attempting to implement an abstract idea, a new way of working and thinking about the problem, in other words a different paradigm.  You attack the tool and its implementer, on its lack of technical merits, and my inadequate capabilities, without understanding the value of the abstract thinking behind it. The deliverable of bob, is the demonstration and framework that allows you to think using the new paradigm, not fundamentally the tool itself. You say to me, ah ha, Lukas has developed a new CI tool, which is better than yours, so its ok we can ignore you, throwing bob away is justified. But, Lukas is using his new tool simply as a way of automating the old paradigm. Conceptually, you load an image from scratch, load some packages and test them, this is how Lukas is using his tool. You can do this manually, there is no new paradigm demonstrated here.
> 


I think that the above is more like an amalgam of various things that people have said to you, not things that I myself have written (I've never heard of the CI tool that Lukas created, and don't have any opinion on it vs Bob).  But fair enough.

The only part I would take issue with is that I'm only criticizing the implementation, not the idea.  I think that many of my posts indicate that I'm trying to grapple with the idea.


> Lets try again once more.


BTW, in advance of my comments and questions below, let me thank you for making the effort to lay everything out at once.  It doesn't clear up all of my questions and concerns, but it offers a real chance that we can stop going in circles.

Also, I'm making a deliberate effort not to say anything controversial in this post.  The goal is to establish a fixed point for subsequent conversation, so that we can stop going in circles.  


> 
> 1.
> 
> Conceptually, there is one Seaside3 repository held centrally, and one team developing it, and there is one Seaside3 Head, for the entire world. This is the essence of the centralised approach. (Of course, anyone can do what they want locally) Linux may use distributed tools, but conceptually it is being controlled by one individual Linus, so it is also fundamentally a centralised model of thinking, no matter what the tools are able to achieve in practice.
> 
> Your point that Monticello can work in a distributed fashion is fine, of course it can, but in practice, it isn't used that way. There is only one head for Seaside3, and there is only one head for Linux, since effectively the Linux kernel is only one product with one gate keeper, and there is only one head for "trunk" too.
> 
> Fundamentally 3 years ago a small group of us recognised that innovating new features on just "one head" was a thing of the past for squeak, we already have 8 or more maintained forks of varying descriptions. That's what the pharo guys were doing too just innovating for themselves, and with every commit pharo was making it harder to maintain packages in common with squeak. The pharo team's response to this dilemma was to say "we dont care", the squeak team and the boards response to this dilemma 3 years ago was to say "we do care". We will write tools that fundamentally expect to fix, innovate and test against multiple forks. We can let pharo be the hare, but we can win in the long haul as the tortoise, because our innovations will be more carefully implemented and tested, and will be more useful to more people with existing code bases to support.
> 
> So as I was saying, it is an abstract point, that conceptually the way we think when we do development, is basically to swarm around a centralised resource, be it a repository, an image or a person. When we do this, we are signing up to depend on that resource for the longer term.


OK, I'm with you so far.


> 2.
> 
> Along comes Mercurial, a new kid on the block with a new fundamental concept.
> 
> They put forward a different abstract idea fundamentally. When you use mercurial, every checkout/branch is a head, and is equal at every level and is indistinguishable from the original (In fact unlike, git and bzr, mercurial, only really supports any notion of centralised repositories, and a lightweight checkout because some users demanded it for pragmatic reasons)
> 
> With mercurial when you branch, you get everything. Your "head" is equal to and as good as anyone elses, and you could take your work to the moon, and forget the rest of the world exists, forever.
> 
> The equivalent in the squeak world is that when I branch to develop something on top of seaside3, I get the whole of the seaside repository to take with me, and I get my own seaside3 head. I can now make innovations to seaside locally, and I can provide a new seaside head to the world if I want to.
> 
> Now that is a silly idea isnt it, I hear you think, what would the world want with, and what is the use of, 15 different heads to a product that is obviously being developed by one team.
> 
> However, if you read the documentation, the Mercurial guys then recommend an entirely new approach to working. What you do, is you make a separate branch relative to a fixed point (A), (a release for example) for each new feature or bug fix, (B, C and D), (or developer working on a feature) in your product. This ensures that each feature may be tested and developed by itself, relative to a known starting point, the branch ancestor. 
> 
> The result being that you locally get all the little deltas of each little change (B.1->B.2->B.3) you make when developing that feature, but when you come to publish, you also have one big delta (B.3 - A) for the whole feature. That big delta for that feature is not contaminated with the work on other features.
> 
> So, when the time comes to make a release you make a new fresh branch (A.2) from the old fixed point (A), you pull from each of the innovations in turn and merge them in (A.2 = A + (B3.-A)+(C.3-A)+(D3-A)), (this is the opposite of the above 1, in which the innovators all effectively push to a central repo. or gatekeeper). In addition, you don't then automatically require the developer of that new feature to continue his work from your new head (AA). He can continue to produce B.4 B.5 and B.6, so that when you merge in his work again. You merge (A3= A.2 + B.6-B.3).
> 
> Are you getting the point that the mercurial approach is a different paradigm, it is fundamentally distributed.  (even though technically you can use it in a centralised manner)
> 
> So... I was suggesting that with Cuis, we could switch from a mindset that thinks centrally, to a mindset that thinks distributed.
> 
> At present there is conceptually one head for the world for everything, i.e. Squeak3.10 + 700 packages in squeak map.  
> 
> My thought was that when you download a distribution built on cuis, you conceptually get all 700 packages, and all their history. When you work you adapt everything you need to work for cuis, and you ignore the rest of world. You could in fact over a very long weekend delete FileDirectory and port every single package to something better.
> 
> Then when you publish your result, it becomes a brand new distribution "Sooty", equal in status to the first, each of the packages you load is fully manageable in the context that it is being used. Others interested in your innovations can study them in context, and pull them as they desire.


I see what you're saying.

I started to write responses to a couple of other posts, but I realize that there are still things that I don't understand about your proposal.  Please continue to help me understand.

Let's imagine that 3.10.2 just came out, and that we have your process in place.  I'm assuming that we can equate 3.10.2 to A.  The important part that I'm missing is that I don't know the time interval between A and A.2.  Is it on the order of a week or two, or 6 months?  Once I know the answer, I'll have more to say.

Also, you often refer to the codebase that you're developing against A that you will eventually want to move forward to whatever the latest is (maybe A.2, or maybe A.10).  How often do you envision porting your packages forward to the latest?  Every week or two?  Every 6 months?   Does it depend on the specific developer/codebase?


(BTW, I'm curious, can you cite an example of a successful project that uses this approach?  If you can't, that doesn't automatically disqualify your idea (someone has to be first), but I'm still curious.  Please, just a short yes or no, with no loss-of-face or giving-up-ground implied in a negative answer).


> 
> 3.
> 
> When you tell me that my bob process relying on Mantis is no good, as many have attempted to do. This criticism is based upon point scoring because they dont think that technically these tools would do it as well as they would like. Basically saying imho "mantis is crap so anything that uses mantis will also be crap", by definition.
> 
> What you fail to understand is that my bob process is not built on mantis, it is built on the abstract principle, that you start with a known fixed point and develop each innovation as a separate delta.
> 
> Then you can apply those deltas individually to several forks. If for some reason a delta doesn't work in a fork, then the fork can be fixed to address the problem. Fixing the fork actually results in all the forks converging to be closer to the original fixed point, and some essential commonality is preserved. Every innovation that you make for just your own image, that you do not make available for other forks to try, is potentially destroying commonality. Every innovation and patch that you make available for other forks to try is potentially improving commonality.
> 
> Mantis just happens to be the place where the fixes are at the moment. Several developers all innovate relative to that known fixed point. Bug fixers develop bug fixes relative to that known fixed point, and then the next release can be produced by cherry picking innovations that are actually complete, and bug fixes that are complete and tested.
> 
> So here is where "bob" could conceptually comes in...
> 
> With bob you can load your version of package A (e.g. Rio), as you maintain it in your local distribution, in your fork, and automatically test it. You make your "rio testing bob" open, so that anyone can add their distribution to the list. Bob downloads their image from their ftp site when it changes, and adds it to the test schedule. So, you can develop package A, seeing the impact you are making on every derived distribution in bobs list. When you see that package A's new feature is going to work for everyone, you can announce that feature and those who are interested can pull it over. At the same time, each user of package A, is also running the tests, so they get to see if they accidentally undermine package A, breaking by doing something silly. 


You should cut people some slack, because you have been anything but consistent on this point over the past months.  You're focusing on the abstract principles now, but at other times you've said that Bob/Mantis is good to go, and that we should start using it immediately.

Having said that, I'm comfortable with agreeing that criticisms of Mantis/Bob (i.e. the current concrete implementation) are out-of-bounds while we consider the abstract principles involved (of course, if we get to the point where you convince the community that you're correct in principle, then criticizing Mantis/Bob is back on the table as we decide *how* to implement your ideas).


> 
> 5.
> 
> Currently, in its technically limited way, using Bob, and Sake/Tasks, you can write a script made of a series of tasks that you think would pull 200 fixes and update to the latest packages to make 3.10 into 3.11.
> 
> Then you can run that script against 3.9, and low and behold, 95% of that script will execute. Any parts that need a tweak, can be tweeked by subclassing or overriding the task that failed. In this manner some fixes can be omitted, or extra ones loaded if needed.
> 
> Then you can run that same script against your production image based upon 3.9, and it might well pass too. If it does, and you fix your production image to work, then this means, that all the code in your production image, can migrate to 3.11 seamlessly.
> 
> The net result will be, that there will be a new monticello, squeakmap, sunit, in squeak 3.11, but also, there will potentially be a 3.9.2 with that same new monticello, squeakmap and sunit, and also a 3.8.4 with that same monticello, squeakmap and sunit. All generated and tested simultaneously.
> 
> If the pharo guys want to pull your 3.10-3.11 innovations, they simply add a subclass for pharo, let bob build Pharo1.0+(3.10 to 3.11 delta script) and see what works and what doesn't. Or they can see exactly what steps you went through to make 3.11, and they can cherry pick the ones they like. If the pharo guys made Pharo 1, from squeak 3.9 in the same manner then you could pull their innovations too. (this is the principle that the mercurial guys are aiming to support)
> 
> The more different forks, that load common fixes, (and I know not everything will work) the more compatability we will have, even though the kernels of the forks may be different.
> 
> Before you tell me that this wouldn't work, may I point out that LPF already did this for 8(?) different squeak forks, you can download 3.7-lpf, 3.8-lpf, cobalt-lpf etc etc.
> 
> In addition, Sake/Packages knows the package dependencies for all the packages that are loadable in 3.10, and 3.9, so bob can produce. 3.7-lpf + sake/packages = 3.7-build, 3.8-build, 3.9-build, cobalt2.0-build etc.
> 
> So, we can update and maintain 8 different forks to have the same scm tools, the same package management tools, and the same testing tools. 
> 
> So... if we can achieve this by pushing out fixes and packages to different forks, then applying the same principles, of publishing the list of changes we make to move from, 3.10 to 3.11, as code. Other forks based on 3.10 can pull what they want of the changes, but they are not forced to accept everything we do. For example, I want a fork of 3.10 which uses Rio, so I wouldn't pull any of the fixes to FileDirectory.
> 
> The current situation is that "trunk" is the only head, and all contributors are working on it, because they are following their esteemed leader. The chances of removing FileDirectory from trunk is 0, because of the legacy, it will never happen.
> 
> However if I choose to fork 3.10 and make an image which doesn't have FileDirectory, (YAY!), then I am on my own, because "trunk" is no use to me. However if "trunk" innovations were of use to me, then I wouldn't be on my own.
> 
> 3.11 with FileDirectory can be tracked to produce a parallel 3.11 with Rio full integrated and FileDirectory removed..
> Seaside with FileDirectory can be tracked to produce a parallel Seaside with Rio.
> 
> eventually FileDirectory can die a graceful death.
> 
> its just an idea


OK, gotcha.  Are we ready to proceed? (did I manage to avoid saying anything that you violently disagree with?)

Cheers,
Josh



> 
> Keith
> 
> 
> 
> 
> 
> 
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20100124/da7149de/attachment.htm