On Tuesday 24 September 2002 08:38 am, Randal L. Schwartz wrote:

...

...

...

...

...

"Richard" == Richard A O'Keefe ok@cs.otago.ac.nz writes:

Richard> 6. Eventually, if all the necessary machines are up, you'll get a list Richard> of projects in the right-hand pane. Look for "LearningMorphic.015.pr". Richard> Click on that to select it. Richard> 7. You'd really like to copy that to your disc so you can use it over Richard> and over again. You aren't given that choice. Bring up the menu Richard> and select "load as project".

Walkback in current image: "You can't add Modules at the top level".

Try it in 3.2

Hi Scott,

On Tue, 15 Oct 2002, Scott Wallace wrote:

At 8:38 AM -0700 9/24/02, Randal L. Schwartz wrote: This noxious "You can't add Modules at the top level" roadblock probably explains, more than any other single thing, why a large percentage of the Squeak community, including most of our most illustrious and powerful members who until the arrival of Modules had *always* lived and worked in the leading-edge Squeak development system, have steered away from 3.3a.

[Snipped excellent analysis of the reaction to 3.3a's module policy.]

But then it turned out that much old code couldn't file in, so that even people with no interest in Modules, and with no perceived current need for them, could not go on about their business.

I think you're hitting the nail on the head here. We have a technical policy difference that doesn't seamlessly "do the Right thing for niewbies". A user interface that consists of an error message for 99% of the Squeak contributors out there is the worst possible interface. If Exceptions were employed, that would be another thing, but Exceptions aren't integrated into Squeak's environment well enough.

I attach FWIW a tiny workaround that has helped lower *my* blood pressure as I operate in 3.3a, since I have frequent need to load code written in earlier Squeaks.

My solution does not modularize non-module-aware code, but it should at least allow you to *file in* your code. Classes which previously generated the "You can't add Modules at the top level" error will now be graciously accepted and placed in submodules of People. Once you have the code on board, you can proceed to use the Modules tools to make your code more module-savvy when and if you want to.

I'd be interested in hearing feedback on this naive but expedient little hack. Should it go into the image? It does not deal with any potentially awkward issues, it just brings code in, rather than rejecting it, so that you can start working with it from within Squeak.

Or would this be too much of an affront to the integrity of the Modules system? Or does anyone care anymore?

I'd rather ask "Why wasn't this added before?". I definitely do care and would like to see this policy incorporated.

Cheers,

-- Scott

Thanks! Brian Rice ~

Hi Scott and all!

Scott Wallace scott.wallace@squeakland.org wrote:

At 8:38 AM -0700 9/24/02, Randal L. Schwartz wrote:

...

...Walkback in current image: "You can't add Modules at the top level".

{sigh}

Hmm, the original posting hasn't reached me yet, but anyway.

[SNIP of true observations]

I'd be interested in hearing feedback on this naive but expedient little hack. Should it go into the image? It does not deal with any potentially awkward issues, it just brings code in, rather than rejecting it, so that you can start working with it from within Squeak.

Sure. IMHO, go for it. 3.3a is *alpha* so hey, fix it - that's what alpha is for... :-)

Or would this be too much of an affront to the integrity of the Modules system? Or does anyone care anymore?

Nope, I don't think it would and yes, I care for Modules - but see no problem with improving it! And since it is alpha mistakes are ok - we can always fix it while we go forward.

We all know development on Modules stopped before it was really userfriendly (and no shame on Henrik, he did a lot of work but there was too little support from the rest of us - and that is noones fault) but it's still IMHO pretty solid code - it just need to get finished! :-)

There was a flurry of activity with Andreas carrying the flag but lately it got very silent. Personally I try to spend my time on SqueakMap for 3.2 and when that is "done" in some sense I intend to start looking at 3.3 more closely - I believe there are a whole bunch of .cs floating around for Modules that need to get integrated.

Cheers,

-- Scott

Cheers, Göran

Hi henrik

Just to refresh some memories here. We tried to help if you remember. Alex started to build some tools for modules. Now the question how it is possible that 4 smart guys cannot understand the modules system to the point that they can explain it. I think that this was the reason of the failure of the modules.

I still think that the question of splitting the image in packages is crucial. I'm waiting to see what andreas did because he told me that he was working on that.

Stef On mercredi, octobre 16, 2002, at 12:44 pm, Henrik Gedenryd wrote:

Scott Wallace wrote:

...

I'd be interested in hearing feedback on this naive but expedient little hack. Should it go into the image? It does not deal with any potentially awkward issues, it just brings code in, rather than rejecting it, so that you can start working with it from within Squeak.

Or would this be too much of an affront to the integrity of the Modules system? Or does anyone care anymore?

I don't know whether to laugh or cry. The current block was not in the initial module system. It was only added after SqC people complained about the super-strange things that happened when it would just silently swallow outdated code!!

Memorably, members of SqC got some bugs which they absolutely couldn't understand why they happened. (The code relied on hacking deep down into the old handling of global variables. Gee I wonder why that would fail now.)

Plus there were updates released that corrupted the system, to which I had to release update patches.

But by all means go ahead and release it (and then wait X months for the proposed anti-patch to surface again). As you said, noone probably cares any more anyway. Out of the dozens of people who promised to contribute, 0 (zero) have made good on what they said then. I don't even think the last set of important fixes that I sent out was ever released as updates. But I too do not care.

Henrik

Dr. Stéphane DUCASSE (ducasse@iam.unibe.ch) http://www.iam.unibe.ch/~ducasse/ "if you knew today was your last day on earth, what would you do different? ... especially if, by doing something different, today might not be your last day on earth" Calvin&Hobbes

There have been a several addons/fixes etc made by several people, but we haven't integrated them yet AFAIK. Alex did his stuff, Andreas, Daniel and more have done a lot too. I still think that we can pull this off - we just need to "do it". The problem has probably more to do with people trying to make money to pay the rent etc.

...

Now the question how it is possible that 4 smart guys cannot understand the modules system to the point that they can explain it. I think that this was the reason of the failure of the modules.

1. No - I don't agree, Modules isn't that hard to understand. But you

need to TRY.

Sorry Goran. I do not want to be rude in public. But we tried and how this is possible that I cannot explain the modules system in 5 sentences. Or may be alex, roel, nathanael, andrew and I are just stupid. May be.

I don't claim to understand all of it in depth but at least enough to grasp it. I mean, hey - essentially (cutting away some small details): a) a Module is a bunch of classes (globals) and you need to explicitly "import" the other Modules that you use in your Module. Very similar to import/include in all the {}-languages. b) The Modules are arranged in a global namespace hierarchy. Very similar to Java. c) The only thing remotely hard to understand here is IMHO DeltaModules and they are more or less like a ChangeSet that *only* affects one given Module. Another way of saying this is that a DeltaModule D specifies the difference (like a patch-file) that needs to be applied to a Module A version 1.0 to turn it into version 1.1.

I do not like this idea of difference but this is my problem.

I do not like deltamodules. Then the hierarchical view of modules is just about names reservation and this is mixed with modules structure here.

Ok, what was hard about this? Yes, I am being a bit pushy here because all this talk about Modules being so hard is getting on my nerves. Don't feel offended - nothing personal.

2. Modules has NOT failed. It's in 3.3 *alpha* which means that it

isn't ready for use yet. We just need to finish it.

So I'm waiting to see it failed. Let say it like that. Have you see how many changes made by the community have been harvested recently. How many people are using 3.3a. Not that much.

If you just take a look at the code Henrik wrote and which especially Daniel and Andreas (if I am not mistaken) has hacked a bit further on - you will see that a lot of it works very good.

I'm seeing that the code is bad. I'm saying that the model is complex. because what is needed is a concept for grouping and manipulating class definition and method definition. I discussed with daniel at ESUG and he told me that now he understand the idea behind Ginsu. The model was small and simple. But joseph succeeded to pissed of everybody so this is great.

So cool we are living in a happy world. I looked at the code of henrik. Alex too. Believe me.

regards, Gˆran

Dr. Stéphane DUCASSE (ducasse@iam.unibe.ch) http://www.iam.unibe.ch/~ducasse/ "if you knew today was your last day on earth, what would you do different? ... especially if, by doing something different, today might not be your last day on earth" Calvin&Hobbes

...

how this is possible that I cannot explain the modules system in 5 sentences. Or may be alex, roel, nathanael, andrew and I are just stupid. May be.

Nope, you most definitely are not. But I still don't really see what it is that is so hard. Obviously I have misunderstood it all.

We can understand any hard stuff without problem but why the model is not that simple that I even have to think about it. That's the real point why modules and deltamodules....

...

...

I don't claim to understand all of it in depth but at least enough to grasp it. I mean, hey - essentially (cutting away some small details): a) a Module is a bunch of classes (globals) and you need to explicitly "import" the other Modules that you use in your Module. Very similar to import/include in all the {}-languages. b) The Modules are arranged in a global namespace hierarchy. Very similar to Java. c) The only thing remotely hard to understand here is IMHO DeltaModules and they are more or less like a ChangeSet that *only* affects one given Module. Another way of saying this is that a DeltaModule D specifies the difference (like a patch-file) that needs to be applied to a Module A version 1.0 to turn it into version 1.1.

I do not like this idea of difference but this is my problem.

Well, liking is one thing. But do you understand it? I must say I found it pretty neat.

It has two very nice consequences:

1. Modules (regular ones) could then simply be a bunch of classes.

Period.

But modules could be a bunch of classes and method definitions. Period

No need of deltamodules that have to be located under module...blblbalbalbablabl

2. Any "changes" needed in other code to make a Module happy where

collected in the DeltaModules. This meant that the DeltaModules carried all the "dirty tricks" and made them visible and concrete. This is a good thing.

And finally, since a DeltaModule essentially is the a "cleaned up" version of ChangeSet we could use it for those purposes too. Remember that a DeltaModule is reversible - this is a very important thing to have. According to Henrik (I haven't looked at them much) ChangeSets are pretty weird beasts...

But may be we should simply throw away changeset or used them as a storage mechanism. \

Sidenote: Just look at SqueakMap for 3.2. We now have single-click installation of .cs files working fine. But deinstallation will of course never work since ChangeSets are not reversible.

I would loved but a book is grilling and a smalltalk lectures (lucky me) fall literally on me, so please show it to roel at OOPSLA

...

I do not like deltamodules. Then the hierarchical view of modules is just about names reservation and this is mixed with modules structure here.

AFAIK the hierarchy (currently) has two purposes:

1. A hierachical naming space makes finding stuff easier. Nothing

technical about that - it's just nice to have it hierarchical. 2. Since we then suddenly have a hierarchy we can choose some meaning to attach to the child-relation (or we could choose not to - but why not?). IIRC some operations work on a Module recursively. Loading/unloading/activation/deactivation to be precise. But apart from that I don't think the child-relation has any other "role" in Modules.

Personally I think we could perhaps let the operations mentioned (loading/unloading/activation/deactivation) instead work only through the dependency graph and simply ignoring the child-parent hierarchy. I think Stephen Pair has made that proposal earlier. This would leave the hierarchy as being only for naming.

I agree I think that naming should not be mixed with childparent. What a mess!

If anything I write is wrong - correct me. I don't have these things fresh in my head since I too have not been in 3.3-land for quite some time... :-)

Me too

...

...

Ok, what was hard about this? Yes, I am being a bit pushy here because all this talk about Modules being so hard is getting on my nerves. Don't feel offended - nothing personal.

2. Modules has NOT failed. It's in 3.3 *alpha* which means that it

isn't ready for use yet. We just need to finish it.

So I'm waiting to see it failed. Let say it like that. Have you see how many changes made by the community have been harvested recently. How many people are using 3.3a. Not that much.

I would say that harvesting has been down for *both* 3.2 and 3.3 - that has nothing to do with Modules.

Yes it has because as an harvester I do not know how to proceed.

In fact almost all community efforts in Squeak have been down and I don't think it has much to do with technical reasons - it has to do with the IT-industry.

not only

...

...

If you just take a look at the code Henrik wrote and which especially Daniel and Andreas (if I am not mistaken) has hacked a bit further on - you will see that a lot of it works very good.

I'm seeing that the code is bad. I'm saying that the model is complex.

It the code really "bad"? I didn't think so, of course code can always be improved but I didn't think it was "bad". And yes - the model is complex - but is it *unnecessarily* complex? I am not so sure.

...

because what is needed is a concept for grouping and manipulating class definition and method definition. I discussed with daniel at ESUG and he told me that now he understand the idea behind Ginsu. The model was small and simple. But joseph succeeded to pissed of everybody so this is great.

I am not so sure - Ginsu is a model describing Smalltalk source code - separately from the Smalltalk model itself. At least that is my understanding.

But we could have retrofitted in Squeak. The separation was not the key point. The key point was that we could manipulate code such a global Variable definition.

Modules is instead an extension to the Smalltalk model of classes. Way back when I talked with Henrik about this I found his arguments and model much more appealing than having an "extra" model like Ginsu.

Henrik could find convincing arguments but now what do we need: a good packaging mechanism and a simple one.

For example - the ability to have classes loaded in the image (fully compiled etc so that tools work and so on) but not having them activated in the name space seemed like a brilliant approach. And activation through a big become-operation seemed very neat.

Indeed

I talked with Henrik about Ginsu and he said that he had looked at it and thought that similar goals could be achieved better using other techniques. And then he wrote Modules. Since he knows this stuff infinitely better than me (especially compared with Ginsu) I can only hope he posts his thoughts.

...

So cool we are living in a happy world. I looked at the code of henrik. Alex too. Believe me.

I believe you. And sorry for my tone in the last post - I was a bit... wound up. ;-)

regards, Gˆran

Dr. Stéphane DUCASSE (ducasse@iam.unibe.ch) http://www.iam.unibe.ch/~ducasse/ "if you knew today was your last day on earth, what would you do different? ... especially if, by doing something different, today might not be your last day on earth" Calvin&Hobbes

But you could have method definition and removal in a module without needing the distinction between delta modules and modules. How do we do that with parcel, envy applications exactly like that. I say I take the module 1.29a then I removed the method xxx and I version it again as 1.29b

For me the distinction between modules that are clean and delta that are only methods has always been strange. Stef

On mercredi, octobre 16, 2002, at 04:01 pm, Ned Konz wrote:

On Wednesday 16 October 2002 06:52 am, Stephane Ducasse wrote:

...

But modules could be a bunch of classes and method definitions. Period

No need of deltamodules that have to be located under module...blblbalbalbablabl

Hi Stephen, I'm not up to speed with modules, I guess, but how (without DeltaModule or something like it) would you express a patch/fix to an existing module that included method *removal*? -- Ned Konz http://bike-nomad.com GPG key ID: BEEA7EFE

Dr. Stéphane DUCASSE (ducasse@iam.unibe.ch) http://www.iam.unibe.ch/~ducasse/ "if you knew today was your last day on earth, what would you do different? ... especially if, by doing something different, today might not be your last day on earth" Calvin&Hobbes

So do not see why we need modules and just keep deltamodules, and rename it delta. This distinction is ad-hoc. But if you believe it is worth the conceptual mess go for it. I imagine me teaching it....dramatic. Stef

On mercredi, octobre 16, 2002, at 06:34 pm, goran.hultgren@bluefish.se wrote:

Ok, perhaps a long post but anyway - IMHO it's important that we have this cleared up.

Stephane Ducasse ducasse@iam.unibe.ch wrote:

...

But you could have method definition and removal in a module without needing the distinction between delta modules and modules.

But why mix those together? It will only mean that: a) I can not make the assumption (as you can today) that a Module is *stand alone* and can't possibly conflict with anything else. b) A Module turns into some form of ChangeSet (recording of operations) instead of a snapshot of classes.

...

How do we do that with parcel, envy applications exactly like that. I say I take the module 1.29a then I removed the method xxx and I version it again as 1.29b

Hep - that is different (I assume). 1.29b does not contain a *removal* of a method. It simply does not contain the method you removed. Big difference.

...

For me the distinction between modules that are clean and delta that are only methods has always been strange.

Eh, a DeltaModule does definitely NOT contain "only methods". Let me repeat this one more time: A DeltaModule is more or less like a ChangeSet. And as you know this means that:

1. It can contain loose method additions/removals.
2. It can contain class definition changes like variable

additions/removals, superclass changes etc. 3. It can contain complete additional classes. 4. In fact it can contain any conceivable change to a Module.

(the above might not be implemented yet, but this is the plan/design)

The whole point of the DeltaModule is that it contains all changes you would like to do to *another* Module that is not yours in order to make *your* Module happy. So think of it as a ChangeSet with the restriction that it can only change *one* Module. And then add the very important aspect of reversability. A DeltaModule is activated and deactivated - when it is activated its changes are applied and when it is deactivated the changes are removed.

So again, what are the nice properties of this scheme?

1. A Module is thus tremendously simple. It is just a bunch of classes.

No history, no additions/removals or anything - it is very much like a category .st-fileout. It can't get simpler than that, and simplicity is our friend.

2. A Module can't possibly conflict with another Module since it is in

it's own namespace. This is a very important aspect. You can load all and any Modules into the image without any worry whatsoever, after all it's only classes and simply filing in classes never hurt anybody (unless the class initialization does dumb things, but that is another story).

3. As we all know somewhere the "dirty business" must be taken care of,

otherwise all these Modules turn into isolated islands of no connection at all and just sits there like dumb ducks. :-) And this is the job of the DeltaModule. Think of it as a ChangeSet which can be reverted (activated/deactivated) and that can only affect ONE Module. Since DeltaModules carry all your changes in Modules that are not your own they by definition contain the areas of your code that CAN conflict with others.

So - the separation is very clear to me. When I write code everything I write can be separated into two groups:

1. My own classes.
2. Changes in other peoples classes.

Voila! Modules and DeltaModules. Since I control my own classes I can represent that code as a snapshot of the classes. But I don't control other peoples code so those changes are best represented as *recorded changes* to those classes.

Ok, now if you do not like DeltaModules - what do you propose instead?

You can't just say - just put it in the Modules, because then you are simply moving the problem into the currently nice, clean and simple Modules. And you can't just say "use ChangeSets" because they are not reversible and if I am not mistaken have other weird problems... The DeltaModules are meant to BE the ChangeSets of the future.

regards, Gˆran

Dr. Stéphane DUCASSE (ducasse@iam.unibe.ch) http://www.iam.unibe.ch/~ducasse/ "if you knew today was your last day on earth, what would you do different? ... especially if, by doing something different, today might not be your last day on earth" Calvin&Hobbes

On Wed, 16 Oct 2002, [ISO-8859-1] G�ran Hultgren wrote:

Ok, what about all you other people out there? Are my explanations making any sense? Henrik - could you please acknowledge that I haven't made any gross factual errors? Daniel, what do you think?

I continue to stand by the Modules codebase as a good start, but I am always open for improvements.

It seems to me that part of the problem with these discussions of modules is that nobody has been terribly clear about what they hope to get out of them. Perhaps if we make our goals clearer, it will be easier to talk about the possible solutions. Here are a few things that I would think a module system might provide:

1. A way to specify related pieces of code. This has to include both entirely new classes and patches/additions to existing classes. Let's call these "packages". 2. A way to easily save and load these packages to and from easily distributable files. 3. A way to cleanly update an image with a new version of a package. 4. A way to cleanly unload a package from an image. 5. A way to analyze dependencies between packages. 6. A way to specify dependencies between packages. 7. A way to protect packages from name clashes. 8. A way to organize packages hierarchically.

IMO, 1-3 are the most important. Not surprisingly, these are also what DVS mostly provides, although it works better for straight additions to existing classes than for patches to them. The interesting thing is that this can be provided without any changes to the base image, as a simple addon package.

4, I think, could also be done fairly easily on top of 3.2, but it seems much less important to me: once I've loaded something into an image I rarely want to remove it. If I'm testing a new package, I use a throwaway image. I guess that assumes a minimal base image to load things into.

Daniel has already done 5, again in 3.2. I don't see why 6 would be very difficult - it ties into what Daniel and I were talking about with subclasses of Module to store metadata. 6 also seems like something that SqueakMap might be extended to do.

7 is, of course, the big one: it is the real justification, as far as I'm concerned, of the work that's beem put into 3.3a. But how big a deal is it? Are people routinely struggling with naming conflicts? I tend to throw a 2 letter prefix in front of my class names and forget about it. But maybe that's just me.

8 would be nice, but the major benefits (being able to load/unload groups of packages at once) can be gotten through dependecies + dummy packages, the same way most of the linux packaging systems work.

This is a highly personal take on things, of course - I'm not pretending that these preferences will be the same for everyone. I'd be curious to hear what, for example, Stef's or Henrik's version of this list would look like. But me, I'd rather see a lightweight module system in 3.2 without namespaces or hierarchy get widely used first, and then start to think about the full Module system a la 3.3a.

Avi

On Wed, Oct 16, 2002 at 10:52:03PM +0200, Göran Hultgren wrote: <snip>

Oh yeah, forgot to add my list. One thing off the top of my head though: Image building instead of image stripping. That is a biggie in my book.

If we want this, it would be good to store objects as well as classes/methods in modules. For example, say that there is a Wonderland module. Where would the bunny come from? A method to generate the mesh is conceivable, but ugly. It would be better to have a serialized mesh object that can be understood once the code from the module is read in.

I think that this would be great, but would add complexity to a problem that has already shown itself to be difficult enough. Eventually, it will probably be far easier to add this feature to Modules than it will be to refactor the image into modules that can be sequentially filed in.

Joshua

regards, Göran

Göran Hultgren, goran.hultgren@bluefish.se GSM: +46 70 3933950, http://www.bluefish.se "Department of Redundancy department." -- ThinkGeek

On Wed, 16 Oct 2002, [ISO-8859-1] G�ran Hultgren wrote:

A sidenote: One reason for the Modules work being so... uncoordinated is the lack of a good "CM" tool like CVS or similar. Would you say (I trust your word) that DVS is the best thing going in this respect for Squeak currently? If that is the case, then perhaps we should consider using it for all the Modules-related cooperative work that needs to be done in 3.3. Currently there are a whole slew of .cses floating around that needs to be integrated etc. The lack of something CVS-ish (and I am talking about the very simple update/commit-cycle) is IMHO a big hurdle for real cooperative development in the Squeak image and the Modules area is a good example where we really need this.

Yes, I would say that DVS is the best thing currently going (the only real alternative I know of is CVSTproj, and DVS is much more usable currently). However, DVS is aimed squarely at 3.2 - I have no idea if it makes any sense at all in a 3.3 context, so doing work on the 3.3 modules with it may be a non-starter. You'll have to tell me.

...

Daniel has already done 5, again in 3.2. I don't see why 6 would be very difficult - it ties into what Daniel and I were talking about with subclasses of Module to store metadata. 6 also seems like something that SqueakMap might be extended to do.

True but I have steered away from that since I didn't want to duplicate efforts

• Modules already has functionality for it.

It would be a pity if people started to "rebuild" stuff (that is already implemented partly in 3.3 Modules) in 3.2 instead of helping out and finishing the job in 3.3...

Well, it not only would be a pity, it *is* a pity, because that's exactly what's happening. I avoided for a long time doing anything that seemed like it was duplicating the 3.3 Modules, but here's the problem: I need this functionality *now*, and I need it to work with all the Squeak tools I currently use. Reimplementing the 10% of the module system I needed in 3.2 was *way* less work than finishing the 3.3 module system + getting all the current 3.2 tools I need updated to work with it. IMO, if a module system is going to get accepted, it's important that a) whatever features it has work, and b) it's backwards compatible. That way people can slowly migrate to it, rather than freaking out because it's currently too broken to work in. Incrementality is the key here. I realize there may be people for whom this isn't true, and so much the better, but for me: there's no chance I can give time to working *on* a version of Squeak that I can't also work *in*.

...

7 is, of course, the big one: it is the real justification, as far as I'm concerned, of the work that's beem put into 3.3a. But how big a deal is it? Are people routinely struggling with naming conflicts? I tend to throw a 2 letter prefix in front of my class names and forget about it. But maybe that's just me.

I agree but if you have read the Swiki pages about Modules you realize that there are a lot of cool stuff there. The ability to separate loading from activation, working on different versions of classes/modules, clean atomic stages of downloading all dependencies, loading them all into the image and finally activating it all, the global namespace and so on.

Sure, that stuff's all extremely cool. The problem with 3.3 modules is not necessarily technical, but social: I'm beginning to repeat myself, but I'd be much happier with a tiny subset of that *that people actually used*.

Cheers, Avi

----- Original Message ----- From: "John Sarkela" sarkela@cox.net To: squeak-dev@lists.squeakfoundation.org Sent: Wednesday, October 16, 2002 2:17 PM Subject: Re: Squeak Starter

Hi gang,

After a year away from Smalltalk I have come back to the light. Still catching up on all the changes... phew quite a task. Still much of the chatter reminds me of my reaction to Les Miserables... "such a lot of fuss over a piece of toast!" ;-}>

Well, I am sorry to see things going like this. I don't recall volunteering any effort on the current incarnation of Modules in Squeak, but I do recall warning of the difficulty of this task based on my own experiences with Oasis. I look at these posts and to me it seems that folks have seriously underestimated this problem- I get the distinct impression that everyone involved expected this to be a slam-dunk. It's not. There are a lot of folks, including myself, who have a distinct interest in this. We are all blind. Please do not get too excited when one of us says that "Modules are Trees!" while another says that "Modules are Ropes!".

As for complaints about a lack of assistance... this is a recurring theme in most large Squeak projects. I've posted many times over the years about what I was doing with Oasis- to me, it seemed that noone was interested. However, folks were interested... but busy. This happens. Happened to me, happened to Eric Clayberg ( I think... re: Pocket St ). Happened to the StrongTalk foks, I'd imagine - I'm very interested in that project, but busy. Happened to lots of people. Don't undertake a large project and expect a lot of feedback- lack of it does not mean that the project has no interest, just that you need to approach the buy-in a bit differently. The techniques for gaining buy-in that I tried failed, but that doesn't mean that what I was doing was of no value. I thought it was sufficient to let people know what I was doing- that was not correct.

We are a small community, but a rather intelligent one. The problems we routinely tackle are not always easy ones. There have been a lot of constructive contributions to the modules discussions, and those should all be considered. Somewhere in there, a system that makes sense for the highly dynamic Smalltalk world is lurking.

I'd be happy to talk with anyone at OOPSLA this year about Squeak & Modules. I can only discuss my own efforts regarding Oasis ( as I said, I've been rather busy ), but I am always happy to share whatever meaningful ( or meaningless ) insights I may have.

- les

On Thursday 17 October 2002 02:33 am, goran.hultgren@bluefish.se wrote:

...

I'd be happy to talk with anyone at OOPSLA this year about Squeak & Modules.

I will be there! :-) And so will Doug Way and probably a whole bunch of other people. Why don't we simply (separated from the SqueakBOF) do a Modules BOF or something? I think that would be good.

OK, when and where?

Since 3.2 will be the stable release for quite some time this is good to hear. I still have to look into DVS.

A sidenote: One reason for the Modules work being so... uncoordinated is the lack of a good "CM" tool like CVS or similar. Would you say (I trust your word) that DVS is the best thing going in this respect for Squeak currently? If that is the case, then perhaps we should consider using it for all the Modules-related cooperative work that needs to be done in 3.3. Currently there are a whole slew of .cses floating around that needs to be integrated etc. The lack of something CVS-ish (and I am talking about the very simple update/commit-cycle) is IMHO a big hurdle for real cooperative development in the Squeak image and the Modules area is a good example where we really need this.

Even more on the side: Of course what I REALLY would want is an interactive update/resolve conflicts/commit-cycle using DeltaModules in 3.3 thus creating the CVS-optimistic-concurrency model of development with intelligent conflict resolution tools instead of simple "oops, those lines are too close"-logic. And then coupled with a bit of realtime feedback inside the browsers on who is doing what and where and you have my wet dream of a CM system...

They had this in 1980 at PARC, it was called PIE....

Reading the PIE paper at http://www.dolphinharbor.org/dh/smalltalk/documents/ made me realize how applicable the concept is to Modules, Sand Boxing, Name spaces, and other goodies...

http://www.iam.unibe.ch/~schaerli/research/index.htm

See Stephen's work on Traits, and compare to the ideas in Modules, NameSpaces, and PIE.

http://www.iam.unibe.ch/~schaerli/research/index.htm

They all intererlate.

Once again, PARC was ahead of the curve. Should I explain further?

I've had PIE, Traits, and NameSpaces boiling around in my head for a week now, and how each could be used to...

Allow modules Allow namespaces Allow sandboxing ( preventing DL'd code from doing nastyness, a Jail/Chroot environment for Squeak Code ) Allow a cool NameSpace system...

And Traits, well, Traits kick butt....

-Daniel

These issues all interrelate...

Le 2002/10/16 à 21:23, Göran Hultgren goran.hultgren@bluefish.se a écrit:

Hi all!

Hi Goran

Quoting Stephane Ducasse ducasse@iam.unibe.ch:

...

But if you believe it is worth the conceptual mess go for it. I imagine

Ok, what about all you other people out there? Are my explanations making any sense?

Even i¹m not an expert what I readed about modules, and what you wrote about it look really clear to me at least Œconceptualy¹, and in continuity with the last year debate.

Like you I don¹t see why the concept of a DeltaModule is bad. May be it is unusual...to define it as an active element (can reverse what it did) seems even quality. Anyway this is needed, we have to Œroll back¹ or Œforward¹ the code to adapt it and doing this actively instead of by definition only seems a good way, to me.

I try also SqueakMap which is Œan absolute need¹ and a model for it simplicity of use. So, if we can push ahead the Henrik module work and made it simple to use even for new Squeakers, it will be a hit.

I wish I can help in that work but it is not the case. I¹m learning Squeak the hard way (alone) and will begin a small team of two others Squeakers this week-end (the only way not to be alone seems to be Œmake a group¹ ) So i will begin to teach Squeak to two non-programmer but interested people...why not.

I continue to stand by the Modules codebase as a good start, but I am always open for improvements.

regards, Göran

Ciao Goran

On Wed, 16 Oct 2002, Viktor wrote:

Such a small benchmark: Visual C++ (see code bellow) ... 14 ms ... 31 %

2 GHz Pentium Whatever:

15254 (microseconds)

Not quite as fast, but mine is interpreted. :^p) Script appended.

Ian

#!/usr/local/bin/uvm

(require 'timeval)

(define gettimeofday (system.dlsym 0 "gettimeofday")) (define tv0 (timeval.malloc)) (define tv1 (timeval.malloc))

(gettimeofday tv0 0)

(let ([a (system.malloc 4)] [i 10000000]) (while (set! i (- i 1)) (set! (word a 0) 3)))

(gettimeofday tv1 0)

(let ([t0 (+ (* 1000000 (timeval.sec tv0)) (timeval.usec tv0))] [t1 (+ (* 1000000 (timeval.sec tv1)) (timeval.usec tv1))]) (system.printf "%d\n" (- t1 t0)))

On Wed, 16 Oct 2002, Viktor wrote:

Such a small benchmark:

[ | a | a := Array new: 1. 10000000 timesRepeat: [ a at: 1 put: 3.] ] millisecondsToRun

Strongtalk (best time from 10 runs): ... 45 ms ... 100 % VisualWorksR NonCommercial, Release 5i.3 of January 30, 2001 ... 220 ms ... 488 % Dolphin Smalltalk Pro 4.0.1. ... 820 ms ... 1822 % Squeak3.2-4956 ... 2410 ms ... 5355% Kathmandu Objects (my poor Smalltalk) ... 7881 ms ... 17513 % Visual C++ (see code bellow) ... 14 ms ... 31 %

DWORD BegTime = ::GetTickCount(); DWORD *Array = new DWORD [1]; for (DWORD Index = 0; Index < 10000000; Index++) { Array [0] = 3; }; DWORD TotalTime = ::GetTickCount() - BegTime; delete Array;

To be a little bit more fair you should write the Squeak benchmark exactly like the C++ one:

| a | a := Array new: 1. 1 to: 10000000 do: [:i| a at: 1 put: 3.]

Using #to:do: gives a speed-up by 100% here!

-- Bert

PS: of course, we should change the implementation of #timesRepeat: accordingly

Viktor wrote:

...

To be a little bit more fair you should write the Squeak benchmark

exactly

...

like the C++ one:

| a | a := Array new: 1. 1 to: 10000000 do: [:i| a at: 1 put: 3.]

Using #to:do: gives a speed-up by 100% here!

-- Bert

PS: of course, we should change the implementation of #timesRepeat: accordingly

No, it is not "more fair". All Smalltalks run the same code, C++ was here just for... I don't know exactly why... just for fun?

I know that many fortran compilers would just optimize the whole loop away. Are you certain that the compiled C program wasn't also similiarily affected?

On Wed, 16 Oct 2002, Viktor wrote:

...

I know that many fortran compilers would just optimize the whole loop away. Are you certain that the compiled C program wasn't also similiarily affected?

-- Travis Griggs

No, I supposed such optimizations, but I was wrong.

If you look closely, it does, partially.

Here is disassembled code:

436: void CSelfDlg::OnBuildObject() 437: { 0040EF40 53 push ebx 0040EF41 55 push ebp 0040EF42 56 push esi 0040EF43 8B 35 28 40 41 00 mov esi,dword ptr [__imp__GetTickCount@0 (00414028)] 0040EF49 57 push edi 0040EF4A 8B D9 mov ebx,ecx 0040EF4C FF D6 call esi 438: DWORD BegTime = ::GetTickCount(); 439: DWORD *Array = new DWORD [1]; 0040EF4E 6A 04 push 4 0040EF50 8B E8 mov ebp,eax 0040EF52 E8 A5 31 00 00 call operator new (004120fc) 0040EF57 8B F8 mov edi,eax 0040EF59 83 C4 04 add esp,4 0040EF5C B8 80 96 98 00 mov eax,989680h 0040EF61 8B 0F mov ecx,dword ptr [edi] 440: for (DWORD Index = 0; Index < 10000000; Index++) 0040EF63 48 dec eax 441: { 442: Array [0] = 3; 0040EF64 B9 03 00 00 00 mov ecx,3 0040EF69 75 F8 jne CSelfDlg::OnBuildObject+23h (0040ef63) 0040EF6B 89 0F mov dword ptr [edi],ecx 443: }; 444: DWORD TotalTime = ::GetTickCount() - BegTime; 0040EF6D FF D6 call esi 0040EF6F 8B F0 mov esi,eax 445: delete Array; 0040EF71 57 push edi 0040EF72 2B F5 sub esi,ebp 0040EF74 E8 77 31 00 00 call operator delete (004120f0) 0040EF79 83 C4 04 add esp,4

Essentially, the above is:

mov ecx,Array[0] label: dec eax mov ecx,3 jne label mov Array[0],ecx

The compiler did put the array element into a register. This saves a memory write for each step.

--Bert

To be a little bit more fair you should write the Squeak benchmark exactly like the C++ one:

| a | a := Array new: 1. 1 to: 10000000 do: [:i| a at: 1 put: 3.]

Using #to:do: gives a speed-up by 100% here!

-- Bert

Now 1229 ms, 2731 % of Strongtalk.

Code (class method of Time)

microbenchmark

"Time microbenchmark"

^Time millisecondsToRun:

[ | a | a := Array new: 1. 1 to: 10000000 do: [:i| a at: 1 put: 3.] ]

Viktor

Have you seen Andreas browser? It has a notion of "current Module" and whenever you change another Module it automatically creates DeltaModules for you in the correct way. You can go by your daily coding business just like before.

Clearly the way to go but still I do not understand why we need two concepts. henrik was fuzzy explaining that to us when he visited us too. There is no comceptual different between a clean class def and a class def plus a bunch of methods sorry.

regards, Gˆran

Dr. Stéphane DUCASSE (ducasse@iam.unibe.ch) http://www.iam.unibe.ch/~ducasse/ "if you knew today was your last day on earth, what would you do different? ... especially if, by doing something different, today might not be your last day on earth" Calvin&Hobbes

On Wed, 16 Oct 2002, subroto mukherjee wrote:

Hi ! I have recently worked on a project on behalf of my University, which involved Squeak. I was quite intrigued by the software and would like to know what effective role it can play in my career since i've recently completed my postgraduation in Multimedia Information System and also i would like to know if there is any Higher level (M. Phil or PhD) qualification or perhaps another MSc. programme available in Squeak ?

Of course, I'd be interested in this, also. I'm working towards finishing my BS in Biology, with plans to go to grad school for ecology work, perhaps working on ecological models in Squeak. However, I'd be interested in persuing a MS or PhD in CS (my minor), perhaps to work on increasing biologist/ecologist field work effectiveness via special purpose application software running on handhelds in the field. Just an idea. :)

Regards, Aaron

Aaron Reichow :: UMD ACM Pres :: http://www.d.umn.edu/~reic0024/ "the end of the human race will be that it will eventually die of civilization. " :: r. w. emerson

On Wednesday 16 October 2002 22:58, John Hinsley wrote:

On Wednesday 16 October 2002 9:16 pm, Aaron J Reichow wrote:

...

Of course, I'd be interested in this, also. I'm working towards finishing my BS in Biology, with plans to go to grad school for ecology work, perhaps working on ecological models in Squeak. However, I'd be interested in persuing a MS or PhD in CS (my minor), perhaps to work on increasing biologist/ecologist field work effectiveness via special purpose application software running on handhelds in the field. Just an idea. :)

Folks, ecology (albeit in the very broad context of architecture) is one of the areas the astonishing and irrepressible Albertina Lourenci (on the Self list) is involved in.

Jecel, if you think this is of interest to Albertina or that she might like to contribute, do you think you could paste her the thread?

Opening this can of worms might annoy many people. It would be best for anyone interested to look at her papers and contact her directly:

http://www.lsi.usp.br/~lourenci/

-- Jecel

On Thu, 17 Oct 2002, Jecel Assumpcao Jr wrote:

...

Folks, ecology (albeit in the very broad context of architecture) is one of the areas the astonishing and irrepressible Albertina Lourenci (on the Self list) is involved in.

Jecel, if you think this is of interest to Albertina or that she might like to contribute, do you think you could paste her the thread?

Opening this can of worms might annoy many people. It would be best for anyone interested to look at her papers and contact her directly:

http://www.lsi.usp.br/~lourenci/

Oohh, interesting stuff on this thar page. :) An interesting mix of architecture, CS, and urban resource management.

Regards, Aaron

Aaron Reichow :: UMD ACM Pres :: http://www.d.umn.edu/~reic0024/ "Like the creators of sitcoms or junk food or package tours, Java's designers were consciously designing a product for people not as smart as them." :: paul graham

Hey, grad school is a great place to be period! (They usually let you actually do work.)

Cheers,

Alan

-----

At 7:27 AM -0700 10/17/02, JohnMaloney@earthlink.net wrote:

Subroto,

You might consider going for an M.S. or Ph.D. with Mark Guzdial at Georgia Tech. His students all use Squeak from the undergraduate level up, and they've contributed heavily to Squeak over the years. Mark wrote one book on Squeak and co-edited another. I think he'd be a terrific thesis advisor.

The economy in the U.S. is bad right now, so I know a lot of Squeakers who are out of work (along with a lot of C++, Java, and web programmers). When it economy is bad, graduate school is a good place to be. With luck, you come out with a shiny new degree just as things are picking up.

-- John

At 10:03 AM -0700 10/16/02, subroto mukherjee wrote:

...

Hi ! I have recently worked on a project on behalf of my University, which involved Squeak. I was quite intrigued by the software and would like to know what effective role it can play in my career since i've recently completed my postgraduation in Multimedia Information System and also i would like to know if there is any Higher level (M. Phil or PhD) qualification or perhaps another MSc. programme available in Squeak ?

Subroto.

--

The following is really somewhat off topic...

... but if you care about congruence and have access to a good library I suggest you (whoever you are) have a look at a book written by Gerald M. Weinberg:

Quality Software Management Volume 3: Congruent Action http://www.dorsethouse.com/books/qsm3.html

Gerald M. Weinberg also has a website: http://www.geraldmweinberg.com/

Andreas

Hi all!

Quoting Stephane Ducasse ducasse@iam.unibe.ch:

...

Have you seen Andreas browser? It has a notion of "current Module" and whenever you change another Module it automatically creates DeltaModules for you in the correct way. You can go by your daily coding business just like before.

Clearly the way to go but still I do not understand why we need two concepts. henrik was fuzzy explaining that to us when he visited us too.

There is no comceptual different between a clean class def and a class def plus a bunch of methods sorry.

But your statement is simply "wrong". A DeltaModule is NOT, and I repeat - NOT - just a bunch of methods! A DeltaModule is LIKE A CHANGESET.

It contains OPERATIONS like for example a method addition, a method removal, an instvar addition, an instvar removal etc. etc. In short it contains "changes" that can be applied (activated) and also reversed (deactivated). This is very different from a SNAPSHOT of a class.

Your statement can roughly be compared to the statement that "there is no conceptual difference between an .st fileout of a class and ChangeSet". And hopefully you agree with me that there is a difference there?!

So in short - a Module contains a bunch of classes. Nothing more. A DeltaModule contains your changes to classes in other Modules that you do not control.

Is there anyone else out there reading this thread that still do not understand this? If there is, speak up because this is important. We are a community and we decide where we want to go together.

regards, Göran

Göran Hultgren, goran.hultgren@bluefish.se GSM: +46 70 3933950, http://www.bluefish.se "Department of Redundancy department." -- ThinkGeek

Stephane Ducasse ducasse@iam.unibe.ch wrote:

Hi goran

Do not be mad (I know you can have some reason to be apparently I'm a trooooollllllll).

:-) I am not mad! Henrik might be but I am not.

I just do not have the time to reply. I have to finish a complete lecture on OOP for Monday and a kid at hospital (great birthday). What I just want to say is that

Hope he/she gets better. There *are* things much more important than modules in Squeak...

if modules are similar to changeset (i.e., contains method removal, addition, class changes), I do not see the point to have deltamodule and modules, we could only have modules = changeset + namespaces. Having a clean module with no class extensions is cool but why this is different than a module with?

Ok, let's see where this argument goes. Lets smack DMs and Modules together just as a thought experiment. This would mean (among other things) that a Module can contain it's own classes (just like now) and also classes which should be added to other Modules. Hmmm. Ok, so this would mean that each class must include information about in WHICH Module that class should be added, right?

And the same goes for each of the loose methods and the other "changes" that a DM can hold. Hmmm. To me it is quite plain datamodelling telling me that "Hey, wait a minute. Why don't I group these changes in ONE object that can contain the information of which other Module these changes are to be applied to?". And there you go - a DM.

Another good point is this (and I might be on loose ground here because I am not totally sure how this works today):

Activation/deactivation is *per Module*. If all the DMs were contained inside the Module you wouldn't be able to deactivate those DMs separately!

So if Module AddressBook has two DMs - one for fixing some stuff in say one of the base Squeak Modules and another one for good coexistance with the Celeste Module (adding a menu choice in Celeste), what happens when I load and activate AddressBook but don't have Celeste? As I said, I am not sure how it works today, but it could activate the first DM and not the second (because Celeste isn't there).

Now - how would this work if the two DMs didn't exist and the "loose methods" were instead floating around inside the AddressBook Module? Suddenly you would also need to keep track of which of the stuff inside the Module that is activated...

...

...

...

The whole point of the DeltaModule is that it contains all changes

you

...

...

...

would like to do to *another* Module that is not yours in order to

make

...

...

...

*your* Module happy. So think of it as a ChangeSet with the

restriction

...

...

...

that it can only change *one* Module. And then add the very important aspect of reversability. A DeltaModule is activated and deactivated

...

...

...

when it is activated its changes are applied and when it is

deactivated

...

...

...

the changes are removed.

But the reversability is not based on the difference between deltaModules and modules. You could have the same with modules (merging deltaModules and modules). Reservibility is just there because of namespaces.

Eh. No. Ok. This is roughly how it works (today):

Modules are in fact never really "activated". They don't need to be because of what they are. They just contain classes which sits there in their own namespace.

When DeltaModules are activated - lets say one DM with a method removal in it - it actually removes the method in question from the class in that Module. But it keeps that CompiledMethod around in order to "hook it back in" when the DM is deactivated again.

So the deactivation mechanism has nothing to do with namespaces AFAICT - a DM must simply keep track of state in order to be able to do an "undo".

...

...

...

So again, what are the nice properties of this scheme?

...

...

...

1. A Module is thus tremendously simple. It is just a bunch of

classes.

...

...

...

No history, no additions/removals or anything - it is very much

like a

...

...

...

category .st-fileout. It can't get simpler than that, and

simplicity is

...

...

...

our friend.

...

...

...

2. A Module can't possibly conflict with another Module since it is

in

...

...

...

it's own namespace. This is a very important aspect. You can load

all

...

...

...

and any Modules into the image without any worry whatsoever, after

all

...

...

...

it's only classes and simply filing in classes never hurt anybody (unless the class initialization does dumb things, but that is

another

...

...

...

story).

This is important but just linked to the fact that you have namespaces.

No. I can't understand what it has to do with namespaces.

...

...

...

3. As we all know somewhere the "dirty business" must be taken care

of,

...

...

...

otherwise all these Modules turn into isolated islands of no

connection

...

...

...

at all and just sits there like dumb ducks. :-) And this is the job

of

...

...

...

the DeltaModule. Think of it as a ChangeSet which can be reverted (activated/deactivated) and that can only affect ONE Module. Since DeltaModules carry all your changes in Modules that are not your own they by definition contain the areas of your code that CAN conflict

with

...

...

...

others.

...

...

...

So - the separation is very clear to me. When I write code

everything I

...

...

...

write can be separated into two groups:

1. My own classes.
2. Changes in other peoples classes.

...

...

...

Voila! Modules and DeltaModules. Since I control my own classes I

can

...

...

...

represent that code as a snapshot of the classes. But I don't

control

...

...

...

other peoples code so those changes are best represented as

*recorded

...

...

...

changes* to those classes.

What I want to say is that this is a code organization matter. You could do the same with one kind of module. Put your code into your modules and change to the system in other modules.

Hmm, can't see that. More on this below.

...

...

...

Ok, now if you do not like DeltaModules - what do you propose

instead?

...

...

...

You can't just say - just put it in the Modules, because then you

are

...

...

...

simply moving the problem into the currently nice, clean and simple Modules. And you can't just say "use ChangeSets" because they are

not

...

...

...

reversible and if I am not mistaken have other weird problems... The DeltaModules are meant to BE the ChangeSets of the future.

But changeset are not reversible because we do not add them a namespace. Imagine a changeset with a namespace, we could load it and unload it. Now we could also add variable initialization because we will need that. Then you can have class definition (including changes format), then you can have your own clean modules and deltaModules but without the need to have this separation.

Why would a namespace make a ChangeSet reversible? And what does it even "mean" to have a ChangeSet in a namespace? I don't understand that at all. AFAIK only globals are contained in namespaces - and concretely that means more or less classes and Modules.

Explain this to me. If a CS contains a removal of a method in class X and I load the CS, the method is removed, right? Now - what has a namespace got to do with undoing that removal?

Anyway continue your work on SqueakMap I thikn that this is important.

Thanks! I will. When I can stop answering all these posts on the list... The itch is unbearable!

I discussed with daniel at ESUG and he is right big bang changes do not work. So small increments are the way to go. I like the things he did for the dynamic menus and RB.

Me too. And generally I agree. I think Modules with namespaces may be an exception though... It is quite hard to "build incrementally" - some things need probably be done in big swoops. But that is just a guess.

He should also continue because this is important. If you can succeed to build a core of three or four people working together and trusting each other this would be great.

Just for your information. We were really working hard to produce the simplest modules system we could, since the modules discussion I have been reading everything that exist on modules, packages...

But for now I have the impression that we failed, Alex implemented several versions but our model has intrinsic problems and we could not find what we were looking for: asystem that let me play in my corner without impacting other but still been able to change other modules. I wrote a paper that I throw away.

Hmmm, tough!

We asked advices to some guys and were looking at SmallScript, old alen Wirfs-Brocks papers, other papers around. One of my dream would be to have a simple module system for language with extensions. but this is hard. You can discuss with Roel at OOPSLA (I will not attend because I have too much stuff to do).

Ok, I will talk with Roel.

So Henrik can say what he wants on me I do not care. We will see what we can do. I will not reply to other emails because I'm a trooooooolllllllllll

Please do reply. I don't think you are a troll and personally I think you both should "make peace" with each other. IMHO there is just too much waste of energy being upset over these things.

regards, Göran

On Friday 18 October 2002 07:36 am, Roel Wuyts wrote:

Stef has this kind of setup in mind, and in that setup there is no difference between modules and delta-modules. A module is something which has definitions and which you can apply on a system.

But that (a simple declarative system) not sufficient to do what we want to do with the system. There has to be a way to remove an existing method (at least for your code).

So unless you effectively define an entire module which consists of the original module MINUS the method, you don't get the behavior you want.

I _think_ the DM's are a way to do this kind of specification: "this is a module just like module X but with this removed, this added, and this modified".

That they are represented as deltas is an optimization that matches the way people work; one could make a new module by copying the original and then editing it. The addition of namespaces lets different modules with the same name coexist in the system, of course.

Hi doug

but in parcel or packages in VW you can change override methods and the shape of the classes too which is really trickier. Now you can also add conceptually a namespace to a envy applications and you get the unload facilities, this is what was used in VW30 to have parcel unload and automatic load. This is great you can load the parcels in any order the prereq are loading if they are found if one is missing at the end everyhitng is unloaded or not installed.

With Envy/Ginsu-style modules (which I assume are similar to what Roel & Steph are talking about), modules can contain classes, and can also contain "Class Extensions" (a.k.a. loose methods). I would define Class Extensions as methods which are additions to classes in other modules, but they are ONLY additions, NOT changes to existing methods (and not removals either).

DeltaModules, on the other hand, can contain method additions, but also changes to existing methods and method removals too.

So, to generalize a bit, DeltaModules are more powerful, but Class Extensions are quite a bit simpler.

Let's think about a couple of useful properties of modules:

1. Can they be cleanly unloaded?
2. Are they behavior-preserving? In other words, if you load (and

activate) a module, will the rest of the system still run without breaking or behaving differently?

Yes this is a key point. We worked on something that we called classBox that add some nice properties but also some wrong ;)

Regarding #1, 3.3a-style Modules and DeltaModules can be unloaded, and Envy/Ginsu-style modules can too, so no problems there. Changesets cannot be unloaded, which is their big limitation.

Regarding #2, this is one reason for the split between Modules and DeltaModules in 3.3a. If you load a Module (with no DeltaModules), the rest of the system will be unaffected, so they are behavior-preserving. DeltaModules are not behavior-preserving, but that's the point of them... they can be used if you do need to change something else in the system.

Because DeltaModules are not behavior-preserving, they naturally have to change version numbers of the Modules that they modify. So for example if a WebBrowser Module has a DeltaModule which adds the method #asEncodedHtml or changes a method in class String, in the Module "Kernel 2.7", the Kernel module will need to be assigned a new version number such as 2.8. (I believe this is how DeltaModules are supposed to work, correct me if I'm wrong.) This may have big implications for the rest of the modules in the system, which might expect to run on Kernel 2.7 but not know about 2.8.

But the interesting thing is, Envy/Ginsu-style modules w/Class Extensions *are* behavior-preserving, with a few caveats* (see bottom of message). This is because, in general, if you add a new method to an existing class, none of the rest of the system would ever send that new method, so the behavior of the rest of the system is unchanged. So, reusing the example above, if a WebBrowser module has a Class Extension which adds the method #asEncodedHtml (but does not change any method) in class String, in the module "Kernel 2.7", the Kernel module will still remain 2.7. (This is exactly what Envy does. I don't remember now if Ginsu supports version numbers.)

Exact we try to see how a class extension could be only working for the module that define it but in case of method modification that the system only calls your modification when you were invoking the other methods that would call the method you changed. It worked but we were not satisfied. Roel cna explain that at OOPSLA

The other question is then, if we're using Class Extensions, is it good enough to only be able to *add* methods to classes in other modules, not modify/remove them (as DeltaModules can do)?

Well, I guess that's a big question. :-) I would say that most of the time, yes, it's good enough. A module does not often need to change/remove methods in other modules, unless we're directly fixing a bug in that other module. Adding methods is much more useful, and can be specific to the "outer" module (such as the example WebBrowser module adding #asEncodedHtml to String). People who've used Envy know that it can be very handy... although you don't want to abuse it to an extreme

With Envy/Ginsu, if your module *really* needs to change/remove a method in Kernel 2.7, you make your own private version of Kernel 2.8 with the change, and/or you lobby the maintainer of Kernel to make the change. But this should be relatively rare aside from bug fixes.

On the other hand, the ability of DeltaModules to handle method changes/removals could be quite nice. And they seem a bit more rigorous, no "caveats" as with Class Extensions below. And namespaces are already built-in. But there is extra complexity cost... For example, DeltaModules have to worry about activation/deactivation, with Class Extensions it is irrelevant. The Class Extensions are always active when the module is active, since they don't affect anything else.

I think that supporting modification of methods is cool but makes things much more complex. Same for changing the shape of a class.

One last thing: As far as splitting up the current Squeak image into modules goes, my hunch is that it would be easier to do with Envy/Ginsu-style modules. This is because you would not need the DeltaModule capability of specifying changed/removed methods when divvying things up, only the ability to specify added methods, which the lighter-weight Class Extensions can also handle without version number worries. For example, String>>asMorph could be quickly dumped into some Morphic module.

(Avi, are the DVS "logical modules" similar to Envy/Ginsu with Class Extensions?)

Am I making sense here? This message is getting too long, sorry. Let me know if I've characterized one of the approaches incorrectly. And I know this is only one aspect of the differences between these two module systems. Anyway, I've often found that I learn more by see two things compared against each other, than by seeing them described separately.

• Doug Way

dway@riskmetrics.com

It does

*Caveat #1: There could be namespace conflicts with Class Extensions, which Envy/Ginsu don't handle. I'm guessing that adding namespace support is one thing that Roel & Steph were working on?

Yes I can send you a so not good paper :)

*Caveat #2: Ideally, to preserve behavior, you would not allow a Class Extension to override a method in a superclass. This would be a simple restriction to add. Otherwise, you could bring an image to its knees by adding a new method #size to SortedCollection which returns nil. :-)

this is price for freedom

*Caveat #3: If a base module class is doing something funky with reflection such that adding a new method changes its behavior, then the Class Extension might not be behavior-preserving. But this falls under the category of Very Rare, IMHO. (I guess an example might be adding a testXXX method to an SUnit test suite class.)

Dr. Stéphane DUCASSE (ducasse@iam.unibe.ch) http://www.iam.unibe.ch/~ducasse/ "if you knew today was your last day on earth, what would you do different? ... especially if, by doing something different, today might not be your last day on earth" Calvin&Hobbes

Hi all!

Quoting Doug Way dway@riskmetrics.com: [SNIP]

Okay, this is getting into a difference between these two approaches which needs to be fleshed out some more (and which I'm somewhat familiar with, at least :-) ).

With Envy/Ginsu-style modules (which I assume are similar to what Roel &

Steph are talking about), modules can contain classes, and can also contain "Class Extensions" (a.k.a. loose methods). I would define Class

Extensions as methods which are additions to classes in other modules, but they are ONLY additions, NOT changes to existing methods (and not removals either).

DeltaModules, on the other hand, can contain method additions, but also

changes to existing methods and method removals too.

And in fact any other possible change too (just so that noone misunderstands that, I know you know this).

So, to generalize a bit, DeltaModules are more powerful, but Class Extensions are quite a bit simpler.

Let's think about a couple of useful properties of modules:

1. Can they be cleanly unloaded?
2. Are they behavior-preserving? In other words, if you load (and

activate) a module, will the rest of the system still run without breaking or behaving differently?

Regarding #1, 3.3a-style Modules and DeltaModules can be unloaded, and Envy/Ginsu-style modules can too, so no problems there. Changesets cannot be unloaded, which is their big limitation.

Regarding #2, this is one reason for the split between Modules and DeltaModules in 3.3a. If you load a Module (with no DeltaModules), the

rest of the system will be unaffected, so they are behavior-preserving.

DeltaModules are not behavior-preserving, but that's the point of them... they can be used if you do need to change something else in the

system.

Because DeltaModules are not behavior-preserving, they naturally have to

change version numbers of the Modules that they modify. So for example

if a WebBrowser Module has a DeltaModule which adds the method #asEncodedHtml or changes a method in class String, in the Module "Kernel 2.7", the Kernel module will need to be assigned a new version number such as 2.8. (I believe this is how DeltaModules are supposed to

work, correct me if I'm wrong.) This may have big implications for the

This is correct AFAIK.

rest of the modules in the system, which might expect to run on Kernel 2.7 but not know about 2.8.

Exactly.

But the interesting thing is, Envy/Ginsu-style modules w/Class Extensions *are* behavior-preserving, with a few caveats* (see bottom of message). This is because, in general, if you add a new method to an existing class, none of the rest of the system would ever send that new method, so the behavior of the rest of the system is unchanged. So, reusing the example above, if a WebBrowser module has a Class Extension which adds the method #asEncodedHtml (but does not change any method) in class String, in the module "Kernel 2.7", the Kernel module will still remain 2.7. (This is exactly what Envy does. I don't remember now if Ginsu supports version numbers.)

Yes, a very crucial observation. Just a note though: You listed a few caveats but there is one more - the new behaviour may of course affect the state of the instance so that other "modules" are affected. But that is probably more of a "bug" than a "source code conflict". Whatever.

The other question is then, if we're using Class Extensions, is it good enough to only be able to *add* methods to classes in other modules, not modify/remove them (as DeltaModules can do)?

Well, I guess that's a big question. :-) I would say that most of the time, yes, it's good enough. A module does not often need to change/remove methods in other modules, unless we're directly fixing a bug in that other module. Adding methods is much more useful, and can be specific to the "outer" module (such as the example WebBrowser module adding #asEncodedHtml to String). People who've used Envy know that it can be very handy... although you don't want to abuse it to an extreme.

With Envy/Ginsu, if your module *really* needs to change/remove a method in Kernel 2.7, you make your own private version of Kernel 2.8 with the change, and/or you lobby the maintainer of Kernel to make the change. But this should be relatively rare aside from bug fixes.

On the other hand, the ability of DeltaModules to handle method changes/removals could be quite nice. And they seem a bit more rigorous, no "caveats" as with Class Extensions below. And namespaces are already built-in. But there is extra complexity cost... For example, DeltaModules have to worry about activation/deactivation, with Class Extensions it is irrelevant. The Class Extensions are always active when the module is active, since they don't affect anything else.

One last thing: As far as splitting up the current Squeak image into modules goes, my hunch is that it would be easier to do with Envy/Ginsu-style modules. This is because you would not need the DeltaModule capability of specifying changed/removed methods when divvying things up, only the ability to specify added methods, which the lighter-weight Class Extensions can also handle without version number worries. For example, String>>asMorph could be quickly dumped into some Morphic module.

There might also be another problem lurking with DMs - at least when trying to use them to chop up the image. Since a DM branches off the affected base Module then it would seem to me that we will end up with tons of "branches" all over the place. Module C has a DM which branches off Kernel to 2.8 and Module D has another DM that branches off Kernel to 2.9 - oops...

(Avi, are the DVS "logical modules" similar to Envy/Ginsu with Class Extensions?)

Yes, that is my sense of DVS. But using category coventions and thus staying "compatible" with good old 3.2. But hey, I don't think DVS has any form of conflict resolution capabilites or anything like that.

Am I making sense here? This message is getting too long, sorry. Let me know if I've characterized one of the approaches incorrectly. And I know this is only one aspect of the differences between these two module systems. Anyway, I've often found that I learn more by see two things compared against each other, than by seeing them described separately.

You are making sense, and it was an interesting angle. Personally I think I have understood Henrik's Modules but I have always been wondering what will happen when we reach the real interesting area - conflict resolution...

The following are a few unstructured thoughts:

Currently I think that producing Envy/Ginsu style packages (for example using DVS) consisting of standalone classes and loose method extensions will work quite fine for many of us (as you also noted) - especially if you build stuff "for" 3.2, compared to trying to detangle 3.2 itself...

But I would like to have these "loose methods" contained within a new smarter kind of ChangeSet that I will call a "Patch" here below. I will try to explain why and what such a beast would be.

- A Patch would be similar to a DM in that it is not a recording of a sequence of changes - it is instead a condensed set of changes, much like a patch-file in Unix.

- On the other hand a Patch would not be a Module (as DMs are) or anything like that in itself. Instead a Patch is an "intermediate object" like DVS and other tools could produce when they want to represent "loose method additions" or "loose method removals" or "class instvar additions" etc. Patches would be able to represent themselves in a simple format that can be used inside fileouts and of course they would be instantiated when filing in such fileouts.

So the Patch would not live in the image like classes do but tools could produce them as an intermediate object when filing in or out. And when installing them it may be a good idea to keep them around.

Ok. A Patch would not try to replace ChangeSets in any way. A ChangeSet is a "recorded session" in some sense and thus IMHO more of a development tool than a packaging/deployment tool. I think we should keep using them for source code management but perhaps we should try to avoid using them for deployment/packaging and instead rely on .st files containing full classes and Patches.

Here are a few crucial points... A Patch object should:

1. Keep enough state about the changes so that it can be used to detect conflicts against other Patch objects and packages. For example: One Patch adding a loose method x in class A would not conflict with another Patch adding method y in class A - they would both be additions. A Patch changing a method x in class A (belonging to package AA) would on the other hand potentially conflict with a package BB that depends on package AA. But perhaps we could even add classifications of "method changes" - one being "no behaviour change" (changing comments, formatting, optimization etc) and then it could be resolved into a "theoretical conflict" (likely not a conflict but could be if the method change is actually changing behaviour nevertheless) instead of a "conflict". So in short - we can arm these Patch objects with a lot of "smarts" for detecting conflicts and also help us resolving them.

2. Keep enough state for deinstallation. This part is (as I have argued in other posts) not truly critical but if we try to implement these little fellows we could at least give it a stab. It would be nice.

Both of the above features would need the Patch objects to stay around after filing in, not necessarily in the image though - we could log them in a file or something.

So in short, the proposal of a lightweight module system could be like this:

1. We implement the Patch object as described above. 2. Avi :-) incorporates Patch objects into DVS. 3. We start using .st files with Patches inside them for packaging/deployment. One way would be by using Envy/Ginsu style packages with DVS. 4. We add dependencies to SqueakMap between packages. We would also need to add versions of packages of course because currently SqueakMap only knows about the latest release).

In Henrik's Modules dependencies were declared "per Module". Perhaps that level is too fine grained? If we add dependencies in SM it would be "per package" which is a larger and perhaps more appropriate level.

All this would mean that we can continue using ChangeSets if we like. Using them for deployment would lead to "uninstallable" packages with much less capability of conflict resolution.

Then when we try to install a package using SM it holds information on what packages we already have installed. Packages using .cs or "interactive" .st-files can't really participate in the conflict detection - hopefully those kind of packages will "phase out". But the "new" .st format with Patches complemented with dependency information in SM should be both uninstallable and especially conflict detection between them should work nicely.

In short: The new Patch object is IMHO the key for successful packages/modules. And the simplest Patch is a "loose method addition" which will give us Envy/Ginsu style modules. But then nothing stops us from evolving the Patch concept with more smarts.

Wow, what a long post this became...

regards, Göran

Göran Hultgren, goran.hultgren@bluefish.se GSM: +46 70 3933950, http://www.bluefish.se "Department of Redundancy department." -- ThinkGeek

Hi Doug and all!

Doug Way dway@riskmetrics.com wrote:

Göran Hultgren wrote:

...

Hi all!

Quoting Doug Way dway@riskmetrics.com: [SNIP]

...

DeltaModules, on the other hand, can contain method additions, but also changes to existing methods and method removals too.

And in fact any other possible change too (just so that noone misunderstands that, I know you know this).

Right, I forgot to mention that. DeltaModules can also add/remove instance variables, etc.

Since people seem to have a problem grasping DMs I just wanted to be picky. ;-)

...

...

Let's think about a couple of useful properties of modules:

1. Can they be cleanly unloaded?
2. Are they behavior-preserving? In other words, if you load (and

activate) a module, will the rest of the system still run without breaking or behaving differently?

[SNIP]

But the interesting thing is, Envy/Ginsu-style modules w/Class Extensions *are* behavior-preserving, with a few caveats* (see bottom of message). This is because, in general, if you add a new method to an existing class, none of the rest of the system would ever send that new method, so the behavior of the rest of the system is unchanged. So, reusing the example above, if a WebBrowser module has a Class Extension which adds the method #asEncodedHtml (but does not change any method) in class String, in the module "Kernel 2.7", the Kernel module will still remain 2.7. (This is exactly what Envy does. I don't remember now if Ginsu supports version numbers.)

Yes, a very crucial observation. Just a note though: You listed a few caveats but there is one more - the new behaviour may of course affect the state of the instance so that other "modules" are affected. But that is probably more of a "bug" than a "source code conflict". Whatever.

Hmm, I'm not sure exactly what you mean here... could you provide an example?

Aside from the caveats I mentioned, adding method extensions should not affect the behavior of other modules. For example, let's say we have a system made up of modules A, B and C, where module A is a prerequisite of B and C. If we also load module D which has Class Extensions on classes in module A, the code in the original system (A B and C) should still work just as it did before (aside from the caveats), because it won't have any references to the newly added methods. (Thus, we don't have to update the version number of module A.)

What I meant is that when module D (adding methods to classes in A) is running it will call the methods in question and that can affect the state of those instances. If those same instances are used by modules A, B or C then those modules will be affected. In short - there is no memory protection between the modules. So if my methods introduce instance states that are faulty or at least unknown to modules A, B and C then those modules can very well break.

So we can't be SURE that modules A, B and C are unaffected. But we can of course say "shit happens". :-)

...

...

One last thing: As far as splitting up the current Squeak image into modules goes, my hunch is that it would be easier to do with Envy/Ginsu-style modules. This is because you would not need the DeltaModule capability of specifying changed/removed methods when divvying things up, only the ability to specify added methods, which the lighter-weight Class Extensions can also handle without version number worries. For example, String>>asMorph could be quickly dumped into some Morphic module.

There might also be another problem lurking with DMs - at least when trying to use them to chop up the image. Since a DM branches off the affected base Module then it would seem to me that we will end up with tons of "branches" all over the place. Module C has a DM which branches off Kernel to 2.8 and Module D has another DM that branches off Kernel to 2.9 - oops...

Yes, this is exactly the problem I was thinking of when I said "version number worries". This seems like it could be a major problem. I'm not sure how things like String>>asMorph were intended to be dealt with when the image is partitioned... perhaps most of them would need to be removed completely, requiring a lot of refactoring, so that we don't have branches all over the place?

Envy/Ginsu-style modules don't have this particular problem, because the additive-only Class Extensions don't require the base module to "branch" (change its version number).

Exactly.

...

[SNIP] The following are a few unstructured thoughts:

Currently I think that producing Envy/Ginsu style packages (for example using DVS) consisting of standalone classes and loose method extensions will work quite fine for many of us (as you also noted) - especially if you build stuff "for" 3.2, compared to trying to detangle 3.2 itself...

But I would like to have these "loose methods" contained within a new smarter kind of ChangeSet that I will call a "Patch" here below. I will try to explain why and what such a beast would be.

• A Patch would be similar to a DM in that it is not a recording of a sequence

of changes - it is instead a condensed set of changes, much like a patch-file in Unix.

• On the other hand a Patch would not be a Module (as DMs are) or anything like

that in itself. Instead a Patch is an "intermediate object" like DVS and other tools could produce when they want to represent "loose method additions" or "loose method removals" or "class instvar additions" etc. Patches would be able to represent themselves in a simple format that can be used inside fileouts and of course they would be instantiated when filing in such fileouts.

So the Patch would not live in the image like classes do but tools could produce them as an intermediate object when filing in or out. And when installing them it may be a good idea to keep them around.

Ok. A Patch would not try to replace ChangeSets in any way. A ChangeSet is a "recorded session" in some sense and thus IMHO more of a development tool than a packaging/deployment tool. I think we should keep using them for source code management but perhaps we should try to avoid using them for deployment/packaging and instead rely on .st files containing full classes and Patches.

Here are a few crucial points... A Patch object should:

1. Keep enough state about the changes so that it can be used to detect

conflicts against other Patch objects and packages. For example: One Patch adding a loose method x in class A would not conflict with another Patch adding method y in class A - they would both be additions. A Patch changing a method x in class A (belonging to package AA) would on the other hand potentially conflict with a package BB that depends on package AA. But perhaps we could even add classifications of "method changes" - one being "no behaviour change" (changing comments, formatting, optimization etc) and then it could be resolved into a "theoretical conflict" (likely not a conflict but could be if the method change is actually changing behaviour nevertheless) instead of a "conflict". So in short - we can arm these Patch objects with a lot of "smarts" for detecting conflicts and also help us resolving them.

This could be interesting. (The "no behavior change" message classification sounds neat... renaming a temporary variable might be another example.)

Right.

...

2. Keep enough state for deinstallation. This part is (as I have argued in other

posts) not truly critical but if we try to implement these little fellows we could at least give it a stab. It would be nice.

Both of the above features would need the Patch objects to stay around after filing in, not necessarily in the image though - we could log them in a file or something.

So in short, the proposal of a lightweight module system could be like this:

1. We implement the Patch object as described above.
2. Avi :-) incorporates Patch objects into DVS.
3. We start using .st files with Patches inside them for packaging/deployment.

One way would be by using Envy/Ginsu style packages with DVS. 4. We add dependencies to SqueakMap between packages. We would also need to add versions of packages of course because currently SqueakMap only knows about the latest release).

In Henrik's Modules dependencies were declared "per Module". Perhaps that level is too fine grained? If we add dependencies in SM it would be "per package" which is a larger and perhaps more appropriate level.

All this would mean that we can continue using ChangeSets if we like. Using them for deployment would lead to "uninstallable" packages with much less capability of conflict resolution.

Then when we try to install a package using SM it holds information on what packages we already have installed. Packages using .cs or "interactive" .st-files can't really participate in the conflict detection - hopefully those kind of packages will "phase out". But the "new" .st format with Patches complemented with dependency information in SM should be both uninstallable and especially conflict detection between them should work nicely.

In short: The new Patch object is IMHO the key for successful packages/modules. And the simplest Patch is a "loose method addition" which will give us Envy/Ginsu style modules. But then nothing stops us from evolving the Patch concept with more smarts.

This type of conflict resolution could be very handy. I don't have time at the moment to comment more on it... and I should probably think about it some more first anyway. :)

In any case, I suppose one lesson learned is that there are many ways to implement a modules system in Smalltalk. I was going to try to summarize/categorize them, but that seems like a task for another time. :) And it may be that some of the problems I talk about above could be addressed by Henrik's module system with some tweaks... I guess I wouldn't rule anything out yet.

True. After having described the above ideas with Patch etc I also (don't remember if I posted or just emailed Daniel about it) realized that a very simple way to achieve something similar is to start classifying updates ChangeSets. If we introduce update queues for packages in SM (which seems pretty high up on the todolist right now) we could use the category system in SM to classify those updates much like I described above (no behaviour change, should be backwards compatible, not backwards compatible etc) and then we could get "stretchy dependencies" from that.

If I didn't introduce that term it means more or less that if a package A is dependent on version 1.22 of B then if B moves to 1.25 by 3 update ChangeSets all classified as "no behaviour change" then we can be sure that A will work with B 1.25 too. And if say the last one is classified as "should be backwards compatible" we can still be pretty safe, etc. So, the dependency can "stretch".

• Doug Way dway@riskmetrics.com

regards, Göran

Hi all!

Avi Bryant avi@beta4.com wrote:

On Wed, 23 Oct 2002 goran.hultgren@bluefish.se wrote:

...

If I didn't introduce that term it means more or less that if a package A is dependent on version 1.22 of B then if B moves to 1.25 by 3 update ChangeSets all classified as "no behaviour change" then we can be sure that A will work with B 1.25 too. And if say the last one is classified as "should be backwards compatible" we can still be pretty safe, etc. So, the dependency can "stretch".

Can't this be done more simply by using a major.minor.patch convention for version numbers? You could have a dependency on 1.2, which would accept 1.2.0, 1.2.2, 1.2.5, etc...

Yes, true it can.

Backtracking: My thinking started with making ChangeSets smarter and better, and then I thought that a new "beast" would be simpler to implement than hacking in ChangeSet and I started thinking in terms of Patch (a new class similar to ChangeSet with these capabilities) and then someone said "Simpler!" and then I said that ok, we can use ChangeSets if we classify them manually instead (some of this can be automated of course). And finally if we make it even simpler we can drop the idea of classifying the updates and just classify "released" package versions. And finally we can drop the idea of using classifications at all and instead rely on a numbering scheme that we *all agree on*. He, that last part might get funny.

Anyway, the idea of classifying the updates one by one would allow a cumulative aspect which is nice. Let me insert something from an email with Daniel: ----

Patches could be "graded" like:

1. Guaranteed behavior preserving. (only class comments, method

comments, method formatting, category changes - we could even compare bytecodes of methods before and after to make sure!) 2. Behavior extensions. (the developer has added behavior as for example added methods/classes but hasn't changed existing methods)

These two are definitely computable, and definitely as computable from differences as from stored sequences. They definitely help - if software promises that version is definitely compatible, you (the integrator) will simply not do (as much) testing, and focus on understanding the additions/naming mods.

3. Likely to be behavior preserving. (the developer has done changes but

*think* they don't affect behavior) 4. Compatible. (the developer has done changes to existing code that does indeed change behavior but should still be compatible) 5. Likely non compatible. (the developer has changed protocols and think that at least some packages will be affected) 6. Non compatible. (the developer is sure that dependent packages are non compatible)

These aren't computable, and as an Integrator, I wouldn't change my behavior much based on seeing them proclaimed. Maybe an SM category on an update would convey this fairly enough.

In fact - we could even start using the above classification of ChangeSets and then use that as "Patch" to begin with. As you realize this will mean that packages/users will have better control - if SqueakMap says to you "You are installing package X which is dependent on Comanche 4.12 but you have packages depending on the installed Comanche 5.0. Comanche 5.0 is classified as a "compatible" with Comanche 4.12 - would you like to proceed?" wouldn't that be nice? :-)

Yes, definitely, but it would be nicer still if the configuration maintainer faced that question, and I just did apt-get upgrade... :-) -------

Anyway, as you see the 6 levels described are like thresholds in "change severity" which means that for example, given package Y with history like this:

Release 1.22 Update 1: classified as 1 (Guaranteed behavior preserving) Update 2: classified as 2 (Behavior extensions) Update 3: classified as 1 (Guaranteed behavior preserving) Release 1.23 (max level above is 2 so the release is classified as "Behavior extensions" compared to 1.22) Update 4: classified as 3 (Likely to be behavior preserving) Update 5: classified as 2 (Behavior extensions) Update 6: classified as 4 (Compatible) Release 1.24 (max level above is 4 so the release is classified as "Compatible" compared to 1.23) Update 7: classified as 5 (Likely non compatible) Update 8: classified as 4 (Compatible) Release 1.25 (max level above is 5 so the release is classified as "Likely non compatible" compared to 1.24) Update 9: classified as 6 (Non compatible) Update 10: classified as 1 (Guaranteed behavior preserving) Release 1.3 (max level above is 6 so the release is classified as "Non compatible" compared to 1.25)

So if package X is dependent on Y 1.22 then it could definitely stretch to 1.23, note also that level 1-2 are automatically computable. With good probability, but we need to trust the developer of X on this we could even try out 1.24. Going to 1.25 is gambling, but still has a chance of working. 1.3 is non compatible so if that works the developer of X is simply wrong in his judgement! About 0.001 chance I guess.

Personally I think this could be really useful and when/if we decide to add support for individual package update streams in SqueakMap (which I think we should ASAP) then this little classification scheme above would fit perfectly. And note - it would still be optional and we would still be using ChangeSets. :-)

But sure, a version numbering scheme is quite ok too. And of course it would be optional to follow.

regards, Göran

On Wednesday 23 October 2002 10:58 am, Avi Bryant wrote:

On Wed, 23 Oct 2002 goran.hultgren@bluefish.se wrote:

...

Personally I think this could be really useful and when/if we decide to add support for individual package update streams in SqueakMap (which I think we should ASAP)

Well, I'm not so interested in package update streams, since they're pretty much unnecessary if you use DVS instead - I find it much more reliable to have DVS figure out the differences between the current version loaded into an image and whatever the newest version is, than to depend on the developer having produced that patch by hand.

But that only works for code packaged with DVS, of course.

How would you use DVS for a package that included a removal of a method that's shipped with some other (non-DVS) package? I ran into this just now, as I'd needed to do a little re-factoring on some system classes to make something work.

Since DVS doesn't seem to be able to express method removals, it would seem to be unable to do this...

Avi Bryant avi@beta4.com wrote:

On Wed, 23 Oct 2002 goran.hultgren@bluefish.se wrote:

...

Personally I think this could be really useful and when/if we decide to add support for individual package update streams in SqueakMap (which I think we should ASAP)

Well, I'm not so interested in package update streams, since they're pretty much unnecessary if you use DVS instead - I find it much more reliable to have DVS figure out the differences between the current version loaded into an image and whatever the newest version is, than to depend on the developer having produced that patch by hand.

But... Hmmm. So how would/could we get the "stretchability" I am after then? Your proposal with version numbers is ok, but that would require a new full "release" of the package and some people like to work with updates I think. And also like to choose their own version numbering scheme...

And how would we let people contribute fixes and enhancements? I was thinking of a sort of "incoming" queue for each package which the maintainer easily could load/review/reject/integrate from.

Perhaps we could have a combination of the approaches:

1. For those using DVS and less frequent releases that are full and don't use update streams we could add the 1-6 level classification as an attribute of the version. (this would still let people use whatever version numbering scheme they like, hmmm - have to take a look at Stephens code for that would be a shame if it got wasted)

2. For those who like update streams we can have the same classification per update changeset and let SqueakMap do the "max" calculation of a sequence as I described etc.

How does this sound? SqueakMap should try to accommodate all variants - that is the idea (within limits of course).

But that only works for code packaged with DVS, of course.

Right.

Avi

regards, Göran

Sorry, a long reply on this mail... my comments are throughout the mail, wherever they make sense.

On Saturday, October 19, 2002, at 09:15 AM, Doug Way wrote:

Ned Konz wrote:

...

On Friday 18 October 2002 07:36 am, Roel Wuyts wrote:

...

Stef has this kind of setup in mind, and in that setup there is no difference between modules and delta-modules. A module is something which has definitions and which you can apply on a system.

But that (a simple declarative system) not sufficient to do what we want to do with the system. There has to be a way to remove an existing method (at least for your code).

So unless you effectively define an entire module which consists of the original module MINUS the method, you don't get the behavior you want.

I _think_ the DM's are a way to do this kind of specification: "this is a module just like module X but with this removed, this added, and this modified".

Okay, this is getting into a difference between these two approaches which needs to be fleshed out some more (and which I'm somewhat familiar with, at least :-) ).

With Envy/Ginsu-style modules (which I assume are similar to what Roel & Steph are talking about),

No, really not. We are not the proponents of Ginsu.... In our system there are not only class extensions, but complete redefinitions are allowed and work. Think more of the parcels in VisualWorks, though I hesitate to say this, because now will think that we want parcels :-) :-) What we did not add yet is method removals, but that is trivial to add in our system. It even has a form of selector namespaces...

modules can contain classes, and can also contain "Class Extensions" (a.k.a. loose methods). I would define Class Extensions as methods which are additions to classes in other modules, but they are ONLY additions, NOT changes to existing methods (and not removals either).

DeltaModules, on the other hand, can contain method additions, but also changes to existing methods and method removals too.

So, to generalize a bit, DeltaModules are more powerful, but Class Extensions are quite a bit simpler.

Let's think about a couple of useful properties of modules:

1. Can they be cleanly unloaded?
2. Are they behavior-preserving? In other words, if you load (and

activate) a module, will the rest of the system still run without breaking or behaving differently?

Regarding #1, 3.3a-style Modules and DeltaModules can be unloaded, and Envy/Ginsu-style modules can too, so no problems there. Changesets cannot be unloaded, which is their big limitation.

In our system, modules can be unloaded. This is actually one of the key points: you want to able to compose as well as decompose a system again. I don't need Celeste at the moment, since I am going to play with Wonderland: unload one, load the other.

Regarding #2, this is one reason for the split between Modules and DeltaModules in 3.3a. If you load a Module (with no DeltaModules), the rest of the system will be unaffected, so they are behavior-preserving. DeltaModules are not behavior-preserving, but that's the point of them... they can be used if you do need to change something else in the system.

Yes, I understood that now. I just don't fully agree, but I think it will be easier to discuss this on a BOF at oopsla, and produce a Swiki page with the results: something REALLY tangible.

Because DeltaModules are not behavior-preserving, they naturally have to change version numbers of the Modules that they modify. So for example if a WebBrowser Module has a DeltaModule which adds the method #asEncodedHtml or changes a method in class String, in the Module "Kernel 2.7", the Kernel module will need to be assigned a new version number such as 2.8. (I believe this is how DeltaModules are supposed to work, correct me if I'm wrong.) This may have big implications for the rest of the modules in the system, which might expect to run on Kernel 2.7 but not know about 2.8.

But the interesting thing is, Envy/Ginsu-style modules w/Class Extensions *are* behavior-preserving, with a few caveats* (see bottom of message). This is because, in general, if you add a new method to an existing class, none of the rest of the system would ever send that new method, so the behavior of the rest of the system is unchanged. So, reusing the example above, if a WebBrowser module has a Class Extension which adds the method #asEncodedHtml (but does not change any method) in class String, in the module "Kernel 2.7", the Kernel module will still remain 2.7. (This is exactly what Envy does. I don't remember now if Ginsu supports version numbers.)

The other question is then, if we're using Class Extensions, is it good enough to only be able to *add* methods to classes in other modules, not modify/remove them (as DeltaModules can do)?

Well, I guess that's a big question. :-) I would say that most of the time, yes, it's good enough. A module does not often need to change/remove methods in other modules, unless we're directly fixing a bug in that other module. Adding methods is much more useful, and can be specific to the "outer" module (such as the example WebBrowser module adding #asEncodedHtml to String). People who've used Envy know that it can be very handy... although you don't want to abuse it to an extreme.

With Envy/Ginsu, if your module *really* needs to change/remove a method in Kernel 2.7, you make your own private version of Kernel 2.8 with the change, and/or you lobby the maintainer of Kernel to make the change. But this should be relatively rare aside from bug fixes.

Exactly. It is with bugfixes in mind that we added the semantics that you can redefine methods.

On the other hand, the ability of DeltaModules to handle method changes/removals could be quite nice. And they seem a bit more rigorous, no "caveats" as with Class Extensions below. And namespaces are already built-in. But there is extra complexity cost... For example, DeltaModules have to worry about activation/deactivation, with Class Extensions it is irrelevant. The Class Extensions are always active when the module is active, since they don't affect anything else.

That is way we combined a namespace mechanism, local to the module. Whatever change you make is local to your module. If you choose to redefine a method, than this redefinition is only used in your module. The method you changed is still there and is used by the rest of the system. So you can have the bugfix that you need local to your module. However, you also want the possibility to have a global change sometimes. We believe that these semantics are easy to grasp... Alex has implemented this system in Squeak and is playing with it. I might do a demo at OOPSLA to give you an idea about what it looks like.

One last thing: As far as splitting up the current Squeak image into modules goes, my hunch is that it would be easier to do with Envy/Ginsu-style modules. This is because you would not need the DeltaModule capability of specifying changed/removed methods when divvying things up, only the ability to specify added methods, which the lighter-weight Class Extensions can also handle without version number worries. For example, String>>asMorph could be quickly dumped into some Morphic module.

(Avi, are the DVS "logical modules" similar to Envy/Ginsu with Class Extensions?)

Am I making sense here? This message is getting too long, sorry. Let me know if I've characterized one of the approaches incorrectly. And I know this is only one aspect of the differences between these two module systems. Anyway, I've often found that I learn more by see two things compared against each other, than by seeing them described separately.

Yes, now with my reply it becomes even longer, but I could not find a good way to chop up the mail :-)

• Doug Way

dway@riskmetrics.com

*Caveat #1: There could be namespace conflicts with Class Extensions, which Envy/Ginsu don't handle. I'm guessing that adding namespace support is one thing that Roel & Steph were working on?

Yes, they were in there from the beginning.

*Caveat #2: Ideally, to preserve behavior, you would not allow a Class Extension to override a method in a superclass. This would be a simple restriction to add. Otherwise, you could bring an image to its knees by adding a new method #size to SortedCollection which returns nil. :-)

There are so many ways that you can bring an image to its knees :-) We don't do these kinds of checks at the moment, since we give the responsibility to the developer. But rules like these are easy to add if it is needed.

*Caveat #3: If a base module class is doing something funky with reflection such that adding a new method changes its behavior, then the Class Extension might not be behavior-preserving. But this falls under the category of Very Rare, IMHO. (I guess an example might be adding a testXXX method to an SUnit test suite class.)

We have a MOP for the modules, and then there should not be problems, in the sense that whatever you can do is described by the MOP. But, again, you can shoot yourself in the foot :-)

Roel Wuyts Software Composition Group roel.wuyts@iam.unibe.ch University of Bern, Switzerland http://www.iam.unibe.ch/~wuyts/ Board Member of the European Smalltalk User Group: www.esug.org

wiljo@mac.com wrote:

Hi gang,

After a year away from Smalltalk I have come back to the light. Still catching up on all the changes... phew quite a task. Still much of the chatter reminds me of my reaction to Les Miserables... "such a lot of fuss over a piece of toast!" ;-}>

On Wednesday, October 16, 2002, at 07:32 AM, goran.hultgren@bluefish.se wrote:

...

I am not so sure - Ginsu is a model describing Smalltalk source code - separately from the Smalltalk model itself. At least that is my understanding. Modules is instead an extension to the Smalltalk model of classes. Way back when I talked with Henrik about this I found his arguments and model much more appealing than having an "extra" model like Ginsu.

Well, you have missed the key distinction. "Ginsu" was about a declaration of the model of the semantics of a Smalltalk program. A non-reflective

No I didn't miss that. Joseph was even here and showed it to me.

description language of a reflective execution environment. The essential distinction is the separation of the declaration from *any* implementation of the semantic model specified. This first distinction is the key to allowing the revision of the state of a Smalltalk program... independent of any particular image or runtime implementation.

Yes, I know this.

The draft of the ANSI Smalltalk standard v1.9 has sidebar comments that go into some detail in this regard. Also, OOPSLA '88 Alan Wirfs-Brock and Brian Wilkerson wrote the seminal paper on Modular Smalltalk which is the mother of the ginsu approach to Smalltalk modularity.

<aside for the interested reader> This subtle distinction is a profound one when considering the evolution of form. (cf G.-Spencer Brown "The Laws of Form") Follow all the injunctions, contemplate the declaration of formal indication, the consequent arithmetic and algebra and finally the introduction of formal self reference in a form that experiences temporal evolution of signal. </aside for the interested reader>

...

For example - the ability to have classes loaded in the image (fully compiled etc so that tools work and so on) but not having them activated in the name space seemed like a brilliant approach. And activation through big become-operation seemed very neat.

How is that distinct from having tools that operate on declarations of the program with all methods fully compilable and loadable? At Digitalk,

"distinct" it is not. But it felt like a much more lightweight approach that delivers much the same benefits of Ginsu but without an "extra model" of the code. To me, Henrik and several others (including Dan) this felt more in line with what we need/want in Squeak. And according to Henrik it is much more lightweight not only codewise but also in RAM. And it built further upon the Environment stuff that Dan had started.

Steve Messick did some very cool remote development on headless images. Utilities like "cp" could be as small as a 30k image, with the

Yes, I know this can then be done (Joseph also talked alot about "tethered compilation" etc) - but I can't actually see what this has to do with anything. It's trivial to hook up to a headless image using sockets and bang anything over there - I can do that today without any of Ginsu or Modules... Or perhaps I missed something?!

capability of attaching the program to the declaration giving full debug and development support. (Imagine programs being a kind of project with a completely separate namespace living in a separate runtime process somewhere on the network... all with development ui's switchable the way in which we switch between projects.) That is my definition of neat.

But... what has this capability (remove development) to do with modules (small m for the general meaning)?

OBTW, the development environment itself was just another program...

...

I talked with Henrik about Ginsu and he said that he had looked at it and thought that similar goals could be achieved better using other techniques. And then he wrote Modules. Since he knows this stuff infinitely better than me (especially compared with Ginsu) I can only hope he posts his thoughts.

Perhaps that is indeed the case. Nevertheless, dig up a copy of VSE if you can. Load Team/V. Open a standard class browser and edit away. Use the disk browser and file in a bunch of module unaware code. All of that stuff winds up in a module called, *Unpackaged*. Most folks never knew that the old fashioned browser was no longer editing the classes and methods themselves, but instead declarations of the aforesaid. I suppose my point is that all of this is based upon many years of experience with making Smalltalk more modular.

Yes, I am aware of that. And it is a very CM oriented approach. Even Joseph has said that Ginsu is "Envy done right". But that is not what Squeak needs! Well, some of use do need a good CM tool - that is not what I meant. I meant that it's more about packaging, deployment, image stripping/building, namespaces, runtime components, dependencies between modules, conflict detection and *above all* detangling the darn image.

And almost all of the above has nothing to do with classic CM problems which more or less focus on "how do we build a BIG system using 20 developers". That is where Envy comes in. And Team/V - just listen to the name of the product.

Anyway, I am blabbering now - thanks for clearing up some parts about the Ginsu origins.

regards, Göran