[reordered for convinience] Henrik Gedenryd Henrik.Gedenryd@lucs.lu.se wrote:
danielv@netvision.net.il wrote:
Have you read the previous postings where I explained the basic design?
Yes, but good idea. I just read them again, with my interest level higher. Quite interesting.
OTGH, DMs are supposed to replace change sets, but when I make a change set, it usually touches a few logical modules, not one. At least eventually, it is the conjunction of deltas to several modules, not one.
Yes, to your project module, you may add multiple deltas, one for each module you "touch". This also automatically handles the equivalent of grouping and managing multiple change sets, without having to add another ad hoc mechanism for doing that.
Ah. So something that's born as a change-set (side effect of a lazy afternoon's hacking) later evolves into a project module with several delta modules, each holding the extensions the author believes should be integrated next into various modules he may not be responsible for. Right?
IIUC, I like this concept, it relates to how I'm working now on the RB module.
BTW, would these DM be submodules or imports? can DMs be submodules? (I'm still trying to pin those concepts down...)
On one hand, they represent the same thing as a Module, except stored as deltas from a specific module. OTOH, DMs can be extensions, but Modules can't. ?!
A Module can't be an extension since it isn't an extension of something. A Delta is by definition an extension from a base (module).
In your String>>asUrl example - String is probably a far more long-lived concept than the BasicTextHandling-V.26 module. Would I have to extend a specific Module and version? this seems to flow from your definitions, but practically speaking, asUrl probably doesn't depend on many features of a specific module version, and it would mighty tedious maintenance to be constantly rereleasing the same code with updates requirements.
Might be a good idea to have looser definitions of requirements - like BasicTextHandling >= V.26 (this is done in the Debian package system, at least, I don't know about ENVY/Team).
But specifically in the case of extending String - This class is so long lived, so basic, that probably even just specifying the name of the module relative to which extension must be defined, is limiting it's use unneccessarily. So IMHO, we should have module that include extensions of classes, even if no module is reference in which the class is defined. Maybe there's a better solution.
I don't know if this is what you mean by the half implied reference to DeltaClasses in the DeltaModule comment...
If you look at the code I posted there is an example showing how it is supposed to work.
Sorry, I wasn't able to resolve that reference... what message, what code do you mean?
Having DMs, I've decided to leave change sets untouched. So people can continue using them like now, but a major point in this discussion has been the weaknesses of change sets, which is what DMs are meant to remedy.
Well, good, that's what wasn't clear.
What do you mean when you say your are/will be built for a "internet-distributed, collaborative open-source project"?
Well, "delivering runtime (end-user) pieces of software" represents a very small fraction of what Squeak is used for.
Agreed totally.
There are numerous other aspects that we need to deal with, as have been stated by others already. Handling contributions, bug fixes, changes to system classes, handling conflicts, shared repositories, and allowing multiple uncoordinated projects to proceed independently are some such things.
I like what I understand from your proposals so far.
So far the idea of "no class extensions unless you specify baseline module and version" seems a bit tighter than my comfort level, otherwise very interesting.
Henrik
danielv@netvision.net.il wrote:
BTW, would these DM be submodules or imports? can DMs be submodules? (I'm still trying to pin those concepts down...)
I think they should be submodules of your project, as they are "part" of it.
In your String>>asUrl example - String is probably a far more long-lived concept than the BasicTextHandling-V.26 module. Would I have to extend a specific Module and version? this seems to flow from your definitions, but practically speaking, asUrl probably doesn't depend on many features of a specific module version, and it would mighty tedious maintenance to be constantly rereleasing the same code with updates requirements.
Might be a good idea to have looser definitions of requirements - like BasicTextHandling >= V.26 (this is done in the Debian package system, at least, I don't know about ENVY/Team).
From the previous discussion here, this seems like an unsolved problem in
version management. We may consider specifying version dependencies with varying precision, so that "1" will be any version 1.x, "1.1" any 1.1x, 1.1014 just that, and version unspecified will be any version.
However, note that String should change very slowly. Practically never.
But specifically in the case of extending String - This class is so long lived, so basic, that probably even just specifying the name of the module relative to which extension must be defined, is limiting it's use unneccessarily.
I don't see why it would be a problem to associate String changes with the module that String is in? Surely which module it is in shouldn't change much either? Not having this info would make conflict resolution much uglier.
If you look at the code I posted there is an example showing how it is supposed to work.
Sorry, I wasn't able to resolve that reference... what message, what code do you mean?
In the Module explorer that opens when you load the code, (first do all the conversions, then) select a module, e.g. #(Squeak MVC), and execute "self collectUpstreamMethodsOutside: self" in the lower pane. Then look at the new submodule(s). Now, all outside methods referring to MVC globals are collected into MVC DeltaModules--currently 72 of them, which tells us something!
Henrik
Hi!
I'm printing out a set of results to the Transcript window when I suddenly realize that I don't need it. I hit cmd-. to stop it. The next time I try running it again, I get a freeze (probably a semaphore is still being held someplace). Is there anyway to recover the use of the Transcript window?
Jerome Chan wrote:
I'm printing out a set of results to the Transcript window when I suddenly realize that I don't need it. I hit cmd-. to stop it. The next time I try running it again, I get a freeze (probably a semaphore is still being held someplace). Is there anyway to recover the use of the Transcript window?
Can you please check if the attached change fixes your problem?
Thanks
Michael
Dear Henrik,
I was reading about the concept of "Delta Modules" on the Swiki (at http://minnow.cc.gatech.edu/squeak/2063).
The idea of separating the loading of a DeltaModule into an image from the activation of the features in that DelatModule is something that I had not thought about before. I can see this as having two benefits.
(1) A DeltaModule could be loaded, so that I can browse it and see what it does, using all of the usual tools, without having to activate it. (So this would subsume the functionality of the Package Browser.)
(2) Activation and de-activation might be very much faster than loading and unloading.
Are these the only advantages, or are there others?
If these advantages make the separation of loading from activation and unloading from deactivation worthwhile, then shouldn't we make the same separation for Modules themselves, not just for DeltaModules.
Finally, what is the motivation for introducing DeltaModules in the first place? I can see it as a storage optimization (like using diffs in RCS or SCCS rather than storing multiple copies of the same stuff), but that does not mean that it needs to be visible to the Squeak image side of things. If it is meant as a communication protocol optimization, then it would need to be visible to Squeak, since the composing of the base and the delta would have to happen in the image. Is that the reason?
Andrew
Andrew P. Black wrote:
I was reading about the concept of "Delta Modules" on the Swiki (at http://minnow.cc.gatech.edu/squeak/2063).
...
Are these the only advantages, or are there others?
Finally, what is the motivation for introducing DeltaModules in the first place?
Andrew,
To try to answer these questions, I fleshed out that page, this should hopefully be useful to you.
http://minnow.cc.gatech.edu/squeak/2063
If these advantages make the separation of loading from activation and unloading from deactivation worthwhile, then shouldn't we make the same separation for Modules themselves, not just for DeltaModules.
I added a new page for this.
"What about conflicts between modules?"
http://minnow.cc.gatech.edu/squeak/2071
(Yes, I am pushing these swiki pages right now!)
Henrik
Dear Henrik,
It is good that you are improving the presentation on the Wiki, because that is a better place to find answers that the mail archive. (But it does take so much longer to edit the Wiki ... lucky I have cable Internet in my flat)
Your changes do clarify the meaning of what is there quite a bit, so thank you for that. But they do not answer my questions, only confirm that I do really still have those questions. So, let me try rephrasing my questions so it is clearer what I am asking.
Question the first:
I understand that "DeltaModules can be un/installed, but the un/install operation has no meaning for regular Modules" [1]. My question is: Why. If un/install is such a good idea, why not have it for all Modules, not just for DeltaModules?
Question the second:
DeltaModules are a kind of Module[2]. They are distinguished from base modules by being defined by difference. So, for example, StringWithUrlOps might be defined as being the same as String_V1.2 with the addition of a specific set of Url manipulation methods. But the DeltaModule does not do this as a set of changes, but rather as a definition of the final state. Right so far? So, my question is: why do you consider DelatModules as being conceptually different from Modules at all? Why do they have different properties, just because they happen to have a different representation? After all, v2.3 of a text document is still a text document, regardless of whether it is defined by deltas from version v2.2 or by giving the full text. In OO terms, the representation (whether by delta or by absolute) is hidden, only the protocol is important. Why do DeltaModules and Module not have identical protocols? Why are DeltaModules and regular Modules not just tow implementation classes with the same interface?
References:
[1] http://minnow.cc.gatech.edu/squeak/2071
[2] http://minnow.cc.gatech.edu/squeak/2063
Andrew P. Black wrote:
I was reading about the concept of "Delta Modules" on the Swiki (at http://minnow.cc.gatech.edu/squeak/2063).
...
Finally, what is the motivation for introducing DeltaModules in the first place?
Andrew,
To try to answer these questions, I fleshed out that page, this should hopefully be useful to you.
http://minnow.cc.gatech.edu/squeak/2063
If these advantages make the separation of loading from activation and unloading from deactivation worthwhile, then shouldn't we make the same separation for Modules themselves, not just for DeltaModules.
I added a new page for this.
"What about conflicts between modules?"
I don't see what conflict resolution has to do with my question at all. But maybe you thought that I was asking a different question; sorry for being unclear.
I don't understand your page about conflicts either, but I made a comment on the page.
Andrew
Andrew,
[Re: Delta Modules]
So, my question is: why do you consider DelatModules as being conceptually different from Modules at all?
A good way to think about DMs may be the following (which happens to be the way how I think about them ;-)
When we do some work in Squeak we usually have two parts in it. One being the "core" of our module, the classes and methods defined intrinsically. If not for class Object, the module should be able to run "within itself", that is being a self-contained entity (plus the modules and classes it imports).
Then the mess starts. It starts because we need to interface our module with the existing system. This usually includes adding methods to various places, methods that - given the absence of the things we modify - don't even need to be there. As an example, if there wouldn't be a file list or a world menu in Squeak we'd never have the need to extend it to be able to load some file that our module handles. If there were no other classes besides the ones defined in our module, we would never have the need to extend Object with one of those common "isMumble" methods. Etc. etc. etc.
Thus, interfacing with the environment is where all the mess starts and where it ends. Delta Modules do nothing but encapsulate this uglyness in a nice way. Delta Modules say "okay, for making this module work in a given environment I need some modification of x or y in a different module" and that is conceptually very different from defining a set of useful classes which - given the absence of the environment - could perfectly well stand on their own.
Hope this helps, - Andreas
-----Original Message----- From: squeak-dev-admin@lists.squeakfoundation.org [mailto:squeak-dev-admin@lists.squeakfoundation.org]On Behalf Of Andrew P. Black Sent: Wednesday, October 03, 2001 3:57 PM To: squeak-dev@lists.squeakfoundation.org Subject: Re: [ENH][Modules] Delta Modules [was: Another version]
Dear Henrik,
It is good that you are improving the presentation on the Wiki, because that is a better place to find answers that the mail archive. (But it does take so much longer to edit the Wiki ... lucky I have cable Internet in my flat)
Your changes do clarify the meaning of what is there quite a bit, so thank you for that. But they do not answer my questions, only confirm that I do really still have those questions. So, let me try rephrasing my questions so it is clearer what I am asking.
Question the first:
I understand that "DeltaModules can be un/installed, but the un/install operation has no meaning for regular Modules" [1]. My question is: Why. If un/install is such a good idea, why not have it for all Modules, not just for DeltaModules?
Question the second:
DeltaModules are a kind of Module[2]. They are distinguished from base modules by being defined by difference. So, for example, StringWithUrlOps might be defined as being the same as String_V1.2 with the addition of a specific set of Url manipulation methods. But the DeltaModule does not do this as a set of changes, but rather as a definition of the final state. Right so far? So, my question is: why do you consider DelatModules as being conceptually different from Modules at all? Why do they have different properties, just because they happen to have a different representation? After all, v2.3 of a text document is still a text document, regardless of whether it is defined by deltas from version v2.2 or by giving the full text. In OO terms, the representation (whether by delta or by absolute) is hidden, only the protocol is important. Why do DeltaModules and Module not have identical protocols? Why are DeltaModules and regular Modules not just tow implementation classes with the same interface?
References:
[1] http://minnow.cc.gatech.edu/squeak/2071
[2] http://minnow.cc.gatech.edu/squeak/2063
Andrew P. Black wrote:
I was reading about the concept of "Delta Modules" on the Swiki (at http://minnow.cc.gatech.edu/squeak/2063).
...
Finally, what is the motivation for introducing
DeltaModules in the
first place?
Andrew,
To try to answer these questions, I fleshed out that page,
this should
hopefully be useful to you.
http://minnow.cc.gatech.edu/squeak/2063
If these advantages make the separation of loading from activation and unloading from deactivation worthwhile, then shouldn't we make the same separation for Modules themselves, not just for
DeltaModules.
I added a new page for this.
"What about conflicts between modules?"
I don't see what conflict resolution has to do with my question at all. But maybe you thought that I was asking a different question; sorry for being unclear.
I don't understand your page about conflicts either, but I made a comment on the page.
Andrew
Hi andreas
So, my question is: why do you consider DelatModules as being conceptually different from Modules at all?
A good way to think about DMs may be the following (which happens to be the way how I think about them ;-)
When we do some work in Squeak we usually have two parts in it. One being the "core" of our module, the classes and methods defined intrinsically. If not for class Object, the module should be able to run "within itself", that is being a self-contained entity (plus the modules and classes it imports).
Then the mess starts. It starts because we need to interface our module with the existing system. This usually includes adding methods to various places, methods that - given the absence of the things we modify - don't even need to be there. As an example, if there wouldn't be a file list or a world menu in Squeak we'd never have the need to extend it to be able to load some file that our module handles. If there were no other classes besides the ones defined in our module, we would never have the need to extend Object with one of those common "isMumble" methods. Etc. etc. etc.
Have you worked with VW and/or Envy? in envy this is called class extension, in envy they cannot hide pre existing method nor change the class definition, while in VW with Parcel you can hide 9and get it back on unload) a method and may be change the class definition I have to check,
Thus, interfacing with the environment is where all the mess starts and where it ends. Delta Modules do nothing but encapsulate this uglyness in a nice way. Delta Modules say "okay, for making this module work in a given environment I need some modification of x or y in a different module" and that is conceptually very different from defining a set of useful classes which - given the absence of the environment - could perfectly well stand on their own.
This is not that different, you only do not define a class. But DM should be unloaded when the module they refer to unload. I do not really see why there is a difference between the two. This is just the way program are perceived and this is fun to see that in a world like Smalltalk where normally we can add method anywhere we start to do a conceptual distinction between class and method and method attached to other classes. A module could be just a list of declaration (method, global, class,..... and you do not need to know what is DM or Module) as soon as you know what belongs to a module and you can load it and unload it.
Another question
I have mod1 with extensions (DM1) = String>>asUrl
can I see String>>asUrl from mod2 if there are not relations between the two?
I have the intuition that I should only see asUrl from a module only if this module depends from mod1.
What is your point of view on that?
Hope this helps,
- Andreas
-----Original Message----- From: squeak-dev-admin@lists.squeakfoundation.org [mailto:squeak-dev-admin@lists.squeakfoundation.org]On Behalf Of Andrew P. Black Sent: Wednesday, October 03, 2001 3:57 PM To: squeak-dev@lists.squeakfoundation.org Subject: Re: [ENH][Modules] Delta Modules [was: Another version]
Dear Henrik,
It is good that you are improving the presentation on the Wiki, because that is a better place to find answers that the mail archive. (But it does take so much longer to edit the Wiki ... lucky I have cable Internet in my flat)
Your changes do clarify the meaning of what is there quite a bit, so thank you for that. But they do not answer my questions, only confirm that I do really still have those questions. So, let me try rephrasing my questions so it is clearer what I am asking.
Question the first:
I understand that "DeltaModules can be un/installed, but the un/install operation has no meaning for regular Modules" [1]. My question is: Why. If un/install is such a good idea, why not have it for all Modules, not just for DeltaModules?
Question the second:
DeltaModules are a kind of Module[2]. They are distinguished from base modules by being defined by difference. So, for example, StringWithUrlOps might be defined as being the same as String_V1.2 with the addition of a specific set of Url manipulation methods. But the DeltaModule does not do this as a set of changes, but rather as a definition of the final state. Right so far? So, my question is: why do you consider DelatModules as being conceptually different from Modules at all? Why do they have different properties, just because they happen to have a different representation? After all, v2.3 of a text document is still a text document, regardless of whether it is defined by deltas from version v2.2 or by giving the full text. In OO terms, the representation (whether by delta or by absolute) is hidden, only the protocol is important. Why do DeltaModules and Module not have identical protocols? Why are DeltaModules and regular Modules not just tow implementation classes with the same interface?
References:
[1] http://minnow.cc.gatech.edu/squeak/2071
[2] http://minnow.cc.gatech.edu/squeak/2063
Andrew P. Black wrote:
I was reading about the concept of "Delta Modules" on the Swiki (at http://minnow.cc.gatech.edu/squeak/2063).
...
Finally, what is the motivation for introducing
DeltaModules in the
first place?
Andrew,
To try to answer these questions, I fleshed out that page,
this should
hopefully be useful to you.
http://minnow.cc.gatech.edu/squeak/2063
If these advantages make the separation of loading from activation and unloading from deactivation worthwhile, then shouldn't we make the same separation for Modules themselves, not just for
DeltaModules.
I added a new page for this.
"What about conflicts between modules?"
I don't see what conflict resolution has to do with my question at all. But maybe you thought that I was asking a different question; sorry for being unclear.
I don't understand your page about conflicts either, but I made a comment on the page.
Andrew
Stephane,
Please keep in mind that I said that's how I think about it - Henrik's view might be entirely different (I'm sure it's at least partly different ;-) so forgive me if I can't answer all your questions. I'm not an expert; I actually haven't done anything except asking Henrik about some of the implications.
Have you worked with VW and/or Envy? in envy this is called class extension, in envy they cannot hide pre existing method nor change the class definition, while in VW with Parcel you can hide 9and get it back on unload) a method and may be change the class definition I have to check,
I think that's correct. My feelings about this issue are controversial. From one point of view, it seems inherently dangerous to allow modifications of other classes that go beyound simply adding methods. At the point were any modification besides adding is allowed, you are allowed to break the "API" of that class. I'm not saying that this must be the case but it can be and it's made easy to break the class (be that only because of a completely unintended side effect). However, the other side of it is that modifying the module itself might not always be practical and if there's no mechanism at all allowing for this the entire process can become extremely painful. As an example (we got a nice one in Squeak) take Form>>displayInterpolatedOn: which is supposed to blow up some form in high quality. If the 3D engine is present, then it'll use bi-linear interpolation from there; if not, it's using WarpBlt (which still gives reasonable results). It is really unclear to me how this should be handled in the best (read that as: simplest, most obvious, and easy to maintain) way.
Delta Modules say "okay, for making this module work in a given environment I need some modification of x or y in a different module" and that is conceptually very different from defining a set of useful classes which - given the absence of the environment - could perfectly well stand on their own.
This is not that different, you only do not define a class.
What could be more different? ;-) On the one hand we have a set of well-defined classes and their methods, playing nicely together. On the other we have a number of "intruders"; classes that only need to interact with that module because "they are there". Methods in the module have a well-defined home, they form a well-defined algebra. Those in DMs are being attached to "other" classes; might not even be there etc. etc. etc. How could the two be more different?!
But DM should be unloaded when the module they refer to unload. I do not really see why there is a difference between the two. This is just the way program are perceived and this is fun to see that in a world like Smalltalk where normally we can add method anywhere we start to do a conceptual distinction between class and method and method attached to other classes.
We're not making an arbitrary distinction here. The difference is inherent because the "other classes" might not even exist.
A module could be just a list of declaration (method, global, class,..... and you do not need to know what is DM or Module) as soon as you know what belongs to a module and you can load it and unload it.
It could. But at that point you're moving the complexity into the module itself. One of the beautiful points about Henriks design (at least the way I see it) is that a module is clean per definition. A module doesn't contain any of the "ugly" stuff (modifications to other classes etc) so it's a beauty in itself. If you get the module you got everything that makes it a piece of functionality. For embedding it into the environment you need deltas, that extend the existing environment to cooperate with the module. No point in "messing up" a module with all that stuff. I like this idea.
Another question
I have mod1 with extensions (DM1) = String>>asUrl
can I see String>>asUrl from mod2 if there are not relations between the two?
Good question. First of all what do mean by "see" here? Being able to send #asUrl to String?! I think you'll be able to do this (at least Henriks last version didn't include selector name spaces).
I have the intuition that I should only see asUrl from a module only if this module depends from mod1.
What is your point of view on that?
Generally, I like selector name spaces. But they can be confusing to use. I believe that the major issue here is (as with so many others ;-) the user interface that is provided. If there's a selector name space then we need to make sure we see the right sub-set of selectors when looking at the object. But I figure that's a while in the future.
Cheers, - Andreas
On Thursday, October 4, 2001, at 01:33 am, Andreas Raab wrote: [...]
This is not that different, you only do not define a class.
What could be more different? ;-) On the one hand we have a set of well-defined classes and their methods, playing nicely together. On the other we have a number of "intruders"; classes that only need to interact with that module because "they are there". Methods in the module have a well-defined home, they form a well-defined algebra. Those in DMs are being attached to "other" classes; might not even be there etc. etc. etc. How could the two be more different?!
Ah, but class definitions can collide as easily as method definitions. Also, the superclass may or may not be present. Notice the strong analogy with the problems that extension methods have (also note the reductio ad absurdum). A module system that doesn't address these issues is probably still better than simple change sets, but not qualitatively so.
Hint: There is a solution, but it involves significant changes to fundamental parts of Squeak. Go ahead with lightweight modules until you see the problems first-hand. That'll give some nice context for how to actually solve the problems.
-Mark
Dear Andreas and Henrik,
Forgive me for pushing on this, but the more you explain, the more confused I become.
I have read the discussion between Andreas and Stéphane, and certainly agree that when one is making an extension to a complex system, one has to do two things, which are rather different from each other:
(1) Define some new stuff. This is mostly classes, but might also include globals. Examples might be the classes that make up the HTML manipulation and representation code, or the 3D engine.
(2) Connect one's new stuff to the old stuff. This is mostly adding methods to existing classes, but will also include changing at least one method somewhere, or else the new stuff will never be used (other than from a DoIt). Examples of this are Andreas's Form>>displayInterpolatedOn:, which should change when the 3D engine is available, and String>>asUrl.
Now, as I read Andreas' reply, the intention is that Modules are for (1) and DeltaModules are for (2). In other words, that this difference in usage, or policy, should be reflected by a difference in _mechanism_. Is this right?
And Henrik's reply is that the difference in mechanism is that Modules define a bunch of stuff absolutely, while DeltaModules define a bunch of stuff by difference. Is this right?
Now, I have spent a lot of time in the Operating Systems community, where one of the principles that we value highly is the separation of policy from mechanism. We have found through hard experience that the best thing for the Operating System designer to do is to provide a small number of simple and powerful mechanisms, and let the people who use them come up with various policies for, e.g., the allocation of resources, that use these mechanisms in different ways.
The alternative approach, which seems at first to be attractive, is for the system designer to guess what policies the application writer will want, and "build them in". This turns out not to work, and you can probably guess the reasons that it doesn't work. First, the policies are a lot more complex than the mechanisms, so more complexity (and more bugs) gets pushed down into the lower levels of the system. Second, the system designers are not very good at understanding what the application developers need to do, so the policies aren't quite right, but because they are "built in", and complex, they are very hard to change. And, thirdly, the application developers will insist on trying to implement new kinds of application, including things that the system developers had never thought of, and for which those built-in mechanisms are inappropriate.
I think that Smalltalk has also, for 20 years, adopted a policy-mechanism separation, and this has contributed greatly to its success, power and longevity. Once example of this is inheritance, which is a mechanism that can be used in many different ways. There is no language syntax that distinguishes between an abstract and a concrete superclass: these are indeed two different policies (uses of the class definition mechanism), but there is just one mechanism. Another example is the self-describing "chunk" mechanism used for file in/out, which can be used to build and read both class fileOuts and changeSets. The policy/mechanism separation is a good rule of thumb, and usually gives one a simple powerful system that can be used in a multitude of different ways, including many that have not yet been thought of.
Of course, policy/mechanism separation is only a rule of thumb, and there may be a reason to break this rule. But I believe that we should understand that reason fully, and should think hard, before we choose to do so.
It seems to me that the distinction between Module and DeltaModule, at least as Andreas describes it, is breaking the policy/mechanism separation. I am not (yet) saying that this is wrong. But I _am_ asking pointedly "Why?" What is the compelling reason that makes you want to break the rule? And I am not getting an answer to that question.
Now, I had assumed up until this morning that a Module could contain not only whole classes, but also "class extensions", that is, groups of methods that could be added to existing classes, even though those classes might be defined in different modules. (The String>>asUrl example again, which I had assumed could be part of the HTML module). Now I see that I might be wrong about this, and that Andreas' point of view would say that Modules shall contain _only_ whole classes, and DeltaModules shall contain _only_ class extensions and perhaps class retractions (deleting methods). In spite of the many pages now on the Wiki, I can find only one paragraph that describes what is _in_ a Module:
From http://minnow.cc.gatech.edu/squeak/2069 :
A module
- may contain zero or more submodules.
- may also take zero or more other modules as "input parameters".
- will of course also have contents proper: classes, globals, etc.
I notice that "proper contents" does not include methods. Is this intentional? Is this a mechanism restriction that is intended to make it hard for me to put the "wrong stuff" in my module, that is, to put in "loose methods" that change some other classes? In other words, are you trying to enforce good behaviour on the part of the programmer by restricting the available mechanisms? If so, I admire your good intentions, but I think that you are misguided. Bad programmers are just too ingenious!
Since Modules can contain sub-modules, and some of those sub-Modules can be DeltaModules, then it seems that in any case you have not achieved anything with this restriction. I just put my "loose methods" (like String >> asUrl) into a DeltaModule inside my Module, and the unsuspecting user who imports my module is still surprised that it "damages" other classes like String, or Form, or whatever.
I think that we should indeed do what Henrik writes on the Wiki: build a "minimal and yet powerful semantic model for a Squeak Modules system". I don't think that this policy separation between defining new and modifying old need be part of that model. I think that this distinction, and others that we have not yet thought of, will be displayed by the modules that we write, just as the Abstract/Concrete class distinction is displayed by the classes that we write. But I have yet to hear an argument for putting it in the model.
Andrew
"Andrew P. Black" wrote:
Since Modules can contain sub-modules, and some of those sub-Modules can be DeltaModules, then it seems that in any case you have not achieved anything with this restriction. I just put my "loose methods" (like String >> asUrl) into a DeltaModule inside my Module, and the unsuspecting user who imports my module is still surprised that it "damages" other classes like String, or Form, or whatever.
That is not a bug, it's a feature ;-)
But, thinking about it, you touch a valid point here. In my understanding a module always describes a closed, self contained unit with globals and classes in a name space. Delta Modules describe deltas to one of these units. But, and I had made that point earlier in a discussion, this clean and simple mechanism(?) is broken by having modules be also the unit for packaging arbitrary combinations of other modules and delta modules.
Hmm, again, confused...
I think that we should indeed do what Henrik writes on the Wiki: build a "minimal and yet powerful semantic model for a Squeak Modules system". I don't think that this policy separation between defining new and modifying old need be part of that model. I think that this
I still would think that a delta module is a mechanism. I only hope this thread doesn't end up with us throwing dictionaries at each other ;-)
Michael
Adrew Black raises various concerns about the specification of delta modules that I largely share.
In my experience, one of the concepts that many Smalltalk programmers have difficulty accepting is that it is a good thing to distinguish between the external artifacts that define a functional unit of a program and the actual executable instantiation of that functional unit. (with apologies to Korzybski, the class definition not the class). I think the delta module discussion/specification suffers from the problem of blurring the identify of the specification with the identify of the specified entity.
Consider a complex, deployed class named Foo (I could talk about "modules" but I think everybody on this list has a common understanding of the word "class". I don't think this extends to the word "module").
The Foo might be defined using four distinct units of code: BaseClass + Extension1 + Extension2 +Extension3
where: BaseClass = Implementation of ANSI Smalltalk specification of Foo Extension1 = Essential Squeak extensions to ANSI functionality Extension2 = MVC related methods Extension3 = Morphic related methods
It makes sense to manage these four pieces of code separately because in different situations we may want different compositions of the code units. None of the above code fragments are actually the "class" Foo, they are simply specifications of portions of various potential classes each named Foo. Practically speaking, there are probably five real classes that, depending on the usage situation, might be instantiated from these fragments: Foo = BaseClass Foo = BaseClass+Extension1 Foo = BaseClass+Extension1+Extension2 Foo = BaseClass+Extension1+Extension3 Foo = BaseClass+Extension1+Extension2+Extension3
Regardless of which combination is used in a given situation, semantically we only have a single class Foo that is actually instantiated in any particular image (or single namespace). From the perspective of executing code, we don't care how that class was specified or constructed. From a code management perspective, it should be clear than an extension is not a class.
By analogy we should be able to reason similarly about modules and delta modules.
What confuses me is distinction (if any) between "modules" as runtime semantic elements of the language (namespaces?) and "modules" as the external specifications of fragments (components?) that can be dynamically composed to make complete semantic entities. If the word "module" means the semantic entity that corresponds to a populated namespace then it would make no sense to say that an extension is a module, for it clearly isn't. If "module" means an external specification of a program fragment then an extension could arguably be called a kind of "module".
In my experience, it is very important to clearly make this distinction between the entity and its specification and to use distinct vocabulary when discussing them. When this isn't done, we often get a confused burring of two very different types of abstractions into a more complex multi-role abstraction. Simplicity seems to be a watch word of the Squeak community. Having two simple distinct abstractions is often simpler than having one larger multi-role abstraction.
Allen
At 10:55 AM 10/4/2001 +0200, Andrew P. Black wrote:
Dear Andreas and Henrik,
Forgive me for pushing on this, but the more you explain, the more confused I become.
I have read the discussion between Andreas and Stéphane, and certainly agree that when one is making an extension to a complex system, one has to do two things, which are rather different from each other:
(1) Define some new stuff. This is mostly classes, but might
also include globals. Examples might be the classes that make up the HTML manipulation and representation code, or the 3D engine.
(2) Connect one's new stuff to the old stuff. This is mostly
adding methods to existing classes, but will also include changing at least one method somewhere, or else the new stuff will never be used (other than from a DoIt). Examples of this are Andreas's Form>>displayInterpolatedOn:, which should change when the 3D engine is available, and String>>asUrl.
Now, as I read Andreas' reply, the intention is that Modules are for (1) and DeltaModules are for (2). In other words, that this difference in usage, or policy, should be reflected by a difference in _mechanism_. Is this right?
And Henrik's reply is that the difference in mechanism is that Modules define a bunch of stuff absolutely, while DeltaModules define a bunch of stuff by difference. Is this right?
Now, I have spent a lot of time in the Operating Systems community, where one of the principles that we value highly is the separation of policy from mechanism. We have found through hard experience that the best thing for the Operating System designer to do is to provide a small number of simple and powerful mechanisms, and let the people who use them come up with various policies for, e.g., the allocation of resources, that use these mechanisms in different ways.
The alternative approach, which seems at first to be attractive, is for the system designer to guess what policies the application writer will want, and "build them in". This turns out not to work, and you can probably guess the reasons that it doesn't work. First, the policies are a lot more complex than the mechanisms, so more complexity (and more bugs) gets pushed down into the lower levels of the system. Second, the system designers are not very good at understanding what the application developers need to do, so the policies aren't quite right, but because they are "built in", and complex, they are very hard to change. And, thirdly, the application developers will insist on trying to implement new kinds of application, including things that the system developers had never thought of, and for which those built-in mechanisms are inappropriate.
I think that Smalltalk has also, for 20 years, adopted a policy-mechanism separation, and this has contributed greatly to its success, power and longevity. Once example of this is inheritance, which is a mechanism that can be used in many different ways. There is no language syntax that distinguishes between an abstract and a concrete superclass: these are indeed two different policies (uses of the class definition mechanism), but there is just one mechanism. Another example is the self-describing "chunk" mechanism used for file in/out, which can be used to build and read both class fileOuts and changeSets. The policy/mechanism separation is a good rule of thumb, and usually gives one a simple powerful system that can be used in a multitude of different ways, including many that have not yet been thought of.
Of course, policy/mechanism separation is only a rule of thumb, and there may be a reason to break this rule. But I believe that we should understand that reason fully, and should think hard, before we choose to do so.
It seems to me that the distinction between Module and DeltaModule, at least as Andreas describes it, is breaking the policy/mechanism separation. I am not (yet) saying that this is wrong. But I _am_ asking pointedly "Why?" What is the compelling reason that makes you want to break the rule? And I am not getting an answer to that question.
Now, I had assumed up until this morning that a Module could contain not only whole classes, but also "class extensions", that is, groups of methods that could be added to existing classes, even though those classes might be defined in different modules. (The String>>asUrl example again, which I had assumed could be part of the HTML module). Now I see that I might be wrong about this, and that Andreas' point of view would say that Modules shall contain _only_ whole classes, and DeltaModules shall contain _only_ class extensions and perhaps class retractions (deleting methods). In spite of the many pages now on the Wiki, I can find only one paragraph that describes what is _in_ a Module:
From http://minnow.cc.gatech.edu/squeak/2069 :
A module
- may contain zero or more submodules.
- may also take zero or more other modules as "input parameters".
- will of course also have contents proper: classes, globals, etc.
I notice that "proper contents" does not include methods. Is this intentional? Is this a mechanism restriction that is intended to make it hard for me to put the "wrong stuff" in my module, that is, to put in "loose methods" that change some other classes? In other words, are you trying to enforce good behaviour on the part of the programmer by restricting the available mechanisms? If so, I admire your good intentions, but I think that you are misguided. Bad programmers are just too ingenious!
Since Modules can contain sub-modules, and some of those sub-Modules can be DeltaModules, then it seems that in any case you have not achieved anything with this restriction. I just put my "loose methods" (like String >> asUrl) into a DeltaModule inside my Module, and the unsuspecting user who imports my module is still surprised that it "damages" other classes like String, or Form, or whatever.
I think that we should indeed do what Henrik writes on the Wiki: build a "minimal and yet powerful semantic model for a Squeak Modules system". I don't think that this policy separation between defining new and modifying old need be part of that model. I think that this distinction, and others that we have not yet thought of, will be displayed by the modules that we write, just as the Abstract/Concrete class distinction is displayed by the classes that we write. But I have yet to hear an argument for putting it in the model.
Andrew
Allen wrote:
Consider a complex, deployed class named Foo (I could talk about "modules" but I think everybody on this list has a common understanding of the word "class". I don't think this extends to the word "module"
I would like to propose that we use the definition of Module that Allen proposed, where a Module is a collection of specifications, and then use the word "Environment" for what Henrik, Dan, and I ( among others ) have been calling "Module". I just can't quite get myself to call what I've been doing a "Component"... might as well call it a "Thing", for all the descriptive power that brings.
As far as I know, there are at least four layers of things that can be "modular" in the Smalltalk world. This is a big part of why we are having some confusion in talking about these things. "Modules" and "Environments" fall into the middle two layers. The topmost layer is where we modularize source code management. The lowest layer is where we find ImageSegments modularizing instance space. Good solutions in each of these layers should yield a very powerful system. As far as I know, each layer can be developed rather independently of the others, especially if each layer makes only simplest assumptions about the character of the layer beneath it ( ie, that it is no more complex than the world we have today ), and none about the layers above it.
* ImageSegments should have no knowledge of Environments, Environments should consider themselves to be living in a single ImageSegment.
* Environments should have no knowledge of Modules, Modules should consider themselves being placed into only a single Environment.
* Modules should have no knowledge of the Source Code Management system. Artifacts in the Source Code Management system should only consider themselves as being placed into a single Module.
Coordinating which things from which layers go where is not the job of any of the above. That is a tools issue- only the tools have sufficient context to properly handle that. Tools will be off to the side a bit, with a slightly more comprehensive view of the world, so this will not be a problem.
- les
I have been out of town for a few days without email, this is why I haven't been able to answer before.
Andrew P. Black wrote:
DeltaModules are a kind of Module[2]. They are distinguished from base modules by being defined by difference. ... So, my question is: why do you consider DelatModules as being conceptually different from Modules at all? Why do they have different properties, just because they happen to have a different representation? ... Why do DeltaModules and Module not have identical protocols? Why are DeltaModules and regular Modules not just tow implementation classes with the same interface?
You are right: My specification of DMs was unclear as well as unfinished, but I wanted to put it out early along with the code.
It did mix the implementation aspect with how DMs would be used, in the manner of "here are some problems, and this is how DMs can address them". The implementation difference is not strictly linked to the uses I mentioned, but in practice there will be such a link: the convenience of the delta representation means that class extensions probably will be put in DMs 99% of the time, and so on. But I know it may have looked like there was a harder link there. (I hope this answered your policy vs. mechanism point as well, otherwise let me know.)
You are also right in that the differences between regular and delta modules should be as small as possible, and that we should be specific about what they are. However, the two cannot be identical: a DM needs to be defined relative to a base module, and the concept of a base module is meaningless for a regular module. This ought to be the only real difference. Then we may want to have some tool-like support for viewing DMs in terms of differences wrt the base module, but those are minor issues.
I think that this situation between Modules and DeltaModules is rather common in class/subclass relations, and I think your question can be understood by analogy to other known cases: why aren't the protocols for Array and OrderedCollection identical, and so on. It is because they are very similar but still slightly different.
The result of activating/installing a DM is equivalent to creating a new/different version of the base module by filing in all the contents as a regular Module. (I now think that de/activating a DM is a better term than un/installing it.) You aren't forced to use DMs for class extensions, but filing in five new methods is clearly more convenient than loading a new version of the whole module that contains String.
Activating a DeltaModule does not modify its base module--it creates a new version of the base module, with the modifications installed. Being able to modify a module is a bad thing since different code packages should be able to specify a module by version and be able to rely on its contents. But this distinction is a technical detail.
ducasse stephane wrote:
I have mod1 with extensions (DM1) = String>>asUrl
can I see String>>asUrl from mod2 if there are not relations between the two?
I have the intuition that I should only see asUrl from a module only if this module depends from mod1.
What is your point of view on that?
There is a short answer to this: Yes, that was just how I intended it to be.
More precisely, when the DeltaModule containing your extensions would be made active, this would create a new, modified version of the base module with the changes installed into it. If mod2 wouldn't specify a particular version of the modified module, then it might use the new version as well, but if it specifically wanted the old version then it wouldn't see the changes.
If two different loaded systems want to modify the same module (the same class/es), then you need to switch between the two module versions created by the modifications--you enter into the area of conflict resolution.
Andrew P. Black wrote:
I understand that "DeltaModules can be un/installed, but the un/install operation has no meaning for regular Modules" [1]. My question is: Why. If un/install is such a good idea, why not have it for all Modules, not just for DeltaModules?
The reason is that Modules are independent and self-contained, and therefore cannot be in conflict with each other. However, several packages may want to apply DMs to the same module, and then there is a use for being able to de/activate all but one of these DMs.
But while you could have two versions of Morphic loaded at the same time, with no conflicts since they would be two different modules, you can't have both running at the same time. E.g. who takes care of input events? But this is not a conflict on a module level (which just defines code), but on a component level (i.e. between parts of the system that are running).
I just put my "loose methods" (like String >> asUrl) into a DeltaModule inside my Module, and the unsuspecting user who imports my module is still surprised that it "damages" other classes like String, or Form, or whatever.
I don't agree at all. DMs not only collect all external modifications in a special place, instead of mixing it with the "proper" contents of your module. They also associate such modifications with the modules where they are made, so that you by looking at the list of delta modules can see exactly what parts of the system a certain package modifies. Any "unsuspecting user" would most probably look at what parts of the system a module modifies before they use it. In this way this is made easy. Can you suggest any way to make it clearer?
This also serves to make it very clear to a conflict resolver in what parts of the system conflicts arise with other loaded packages: conflicts need only be handled in those modules where multiple sources want to have modified versions of the same module.
Now, I had assumed up until this morning that a Module could contain not only whole classes, but also "class extensions", that is, groups of methods that could be added to existing classes, even though those classes might be defined in different modules. (The String>>asUrl example again, which I had assumed could be part of the HTML module).
But the very purpose of modularity is to separate a system into smaller, independent, self-contained pieces, whose definitions are not entangled in each other. Modules really should not be allowed to modify the contents of other modules. Allowing this would go against the very idea of modularity--it is really a contradition in terms, more or less. It is the same violation as if an object would be allowed to directly alter the innards of another object, it breaks encapsulation.
A module
- will of course also have contents proper: classes, globals, etc.
I notice that "proper contents" does not include methods. Is this intentional? Is this a mechanism restriction that is intended to make it hard for me to put the "wrong stuff" in my module, that is, to put in "loose methods" that change some other classes?
...
If so, I admire your good intentions, but I think that you are misguided.
This is just because in ordinary Smalltalk you don't put methods anywhere except as part of classes, this just follows that principle. You may consider Smalltalk to be misguided when it enforces this principle, and would then regard C++ as superior in this respect, but many would consider this part of the essence and elegance of Smalltalk. #Smalltalk contains classes and other globals, but not loose methods, it's the same thing with modules. In fact, neither DMs contain loose methods, they are put in class objects too, at least as it is now. In fact it would be more work to support the kind of exception you suggest, than to simply put methods in classes like now.
Dan stated earlier that while method extensions may be supported, they shouldn't be encouraged (something to that effect), and I agree, since it breaks the above principles. I see it as a kind of emergency solution or exception, a concession to realities, it is never strictly necessary I believe--although I don't want to repeat the {} discussion here.
Now I see that I might be wrong about this, and that Andreas' point of view would say that Modules shall contain _only_ whole classes, and DeltaModules shall contain _only_ class extensions and perhaps class retractions (deleting methods).
Since DMs are defined as differences in relation to a base module, per this definition (and for superseding change sets) they strictly ought to be able to define any possible differences in a module, however some changes are harder to support than others. Still, adding new classes should definitely be possible, and it is easy to support (DMs being Modules, I think they already do). But this would probably be used when DMs are used in the role of change sets--a package will rarely need new classes to be added to other modules.
The email of yours in reply to Michael Rueger also said a lot of interesting things about modules being namespaces (not stated on the Wiki) and that these namespaces do not follow the hierarchic structure of the module nesting (not on the Wiki either).
In both cases, the information on how names are looked up was in the first posting I made, and was copied into the first heading listed on the Swiki, "design principles". However, Smalltalk is the only language I can think of where different modules share a single namespace, so I may not have made it as clear, taking it for granted. Again, it seems to go against the very principle of modularity, where modules' contents should be independent per definition. So I've been taking this feature for granted whereas all Smalltalkers obviously don't.
The distinction between BasicModules (as I shall call them) and DelatModules as I now understand it is:
As I said above, it now seems clearer that the only difference ought to be that a Module is self-contained, whereas DeltaModules are represented as differences wrt a base module. However, this apparently small difference will lead to great differences in usage patterns.
- BasicModules can be used only to define whole classes, whereas
DeltaModules can be used only make additions, removals or changes to classes that are defined elsewhere.
Regular Modules should be complete and self-contained, as per the definition of modularity, with e.g. complete classes, as usual in Smalltalk.
DMs should in principle be able to handle all possible differences wrt a base module: new items, deletions, and changes.
- BasicModules can contain (sub) BasicModules, global specifications
and Class specifications, but not DeltaModules.
Regular Modules can be linked to any other Module, regardless of their kind (of course). PackageModules etc. are unnecessary.
- DelataModules can contain (sub) DeltaModules and change
specifications (but not BasicModules)
A DM really ought to only specify changes to the contents, and not itself have any submodules. If your system X modifies module A and A's submodule B, then X should have one delta module DM(A) and one DM(B), this is for clarity, in this way you clearly see what parts X needs to modify.
Henrik
squeak-dev@lists.squeakfoundation.org