Dear Henrick, Andreas, and others who may still be interested:

I'm giving up on this thrust.  It seems clear that I am not the only 
person with qualms about the way that the module system is 
developing, but it is also clear that nothing is going to change 
because of further discussion here.

I was just reading Augustin Mrazik's short paper on structured name 
spaces, which includes the following statement: "... first the 
environments and polymorphic class identification have to be designed 
on a conceptual level, and just then implemented in the system". 
Well, that is perhaps the conventional  way to proceed: the other is 
to implement something, and then try to describe it at a conceptual 
level.  This seems to be what is happening with modules: the only 
effective description of what a module is and does is in the code.

I belive in XP, and I've seen XP work well, and XP certainly has the 
flavour of implementation first, description later, if at all.  But 
XP has other principles and practices that protect the team while 
working in this style:

	* There is a user story that defines a goal for each coding effort
	* There is continual face-to-face communication between the 
members of the team
	* There are tests, both unit tests and customer-defined 
acceptance tests
	* There is a system metaphor, which guides the implementation
	* The programmers write the simplest thing that could possibly work.

The reason that I'm concerned about the current work to implement a 
module system is that it seems to lack all of these attributes. 
Which is why I thought it important to come to an agreement on a 
simple semantic model before implementation started.

I think that nevertheless something good will come out of this; there 
are good implementors working on the module system and it will 
probably end up serving my purpose.  I'm more concerned that it will 
be so complex that I never figure out how to use it, which is the way 
with many "powerful systems".

It's also clear that email does not take the place of face to face 
conversation.  I've been asking what I though were simple questions, 
and Henrick needed 12 kB of email to respond -- but never actually 
answered them.  I'm not expecting you to spend more time answering 
this, but I hope that you will take the opportunity to come to Bern 
and visit with Stephane and myself, and see if we can't bridge the 
semantic gulf that seems to separate us.

Henrik writes:


>You are also right in that the differences between regular and delta modules
>should be as small as possible, and that we should be specific about what
>they are. However, the two cannot be identical: a DM needs to be defined
>relative to a base module, and the concept of a base module is meaningless
>for a regular module. This ought to be the only real difference. Then we may
>want to have some tool-like support for viewing DMs in terms of differences
>wrt the base module, but those are minor issues.

Can't any regular module be defined as a Delta with respect to the 
null (empty) module?  Then the empty module could be the only regular 
module in the system.   An analogy: this is like defining all lists 
as the catenation of elements on to the front of a single empty list.


>I think that this situation between Modules and DeltaModules is rather
>common in class/subclass relations, and I think your question can be
>understood by analogy to other known cases: why aren't the protocols for
>Array and OrderedCollection identical, and so on. It is because they are
>very similar but still slightly different.

The protocols for Array and OrderedCollection are not identical 
because the reason for the existence of OrderedCollection is to add 
some functionality, specifically, the add: and remove: protocols. 
This seemed completely different from your description of the 
difference between Delta Modules and regular modules, which was that 
they had a different implementation.  That seemed more like the 
difference between a Stream writing to a file vs a Stream writing to 
an OrderedCollection: the implementation is different, but the 
protocol ought to be the same.


>The result of activating/installing a DM is equivalent to creating a
>new/different version of the base module by filing in all the contents as a
>regular Module. (I now think that de/activating a DM is a better term than
>un/installing it.) You aren't forced to use DMs for class extensions, but
>filing in five new methods is clearly more convenient than loading a new
>version of the whole module that contains String.

If filing-in is the mechanism for loading functionality, that is 
certainly true.  In VisualWorks, of course, they load something like 
image segments, and the user can't see the pause.

>
>>  I just put my "loose
>>  methods" (like String >> asUrl) into a DeltaModule inside my Module,
>>  and the unsuspecting user who imports my module is still surprised
>>  that it "damages" other classes like String, or Form, or whatever.
>
>I don't agree at all. DMs not only collect all external modifications in a
>special place, instead of mixing it with the "proper" contents of your
>module. They also associate such modifications with the modules where they
>are made, so that you by looking at the list of delta modules can see
>exactly what parts of the system a certain package modifies. Any
>"unsuspecting user" would most probably look at what parts of the system a
>module modifies before they use it. In this way this is made easy. Can you
>suggest any way to make it clearer?

Yes, I can suggest a way to make it clearer.  It would be clearer to 
have a module browser tool that examines a module and tells me what 
the module does -- what classes it adds,and what classes it changes 
-- before I load it.  If the module that I'm browsing contains 
sub-modules, I can see their structure too.  I can also selectively 
turn off (i.e., elect not to install) any of the components.  Think 
of a Mac installer, where you can do the "Easy Install", or you can 
go to a Custom Install option that lets you check boxes for each of 
the sub-components that you want, and gives you information about 
each of them.

The reason that I would prefer such a tool is that it would tell me 
what is _actually_ in the module, not what some programmer said was 
in it before the last round of changes!  It doesn't rely on 
conventions, like "additions go only in Delta Modules".  Hence,  I 
can't be confounded because someone doesn't observe the conventions.


>This also serves to make it very clear to a conflict resolver in what parts
>of the system conflicts arise with other loaded packages: conflicts need
>only be handled in those modules where multiple sources want to have
>modified versions of the same module.
>
>>  Now, I had assumed up until this morning that a Module could contain
>>  not only whole classes, but also "class extensions", that is, groups
>>  of methods that could be added to existing classes, even though those
>>  classes might be defined in different modules.  (The String>>asUrl
>>  example again, which I had assumed could be part of the HTML module).
>
>But the very purpose of modularity is to separate a system into smaller,
>independent, self-contained pieces, whose definitions are not entangled in
>each other. Modules really should not be allowed to modify the contents of
>other modules. Allowing this would go against the very idea of
>modularity--it is really a contradition in terms, more or less. It is the
>same violation as if an object would be allowed to directly alter the
>innards of another object, it breaks encapsulation.

So are you saying that Modules cannot modify the contents of other 
Modules, or that they can?  I'm confused again.  Is a Delta Module a 
Module, or not?  It sure sounded like a module up until now.

>
>>>  A module
>>>  * will of course also have contents proper: classes, globals, etc.
>>
>>  I notice that "proper contents" does not include methods.  Is this
>>  intentional?  Is this a mechanism restriction that is intended to
>>  make it hard for me to put the "wrong stuff" in my module, that is,
>>  to put in "loose methods" that change some other classes?
>...
>>  If so, I admire
>>  your good intentions, but I think that you are misguided.
>
>This is just because in ordinary Smalltalk you don't put methods anywhere
>except as part of classes, this just follows that principle. You may
>consider Smalltalk to be misguided when it enforces this principle, and
>would then regard C++ as superior in this respect, but many would consider
>this part of the essence and elegance of Smalltalk.  #Smalltalk contains
>classes and other globals, but not loose methods, it's the same thing with
>modules. In fact, neither DMs contain loose methods, they are put in class
>objects too, at least as it is now. In fact it would be more work to support
>the kind of exception you suggest, than to simply put methods in classes
>like now.


What I meant was not that there could be methods that do not belong 
to any object, but that when I am programming I can not only create 
new classes, but also add methods to existing classes.  This is one 
of the strengths of Smalltalk, because it lets me put the methods in 
the right place.

I think that you are saying that Regular modules can _only_ define 
globals, whole classes, and DelatModules, and that additions and 
deletions to classes can only appear in DeltaModules, either nested 
within a Module or not.  Is that right?

>  > Now I see that I might be wrong about this, and that Andreas' point
>  > of view would say that Modules shall contain _only_ whole classes,
>>  and DeltaModules shall contain _only_ class extensions and perhaps
>>  class retractions (deleting methods).
>
>Since DMs are defined as differences in relation to a base module, per this
>definition (and for superseding change sets) they strictly ought to be able
>to define any possible differences in a module, however some changes are
>harder to support than others. Still, adding new classes should definitely
>be possible, and it is easy to support (DMs being Modules, I think they
>already do). But this would probably be used when DMs are used in the role
>of change sets--a package will rarely need new classes to be added to other
>modules.

So DeltaModules can define globals and whole classes too?  Was I 
wrong again?  I'm frustrated that after all of this discussion, I 
still cannot seem to get a three line summary of what things go in 
which kind of Module.

>
>>  The email of yours in reply to Michael Rueger also said a lot of
>>  interesting things about modules being namespaces (not stated on the
>>  Wiki) and that these namespaces do not follow the hierarchic
>>  structure of the module nesting (not on the Wiki either).
>
>In both cases, the information on how names are looked up was in the first
>posting I made, and was copied into the first heading listed on the Swiki,
>"design principles". However, Smalltalk is the only language I can think of
>where different modules share a single namespace, so I may not have made it
>as clear, taking it for granted. Again, it seems to go against the very
>principle of modularity, where modules' contents should be independent per
>definition. So I've been taking this feature for granted whereas all
>Smalltalkers obviously don't.

I felt really bad that I had missed something obvious.   I went back 
and read it again.  I searched on Namespace.  I still don't see where 
you describe this at all.

I did find the following paragraph under "Module Structure" 
(http://minnow.cc.gatech.edu/squeak/2069)

>Instead of a single path of inheritance for the names available in a 
>Module, it should be possible to delegate name lookup to any 
>(possibly several) imported module or submodule, by marking those 
>modules for delegation. (The present scheme is equivalent to how 
>Self replaces s.i. with its multiple inheritance/delegation scheme, 
>where submodules correspond to slots and delegation corresponds to 
>marking a self slot as a parent, except in that you can delegate 
>also to imported modules.) Simple conflict resolution can be done by 
>specifying an ordered lookup of submodules and imports (in the order 
>of declaration).

Is that what you are referring to?  i read this as describing nested 
scopes, like Algol 60 blocks.

>DMs should in principle be able to handle all possible differences wrt a
>base module: new items, deletions, and changes.
>
>>  *  BasicModules can contain (sub) BasicModules, global specifications
>  > and Class specifications, but not DeltaModules.
>
>Regular Modules can be linked to any other Module, regardless of their kind
>(of course). PackageModules etc. are unnecessary.

I know that PackageModules are unnecessary.  RegualrModules are 
unnecessary too, since DeltaModules can do anything that a Module can 
do, and more besides.  The suggestion of adding PackageModules was 
based on the Andreas' idea that we want a different implementation 
concept for every possible usage.  We have so far identified three 
usages:

	(1) defining new stuff (we do this in a regular module)
	(2) changing old stuff (we do this in a DeltaModule)
	(3) packaging up a subsystem for distribution, which will 
usually require some of (1) and some of (2).

We could do (3) with a RegularModule, but that would destroy the 
purity of RegularModules.  We could do (3) with a DeltaModule,  but 
that would destroy the purity of DeltaModules.   We could  forget the 
idea of having different kinds of module, and instead have a tool 
that tells us what a module would do when loaded.  (My suggestion 
above.)  But if we reject all of these alternatives, then I think the 
consequence is that we need a third kind of module, the 
PackageModule, that just does Packaging, and nothing else.

Do a good job with the implementation, and I'm sure that many people 
will be happy.

	Regards,

		Andrew