Hi all!

Quoting Doug Way <dway at riskmetrics.com>:
[SNIP]
> Okay, this is getting into a difference between these two approaches 
> which needs to be fleshed out some more (and which I'm somewhat familiar
> with, at least :-) ).
> 
> With Envy/Ginsu-style modules (which I assume are similar to what Roel &
> 
> Steph are talking about), modules can contain classes, and can also 
> contain "Class Extensions" (a.k.a. loose methods).  I would define Class
> 
> Extensions as methods which are additions to classes in other modules, 
> but they are ONLY additions, NOT changes to existing methods (and not 
> removals either).
> 
> DeltaModules, on the other hand, can contain method additions, but also
> 
> changes to existing methods and method removals too.

And in fact any other possible change too (just so that noone misunderstands
that, I know you know this).

> So, to generalize a bit, DeltaModules are more powerful, but Class 
> Extensions are quite a bit simpler.
>
> Let's think about a couple of useful properties of modules:
> 
> 1. Can they be cleanly unloaded?
> 2. Are they behavior-preserving?  In other words, if you load (and 
> activate) a module, will the rest of the system still run without 
> breaking or behaving differently?
> 
> Regarding #1, 3.3a-style Modules and DeltaModules can be unloaded, and 
> Envy/Ginsu-style modules can too, so no problems there.  Changesets 
> cannot be unloaded, which is their big limitation.
> 
> Regarding #2, this is one reason for the split between Modules and 
> DeltaModules in 3.3a.  If you load a Module (with no DeltaModules), the
> 
> rest of the system will be unaffected, so they are behavior-preserving.
> 
>  DeltaModules are not behavior-preserving, but that's the point of 
> them... they can be used if you do need to change something else in the
> 
> system.
> 
> Because DeltaModules are not behavior-preserving, they naturally have to
> 
> change version numbers of the Modules that they modify.  So for example
> 
> if a WebBrowser Module has a DeltaModule which adds the method 
> #asEncodedHtml or changes a method in class String, in the Module 
> "Kernel 2.7", the Kernel module will need to be assigned a new version 
> number such as 2.8.  (I believe this is how DeltaModules are supposed to
> 
> work, correct me if I'm wrong.)  This may have big implications for the

This is correct AFAIK.

> rest of the modules in the system, which might expect to run on Kernel 
> 2.7 but not know about 2.8.

Exactly.

> But the interesting thing is, Envy/Ginsu-style modules w/Class 
> Extensions *are* behavior-preserving, with a few caveats* (see bottom of
> message).  This is because, in general, if you add a new method to an 
> existing class, none of the rest of the system would ever send that new
> method, so the behavior of the rest of the system is unchanged.  So, 
> reusing the example above, if a WebBrowser module has a Class Extension
> which adds the method #asEncodedHtml (but does not change any method) in
> class String, in the module "Kernel 2.7", the Kernel module will still 
> remain 2.7.  (This is exactly what Envy does.  I don't remember now if 
> Ginsu supports version numbers.)

Yes, a very crucial observation. Just a note though:
You listed a few caveats but there is one more - the new behaviour may of course
affect the state of the instance so that other "modules" are affected. But that
is probably more of a "bug" than a "source code conflict". Whatever.

> The other question is then, if we're using Class Extensions, is it good
> enough to only be able to *add* methods to classes in other modules, not
> modify/remove them (as DeltaModules can do)?
> 
> Well, I guess that's a big question. :-)  I would say that most of the 
> time, yes, it's good enough.  A module does not often need to 
> change/remove methods in other modules, unless we're directly fixing a 
> bug in that other module.  Adding methods is much more useful, and can 
> be specific to the "outer" module (such as the example WebBrowser module
> adding #asEncodedHtml to String).  People who've used Envy know that it
> can be very handy... although you don't want to abuse it to an extreme.
> 
> With Envy/Ginsu, if your module *really* needs to change/remove a method
> in Kernel 2.7, you make your own private version of Kernel 2.8 with the
> change, and/or you lobby the maintainer of Kernel to make the change. 
>  But this should be relatively rare aside from bug fixes.
> 
> On the other hand, the ability of DeltaModules to handle method 
> changes/removals could be quite nice.  And they seem a bit more 
> rigorous, no "caveats" as with Class Extensions below.  And namespaces 
> are already built-in.  But there is extra complexity cost...  For 
> example, DeltaModules have to worry about activation/deactivation, with
> Class Extensions it is irrelevant.  The Class Extensions are always 
> active when the module is active, since they don't affect anything
> else.
> 
> One last thing: As far as splitting up the current Squeak image into 
> modules goes, my hunch is that it would be easier to do with 
> Envy/Ginsu-style modules.  This is because you would not need the 
> DeltaModule capability of specifying changed/removed methods when 
> divvying things up, only the ability to specify added methods, which the
> lighter-weight Class Extensions can also handle without version number 
> worries.  For example, String>>asMorph could be quickly dumped into some
> Morphic module.

There might also be another problem lurking with DMs - at least when trying to
use them to chop up the image. Since a DM branches off the affected base Module
then it would seem to me that we will end up with tons of "branches" all over
the place. Module C has a DM which branches off Kernel to 2.8 and Module D has
another DM that branches off Kernel to 2.9 - oops...

> (Avi, are the DVS "logical modules" similar to Envy/Ginsu with Class 
> Extensions?)

Yes, that is my sense of DVS. But using category coventions and thus staying
"compatible" with good old 3.2. But hey, I don't think DVS has any form of
conflict resolution capabilites or anything like that.

> Am I making sense here?  This message is getting too long, sorry.  Let 
> me know if I've characterized one of the approaches incorrectly.  And I
> know this is only one aspect of the differences between these two module
> systems.  Anyway, I've often found that I learn more by see two things 
> compared against each other, than by seeing them described separately.

You are making sense, and it was an interesting angle. Personally I think I have
understood Henrik's Modules but I have always been wondering what will happen
when we reach the real interesting area - conflict resolution...

The following are a few unstructured thoughts: 

Currently I think that producing Envy/Ginsu style packages (for example using
DVS) consisting of standalone classes and loose method extensions will work
quite fine for many of us (as you also noted) - especially if you build stuff
"for" 3.2, compared to trying to detangle 3.2 itself...

But I would like to have these "loose methods" contained within a new smarter
kind of ChangeSet that I will call a "Patch" here below. I will try to explain
why and what such a beast would be.

- A Patch would be similar to a DM in that it is not a recording of a sequence
of changes - it is instead a condensed set of changes, much like a patch-file in
Unix.

- On the other hand a Patch would not be a Module (as DMs are) or anything like
that in itself. Instead a Patch is an "intermediate object" like DVS and other
tools could produce when they want to represent "loose method additions" or
"loose method removals" or "class instvar additions" etc. Patches would be able
to represent themselves in a simple format that can be used inside fileouts and
of course they would be instantiated when filing in such fileouts.

So the Patch would not live in the image like classes do but tools could produce
them as an intermediate object when filing in or out. And when installing them
it may be a good idea to keep them around.

Ok. A Patch would not try to replace ChangeSets in any way. A ChangeSet is a
"recorded session" in some sense and thus IMHO more of a development tool than a
packaging/deployment tool. I think we should keep using them for source code
management but perhaps we should try to avoid using them for
deployment/packaging and instead rely on .st files containing full classes and
Patches.

Here are a few crucial points... A Patch object should:

1. Keep enough state about the changes so that it can be used to detect
conflicts against other Patch objects and packages. For example: One Patch
adding a loose method x in class A would not conflict with another Patch adding
method y in class A - they would both be additions. A Patch changing a method x
in class A (belonging to package AA) would on the other hand potentially
conflict with a package BB that depends on package AA. But perhaps we could even
add classifications of "method changes" - one being "no behaviour change"
(changing comments, formatting, optimization etc) and then it could be resolved
into a "theoretical conflict" (likely not a conflict but could be if the method
change is actually changing behaviour nevertheless) instead of a "conflict". So
in short - we can arm these Patch objects with a lot of "smarts" for detecting
conflicts and also help us resolving them.

2. Keep enough state for deinstallation. This part is (as I have argued in other
posts) not truly critical but if we try to implement these little fellows we
could at least give it a stab. It would be nice.

Both of the above features would need the Patch objects to stay around after
filing in, not necessarily in the image though - we could log them in a file or
something.

So in short, the proposal of a lightweight module system could be like this:

1. We implement the Patch object as described above.
2. Avi :-) incorporates Patch objects into DVS. 
3. We start using .st files with Patches inside them for packaging/deployment.
One way would be by using Envy/Ginsu style packages with DVS.
4. We add dependencies to SqueakMap between packages. We would also need to add
versions of packages of course because currently SqueakMap only knows about the
latest release).

In Henrik's Modules dependencies were declared "per Module". Perhaps that level
is too fine grained? If we add dependencies in SM it would be "per package"
which is a larger and perhaps more appropriate level.

All this would mean that we can continue using ChangeSets if we like. Using them
for deployment would lead to "uninstallable" packages with much less capability
of conflict resolution.

Then when we try to install a package using SM it holds information on what
packages we already have installed. Packages using .cs or "interactive"
.st-files can't really participate in the conflict detection - hopefully those
kind of packages will "phase out". But the "new" .st format with Patches
complemented with dependency information in SM should be both uninstallable and
especially conflict detection between them should work nicely.

In short: The new Patch object is IMHO the key for successful packages/modules.
And the simplest Patch is a "loose method addition" which will give us
Envy/Ginsu style modules. But then nothing stops us from evolving the Patch
concept with more smarts. 

Wow, what a long post this became...

regards, Göran

Göran Hultgren, goran.hultgren at bluefish.se
GSM: +46 70 3933950, http://www.bluefish.se
\"Department of Redundancy department.\" -- ThinkGeek