Just to anticipate a certain category of comments ;-)
Top 10 ways in which the modules system sucks (ordering depends on particular tastes).
- Documentation. This is a feature. No really. I had to remove the documentation to make this system Squeak-compliant. But some of the rationale behind the system is to be found at http://minnow.cc.gatech.edu/squeak/modules.
- Things are probably broken in the existing code base. Some parts most certainly are broken: Alice/Wonderlands use pool variables in a way that really isn't supposed to work, now it propbably doesn't. Project un/loading and image segments most likely don't work properly any more. Some eToy player-something also uses pool variables in a naughty way, in classes that modify their own definition.
- UI support. Right now the module explorer is what you get. And that ain't much. In particular, there is little or no ui support for creating delta modules and their contents yet.
- Version management. This barely exists yet.
- DeltaModules. They are not complete. Changing the external modules of the base modules, or the format of a class, would be the next thing to add.
- Image modularity. I have only been able to make the VMConstruction module, and Balloon3D (almost), unloadable.
- Integration with existing dev. tools. I've tried to fix the most obvious things, but cmd-q doesn't seem to find all symbols. For example.
- Repository classes. These could be made complete. And be made to handle errors gracefully. No good forwarding facility yet.
- ModuleInstaller. A cleaner (and complete) message protocol. More solid unistallation and failure recovery (and tests for these). Possibly the uninstall information will hold on to modules to prevent them from being cleared out from the image?
- Tests. The tests I've written so far cover the functionality to a reasonable amount, but they are not exhaustive. Also, the tests for module storage/retrieval from net servers do not pass yet, in large part because the ServerDirectory code doesn't work properly. This is something anyone could work on.
Henrik
Wow! It's great to see this stuff actually coming through. The details are mostly way over my head, but perhaps I can make one small contribution...
On Wed, 30 Jan 2002 11:30:52 +0000, Henrik Gedenryd wrote:
- Image modularity. I have only been able to make the VMConstruction module,
and Balloon3D (almost), unloadable.
On my first reading of this I took "unloadable" to mean "cannot be loaded", but I think perhaps you mean "can be unloaded" ?
(Or, given the early state of the code, perhaps you mean both :-)
Can I suggest "de-loadable" instead?
Best regards, Nick Brown
Henrik,
Just to anticipate a certain category of comments ;-)
You forgot the #1 issue: Filing in old classes is broken ;-) I got a DNU attempting to file in a class which probably wasn't quite module clean. Here it is:
Object subclass: #MumbleObject instanceVariableNames: '' classVariableNames: 'MumbleClassVar' poolDictionaries: 'TextConstants' category:'Foo-Bar'!
!MumbleObject methodsFor: 'mumbling' stamp: ''! doMumble ^ByteArray new! !
[Notice it fails on looking up ByteArray which I _really_ don't understand]
- Things are probably broken in the existing code base. Some
parts most certainly are broken: Alice/Wonderlands use pool variables in a way that really isn't supposed to work, now it propbably doesn't.
I'll fix that if you can give me a few hints at what to look for.
- Image modularity. I have only been able to make the
VMConstruction module, and Balloon3D (almost), unloadable.
Will fix that too if you let me know what the current problems are.
Cheers, - Andreas
Andreas Raab wrote:
Henrik,
Just to anticipate a certain category of comments ;-)
You forgot the #1 issue: Filing in old classes is broken ;-) I got a DNU attempting to file in a class which probably wasn't quite module clean. Here it is:
Object subclass: #MumbleObject instanceVariableNames: '' classVariableNames: 'MumbleClassVar' poolDictionaries: 'TextConstants' category:'Foo-Bar'!
My guess is that this is because Pools don't exist any longer :-/ because they are unnecessary and Modules can do the same thing in a better way. TextConstants, and the other pools, are now Modules. You can send the messages.
The second most probable cause is that you found a bug.
Now you don't really need to have TextConstants as a pool var, do you? ;-)
1. Send messages to TextConstants to get the definitions: "NewYork" --> TextConstants NewYork, or 2. In the preamble to your file, create your module and declare TextConstants as an external module.
(Module fromPath: #(Foo Bar) forceCreate: true) externalModule: #(<...> TextConstants) alias: nil version: nil importNames: true.
This is handled automatically by the module fileOut/storage system. 2 is the right way to do it but a slight nuisance to hack this way.
- Things are probably broken in the existing code base. Some
parts most certainly are broken: Alice/Wonderlands use pool variables in a way that really isn't supposed to work, now it propbably doesn't.
I'll fix that if you can give me a few hints at what to look for.
Alice and Wonderlands use pool variable names that are *lowercase* and they insist that the names should be Strings, not Symbols. The first will work anyway, but I did not want to hack in support for the second one, so this will probably just break some things.
- Image modularity. I have only been able to make the
VMConstruction module, and Balloon3D (almost), unloadable.
Will fix that too if you let me know what the current problems are.
I think it works ok, actually, now that I thought about it.
Henrik
Henrik,
Object subclass: #MumbleObject instanceVariableNames: '' classVariableNames: 'MumbleClassVar' poolDictionaries: 'TextConstants' category:'Foo-Bar'!
My guess is that this is because Pools don't exist any longer :-/ because they are unnecessary and Modules can do the same thing in a better way. TextConstants, and the other pools, are now Modules. You can send the messages.
Okay, thanks for the clarification. If so, then I need to report two bugs and ask a question. First bug is that some classes still use pool dictionaries (oops) and I believe that the conversion process didn't quite get them right. How do we fix those in particular considering that custom images may contain many classes that may reference pools?! The other bug is that apparently the set of dictionary messages sent from the compiler is not understood by modules. I don't think they should be understood (modules are _no_ dictionaries) or at least in a rather limited way (through their exported names?!). But we need to fix those users of dictionary access messages and the above exposes one of them.
The question is: What do we do about backward compatibility?! In my case, the above was just a very simple example of a CS that I was actually filing in, which used TextConstants and was _supposed_ to just work. I think we need to aim at backward compatibility at the point of just filing in classes - it should not be required that in order to just file in an old bit of Squeak which (for instance) happens to use pool dicts we need to rewrite all the references in the old code.
Now you don't really need to have TextConstants as a pool var, do you? ;-)
See above. It's been old code and, yes, I needed it.
- Send messages to TextConstants to get the definitions:
"NewYork" --> TextConstants NewYork, or 2. In the preamble to your file, create your module and declare TextConstants as an external module.
See above. It's been old code and, no, I really don't want to be forced to do this.
I'll fix that if you can give me a few hints at what to look for.
Alice and Wonderlands use pool variable names that are *lowercase* and they insist that the names should be Strings, not Symbols. The first will work anyway, but I did not want to hack in support for the second one, so this will probably just break some things.
Okay, I'll look what I can do - I think I know how to fix this.
Cheers, - Andreas
Andreas Raab wrote:
Henrik,
Object subclass: #MumbleObject instanceVariableNames: '' classVariableNames: 'MumbleClassVar' poolDictionaries: 'TextConstants' category:'Foo-Bar'!
My guess is that this is because Pools don't exist any longer :-/ because they are unnecessary and Modules can do the same thing in a better way. TextConstants, and the other pools, are now Modules. You can send the messages.
Okay, thanks for the clarification. If so, then I need to report two bugs and ask a question. First bug is that some classes still use pool dictionaries (oops) and I believe that the conversion process didn't quite get them right.
Ok that surely wasn't my intention so info on what they are (or a fix :-) is appreciated.
How do we fix those in particular considering that custom images may contain many classes that may reference pools?!
Those class that already use pools should continue to work fine until you recompile pool-referring code--I don't actively destroy existing pools. But creating classes that use pools wouldn't work anymore.
And to be able to recompile (former) pool using classes you need to
A) convert their pools into (pool) modules.
B) Convert existing code that accesses the pool dictionaries directly. This is typically only pool intialization code (unless you write naughty self-modifying code or some such ;-).
To do A you create a subclass of ModuleRefactorer, and fill in the info by overriding #newModulesForPools. Search for the existing implementor of that message to see how to do this. Check ModuleRefactorer 'convert pools' category to see the code that does this (but you shouldn't need to know how it is done). You run this code by executing "<TheRefactorerClass> run". Save the image before running that--you'll probably need a couple of tries to get it right.
To do B, you change the pool initialization code that sets up the pool vars to define them as vars in the pool module instead. The CS released as update 4681 holds the code that converts the necessary code in the standard image. That CS should be a good reference point for doing this, it holds conversions both for simple pool init code and for some more naughty code.
The other bug is that apparently the set of dictionary messages sent from the compiler is not understood by modules. I don't think they should be understood (modules are _no_ dictionaries) or at least in a rather limited way (through their exported names?!).
Right, I intentionally did not imitate the Dictionary protocol.
But we need to fix those users of dictionary access messages and the above exposes one of them.
What are the dict. access messages you are thinking of? Are they the pool initialization code? If so see above, otherwise get back to me.
The question is: What do we do about backward compatibility?! In my case, the above was just a very simple example of a CS that I was actually filing in, which used TextConstants and was _supposed_ to just work. I think we need to aim at backward compatibility at the point of just filing in classes - it should not be required that in order to just file in an old bit of Squeak which (for instance) happens to use pool dicts we need to rewrite all the references in the old code.
Umm, there is unfortunately no simple way to do any automatic conversion, but simple pool initialization code should be simple to convert.
A possible solution is to make pools fileIn-able again. It would mainly mean putting back pool lookups in the code that binds upper case variable names to values. It's on my list (though a list of what, I don't know).
But you won't be able to use this code as proper module code anyway as pools break modularity.
Henrik
Henrik,
The question is: What do we do about backward compatibility?! In my case, the above was just a very simple example of a CS that I was actually filing in, which used TextConstants and was _supposed_ to just work. I think we need to aim at backward compatibility at the point of just filing in classes - it should not be required that in order to just file in an old bit of Squeak which (for instance) happens to use pool dicts we need to rewrite all the references in the old code.
Umm, there is unfortunately no simple way to do any automatic conversion, but simple pool initialization code should be simple to convert.
It's not initialization that bothers me. It's _using_ some existing pool (like TextConstants). What happens is that when you file in such a class (like my MumbleClass example), the file in process stores a _module_ in the shared pools. And then things start to get horribly wrong.
But you won't be able to use this code as proper module code anyway as pools break modularity.
I don't understand this. Why would they? Pools where the only kind of globals that were actually scoped across the class hierarchy in a halfways reasonable way, so what's the difference between using a pool and importing a module and why would using pools break modularity?
Cheers, - Andreas
Andreas Raab wrote:
But you won't be able to use this code as proper module code anyway as pools break modularity.
I don't understand this. Why would they? Pools where the only kind of globals that were actually scoped across the class hierarchy in a halfways reasonable way, so what's the difference between using a pool and importing a module and why would using pools break modularity?
I agree with Andreas- I don't see how you can claim that Pools break modularity. There is nothing about them that I can see that is a problem here. In fact, they act very much like a weak form of modules, in that they are containers of publicly exported variables. The SystemDictionary Smalltalk was the same way, as was Undeclared and a few other "true" Pool dictionaries ( by this, I am excluding the PoolDictionaries that were/are used by Classes to maintain the so-called "Class Variables" - those were a contextually different useage of these dictionaries ). Pools are certainly a special case in that the previous implementation of them was something of a hack, ( ie, Pools wanted to both contain variables in a dictionary-like way, but also wanted to have certain behavior-like capability, in order to define its initialization- this was a "hack" as the pool referenced a "defining class" in order to accomplish this, thereby putting the initialization burden onto another object where it could very well be argued that responsibility really didn't belong ). But I don't see how that makes them "break modularity".
It seems to me that the proper fix is to upgrade Pools to objects which have some dictionary-like and some behavior-like behavior ( really, all that is required is that the pool be able to initialize itself, without having that code living on some other object as is done now ). ( and I suppose that this is where David will point out that this is where he wanted the instance-specific behavior for modules, since really a Pool is a just a (possibly) weakened form of module... or is it that modules are extended forms of Pools? )
- les
Just some thoughts about the experience I got with Pool in old VW.
I think that the problem with Pools is that they are static. (I did not checked in Squeak) but in VW pool have to be defined before the method that make references to them. To that regards Pools are not equal to class methods of a class that would represent the Pool.
So I always thought that Pools were kind of ad-hoc.
I do not know what you think about that but I have the impression that Pools were an optimization rather than something really needed.
In the new VW5, I understood that the semantics of Pool is more dynamic which raises issues of concurrent synchronization. (I cannot tell more because I never look carefully at it).
Stef
Andreas Raab wrote:
Umm, there is unfortunately no simple way to do any automatic conversion, but simple pool initialization code should be simple to convert.
It's not initialization that bothers me. It's _using_ some existing pool (like TextConstants). What happens is that when you file in such a class (like my MumbleClass example), the file in process stores a _module_ in the shared pools. And then things start to get horribly wrong.
Ok I see.
But you won't be able to use this code as proper module code anyway as pools break modularity.
I don't understand this. Why would they? Pools where the only kind of globals that were actually scoped across the class hierarchy in a halfways reasonable way, so what's the difference between using a pool and importing a module and why would using pools break modularity?
Like this: A globally available definition should be defined in exactly one module. But in which module does a pool variable belong? Alsok, the module system has a precise way of controlling access to definitions, but it cannot control pool variables. Besides, pools are redundant now that there are ways of collecting global values and control access to them--with modules.
Henrik
I don't understand this. Why would they? Pools where the only kind of globals that were actually scoped across the class hierarchy in a halfways reasonable way, so what's the difference between using a pool and importing a module and why would using pools break modularity?
Like this: A globally available definition should be defined in exactly one module. But in which module does a pool variable
belong?
Alsok, the module system has a precise way of controlling access to definitions, but it cannot control pool variables. Besides, pools are redundant now that there are ways of collecting global values and control access to them--with modules.
Redundancy is not necessarily a bad thing, as you could see from my message. In fact, I would argue that for the time being there probably should be a PoolCompatibilityModule allowing you to access pools both ways; either through scoped module messages or through just using the pool dictionary. With respect to where a pool variable is defined you've given the answer yourself: In the pool, of course. The pool was the only existing scoping mechanism and while not as fine-grained as a fully blown module there's still something to be said for this level of scoping. What's happening right now is that you're breaking some fundamental relations to ST80 - and I guess that'll raise a lot of questions.
This is a hopefully informative how-to for those who want to migrate their current code to work within the modular system:
http://minnow.cc.gatech.edu/squeak/HowToPortOldCodeToModules
It also links to a page with the info on converting pools.
If the above needs to be done for every least bit of code, I guess we'll have _serious_ problems to convince people to migrate to 3.3. There _has_ to be an easier way to just file in a bit of old code.
A few things to consider in this respect:
* categories vs. modules: Categorization is independent from modules. When I saw that the new scheme makes category == module I was baffled. Why can't I put two classes from different modules into the same category? Or why can't I put classes from the same module into different categories? The current notion of module == category means that we're loosing one useful level of structuring here.
* 'As yet unclassified' module: I think we've all agreed on making the module system as unintrusive as possible for the average weekend hacker. What I'm really missing is some "as yet unclassified" module which ought to be the default for your weekend hacker.
What I'm envisioning here is basically that we have (at first) everything in the "as yet unclassified" module. People will work there exactly as they did before - no harm done if you don't want to live modules. When you _do_ you have tools to re-classify and fix your module definitions in order to get a serious module.
Cheers, - Andreas
Hi
I like the fact, that pools are now globals defined within a module. I didn't like the pools. I as well like that if something new is introduced something old is phased out.
On Sun, 3 Feb 2002, Andreas Raab wrote:
- categories vs. modules: Categorization is independent from modules.
When I saw that the new scheme makes category == module I was baffled. Why can't I put two classes from different modules into the same category? Or why can't I put classes from the same module into different categories? The current notion of module == category means that we're loosing one useful level of structuring here.
I think it is OK to give the class category some real significance. Up until now it was just a way of grouping classes in the browser's class list with no other function. However there should be a way for a smooth transition.
- 'As yet unclassified' module: I think we've all agreed on making the
module system as unintrusive as possible for the average weekend hacker. What I'm really missing is some "as yet unclassified" module which ought to be the default for your weekend hacker.
I don't mind that I have to put the new classes I am doing into a module (people HannesHirzel).
I do not yet fully understand the concept of DeltaModules. If I alter an existing class (for example add a new method to the class String), where does the addition go to? Into a DeltaModule? Which one?
(I read the page on DeltaModules: http://minnow.cc.gatech.edu/squeak/2063)
What I'm envisioning here is basically that we have (at first) everything in the "as yet unclassified" module. People will work there exactly as they did before - no harm done if you don't want to live modules. When you _do_ you have tools to re-classify and fix your module definitions in order to get a serious module.
Probably a chicken and egg problem. We can't avoid to 'muddle through'. Introducing the modules is a chance to clean things up. People who are not feeling comfortable in doing this can use the version 3.3a-4664 (the one before modules got introduced) and wait some time.
To sum up: I think introducing modules will greatly help the Squeak community to grow.
Cheers Hannes Hirzel
Hannes Hirzel wrote:
I like the fact, that pools are now globals defined within a module. I didn't like the pools. I as well like that if something new is introduced something old is phased out.
I tend to agree, assuming the new functionality is roughly a superset of the old.
Although phasing out shouldn't necessarily be instant. :) Maybe some sort of PoolCompatibilityModule would make sense, I don't know.
On Sun, 3 Feb 2002, Andreas Raab wrote:
- categories vs. modules: Categorization is independent from modules.
When I saw that the new scheme makes category == module I was baffled. Why can't I put two classes from different modules into the same category? Or why can't I put classes from the same module into different categories? The current notion of module == category means that we're loosing one useful level of structuring here.
Hmm... the way that classes seem to be organized in modules in 3.3alpha seems pretty similar (though not identical) to how they were organized in categories before. (e.g. the classes in the Language-Core-Objects module are the same ones that were in the Kernel-Objects category) They also have the same sort of hierarchical structure.
I suppose you could have some sort of grouping by category which cross-cuts the grouping by module, but I think people would tend to group classes in modules similarly to how they would have grouped them in categories. So, IMHO, I'd prefer to see categories dissappear as a separate entity.
- 'As yet unclassified' module: I think we've all agreed on making the
module system as unintrusive as possible for the average weekend hacker. What I'm really missing is some "as yet unclassified" module which ought to be the default for your weekend hacker.
I sort of agree with this. Although I confess I haven't really tried hacking around in 3.3alpha enough yet to tell if the weekend hacker is hampered by modules or not. But I agree that it would be nice to have some sort of default module that new classes would go into without having to do anything. (sort of like the current changeset)
Hannes Hirzel wrote:
I do not yet fully understand the concept of DeltaModules. If I alter an existing class (for example add a new method to the class String), where does the addition go to? Into a DeltaModule? Which one?
(I read the page on DeltaModules: http://minnow.cc.gatech.edu/squeak/2063)
I think Goran's recent summary was pretty good. It sounds like the DeltaModule would be relative to String's base module (Language-Collections-Text), but you could include this DeltaModule with your own module.
(That's all fine & dandy, but I actually have some issues with whether DeltaModules will be able to handle purely additive class extensions in such a way that, e.g. the non-core Morph methods could be split off into different DeltaModules which don't interfere with each other, but could still all be loaded at the same time as part of a standard Squeak release. But I need to look into it further, and it should probably be the topic of a separate message...)
Anyway, I'm also glad that the modules system is in place and the divvying up can begin. I'm sure there will be plenty of discussion of the modules system and any possible modifications as everyone starts to really use it...
- Doug Way dway@riskmetrics.com
Doug Way wrote:
(That's all fine & dandy, but I actually have some issues with whether DeltaModules will be able to handle purely additive class extensions in such a way that, e.g. the non-core Morph methods could be split off into different DeltaModules which don't interfere with each other, but could still all be loaded at the same time as part of a standard Squeak release. But I need to look into it further, and it should probably be the topic of a separate message...)
In fact, that is one of the things the system definitely handles at this point. There are specific tests for method extensions-adding, removing, and changing methods. So that could in principle be done now. The problem is that there is no good integration for redirecting interactive changes into DeltaModules yet (like capturing changes).
Henrik
On Wed, 6 Feb 2002, Henrik Gedenryd wrote:
Doug Way wrote:
(That's all fine & dandy, but I actually have some issues with whether DeltaModules will be able to handle purely additive class extensions in such a way that, e.g. the non-core Morph methods could be split off into different DeltaModules which don't interfere with each other, but could still all be loaded at the same time as part of a standard Squeak release. But I need to look into it further, and it should probably be the topic of a separate message...)
In fact, that is one of the things the system definitely handles at this point. There are specific tests for method extensions-adding, removing, and changing methods. So that could in principle be done now. The problem is that there is no good integration for redirecting interactive changes into DeltaModules yet (like capturing changes).
If the integration is good or not doesn't bother me too much at the moment. I would just like to now how to redirect changes into DeltaModules associated with my own newly created module. (see another thread).
Hannes Hirzel
squeak-dev@lists.squeakfoundation.org
-
Andreas Raab -
Doug Way -
ducasse -
Hannes Hirzel -
Henrik Gedenryd -
Les Tyrrell -
Nick Brown