So what are we trying to accomplish? Dan listed a set of "desiderata" that I can whole-heartedly support. However, I would like to step up one level before plunging into the details.
My understanding is that a common goal is the elimination of the all encompassing, monolithic image. I can't argue with that goal and I don't think I need to reiterate all reasons this is desirable. However, there are a lot of different reasons that different constituencies have for wanting to see this happen. I don't necessarily believe that one solution will make everybody happy. I also think this may be one reason that there appears to be a lack of consensus on an approach to modularity. Different people are trying to solve different problems.
If one tool won't do the job then the next simplest thing is to have exactly two tools that collective will. For conquering the monolithic image, I'm going to argue that the two tools we need are "components" and "modules". Clearly, I need to define what I mean by this terms. I'll start by trying to list some distinguishing characteristics:
Function Components Modules ------------------------------------------------------------------------------------------ Organizes behavior definition of behavior When runtime development time composition dynamic static coupling loose tight via object refs/messages inheritance/naming/syntax, etc.
Roughly speaking Components correspond to COM or CORBA objects, DLLs, SLLs in Visual Smalltalk, etc.
Roughly speaking Modules correspond to Java source files, Envy projects, TEAM/V packages, Smalltalk change sets.
Modules are all about organizing and managing code, Components are all about composing functionality (behavior). Modules are used to create components.
Most module systems end up have some (usually weak) component-like characteristics. Most component systems have some (usually weak) module-like characteristics. To me, all of the various systems I've heard described in this discussion appear to fit primarily in one or the other of these categories with some features that are more appropriate to the other. (Here is my first crack assignment based upon limited visibility. Component systems: Environments, OASIS, (I don't really know enough about either but this is my impression) , Pope's selector ideas. Module systems: Pelrine's work, change sets). My impression is that people who are primarily thinking about component issues have problems with module systems because they don't see what they are looking for visa versa. Most of my work with modularizing Smalltalk has been on the module side. However, I think there is a real place for the component view. Many of the rough edge I encountered in module systems have been in areas where they have tried to support component like usage.
So here's my strawman position: Squeaks needs both a module architecture and a component architecture. They should be designed to be complementary to each other. The module system should stick to development time issues. The component system should stick to runtime issues. Globally, some people should be thinking about how to slice the image into components and modules.
Allen
--On Thursday, August 16, 2001 2:12 PM -0700 Allen Wirfs-Brock Allen_Wirfs-Brock@Instantiations.com wrote:
So what are we trying to accomplish? Dan listed a set of "desiderata" that I can whole-heartedly support. However, I would like to step up one level before plunging into the details.
My understanding is that a common goal is the elimination of the all encompassing, monolithic image. I can't argue with that goal and I don't think I need to reiterate all reasons this is desirable. However, there are a lot of different reasons that different constituencies have for wanting to see this happen. I don't necessarily believe that one solution will make everybody happy.
[snipped interesting discussion of a reasonable distinction]
I wish to speak for the all encompassing, monolithic image. It's been pretty good to me, and good to Smalltalk (and similar systems).
Yes yes yes, shrinking is a pain. Etc. etc. etc. I'm with everyone.
But there's something *golden* about having a 4 file install (vm, image, sources, and changes). Wrestingly with VisualWorks packages and paths tends to make me grumpy. Python is way worse. SWI-Prolog (the other system I've been dorking with) is somewhat better, if just because it's less elaborate (at least, my install is).
Anything that sends me scurrying to the filesystem makes me want to HOLLER! (And not in joy :))
I think this is on the side of the "casual hacker" person :) I'm fairly sure all involved are with me, but I feel inclined to make noise about this. The monolithic image has virtues and I'd rather not *lose* them, if at all possible.
Similarly, there's already at least one modularity/polymorphism dynamic going on in the system. (I.e., classes and messages.) (Oh, and projects and changesets and *morphs* and views and... :)) This one is fairly thoroughly integrated with the system and its tools.
Anyway, you get the idea :) I wouldn't mind knowing whether a specific proposal let me end up with the "happy mess" that is the current image *if I want it*. Similarly, I favor solutions that build on the current mechanisms.
Cheers, Bijan Parsia.
At 08:58 PM 8/16/2001 -0400, Bijan Parsia wrote:
I wish to speak for the all encompassing, monolithic image. It's been pretty good to me, and good to Smalltalk (and similar systems).
Yes yes yes, shrinking is a pain. Etc. etc. etc. I'm with everyone.
But there's something *golden* about having a 4 file install (vm, image, sources, and changes). Wrestingly with VisualWorks packages and paths tends to make me grumpy. Python is way worse. SWI-Prolog (the other system I've been dorking with) is somewhat better, if just because it's less elaborate (at least, my install is). ... Cheers, Bijan Parsia.
Just to clarify, I wasn't talking about physical deployment structure. I also like single file deployment. There nothing as convenient as a self-contained exe.
What I was saying, it the in some situations, tightly bound modular source code structuring is the right solution. In other situations, loosely bound highly encapsulated components (think objects without inheritance) work better. In either case, where the situation warrants it, you should be able to create a self contained deployable application.
Allen
Bijan Parsia wrote:
[snipped interesting discussion of a reasonable distinction]
I wish to speak for the all encompassing, monolithic image. It's been pretty good to me, and good to Smalltalk (and similar systems).
Yes yes yes, shrinking is a pain. Etc. etc. etc. I'm with everyone.
But there's something *golden* about having a 4 file install (vm, image, sources, and changes). Wrestingly with VisualWorks packages and paths tends to make me grumpy. Python is way worse. SWI-Prolog (the other system I've been dorking with) is somewhat better, if just because it's less elaborate (at least, my install is).
Anything that sends me scurrying to the filesystem makes me want to HOLLER! (And not in joy :))
This would be one of the Scenarios that I mentioned wanting to write about, so since you've brought it up...
Scenario: Bewildering Batches Files
( see above plaintive cry - I don't have to write this part ).
THEREFORE Be it resolved that we do NOT wish to force the user to wade through a bewildering sea of files.
( the temptation to talk about solutions is overwhelming, but for me the purpose of the Scenario is to highlight something, illustrating clearly what properties we do or do not wish for our system to have - solutions must be brought up elsewhere. The scenario can then be used to judge the merits of various approaches, and hopefully to act as guides in the development process when the technical issues start getting tough, which I can assure you they will ).
No one is advocating that the image disappear, nor that you should be prevented from working the way you want to today if you wish. We all appreciate flexibilty and performance of the image.
In my ideal world the image would be a cache which is materialized from a well structure set of modules. The choice of when to materialize the image (including just download file from Squeak.org) and save it in my directory should be absolutely yours. In the past images have been packaged as everything from roms/flash (my favorite :)) to .exes, DLLs etc.
Dave Thomas
Bijan Parsia wrote:
--On Thursday, August 16, 2001 2:12 PM -0700 Allen Wirfs-Brock Allen_Wirfs-Brock@Instantiations.com wrote:
So what are we trying to accomplish? Dan listed a set of "desiderata" that I can whole-heartedly support. However, I would like to step up one level before plunging into the details.
My understanding is that a common goal is the elimination of the all encompassing, monolithic image. I can't argue with that goal and I don't think I need to reiterate all reasons this is desirable. However, there are a lot of different reasons that different constituencies have for wanting to see this happen. I don't necessarily believe that one solution will make everybody happy.
[snipped interesting discussion of a reasonable distinction]
I wish to speak for the all encompassing, monolithic image. It's been pretty good to me, and good to Smalltalk (and similar systems).
Yes yes yes, shrinking is a pain. Etc. etc. etc. I'm with everyone.
But there's something *golden* about having a 4 file install (vm, image, sources, and changes). Wrestingly with VisualWorks packages and paths tends to make me grumpy. Python is way worse. SWI-Prolog (the other system I've been dorking with) is somewhat better, if just because it's less elaborate (at least, my install is).
Anything that sends me scurrying to the filesystem makes me want to HOLLER! (And not in joy :))
I think this is on the side of the "casual hacker" person :) I'm fairly sure all involved are with me, but I feel inclined to make noise about this. The monolithic image has virtues and I'd rather not *lose* them, if at all possible.
Similarly, there's already at least one modularity/polymorphism dynamic going on in the system. (I.e., classes and messages.) (Oh, and projects and changesets and *morphs* and views and... :)) This one is fairly thoroughly integrated with the system and its tools.
Anyway, you get the idea :) I wouldn't mind knowing whether a specific proposal let me end up with the "happy mess" that is the current image *if I want it*. Similarly, I favor solutions that build on the current mechanisms.
Cheers, Bijan Parsia.
--On Thursday, August 16, 2001 9:45 PM -0400 Dave dave@bedarra.com wrote:
No one is advocating that the image disappear,
Which seems to suggest that *I* was suggesting that someone is so advocating, but:
I'm fairly sure all involved are with me, but I feel inclined to make noise about this.
[snip]
nor that you should be prevented from working the way you want to today if you wish. We all appreciate flexibilty and performance of the image.
[snip]
But it's easy to get carried away when working on "the other side".
Such appriciation hasn't prevented other Smalltalks from producing what is, for me, kinda yucky solutions. Of course, the lack of my advocacy hasn't prevented other Smalltalks from achieving really cool ones :)
But here is my advocacy, nevertheless :)
And as a side reply to Allen: While I mostly mentioned the virtues of the monolithic image in terms of deployment, is it *limited* to deployment? It seems deployment issues (e.g., building vs. stripping) are one modularity issue among many. Perhaps one trickiness about the image is that it spurts over several potentially separate modularity issues...
...and thus I'm led into the "modularize the modularity" camp :)
Cheers, Bijan Parsia.
On Thursday, August 16, 2001, at 08:58 PM, Bijan Parsia wrote:
I agree that the image is a good thing. The pain that I feel when I complain about the "lack of modularity" is related to the ability to UNDO. If I file in a ChangeSet, and find that it did something obnoxious (like redefine Set>>union: -- an actual example) I would like to UNDO that module/component/experiment/heap-of-code . People who live in a file-based code world -- where the program is link-edited, then run, then stops -- have a notion of modularity that is associated with "assembling" the code.
But if you live in an image (body-surfing in the object sea), the assembly happens rarely. It is the DIS-ASSEMBLY that's hard. The solution after a bad exploration winds up being -- start with a fresh image, and file all my stuff back in. Because the ability to unload modules is so weak.
So I'm arguing that the core functionality -- the definition of a Module (and/or component) -- is a unit of Stuff that I can remove. Currently that means Class or Category . But ChangeSet does NOT meet this definition of Module.
r0ml
--On Thursday, August 16, 2001 2:12 PM -0700 Allen Wirfs-Brock Allen_Wirfs-Brock@Instantiations.com wrote:
So what are we trying to accomplish? Dan listed a set of "desiderata" that I can whole-heartedly support. However, I would like to step up one level before plunging into the details.
My understanding is that a common goal is the elimination of the all encompassing, monolithic image. I can't argue with that goal and I don't think I need to reiterate all reasons this is desirable. However, there are a lot of different reasons that different constituencies have for wanting to see this happen. I don't necessarily believe that one solution will make everybody happy.
[snipped interesting discussion of a reasonable distinction]
I wish to speak for the all encompassing, monolithic image. It's been pretty good to me, and good to Smalltalk (and similar systems).
Yes yes yes, shrinking is a pain. Etc. etc. etc. I'm with everyone.
But there's something *golden* about having a 4 file install (vm, image, sources, and changes). Wrestingly with VisualWorks packages and paths tends to make me grumpy. Python is way worse. SWI-Prolog (the other system I've been dorking with) is somewhat better, if just because it's less elaborate (at least, my install is).
Anything that sends me scurrying to the filesystem makes me want to HOLLER! (And not in joy :))
I think this is on the side of the "casual hacker" person :) I'm fairly sure all involved are with me, but I feel inclined to make noise about this. The monolithic image has virtues and I'd rather not *lose* them, if at all possible.
Similarly, there's already at least one modularity/polymorphism dynamic going on in the system. (I.e., classes and messages.) (Oh, and projects and changesets and *morphs* and views and... :)) This one is fairly thoroughly integrated with the system and its tools.
Anyway, you get the idea :) I wouldn't mind knowing whether a specific proposal let me end up with the "happy mess" that is the current image *if I want it*. Similarly, I favor solutions that build on the current mechanisms.
Cheers, Bijan Parsia.
On Thu, 16 Aug 2001, Robert M. Lefkowitz wrote: [snip]
I agree that the image is a good thing. The pain that I feel when I complain about the "lack of modularity" is related to the ability to UNDO. If I file in a ChangeSet, and find that it did something obnoxious (like redefine Set>>union: -- an actual example) I would like to UNDO that module/component/experiment/heap-of-code . People who live in a
[snip]
I was *just* thinking of this scenario when I read your note.
I completely agree.
The big time version of this that comes to *my* mind is SuperSwiki. I'd love to be able to *Dump* projects as easily as loading them.
This is, of course, the famous "Sandbox" situation. Which, I expect, is very different from most of the others.
A runnable CodeBrowser (aka "Oasis") would be very nice in the near future.
Actually, *just* being able to load/clean out projects would be a wonderful boon. Even if it required certain restrictions (i.e., you couldn't have certain modifications, etc.)...it would be nice to know that a project was "sandbox safe" (i.e., easy to dump) vs. "big ole deep change". Espeically as some superswiki projects are just books...
...of course "just books" is weird, espeically when you're explicating code. Browsers can end up pointing to different things in different systems.
So being able to dump *everything* I need into a project would be good, too. Projects as self-containable mini-images :)
Cheers, Bijan Parsia.
[snip]
So I'm arguing that the core functionality -- the definition of a Module (and/or component) -- is a unit of Stuff that I can remove. Currently that means Class or Category . But ChangeSet does NOT meet this definition of Module.
(Okay, I'm speculating wildly here...)
But ChangeSets _could_, couldn't they, with some modification to the way they work?
I'm imagining that when you file in a ChangeSet (let's say Foo.cs) into your image (Bar.image), your image is writing out information to it's Bar.changes file that reflect the things Foo.cs is doing to your image. Then, to back out, you could use a currently fictional "Undo" feature which would allow you to back up steps as recorded in Bar.changes
This may necessitate modifications to userland tools to deal with .changes file, and perhaps some modification to the information and/or format of information stored in .changes files, but I don't think it would be too drastic and I don't think it would have much with the implementation of ChangeSets themselves.
r0ml
[snip]
----- Original Message ----- From: Dan Moniz dnm@pobox.com
I'm imagining that when you file in a ChangeSet (let's say Foo.cs) into your image (Bar.image), your image is writing out information to it's Bar.changes file that reflect the things Foo.cs is doing to your image. Then, to back out, you could use a currently fictional "Undo" feature which would allow you to back up steps as recorded in Bar.changes
There are a few things that would need to be done when unloading a module. For one, you would like a rapid way of gc'ing or otherwise mutating any instances defined by that module. Then, you will want a rapid way of removing the definitions found in that module. ChangeSets are no better, and no worse, at doing this than any other scheme. There is no need to iterate over a change log- the ChangeSet already has an accounting of what it has modified. So would an OasisModule, or a Collage Layer.
However, while associating a ChangeSet with either a Collage Layer or an OasisModule seems reasonable to me, trying to warp ChangeSet into becoming more like these things does not seem reasonable.
This may necessitate modifications to userland tools to deal with .changes file, and perhaps some modification to the information and/or format of information stored in .changes files, but I don't think it would be too drastic and I don't think it would have much with the implementation of ChangeSets themselves.
What's wrong with the idea of implementing the roles of a Components and Modules? In Oasis, a Module is a component that keeps track of Shared Globals, Undeclared variables, Pools, and a few other things. ChangeSets don't do this.
The role of a changeset is to keep track of changes... the role of an OasisModule is to act as a place in which those changes take effect. These are separate roles, so I would not reccomend mixing them. ChangeSets are just fine the way they are, at least in the sense of the scoping of their responsibilities to tracking changes, and no more.
Using ChangeSets as a means of identifying chunks of code to turn into a package is another issue, as far as I know there is nothing wrong with that.
- les
1. The mechanism for unloading is indeed a good deal more complex than loading and was one of the throny problems OTI had to address in building embedded Smalltalk and Java.
Perhaps the issue of unloading is something to be deferred until after the basic module and component mechanisms are defined and tested?
2. To my knowledge the only existing system that supports current execution of multiple versions classes/methods is MS.NET with their assemblies. This itself gets problematic however if you want to run shared assemblies. I posted the references earlier for those interested in the problem.
Regards, Dave
Les Tyrrell wrote:
----- Original Message ----- From: Dan Moniz dnm@pobox.com
I'm imagining that when you file in a ChangeSet (let's say Foo.cs) into your image (Bar.image), your image is writing out information to it's Bar.changes file that reflect the things Foo.cs is doing to your image. Then, to back out, you could use a currently fictional "Undo" feature which would allow you to back up steps as recorded in Bar.changes
There are a few things that would need to be done when unloading a module. For one, you would like a rapid way of gc'ing or otherwise mutating any instances defined by that module. Then, you will want a rapid way of removing the definitions found in that module. ChangeSets are no better, and no worse, at doing this than any other scheme. There is no need to iterate over a change log- the ChangeSet already has an accounting of what it has modified. So would an OasisModule, or a Collage Layer.
However, while associating a ChangeSet with either a Collage Layer or an OasisModule seems reasonable to me, trying to warp ChangeSet into becoming more like these things does not seem reasonable.
This may necessitate modifications to userland tools to deal with .changes file, and perhaps some modification to the information and/or format of information stored in .changes files, but I don't think it would be too drastic and I don't think it would have much with the implementation of ChangeSets themselves.
What's wrong with the idea of implementing the roles of a Components and Modules? In Oasis, a Module is a component that keeps track of Shared Globals, Undeclared variables, Pools, and a few other things. ChangeSets don't do this.
The role of a changeset is to keep track of changes... the role of an OasisModule is to act as a place in which those changes take effect. These are separate roles, so I would not reccomend mixing them. ChangeSets are just fine the way they are, at least in the sense of the scoping of their responsibilities to tracking changes, and no more.
Using ChangeSets as a means of identifying chunks of code to turn into a package is another issue, as far as I know there is nothing wrong with that.
- les
Dave dave@bedarra.com is widely believed to have written:
- The mechanism for unloading is indeed a good deal more complex than
loading and was one of the throny problems OTI had to address in building embedded Smalltalk and Java.
Perhaps the issue of unloading is something to be deferred until after the basic module and component mechanisms are defined and tested?
I was just thinking something similar; that maybe _un_loading isn't really all that interesting. Here's why - the typical application distribution job is to assemble one's code along with enough support code to run properly. As long as we have a system that we can put together easily, and that has tools/capabilities to allows us to be sure that any particular app is 'clean', then we ought to be safe in assuming that assembling a 'user' system from the appropriate base components plus our own app code will produce a workable program. In most cases I rather suspect that this would be equivalent to something like base + gui + programming tools + myApp works therefore base + gui + myApp will work I'm not at all sure I've written this terribly well, but you can probably see what I'm getting at.
tim - no longer on fire :-)
Simon Michael simon@joyful.com is widely believed to have written:
Tim Rowledge tim@sumeru.stanford.edu writes:
tim - no longer on fire :-)
Phew! :)
Indeed - although I've just been told the fire has now reached 12,000 acres, destroyed eight houses and threatens many more. Not good. tim
Actually, what's really hard is what Team/V called "migration". That is taking a module or set of modules and atomically replacing them with different versions of the same modules. Envy does something similar. In VisualAge Java they call it "replace".
While it's hard, I see this a pretty essential operation in a collaborative development environment. It's one of those things like working with incomplete or inconsistent modules that isn't strictly necessary for deployment but is integral to incremental development.
Alllen
At 09:06 PM 8/20/2001 -0400, Dave wrote:
- The mechanism for unloading is indeed a good deal more complex than
loading and was one of the throny problems OTI had to address in building embedded Smalltalk and Java.
Perhaps the issue of unloading is something to be deferred until after the basic module and component mechanisms are defined and tested?
- To my knowledge the only existing system that supports current
execution of multiple versions classes/methods is MS.NET with their assemblies. This itself gets problematic however if you want to run shared assemblies. I posted the references earlier for those interested in the problem.
Regards, Dave
Les Tyrrell wrote:
----- Original Message ----- From: Dan Moniz dnm@pobox.com
I'm imagining that when you file in a ChangeSet (let's say Foo.cs) into your image (Bar.image), your image is writing out information to it's Bar.changes file that reflect the things Foo.cs is doing to your image. Then, to back out, you could use a currently fictional "Undo" feature which would allow you to back up steps as recorded in Bar.changes
There are a few things that would need to be done when unloading a
module. For
one, you would like a rapid way of gc'ing or otherwise mutating any
instances
defined by that module. Then, you will want a rapid way of removing the definitions found in that module. ChangeSets are no better, and no
worse, at
doing this than any other scheme. There is no need to iterate over a
change
log- the ChangeSet already has an accounting of what it has
modified. So would
an OasisModule, or a Collage Layer.
However, while associating a ChangeSet with either a Collage Layer or an OasisModule seems reasonable to me, trying to warp ChangeSet into
becoming more
like these things does not seem reasonable.
This may necessitate modifications to userland tools to deal with .changes file, and perhaps some modification to the information and/or format of information stored in .changes files, but I don't think it would be too drastic and I don't think it would have much with the implementation of ChangeSets themselves.
What's wrong with the idea of implementing the roles of a Components and Modules? In Oasis, a Module is a component that keeps track of Shared
Globals,
Undeclared variables, Pools, and a few other things. ChangeSets don't
do this.
The role of a changeset is to keep track of changes... the role of an OasisModule is to act as a place in which those changes take
effect. These are
separate roles, so I would not reccomend mixing them. ChangeSets are
just fine
the way they are, at least in the sense of the scoping of their
responsibilities
to tracking changes, and no more.
Using ChangeSets as a means of identifying chunks of code to turn into
a package
is another issue, as far as I know there is nothing wrong with that.
- les
I agree this too is "hard" and agree that for any useful image based system there needs to support for a limbo state.
Allen Wirfs-Brock wrote:
Actually, what's really hard is what Team/V called "migration". That is taking a module or set of modules and atomically replacing them with different versions of the same modules. Envy does something similar. In VisualAge Java they call it "replace".
While it's hard, I see this a pretty essential operation in a collaborative development environment. It's one of those things like working with incomplete or inconsistent modules that isn't strictly necessary for deployment but is integral to incremental development.
Alllen
At 09:06 PM 8/20/2001 -0400, Dave wrote:
- The mechanism for unloading is indeed a good deal more complex than
loading and was one of the throny problems OTI had to address in building embedded Smalltalk and Java.
Perhaps the issue of unloading is something to be deferred until after the basic module and component mechanisms are defined and tested?
- To my knowledge the only existing system that supports current
execution of multiple versions classes/methods is MS.NET with their assemblies. This itself gets problematic however if you want to run shared assemblies. I posted the references earlier for those interested in the problem.
Regards, Dave
Les Tyrrell wrote:
----- Original Message ----- From: Dan Moniz dnm@pobox.com
I'm imagining that when you file in a ChangeSet (let's say Foo.cs) into your image (Bar.image), your image is writing out information to it's Bar.changes file that reflect the things Foo.cs is doing to your image. Then, to back out, you could use a currently fictional "Undo" feature which would allow you to back up steps as recorded in Bar.changes
There are a few things that would need to be done when unloading a
module. For
one, you would like a rapid way of gc'ing or otherwise mutating any
instances
defined by that module. Then, you will want a rapid way of removing the definitions found in that module. ChangeSets are no better, and no
worse, at
doing this than any other scheme. There is no need to iterate over a
change
log- the ChangeSet already has an accounting of what it has
modified. So would
an OasisModule, or a Collage Layer.
However, while associating a ChangeSet with either a Collage Layer or an OasisModule seems reasonable to me, trying to warp ChangeSet into
becoming more
like these things does not seem reasonable.
This may necessitate modifications to userland tools to deal with .changes file, and perhaps some modification to the information and/or format of information stored in .changes files, but I don't think it would be too drastic and I don't think it would have much with the implementation of ChangeSets themselves.
What's wrong with the idea of implementing the roles of a Components and Modules? In Oasis, a Module is a component that keeps track of Shared
Globals,
Undeclared variables, Pools, and a few other things. ChangeSets don't
do this.
The role of a changeset is to keep track of changes... the role of an OasisModule is to act as a place in which those changes take
effect. These are
separate roles, so I would not reccomend mixing them. ChangeSets are
just fine
the way they are, at least in the sense of the scoping of their
responsibilities
to tracking changes, and no more.
Using ChangeSets as a means of identifying chunks of code to turn into
a package
is another issue, as far as I know there is nothing wrong with that.
- les
I compare some of the this to the current situation with global references. It is easy to get the image into an inconsistent state and it even keeps running. BUT there are tools that make it easy to find and current this. Adding modules together with imports and exports results in more ways the image can be inconsistent but rather than try and prevent them the focus should be on tools to find and correct them.
Joerg -----Original Message----- From: squeak-dev-admin@lists.squeakfoundation.org [mailto:squeak-dev-admin@lists.squeakfoundation.org]On Behalf Of Dave Sent: August 21, 2001 4:55 AM To: Allen Wirfs-Brock Cc: squeak-dev@lists.squeakfoundation.org; modsqueak@bluefish.se Subject: Re: [Modules] Unloading and Conflicting Versions
I agree this too is "hard" and agree that for any useful image based system there needs to support for a limbo state.
Allen Wirfs-Brock wrote:
Actually, what's really hard is what Team/V called "migration". That is taking a module or set of modules and atomically replacing them with different versions of the same modules. Envy does something similar. In VisualAge Java they call it "replace".
While it's hard, I see this a pretty essential operation in a collaborative development environment. It's one of those things like working with incomplete or inconsistent modules that isn't strictly necessary for deployment but is integral to incremental development.
Alllen
At 09:06 PM 8/20/2001 -0400, Dave wrote:
- The mechanism for unloading is indeed a good deal more complex than
loading and was one of the throny problems OTI had to address in building embedded Smalltalk and Java.
Perhaps the issue of unloading is something to be deferred until after the basic module and component mechanisms are defined and tested?
- To my knowledge the only existing system that supports current
execution of multiple versions classes/methods is MS.NET with their assemblies. This itself gets problematic however if you want to run shared assemblies. I posted the references earlier for those interested in the problem.
Regards, Dave
Les Tyrrell wrote:
----- Original Message ----- From: Dan Moniz dnm@pobox.com
I'm imagining that when you file in a ChangeSet (let's say Foo.cs) into your image (Bar.image), your image is writing out information to it's Bar.changes file that reflect the things Foo.cs is doing to your image. Then, to back out, you could use a currently fictional "Undo" feature which would allow you to back up steps as recorded in Bar.changes
There are a few things that would need to be done when unloading a
module. For
one, you would like a rapid way of gc'ing or otherwise mutating any
instances
defined by that module. Then, you will want a rapid way of removing the definitions found in that module. ChangeSets are no better, and no
worse, at
doing this than any other scheme. There is no need to iterate over a
change
log- the ChangeSet already has an accounting of what it has
modified. So would
an OasisModule, or a Collage Layer.
However, while associating a ChangeSet with either a Collage Layer or an OasisModule seems reasonable to me, trying to warp ChangeSet into
becoming more
like these things does not seem reasonable.
This may necessitate modifications to userland tools to deal with .changes file, and perhaps some modification to the information and/or format of information stored in .changes files, but I don't think it would be too drastic and I don't think it would have much with the implementation of ChangeSets themselves.
What's wrong with the idea of implementing the roles of a Components and Modules? In Oasis, a Module is a component that keeps track of Shared
Globals,
Undeclared variables, Pools, and a few other things. ChangeSets don't
do this.
The role of a changeset is to keep track of changes... the role of an OasisModule is to act as a place in which those changes take
effect. These are
separate roles, so I would not reccomend mixing them. ChangeSets are
just fine
the way they are, at least in the sense of the scoping of their
responsibilities
to tracking changes, and no more.
Using ChangeSets as a means of identifying chunks of code to turn into
a package
is another issue, as far as I know there is nothing wrong with that.
- les
So I'm arguing that the core functionality -- the definition of a Module (and/or component) -- is a unit of Stuff that I can remove. Currently that means Class or Category . But ChangeSet does NOT meet this definition of Module.
Dan Moniz dnm@pobox.com responded...
(Okay, I'm speculating wildly here...)
But ChangeSets _could_, couldn't they, with some modification to the way they work?
I'm imagining that when you file in a ChangeSet (let's say Foo.cs) into your image (Bar.image), your image is writing out information to it's Bar.changes file that reflect the things Foo.cs is doing to your image. Then, to back out, you could use a currently fictional "Undo" feature which would allow you to back up steps as recorded in Bar.changes
This may necessitate modifications to userland tools to deal with .changes file, and perhaps some modification to the information and/or format of information stored in .changes files, but I don't think it would be too drastic and I don't think it would have much with the implementation of ChangeSets themselves.
Actually, there is a great deal of support for removal already in changeSets. In fact, in the few milliseconds it takes to enter or leave an isolated project, all the changes get asserted and reverted. This is done by saving a copy of the method dictionaries and organizations before the changes. This is expensive in space, but incredibly effective for reverting. I can imagine a number of alternative solutions such as...
using the file pointers to prior versions which would require some compilation before the revert.
The (incomplete) uninstall command to ChangeSets does this.
saving the revert state in an image segment on file. You could load this quickly to do a revert.
Basically, I'm agreeing -- yes, ChangeSets could be made truly removable, especially in the context of a module architecture.
- Dan
On Monday 20 August 2001 05:33 pm, Dan Ingalls wrote:
Actually, there is a great deal of support for removal already in changeSets. In fact, in the few milliseconds it takes to enter or leave an isolated project, all the changes get asserted and reverted. This is done by saving a copy of the method dictionaries and organizations before the changes.
So what happens to instances of classes that were defined in the isolated project if they don't get GC'd?
I wonder whether (at least in terms of what's required to assemble various images) it's even important to support clean unloading. I for one wouldn't mind a scheme where I could build up a custom image with my desired modules (as long as that worked reliably).
But then maybe I'm too used to cross-compilation, etc. from embedded systems.
Bijan Parsia wrote:
I wish to speak for the all encompassing, monolithic image. It's been pretty good to me, and good to Smalltalk (and similar systems).
Yes yes yes, shrinking is a pain. Etc. etc. etc. I'm with everyone.
In many respect I agree. HOwever, a good packaging/module/froobler system should allow the best of both worlds - chunks of stuff (love those technical terms baby!) outside in some nice safe repositoy cared for by the likes of Cees or bound into your very own single file jealously guarded by the mighty security of WinMe. The chief difference from the current situation ought to be that shrinking does not involve illicit firking around but simply letting go of unneeded attachments on a positively zenlike manner. A modular system would provide for lighter linkages between parts.
tim
Bijan Parsia bparsia@email.unc.edu wrote...
I wish to speak for the all encompassing, monolithic image. It's been pretty good to me, and good to Smalltalk (and similar systems).
Yes yes yes, shrinking is a pain. Etc. etc. etc. I'm with everyone.
But there's something *golden* about having a 4 file install (vm, image, sources, and changes). Wrestingly with VisualWorks packages and paths tends to make me grumpy. Python is way worse. SWI-Prolog (the other system I've been dorking with) is somewhat better, if just because it's less elaborate (at least, my install is).
Anything that sends me scurrying to the filesystem makes me want to HOLLER! (And not in joy :))
Bijan -
I'm with you on this. Regardless of how well we succeed with breaking the image down into modules, I believe we will always want to have both a decomposed and a full-blown image as options for the release. Folks with big machines who don't care about packages will choose Squeak as it has been for years. Folks who are doing a headless server or mini PDA app will want the fully factored version, and someone who wants to rework the file system in collaboration with 2 other people in other countries will want this plus a link to some CVS-style database.
But as long as I'm around, we'll have an all-in-one option as part of our release process.
- Dan
on 8/16/01 5:58 PM, Bijan Parsia at bparsia@email.unc.edu wrote:
I wish to speak for the all encompassing, monolithic image. It's been pretty good to me, and good to Smalltalk (and similar systems).
Yes yes yes, shrinking is a pain. Etc. etc. etc. I'm with everyone.
But there's something *golden* about having a 4 file install (vm, image, sources, and changes). Wrestingly with VisualWorks packages and paths tends to make me grumpy. Python is way worse. SWI-Prolog (the other system I've been dorking with) is somewhat better, if just because it's less elaborate (at least, my install is).
What's wrong with having a simple database of components/modules that you can selectively install via a simple dialog box during the initial install process?
Default Install (all) () Minimal Install () Custom Install ()
That way, you get the advantage of a monolithic image with the flexibility of the component/module architecture.
Bijan Parsia wrote:
But there's something *golden* about having a 4 file install (vm, image, sources, and changes). Wrestingly with VisualWorks packages and paths
tends
to make me grumpy. Python is way worse. SWI-Prolog (the other system I've been dorking with) is somewhat better, if just because it's less elaborate (at least, my install is).
Anything that sends me scurrying to the filesystem makes me want to
HOLLER!
(And not in joy :))
I think this is on the side of the "casual hacker" person :) I'm fairly sure all involved are with me, but I feel inclined to make noise about this. The monolithic image has virtues and I'd rather not *lose* them, if at all possible.
Here, here. If 4 file install is that good, I guess 3 file install is even better. I heard that the image can be embedded in the VM.
Is it possible to merge 'Source' and 'Changes' ? Why do we need two separate textfiles (?) for the REAL source ? This will give us a 2 file install. That's twice as good ;-)
I think this is on the side of the "casual hacker" person :) I'm fairly sure all involved are with me, but I feel inclined to make noise about this. The monolithic image has virtues and I'd rather not *lose* them, if at all possible.
If it is possible to embed a binary file (image) in another binary (VM) then it's no brainer (?) to embed the merged real source into that VM with embedded image (I think REBOL has about 2Meg. of source text in the Rebol.exe, VM + compressed text source about 500K). Voila, THE ULTIMATE really truly monolithic Squeak ;-)
Any taker ?
OTOH, I also like the idea of 1 file install Squeak. This is truly 1 file, not a zip, tar or self extract archive of any sort. It is truly one Windows executable, Squeak.exe, the 'MobVM'.
I have a couple of questions to the list (especially the creator of the [Modules] tag) :
1/- Is it appropriate to discuss about the 'MobVM' now or should I wait.
2/- Is it approriate to do so within the [Modules] tag or should it be elsewhere ?
Cheers,
PhiHo.
----- Original Message ----- From: "Bijan Parsia" bparsia@email.unc.edu To: squeak-dev@lists.squeakfoundation.org Cc: modsqueak@bluefish.se Sent: Thursday, August 16, 2001 8:58 PM Subject: Re: [Modules] Components or Modules??
--On Thursday, August 16, 2001 2:12 PM -0700 Allen Wirfs-Brock Allen_Wirfs-Brock@Instantiations.com wrote:
So what are we trying to accomplish? Dan listed a set of "desiderata" that I can whole-heartedly support. However, I would like to step up
one
level before plunging into the details.
My understanding is that a common goal is the elimination of the all encompassing, monolithic image. I can't argue with that goal and I don't think I need to reiterate all reasons this is desirable. However, there are a lot of different reasons that different constituencies have for wanting to see this happen. I don't necessarily believe that one solution will make everybody happy.
[snipped interesting discussion of a reasonable distinction]
I wish to speak for the all encompassing, monolithic image. It's been pretty good to me, and good to Smalltalk (and similar systems).
Yes yes yes, shrinking is a pain. Etc. etc. etc. I'm with everyone.
But there's something *golden* about having a 4 file install (vm, image, sources, and changes). Wrestingly with VisualWorks packages and paths
tends
to make me grumpy. Python is way worse. SWI-Prolog (the other system I've been dorking with) is somewhat better, if just because it's less elaborate (at least, my install is).
Anything that sends me scurrying to the filesystem makes me want to
HOLLER!
(And not in joy :))
I think this is on the side of the "casual hacker" person :) I'm fairly sure all involved are with me, but I feel inclined to make noise about this. The monolithic image has virtues and I'd rather not *lose* them, if at all possible.
Similarly, there's already at least one modularity/polymorphism dynamic going on in the system. (I.e., classes and messages.) (Oh, and projects
and
changesets and *morphs* and views and... :)) This one is fairly thoroughly integrated with the system and its tools.
Anyway, you get the idea :) I wouldn't mind knowing whether a specific proposal let me end up with the "happy mess" that is the current image *if I want it*. Similarly, I favor solutions that build on the current mechanisms.
Cheers, Bijan Parsia.
On Sun, 19 Aug 2001, PhiHo Hoang wrote:
Bijan Parsia wrote:
But there's something *golden* about having a 4 file install (vm, image, sources, and changes). Wrestingly with VisualWorks packages and paths
[snip]
Anything that sends me scurrying to the filesystem makes me want to
HOLLER!
(And not in joy :))
[snipped stuff not directly related to this point]
Here, here. If 4 file install is that good, I guess 3 file install iseven better.
Not clearly.
I heard that the image can be embedded in the VM.
Yes. And in spite of your sarcasm/reductio attempt (:)), it's been *seriously argued* that the 4 file install is too much.
[snip]
I think this is on the side of the "casual hacker" person :) I'm fairly sure all involved are with me, but I feel inclined to make noise about this. The monolithic image has virtues and I'd rather not *lose* them, if at all possible.
If it is possible to embed a binary file (image) in another binary (VM)then it's no brainer (?) to embed the merged real source into that VM with embedded image (I think REBOL has about 2Meg. of source text in the Rebol.exe, VM + compressed text source about 500K). Voila, THE ULTIMATE really truly monolithic Squeak ;-)
Any taker ?
Not me, but there have been many who've so argued :)
[snip]
One thing about the 4 files is that they *are* reasonable modular...I tend to run multiple image/changes with 1 vm/sources, and several vms with 1 sources. So image+changes...that's definitely interesting, but not critical imho :)
I *would* like smarter decompiling (i.e., storing variable names & comments in the image), but that's a different issue.
Cheers, Bijan Parsia.
Hi,
I was filing in the new updates when I got a walkback MethodDictionary>>grow. The method is shown below. The problem occured because in the first line, the species of MethodDictionary is Set. Then when "newSelf at: key put: (array at: i)" is executed, it fails because key is not an integer. I solved the problem enough to get through the updates by creating the method MethodDictionary>>species that returns MethodDictionary. But I'm not sure if this is the correct solution.
grow | newSelf key | newSelf _ self species new: self basicSize. "This will double the size" 1 to: self basicSize do: [:i | key _ self basicAt: i. key == nil ifFalse: [newSelf at: key put: (array at: i)]]. self become: newSelf! !
Bijan Parsia wrote:
I heard that the image can be embedded in the VM.
Yes. And in spite of your sarcasm/reductio attempt (:)), it's been *seriously argued* that the 4 file install is too much.
I hope 'MobSqueak' will have your support to make it a working truely 1 file install Squeak.
If the core bootstrap networking stuff in the micro kernel does not work out as expected, I think I know where to turn for help ;-)
Cheers,
PhiHo
----- Original Message ----- From: "Bijan Parsia" bparsia@email.unc.edu To: squeak-dev@lists.squeakfoundation.org Sent: Monday, August 20, 2001 8:41 AM Subject: Re: [Modules] Introducing MobVM (was Components or Modules??)
On Sun, 19 Aug 2001, PhiHo Hoang wrote:
Bijan Parsia wrote:
But there's something *golden* about having a 4 file install (vm,
image,
sources, and changes). Wrestingly with VisualWorks packages and paths
[snip]
Anything that sends me scurrying to the filesystem makes me want to
HOLLER!
(And not in joy :))
[snipped stuff not directly related to this point]
Here, here. If 4 file install is that good, I guess 3 file install
is
even better.
Not clearly.
I heard that the image can be embedded in the VM.
Yes. And in spite of your sarcasm/reductio attempt (:)), it's been *seriously argued* that the 4 file install is too much.
[snip]
I think this is on the side of the "casual hacker" person :) I'm
fairly
sure all involved are with me, but I feel inclined to make noise about this. The monolithic image has virtues and I'd rather not *lose* them,
if
at all possible.
If it is possible to embed a binary file (image) in another binary
(VM)
then it's no brainer (?) to embed the merged real source into that VM
with
embedded image (I think REBOL has about 2Meg. of source text in the Rebol.exe, VM + compressed text source about 500K). Voila, THE ULTIMATE really truly monolithic Squeak ;-)
Any taker ?Not me, but there have been many who've so argued :)
[snip]
One thing about the 4 files is that they *are* reasonable modular...I tend to run multiple image/changes with 1 vm/sources, and several vms with 1 sources. So image+changes...that's definitely interesting, but not critical imho :)
I *would* like smarter decompiling (i.e., storing variable names & comments in the image), but that's a different issue.
Cheers, Bijan Parsia.
"PhiHo Hoang" phiho.hoang@home.com wrote...
OTOH, I also like the idea of 1 file install Squeak. This is truly 1 file, not a zip, tar or self extract archive of any sort. It is truly one Windows executable, Squeak.exe, the 'MobVM'.
I have a couple of questions to the list (especially the creator of the [Modules] tag) :
1/- Is it appropriate to discuss about the 'MobVM' now or should I wait.
Why wait?
2/- Is it approriate to do so within the [Modules] tag or should it be elsewhere ?
I think it should be elsewhere. An appropriate tag would probably be [Squeak Installer].
We had some interesting discussion about this almost two years ago, and I think a fair amount of it wound up on the Squeak swiki as a "SqC project". Little was done thereafter, except that the ability to compress and decompress multiple files has been added to squeak since then.
From the client's point of view, it's pretty simple already to get Squeak as a self extracting archive. I do this on a Mac, and it comes as just one file, is well compressed for the download, and expands into a convenient folder.
Most of the motivation for a Squeak installer was to make life easier for the folks who put out a release. If the clients already have a squeak VM, then the release could be packaged as a single file, that would be the *same for every platform*. This means that maintaining any given version amounts to only a set of VMs for each platform (which we require now), and one other file that is a Squeak-compressed and managed combination image and compressed changes etc.
Hope this helps
- Dan
"Dan Ingalls" Dan@SqueakLand.org wrote:
1/- Is it appropriate to discuss about the 'MobVM' now or should I
wait.
Why wait?
Because I think there are lot of things in the VM depends on the way the whole Squeak is architect-'ed' and not sure if it is advantageous to have discussion on the VM in sync with discussion of the architecture of Squeak.
2/- Is it approriate to do so within the [Modules] tag or should it
be
elsewhere ?
I think it should be elsewhere. An appropriate tag would probably be
[Squeak Installer].
That's it !!! It's the 'Squeak Installer'. Thanks for the suggestion, Dan.
The micro kernel (with the help of a few bootstrap plugins) dynamically installs other plugins on demand just enough to support a preconfigured image built from a set of modules. All in memory.
One thing that bothers me though. People used to think of Squeak.exe (for Windows) as the Squeak VM. Now it's promoted to Squeak Installer and then the Squeak VM now becomes invisible (dynamically assembled).
Another thought, maybe, the micro kernel will retain the identity as a VM. The Squeak Installer will be actually a truly micro kernel image requiring just the micro kernel, PlugManPlugin (Plugin Manager), ModManPlugin (Module Manager), PlatformPlugin (platform dependent utility routines), ZipPlugin and a bootstrap interpreter with just enough of bytecodes and named primitives to support the Squeak Installer to install the rest (including an approriate interpreter for the target image).
Maybe, the PlatformPlugin is not necessary at all. If the SqueakFFIPrims is used instead, then all platform functionalities should be available from within the Squeak Installer image.
Cheers,
PhiHo.
----- Original Message ----- From: "Dan Ingalls" Dan@SqueakLand.org To: squeak-dev@lists.squeakfoundation.org Sent: Monday, August 20, 2001 3:44 PM Subject: Re: [Modules] Introducing MobVM (was Components or Modules??)
"PhiHo Hoang" phiho.hoang@home.com wrote...
OTOH, I also like the idea of 1 file install Squeak. This is truly 1 file, not a zip, tar or self extract archive of any sort. It is truly one Windows executable, Squeak.exe, the 'MobVM'.
I have a couple of questions to the list (especially the creator of
the
[Modules] tag) :
1/- Is it appropriate to discuss about the 'MobVM' now or should I
wait.
Why wait?
2/- Is it approriate to do so within the [Modules] tag or should it
be
elsewhere ?
I think it should be elsewhere. An appropriate tag would probably be
[Squeak Installer].
We had some interesting discussion about this almost two years ago, and I
think a fair amount of it wound up on the Squeak swiki as a "SqC project". Little was done thereafter, except that the ability to compress and decompress multiple files has been added to squeak since then.
From the client's point of view, it's pretty simple already to get Squeak
as a self extracting archive. I do this on a Mac, and it comes as just one file, is well compressed for the download, and expands into a convenient folder.
Most of the motivation for a Squeak installer was to make life easier for
the folks who put out a release. If the clients already have a squeak VM, then the release could be packaged as a single file, that would be the *same for every platform*. This means that maintaining any given version amounts to only a set of VMs for each platform (which we require now), and one other file that is a Squeak-compressed and managed combination image and compressed changes etc.
Hope this helps
- Dan
--
Hi List,
There is some progress. Not much though, merely 11KB. The codes were mainly stolen from the existing InterpreterPlugin and PlatformPlugin.
What it does is looking around for a Squeak image, if not found it will bring up a file open dialog box so that the user can pick up an image somewhere on the local file system. It then loads the image, read some saved information, then initialised the ObjectMemory, CompilerHooks...prepare the scene for interpret().
This new plugin, for now, is named 'ModManPlugin' (Module Manager). Of course this is only a place holder for the true Module Manager. I hope at some stage, with help from the list, this plugin will be able to look for, instead of a pre-built image, the modules (Smalltalk source files) whether on the local file system or on the internet (via http/ftp ), pass them to the compiler, get back the result from the compiler, building up an image (?) in memory......prepare the scene for interpret().
Please find attached the file 'ObjectMemoryStartup.zip' to see what was stolen from where. Beware, it's pretty dirty with '#if 1' and '#if 0' and other commented out things.
If you are curious to see if it really works or you want to help beating it to death (in the process of bug hunting ;-), please email me. I will send you Squeak.exe and 33 plugins, excluding SqueakFFIPrims.dll and Mpeg3Plugin.dll, you should get these from the official distribution (they have been external all time).
If you have any external plugin of your own, you are encouraged to try them with MobVM. I would like to hear your comments. it should be fully compatible with Squeak-D3D (625,644 bytes dated May-17-2001).
The file MobVM.zip is 328KB. I am sorry, it's for Win32 only (tested on Win2K Pro, somewhat tested on Win2KAS and WinME. It should just work on YourWin ;-)
Cheers,
PhiHo
P.S: Please ensure that you do not have the official VM and MobVM in the same directory. This should do no harm to MobVM but the official VM will just display a black screen because it found the external plugins (belonging to MobVM, requiring a higher minor version for the interpreter) and cannot use it (because of the version). The official VM should fall back to the internal plugins and proceed as if there were no (useless) external plugins. Andreas, please consider this as a bug report ;-)
Hi List,
I wrote:
This new plugin, for now, is named 'ModManPlugin' (Module Manager). Ofcourse this is only a place holder for the true Module Manager. I hope at some stage, with help from the list, this plugin will be able to look for, instead of a pre-built image, the modules (Smalltalk source files) whether on the local file system or on the internet (via http/ftp ), pass them to the compiler, get back the result from the compiler, building up an image (?) in memory......prepare the scene for interpret().
The missing piece in this picture is the compiler. Browsing through 'interp.c' I couldn't find much code for the compiler in there. The 'System-Compiler' category, on the other hand has a lot.
Is it true that the Squeak compiler is implemented in Squeak only, there is no plugin ? Is it because there is no need for speed in compiling ? Or is it impossible to implement the compiler outside of the image ?
I need a compiler outside of the Squeak image for bootstrapping purpose. Is there a mechanism to translate the 'System-Compiler' category into a plugin ? If not, how else can I get a standalone Squeak compiler from the available Squeak codes ?
Thanks for your help.
Cheers,
PhiHo
On Saturday, August 25, 2001, at 02:34 am, PhiHo Hoang wrote: [...] Is it true that the Squeak compiler is implemented in Squeak only, there
is no plugin ? Is it because there is no need for speed in compiling ? Or is it impossible to implement the compiler outside of the image ?
I haven't been up on Squeak lately, but unless I'm really mistaken, the compiler within Squeak is the only one out there.
I need a compiler outside of the Squeak image for bootstrappingpurpose. Is there a mechanism to translate the 'System-Compiler' category into a plugin ? If not, how else can I get a standalone Squeak compiler from the available Squeak codes ?
My fourth year project for my B.C.S. was bootstrapping a Smalltalk system. I wrote a *simple* Smalltalk compiler in C and used it to directly grow an image from nothing more than a text file with a parenthesized hierarchical list of classes (with instance variable names), and all method source code for all the classes.
Seriously, writing the compiler in C shouldn't be hard at all (a day to a week, depending on experience). The parser can be simple recursive descent, and your tokens don't even have to be allocated as Squeak objects. You don't have to produce "optimal" code, using all the latest and greatest bytecodes. My Smalltalk-in-C compiler didn't even bother optimizing conditionals (or maybe I added that to it later). That kind of thing can be dealt with as a final "linking" stage, after all your modules have been compiled.
It's been a while since I wrote that code ('88-'89), but I recall an issue was how to survive a garbage collection during initial image construction (the C code had to point into the image a lot while it was being constructed). If I had it to do over today, I would simply use smart pointers that add themselves to a global bi-directional ring in their constructor, and remove themselves in their destructor (I use this technique in my Avail primitives). I think that's not a good idea with Squeak, due to unavailability of C++ compilers on some platforms. In my old Smalltalk system I simply banned garbage collection during image construction -- it wasn't a serious problem, even in 1MB (Atari 1040ST).
Here's an idea: Extend Slang to be able to translate the Squeak compiler. Most of it is fairly simple code, and the stuff that's more complex can be made simple. Even if everything won't translate, you can always fake the rest with a few C functions. Don't worry about memory leaks initially. Eventually you can use your own malloc substitute that allocates a "space" for the temporary structures, and then bulk deallocates the whole space after each method compilation. The advantage of translating the existing compiler is that as bytecodes change, your code will continue to work.
Here's an alternative: Use a Squeak image to grow your fetal Squeak image. The compiler produces a CompiledMethod which you can then trace through and copy into the new image. SystemTracer might help you with that (and you might help SystemTracer with that, too). The Smalltalk compiler probably runs within an order of magnitude as fast as a compiler written hastily in C, and that should be fast enough.
You don't need to simulate image memory in an Array or anything so severe. Just keep a few roots pointing to the key data structures of your fetal "image", and be prepared to do a little extra work separating your data from the running image when producing an image file. If you want to do this live, create all your data structures (sharing immutables like Symbols if you want), then invoke a new magic primitive whose purpose is to do a big "context switch" of all the key Smalltalk roots (Processor, etc). Two images worth of data can live in one actual image without much trouble. Hm. On second thought, don't share Symbols or you'll run into method lookup problems. Hm. Even SmallIntegers will be a problem (and you can't really build a class like that). You'll have to switch method dictionaries for all the "known to the VM" classes atomically inside the context switch primitive.
-Mark
Hi Mark,
Thanks for your reply. I am sorry that I could not get back to you sooner. I was dragged to a camp, no, not Camp Smalltalk ;-), but there were some small talks among the better halves and bigger talks (even swearing when a big fish got off the hook, we went fishing) among the fishers.
[BIG snip on helpful detailed instructions for playing with a Smalltalk compiler]
I am saving your notes for a later stage. Right now it would be very inappropriate for a Squeak newbie to try this stunt even in the basement , implementing a Squeak compiler for the washing machine, let alone implementing it for the Squeak Installer publicly on The Squeak List.
However, this does not rule out the possibilities that I will keep looking around for codes to steal ;-) for this compiler. This is exactly how I got MobVM. The codes were stolen from Andreas' Windows VM distribution (BIG THANKS , Andreas). Most of the things were already there and pretty modular at that. All I did was shufling them, rearanging them around the plugin proxies and add some glue to hold them together. It was a joy to work with his codes.
I am setting my eyes on GNU Smalltalk. Does it have a compiler outside of the image ? How compatible is it to Squeak compiler ? What's involved to make it work for Squeak ?
On the other hand, someone may come up with a scheme to bootstrap Squeak that do not need this compiler out of the image at all. That would make me a lot happier (I can stop that bad habit of stealing codes ;-)
Once again, your help is very much appreciated.
Cheers,
PhiHo
I recently installed YAX XML and tried to get the examples to work, but [apparently] nothing happened. I updated to the very latest set that has been released, and I'm using the YAX XML library + SocketStream library as found at http://squeaklet.com/Yax/index.htm
I can't figger out how to get it to work/display/jump/leap/run. I'm sure it is something very trivial, but I can't seem to make it work... :(
Thanks in advance...
Lawson English wrote:
I recently installed YAX XML and tried to get the examples to work, but [apparently] nothing happened. I updated to the very latest set that has been released, and I'm using the YAX XML library + SocketStream library as found at http://squeaklet.com/Yax/index.htm
I updated the site and put up a new version (fixed a few bugs and speeded up the parsing process) along with better examples.
Michael
P.S. You actually don't need the SocketStream. It was orignally intended to go with the rudimentary Jabber implementation.
on 8/26/01 6:49 PM, Michael Rueger at m.rueger@acm.org wrote:
Lawson English wrote:
I recently installed YAX XML and tried to get the examples to work, but [apparently] nothing happened. I updated to the very latest set that has been released, and I'm using the YAX XML library + SocketStream library as found at http://squeaklet.com/Yax/index.htm
I updated the site and put up a new version (fixed a few bugs and speeded up the parsing process) along with better examples.
Michael
P.S. You actually don't need the SocketStream. It was orignally intended to go with the rudimentary Jabber implementation.
Thanks much. XML parsing is essential to implementing a World Forge client, and I'd rather not try to do THAT from scratch...
The compiler is only in the image. It is completely unsuited for conversion via the CCodeGenreator.
Not that you could possibly have a reasonable use for such a thing; there is no need nor sense in having modules as source code. A format like ImageSegments is entirely more likely to be useful.
Howdy, Tim,
The compiler is only in the image. It is completely unsuited for conversion via the CCodeGenreator.
Thanks for your confirmations. One should look else where for a Squeak compiler outside of the image, unless...
Not that you could possibly have a reasonable use for such a thing; there is no need nor sense in having modules as source code. A format like ImageSegments is entirely more likely to be useful.
We can bootstrap Squeak with binary (?) ImageSegment modules instead of textual Smalltalk source modules ?
Are you saying that one can use a full blown Squeak image, create an ImageSegment consisting of, say, ProtoObject, and Greetings (a subclass of 'ProtoObject' with just one method 'sayHello') classes.
From this ImageSegment, theoretically, it is possible to construct a micro image ?
If this is viable, definitely, I agree with you, it's a much better approach.
It is very much appreciated if you or other ImageSegment gurus would elaborate on this issue, probably with some examples and exercises. Especially, how one should go about to construct an image out of a set of ImageSegment modules. ( A draft for the chapter on OE tour, second edition ? :-)
Take care.
Cheers,
PhiHo
----- Original Message ----- From: "Tim Rowledge" tim@sumeru.stanford.edu To: squeak-dev@lists.squeakfoundation.org Sent: Saturday, August 25, 2001 11:18 AM Subject: Re: [Squeak Installer] The Compiler, The Final Frontier(?)
The compiler is only in the image. It is completely unsuited for conversion via the CCodeGenreator.
Not that you could possibly have a reasonable use for such a thing; there is no need nor sense in having modules as source code. A format like ImageSegments is entirely more likely to be useful.
-- Tim Rowledge, tim@sumeru.stanford.edu, http://sumeru.stanford.edu/tim Drugs may lead to nowhere, but at least it's the scenic route.
Tim Rowledge wrote :
The compiler is only in the image. It is completely unsuited for conversion via the CCodeGenreator.
Joseph Pelrine came up with a simple yet powerful and a-la-mode solution: Web Service.
John M McIntosh reported from Camp Smalltalk Essen ESUG 2001:
"Did I mention the morning started late? So lunch is where all thirty plus of us endured poor service at an Italian German restaurant, however I had some interesting discussions with Joseph about tethered Smalltalk images. His vision is to have a very small basic image that supports only the kernel, and the socket layer. By talking to a fully functional image that exists on the Internet somewhere it could request that server to load modules from a repository, compile them and sent it the bytecodes to install. That way the small image could start from a very small size, and add functionality to meet it's business needs, and forgo loading dangerous things like compilers."
A format like ImageSegments is entirely more likely to be useful.
Hi John,
Thanks for your report, I am looking forward for the next installment. If it is not too much trouble for you, would you please try to get an answer for the following question:
Now, that Web Service can deliver compiled modules in either CompiledMethod (?) format or ImageSegment format. Which one would be more preferrable, simpler for the ModuleManger in creating a bootstrap image ?
Cheers,
PhiHo
----- Original Message ----- From: "Tim Rowledge" tim@sumeru.stanford.edu To: squeak-dev@lists.squeakfoundation.org Sent: Saturday, August 25, 2001 11:18 AM Subject: Re: [Squeak Installer] The Compiler, The Final Frontier(?)
The compiler is only in the image. It is completely unsuited for conversion via the CCodeGenreator.
Not that you could possibly have a reasonable use for such a thing; there is no need nor sense in having modules as source code. A format like ImageSegments is entirely more likely to be useful.
-- Tim Rowledge, tim@sumeru.stanford.edu, http://sumeru.stanford.edu/tim Drugs may lead to nowhere, but at least it's the scenic route.
Hi,
I just downloaded and installed ComSwiki. Browsing through the documentation pages, I couldn't find any references to Squeak Server Page (from Stephen Pair ?).
Does Squeak Server Page work with ComSwiki or any other Web Server ? Where can I get the Squeak Server Page and documentation ?
Thanks,
PhiHo
Bolot updated SSP to work with Squeak 2.9 (I believe some compiler changes in 2.9 broke it). That should work with 3.1 as well (I should write some SUnits for it so that we can be sure).
The original, and Bolot's 2.9 update can be found at: http://ssp-squeak.swiki.net ...I thought that ComSwiki includes SSP, but maybe not.
- Stephen
-----Original Message----- From: squeak-dev-admin@lists.squeakfoundation.org [mailto:squeak-dev-admin@lists.squeakfoundation.org] On Behalf Of PhiHo Hoang Sent: Monday, August 27, 2001 1:10 AM To: squeak-dev@lists.squeakfoundation.org Subject: Squeak Server Page
Hi,
I just downloaded and installed ComSwiki. Browsingthrough the documentation pages, I couldn't find any references to Squeak Server Page (from Stephen Pair ?).
Does Squeak Server Page work with ComSwiki or any otherWeb Server ? Where can I get the Squeak Server Page and documentation ?
Thanks, PhiHo
Hi Stephen,
The original, and Bolot's 2.9 update can be found at: http://ssp-squeak.swiki.net ...I thought that ComSwiki includes SSP, but maybe not.
Thanks for the link. I downloaded the updated version and tried to filein (ComSwiki image, latest update #4282)
There seems to be HTML tags in this change set that caused many syntax errors. After cleaning the header part, I got down to this syntax error:
" formattedNameAndAddressOn: strm <ssp on: strm>
double return expected ->A Person: Name: <%= name %> Address: <%= address %> "
Cheers,
PhiHo
----- Original Message ----- From: "Stephen Pair" spair@advantive.com To: squeak-dev@lists.squeakfoundation.org Sent: Monday, August 27, 2001 11:50 AM Subject: RE: Squeak Server Page
Bolot updated SSP to work with Squeak 2.9 (I believe some compiler changes in 2.9 broke it). That should work with 3.1 as well (I should write some SUnits for it so that we can be sure).
The original, and Bolot's 2.9 update can be found at: http://ssp-squeak.swiki.net ...I thought that ComSwiki includes SSP, but maybe not.
- Stephen
-----Original Message----- From: squeak-dev-admin@lists.squeakfoundation.org [mailto:squeak-dev-admin@lists.squeakfoundation.org] On Behalf Of PhiHo Hoang Sent: Monday, August 27, 2001 1:10 AM To: squeak-dev@lists.squeakfoundation.org Subject: Squeak Server Page
Hi,
I just downloaded and installed ComSwiki. Browsingthrough the documentation pages, I couldn't find any references to Squeak Server Page (from Stephen Pair ?).
Does Squeak Server Page work with ComSwiki or any otherWeb Server ? Where can I get the Squeak Server Page and documentation ?
Thanks, PhiHo
I wrote:
If you have any external plugin of your own, you are encouraged to try them with MobVM. I would like to hear your comments. it should be fully compatible with Squeak-D3D (625,644 bytes dated May-17-2001).
I forgot a little issue, in Squeak-D3D, the plugin B3DAcceleratorPlugin was already 'external-ready'. However, I had to change a bit:
//static HWND *theSTWindow = NULL; /* a reference to Squeak's main window */ // theSTWindow = (HWND*) interpreterProxy->ioLoadFunctionFrom("stWindow","");
static HWND stWindow = NULL; /* Squeak's main window */ static HWND *theSTWindow = &stWindow ; /* a reference to Squeak's main window */
stWindow = windowProxy->getStWindow();
// preMessageHook = (messageHook*) interpreterProxy->ioLoadFunctionFrom("preMessageHook",""); preMessageHook = windowProxy->getPreMessageHookAddress() ;
The main reason for this incompatibility is that I am not sure how to best handle things like this. So I just settle for uniformity in the way properties are accessed, through getters and setters. Somehow, I feel a bit uneasy in using 'interpreterProxy->ioLoadFunctionFrom' to get at a property.
So, if your plugin use 'interpreterProxy->ioLoadFunctionFrom' to get at a property, it will not work with MobVM.
P.S: Please ensure that you do not have the official VM and MobVM in the same directory. This should do no harm to MobVM but the official VM will just display a black screen because it found the external plugins
(belonging
to MobVM, requiring a higher minor version for the interpreter) and cannot use it (because of the version). The official VM should fall back to the internal plugins and proceed as if there were no (useless) external
plugins.
Andreas, please consider this as a bug report ;-)
New feature to avoid this situation, all plugins distributed with MobVM will reside in the subdirectory 'plugins' below the directory where Squeak.exe stay. MobVM will look for the plugins in the usual places, if not found, will look in this subdirectory 'plugins'.
One question, to ensure that MobVM and official VM can coexist peacefully in the same place, should the traditional name 'Squeak.exe' be changed to say 'MobSqueak.exe' or 'SqueakMobVM.exe' or 'SqueakInstaller.exe' ?
Cheers,
PhiHo
----- Original Message ----- From: "PhiHo Hoang" phiho.hoang@home.com To: squeak-dev@lists.squeakfoundation.org Sent: Saturday, August 25, 2001 12:17 AM Subject: [Squeak Installer] Progress Report
Hi List,
There is some progress. Not much though, merely 11KB. The codes weremainly stolen from the existing InterpreterPlugin and PlatformPlugin.
What it does is looking around for a Squeak image, if not found it
will
bring up a file open dialog box so that the user can pick up an image somewhere on the local file system. It then loads the image, read some saved information, then initialised the ObjectMemory, CompilerHooks...prepare the scene for interpret().
This new plugin, for now, is named 'ModManPlugin' (Module Manager). Ofcourse this is only a place holder for the true Module Manager. I hope at some stage, with help from the list, this plugin will be able to look for, instead of a pre-built image, the modules (Smalltalk source files) whether on the local file system or on the internet (via http/ftp ), pass them to the compiler, get back the result from the compiler, building up an image (?) in memory......prepare the scene for interpret().
Please find attached the file 'ObjectMemoryStartup.zip' to see what
was
stolen from where. Beware, it's pretty dirty with '#if 1' and '#if 0' and other commented out things.
If you are curious to see if it really works or you want to help
beating
it to death (in the process of bug hunting ;-), please email me. I will
send
you Squeak.exe and 33 plugins, excluding SqueakFFIPrims.dll and Mpeg3Plugin.dll, you should get these from the official distribution (they have been external all time).
If you have any external plugin of your own, you are encouraged to trythem with MobVM. I would like to hear your comments. it should be fully compatible with Squeak-D3D (625,644 bytes dated May-17-2001).
The file MobVM.zip is 328KB. I am sorry, it's for Win32 only (testedon Win2K Pro, somewhat tested on Win2KAS and WinME. It should just work on YourWin ;-)
Cheers, PhiHoP.S: Please ensure that you do not have the official VM and MobVM in the same directory. This should do no harm to MobVM but the official VM will just display a black screen because it found the external plugins
(belonging
to MobVM, requiring a higher minor version for the interpreter) and cannot use it (because of the version). The official VM should fall back to the internal plugins and proceed as if there were no (useless) external
plugins.
Andreas, please consider this as a bug report ;-)
Allen Wirfs-Brock wrote:
So what are we trying to accomplish? Dan listed a set of "desiderata" that I can whole-heartedly support. However, I would like to step up one level before plunging into the details.
Yes, let us avoid the mad rush into technical details before we have a good sense of where we're going. There are lots of good ideas being thrown about, but I'd like to avoid getting sucked into that just yet. As I mention in another message, I would like to see more scenarios posted, edited, commented upon, etc, to use as motivational examples or counter-examples regarding the way we want our system to work. I have a few that I'd like to post before wrangling over technical approaches.
[snip]
Roughly speaking Components correspond to COM or CORBA objects, DLLs, SLLs in Visual Smalltalk, etc.
Roughly speaking Modules correspond to Java source files, Envy projects, TEAM/V packages, Smalltalk change sets.
Modules are all about organizing and managing code, Components are all about composing functionality (behavior). Modules are used to create components.
[snip]
(Here is my first crack assignment based upon limited visibility. Component systems: Environments, OASIS, (I don't really know enough about either but this is my impression) , Pope's selector ideas. Module systems: Pelrine's work, change sets).
I'd definitely agree that Oasis in present form is a component system in your scheme. The Module aspect of that system has not been built, although that does not mean that I am without ideas about how that part of the system should work. More on this as we go ;^)
[snip]
So here's my strawman position: Squeaks needs both a module architecture and a component architecture. They should be designed to be complementary to each other. The module system should stick to development time issues. The component system should stick to runtime issues.
I very strongly agree with you on this- in fact, I just posted another message along those lines ( perhaps not as clearly ).
Globally, some people should be thinking about how to slice the image into components and modules.
That's a mighty big topic- I'll have to come back to that.
Take care!
- les
Allen Wirfs-Brock Allen_Wirfs-Brock@Instantiations.com wrote...
So here's my strawman position: Squeaks needs both a module architecture and a component architecture. They should be designed to be complementary to each other. The module system should stick to development time issues. The component system should stick to runtime issues. Globally, some people should be thinking about how to slice the image into components and modules.
I agree about this. The thinking in SqC has generally been moving toward projects as the unit for Components. I don't know if this will cover everything, but it's quite compelling when you browse Squeak content over the web in project-sized chunks.
The very next thing we aim to do is to place a name scope in Project (in fact, you'll see an instVar named 'environment' already there). Two immediate goals are a) guarantee that two projects cannot interfere with one another if loaded at the same time, and b) that if you unload a project, there is no memory in the system that it had been there.
- Dan
On Thu, 16 Aug 2001, Dan Ingalls wrote: [snip]
Two immediate goals are a) guarantee that two projects cannot interfere with one another if loaded at the same time, and b) that if you unload a project, there is no memory in the system that it had been there.
Dan is, as usual, taking deep personal care of my specific interests :)
Sounds great.
Cheers, Bijan Parsia.
Hi all!
--- Allen Wirfs-Brock Allen_Wirfs-Brock@Instantiations.com wrote: [MEGASNIP of very good stuff]
other. (Here is my first crack assignment based upon limited visibility. Component systems: Environments, OASIS, (I don't really know enough about either but this is my impression) , Pope's selector ideas. Module systems: Pelrine's work, change sets). My impression is that people
Yes, the discussion has definitely mixed these two different areas and I think that we should perhaps focus on the sourcemanagement side of it first. But if I am not totally off I would have guessed that OASIS should be placed among the module systems - it's about source code analysis, type inference, interface detection etc. etc., or did I get the wrong impression when I looked at it?
[SNIP]
So here's my strawman position: Squeaks needs both a module architecture and a component architecture. They should be designed to be complementary to each other. The module system should stick to development time issues. The component system should stick to runtime issues. Globally, some people should be thinking about how to slice the image into components and modules.
Amen.
regards, G�ran
===== G�ran Hultgren, goran.hultgren@bluefish.se GSM: +46 70 3933950, http://www.bluefish.se "Department of Redundancy department." -- ThinkGeek
__________________________________________________ Do You Yahoo!? Make international calls for as low as $.04/minute with Yahoo! Messenger http://phonecard.yahoo.com/
Hi!
--- G�ran Hultgren gohu@rocketmail.com wrote:
Hi all!
--- Allen Wirfs-Brock Allen_Wirfs-Brock@Instantiations.com wrote: [MEGASNIP of very good stuff]
other. (Here is my first crack assignment based upon limited visibility. Component systems: Environments, OASIS, (I don't really know enough about either but this is my impression) , Pope's selector ideas. Module systems: Pelrine's work, change sets). My impression is that people
Yes, the discussion has definitely mixed these two different areas and I think that we should perhaps focus on the sourcemanagement side of it first. But if I am not totally off I would have guessed that OASIS should be placed among the module systems - it's about source code analysis, type inference, interface detection etc. etc., or did I get the wrong impression when I looked at it?
And then I read what Les wrote and realized that I truly must be "totally off". Ok! :-)
I'll just shut up and perhaps start to create a "digest" of what people are saying. That would be useful because right now ideas and views are flying by in an awful speed... Somebody must collect.
regards, G�ran
===== G�ran Hultgren, goran.hultgren@bluefish.se GSM: +46 70 3933950, http://www.bluefish.se "Department of Redundancy department." -- ThinkGeek
__________________________________________________ Do You Yahoo!? Make international calls for as low as $.04/minute with Yahoo! Messenger http://phonecard.yahoo.com/
OOPS... I said that Oasis is, in Allen's terms, about Modules. WRONG. It is about Components... that's what you get when you go and change the semantics of the words I've been using to name my classes !!!
However, the so-called "third piece" of Oasis *is* about Modules, in Allen's terms. That has not been built- so now is a good time to consider that, for me.
- les
At 09:38 -0700 2001.08.17, Les Tyrrell wrote:
OOPS... I said that Oasis is, in Allen's terms, about Modules. WRONG. It is about Components... that's what you get when you go and change the semantics of the words I've been using to name my classes !!!
Les,
I'm so glad that you corrected yourself before I started banging my head against the table!
Andrew
Göran Hultgren wrote:
Hi all!
--- Allen Wirfs-Brock Allen_Wirfs-Brock@Instantiations.com wrote: [MEGASNIP of very good stuff]
other. (Here is my first crack assignment based upon limited visibility. Component systems: Environments, OASIS, (I don't really know enough about either but this is my impression) , Pope's selector ideas. Module systems: Pelrine's work, change sets). My impression is that people
...
Yes, the discussion has definitely mixed these two different areas and I think that we should perhaps focus on the sourcemanagement side of it first. But if I am not totally off I would have guessed that OASIS should be placed among the module systems - it's about source code analysis, type inference, interface detection etc. etc., or did I get the wrong impression when I looked at it?
Definitely, what I have done so far in Oasis is would fall in Allen's module camp. And yes, those things you mention are what I have been doing quite a lot of... 5 years ago I thought that they would be Really Important things that would have to be solved in dealing with large, non-modular systems... 5 years later, the shininess has worn off that notion a bit, but I am still plinking away at it from time to time. Discussions like these help a lot in getting me all riled up again to go attack some of those problems.
[SNIP]
So here's my strawman position: Squeaks needs both a module architecture and a component architecture. They should be designed to be complementary to each other. The module system should stick to development time issues. The component system should stick to runtime issues. Globally, some people should be thinking about how to slice the image into components and modules.
Amen.
I will ( once again ) second that!
- les
squeak-dev@lists.squeakfoundation.org
-
Allen Wirfs-Brock -
Andrew P. Black -
Bijan Parsia -
Dan Ingalls -
Dan Moniz -
Dave -
G�ran Hultgren -
Joerg Beekmann -
Lawson English -
Les Tyrrell -
Mark van Gulik -
Michael Rueger -
Ned Konz -
PhiHo Hoang -
Robert M. Lefkowitz -
Simon Michael -
Stephen Pair -
Steve S -
Tim Rowledge