Hi all!
Yesterday we presented the Team model and now we want to follow that up with proposing two quite important Teams.
As you have seen Ken Causey has already picked up the ball and is proposing a Team for tackling one of our biggest challenges - Harvesting. Both in the short and long term. Perfect.
Now... after chatting on IRC with various people etc we want to propose to form the following two Teams.
Packages ---------- This is a very central baby to many of us. The Team would focus on the following issues:
- PackageInfo/Package etc. There is new code from Alexandre Bergel etc, we need to get something nailed down and into the 3.9 stream. - TFNR. (Task Force November Revolution, see Swiki) In short - the idea of partitioning the image and assigning maintainers (with responsibility and authority) to all parts (sometimes referred to as Stewards trying to convey that being a maintainer of a base image part is a slightly different role than just being a "maintainer"). This is not primarily about ripping out packages. - SqueakMap. Moving SqueakMap forward, lots of stuff there including dependencies. - Universes. Lex's idea about maintaining coherent "distributions" of packages. (well, he can correct me, but anyway :) ). - ...and various other related things like Monticello and its use in all this etc.
The overall theme of this Team is to evolve the technology we already have and to make working solutions for the current and near term Squeak releases. And of course to make it all fit together. Without me knowing if people are interested I can *at least* see the following names in this Team:
Doug Way, Lex Spoon, Göran Krampe, Avi Bryant, Alexandre Bergel, Ned Konz
I am not going to venture into talking about who should be Team leader here - I have a favorite though and its not me ;). But I am definitely signing up as a very active member.
FutureModules ---------------- This is a Team focusing on more "future" technologies in this area. Examples that have come up are:
- Spoon. - Islands. - E and similar idea. - Environments. - Namespaces (various models and solutions thereof) - ClassBoxes and other stuff from Berne in this regard.
Well, I am surely missing something but you get the picture. Dan Ingalls has expressed a solid interest in this Team and Craig Latta too. And some of us from the Packages team might be found here too - if not only just to be able to reflect on it.
So... the list seems quiet compared to earlier - have we scared everyone away? :) Feedback please!!!
regards, Göran
Gran wrote...
FutureModules
This is a Team focusing on more "future" technologies in this area. Examples that have come up are:
- Spoon.
- Islands.
- E and similar idea.
- Environments.
- Namespaces (various models and solutions thereof)
- ClassBoxes and other stuff from Berne in this regard.
Well, I am surely missing something but you get the picture. Dan Ingalls has expressed a solid interest in this Team and Craig Latta too. And some of us from the Packages team might be found here too - if not only just to be able to reflect on it.
[and I would include Universes in the above list too]
Thanks for the intro, Gran. It's true; I have a solid interest here.
This list wraps around the world, so Gran's proposal has been out there for several hours, but I've only just woken up and finished reading my email. I'll try to articulate my perspective on this and send out a message within the next few hours , so that people can decide whether they share that perspective, or want to approach it differently.
If I become leader of this project, though, my first move will be to strike "future" from the title. While I want it to take us into the future, I want the design done in 3 months, and I want something working in 6 months.
More later
- Dan
Heh heh...
About six hours ago, I said I'd write a quick summary of my thoughts on the subject, and I'm just getting started. It's not that I'm writing more, but I'm trying to get it right which, as you know, takes longer.
Meanwhile, I'd like to proceed simply to ask for people interested in putting modules into Squeak.
As part of any positive response, I'd like to hear
What would you like to see in such a system?
Are you mainly interested as a designer, implementor, or user?
Anything else you'd like to say.
Thanks - Dan
PS: For now, please don't worry about overlap with current packages work. The two are closely related, and they will come together in the end, but they can be worked on independently (hopefully with cross fertilization) until then. I'm hoping to see many of the package folks among those interested.
PPS: And, lest people freak out about earthshaking changes, let me say that one of my top priorities is that, however it works, such a system should be unnoticeable to casual users of Squeak.
Hi Dan and all!
Dan Ingalls Dan@SqueakLand.org wrote:
Heh heh...
About six hours ago, I said I'd write a quick summary of my thoughts on the subject, and I'm just getting started. It's not that I'm writing more, but I'm trying to get it right which, as you know, takes longer.
No rush, sorry for putting on the pressure by my post. I just thought it was a good move - both with the momentum and also to show what we mean with Teams more concretely. As in - they are For Real. :)
For the record it should be said that the idea of two Teams like this originates from Dan.
Meanwhile, I'd like to proceed simply to ask for people interested in putting modules into Squeak.
I just created a new list called "modules" for this Team:
modules-subscribe@discuss.squeakfoundation.org
As part of any positive response, I'd like to hear
I am in. :)
What would you like to see in such a system?
Personally - simplicity and... simplicity.
Are you mainly interested as a designer, implementor, or user?
Partly designer/implementor, but no hardcore. Of course SM and its future place/direction/area and also my little Namespace thingy (almost scared of mentioning it nowadays :)) are my personal interests in all this.
And, unless I haven't made that absolutely clear by now, I don't have any special "standing", I don't expect my interests or proposals/solutions to be treated differently than anyone elses.
Anything else you'd like to say.
Hmmm, well, one important thing, given the "risks" with this work (we all remember the past), as with MinimalMorphic one smart idea is to run the Team "in parallell" to the regular stream as long as it can be done. Meaning that if there is a crash and burn, we aren't sitting with a half baked mess in the update stream :). But that is probably obvious.
Thanks
- Dan
PS: For now, please don't worry about overlap with current packages work. The two are closely related, and they will come together in the end, but they can be worked on independently (hopefully with cross fertilization) until then. I'm hoping to see many of the package folks among those interested.
Yes, fully agree. And I am in and I assume Lex and Alexandre are too etc.
PPS: And, lest people freak out about earthshaking changes, let me say that one of my top priorities is that, however it works, such a system should be unnoticeable to casual users of Squeak.
Yes, indeed.
regards, Göran
Dan Ingalls wrote:
As part of any positive response, I'd like to hear
What would you like to see in such a system?
I'd like to see it take a Monticello package and spit out a module that loads as fast as an image segment, and permits a "postscript" ala SAR/changesets.
Are you mainly interested as a designer, implementor, or user?
Yes.
Anything else you'd like to say.
Could you make a statement about how you see your work on modules relating to Craig Latta's work? I know you said "don't worry", but just a sentence or two would be appreciated.
Thanks.
On Wed, 23 Feb 2005 12:36:16 -0500, Steven Swerling sswerling@yahoo.com wrote:
I'd like to see it take a Monticello package and spit out a module that loads as fast as an image segment, and permits a "postscript" ala SAR/changesets.
Heh. Some attention to fast loading might be worthwhile, yes. I had to load RB and SRP into the (admittedly sluggish) wxSqueak image, and you wouldn't say that this was a 3.2GHz machine doing the job...
alex proposed a first implementation of byet-code loading (even if image segments may be better) of MC packages that was going 10 faster that normal loading and we got no reaction. If you ask him he can send you that.
Stef
On 23 févr. 05, at 18:58, Cees de Groot wrote:
On Wed, 23 Feb 2005 12:36:16 -0500, Steven Swerling sswerling@yahoo.com wrote:
I'd like to see it take a Monticello package and spit out a module that loads as fast as an image segment, and permits a "postscript" ala SAR/changesets.
Heh. Some attention to fast loading might be worthwhile, yes. I had to load RB and SRP into the (admittedly sluggish) wxSqueak image, and you wouldn't say that this was a 3.2GHz machine doing the job...
On Feb 22, 2005, at 5:53 PM, Dan Ingalls wrote:
As part of any positive response, I'd like to hear
What would you like to see in such a system?
From the deployment side, I would like to see robust dependency mechanisms - for example, in Monticello there is a notion of prerequisites (ignore the fact they are changing and the Monticello isn't a package manager) that can exist on various repositories. The dependency mechanism that G-ran has talked about for the next version of SM would ostensibly allow getting the modules to be loaded from various places, not only a "central" server. Some sort of fallback mechanism for package repositories would be very useful.
Are you mainly interested as a designer, implementor, or user?
Whatever is needed :)
Anything else you'd like to say.
Thanks
- Dan
Hi Dan--
Meanwhile, I'd like to proceed simply to ask for people interested in putting modules into Squeak.
Well, I'm interested, of course... I guess this response is rather routine, but I thank the community for humoring me. :)
What would you like to see in such a system?
I'd like to see a system which, from the human's point of view, is very simple and consistent. I'd like a system which uses messages instead of files, and use that ability to do live inter-system negotiation.
Are you mainly interested as a designer, implementor, or user?
Hard to say. :) I was motivated to develop this stuff by my userly dissatisfaction with traditional mechanisms. So I became a designer and implementor. I guess it still comes down to wanting to *use* the stuff, though.
Anything else you'd like to say.
Let's go!
thanks,
-C
-- Craig Latta improvisational musical informaticist craig@netjam.org www.netjam.org [|] Proceed for Truth!
On Feb 22, 2005, at 7:53 PM, Dan Ingalls wrote:
Meanwhile, I'd like to proceed simply to ask for people interested in putting modules into Squeak.
Hi Dan,
Chalk me up as interested in modularizing Squeak, as designer, implementor and user.
One of the motivations for OmniBrowser was the observation that any modularization scheme would need tool support, and the current browser is an obstacle to that because it's difficult to modify. Indeed, the early versions of OB included package browsers based on PackageInfo packages, but I took those out on feedback from the community.
I agree with Florin that versioning is a key part of organizing Smalltalk code. One of my biggest frustrations with Monticello is that it's not been of any use to the Guides and Harvesters. That is, it's great for applications built on Squeak, but not so good for Squeak its self. I'm now working on a version of Monticello with a redesigned versioning engine, which I hope will be more useful in managing the kernel and core libraries of Squeak. More on this in another post.
I also implemented an experimental packaging system a few months ago, complete with an OB-based set of browsers. It's not quite finished, and I'm not sure it's the way to go, but it's interesting nonetheless.
I quite like Alan Lovejoy's characterization of packages as units of separately deployable code, and I hope that doesn't get overlooked in the all the discussion of versioning, categories, namespaces, dependencies and so on. All those things are necessary to some degree, since they are ways of dealing with the additional complexity that a packaged image brings with it, but they're not end goals in themselves. Separately-deployable packages is what we're after.
Some more specific things I'd like to see.
- Decoupling the organization of the image (packages, environments or whatever) from the compiler's strategy for resolving names. This will make things like compiling in a sandbox, building small images, atomically loading packages and so on much easier.
- Good tool support for packages. Browsers, debuggers, changesorters, testrunners, etc should all be package-aware.
- Good tool support by packages. With a little care, we can make core execution machinery *much* easier to manipulate, and thus make it easier to write good tools. ClassBuilder is a good example: it is very difficult to deal with any other way than through the class definition doIts that appear in the browser.
- A fast-loading and compact format for distributing code.
Thanks for taking this on, Dan. I think it's really important.
Colin
sounds coooooool colin.
Stef
On 24 févr. 05, at 7:25, Colin Putney wrote:
On Feb 22, 2005, at 7:53 PM, Dan Ingalls wrote:
Meanwhile, I'd like to proceed simply to ask for people interested in putting modules into Squeak.
Hi Dan,
Chalk me up as interested in modularizing Squeak, as designer, implementor and user.
One of the motivations for OmniBrowser was the observation that any modularization scheme would need tool support, and the current browser is an obstacle to that because it's difficult to modify. Indeed, the early versions of OB included package browsers based on PackageInfo packages, but I took those out on feedback from the community.
I agree with Florin that versioning is a key part of organizing Smalltalk code. One of my biggest frustrations with Monticello is that it's not been of any use to the Guides and Harvesters. That is, it's great for applications built on Squeak, but not so good for Squeak its self. I'm now working on a version of Monticello with a redesigned versioning engine, which I hope will be more useful in managing the kernel and core libraries of Squeak. More on this in another post.
I also implemented an experimental packaging system a few months ago, complete with an OB-based set of browsers. It's not quite finished, and I'm not sure it's the way to go, but it's interesting nonetheless.
I quite like Alan Lovejoy's characterization of packages as units of separately deployable code, and I hope that doesn't get overlooked in the all the discussion of versioning, categories, namespaces, dependencies and so on. All those things are necessary to some degree, since they are ways of dealing with the additional complexity that a packaged image brings with it, but they're not end goals in themselves. Separately-deployable packages is what we're after.
Some more specific things I'd like to see.
- Decoupling the organization of the image (packages, environments or
whatever) from the compiler's strategy for resolving names. This will make things like compiling in a sandbox, building small images, atomically loading packages and so on much easier.
- Good tool support for packages. Browsers, debuggers, changesorters,
testrunners, etc should all be package-aware.
- Good tool support by packages. With a little care, we can make core
execution machinery *much* easier to manipulate, and thus make it easier to write good tools. ClassBuilder is a good example: it is very difficult to deal with any other way than through the class definition doIts that appear in the browser.
- A fast-loading and compact format for distributing code.
Thanks for taking this on, Dan. I think it's really important.
Colin
On Wednesday 23 February 2005 10:25 pm, Colin Putney wrote:
I quite like Alan Lovejoy's characterization of packages as units of separately deployable code, and I hope that doesn't get overlooked in the all the discussion of versioning, categories, namespaces, dependencies and so on. All those things are necessary to some degree, since they are ways of dealing with the additional complexity that a packaged image brings with it, but they're not end goals in themselves. Separately-deployable packages is what we're after.
Yes, and as Robert Martin points out, that means that if two packages depend on each other, then they may as well be considered parts of the same package, because you can't version them separately.
So much of the refactoring work in complicated systems amounts to removing these circular dependencies so you can develop packages at different rates.
On Thu, 24 Feb 2005 08:35:44 -0800, "Ned Konz" ned@squeakland.org said:
On Wednesday 23 February 2005 10:25 pm, Colin Putney wrote:
I quite like Alan Lovejoy's characterization of packages as units of separately deployable code, and I hope that doesn't get overlooked in the all the discussion of versioning, categories, namespaces, dependencies and so on. All those things are necessary to some degree, since they are ways of dealing with the additional complexity that a packaged image brings with it, but they're not end goals in themselves. Separately-deployable packages is what we're after.
Yes, and as Robert Martin points out, that means that if two packages depend on each other, then they may as well be considered parts of the same package, because you can't version them separately.
So much of the refactoring work in complicated systems amounts to removing these circular dependencies so you can develop packages at different rates.
Agreed...
Just to throw in a concrete example, Goran & I were casually discussing (on a partitioning thread here a month ago) whether Collections should be partitioned as a separate PI package from Kernel. On one hand, you might have a separate group of maintainers taking care of Collections, versus the Kernel, which makes some sense. (At least this was the initial idea with TFNR.) But Collections is not at all separately deployable from Kernel, the two are very dependent on each other. I think the separate deployability aspect is more important, so I'd favor keeping them in the same package.
This is sort of a special case, since the Kernel is not exactly a "package" which can be loaded, it is the base image. (Although it could still be defined as a package or module.)
Also, I'm mixing the simple near-term partitioning project with the longer-term modules project here. (Though it's not *too* long-term as Dan says. :) ) But I think the principles are the same in this case. Our partitioning should still be roughly based on deployable units.
- Doug
Hi people!
"Doug Way" dway@mailcan.com wrote:
Just to throw in a concrete example, Goran & I were casually discussing (on a partitioning thread here a month ago) whether Collections should be partitioned as a separate PI package from Kernel. On one hand, you might have a separate group of maintainers taking care of Collections, versus the Kernel, which makes some sense. (At least this was the initial idea with TFNR.) But Collections is not at all separately deployable from Kernel, the two are very dependent on each other. I think the separate deployability aspect is more important, so I'd favor keeping them in the same package.
This is sort of a special case, since the Kernel is not exactly a "package" which can be loaded, it is the base image. (Although it could still be defined as a package or module.)
And thus we should also remember the idea of TFNR - the idea of making sure interested and people willing to take responsibility of a certain part should be able to do that. In this case there are probably several people interested in maintaining Collections - but not the whole kernel!
So even though I generally agree with the "deployable" attribute, for the parts that we typically can't break out yet - staking Collections out as a separate "part" (as in part-itioning) from kernel doesn't seem to be harmful. It will still be maintained using the update stream.
Sure we can lump it all together in a huge "kernel" part - and assign 20 maintainers to it - but will that give the effect we want? The "effect" being responsible motivated maintainers tending carfully for these classes. I am not sure.
Also, I'm mixing the simple near-term partitioning project with the longer-term modules project here. (Though it's not *too* long-term as Dan says. :) ) But I think the principles are the same in this case. Our partitioning should still be roughly based on deployable units.
"Rougly" yes, but... if this leads to a single "part" (the whole Basic image) then there is no point, is there? :)
- Doug
regards, Göran
On Fri, 25 Feb 2005 10:39:25 +0100, goran.krampe@bluefish.se wrote:
"Rougly" yes, but... if this leads to a single "part" (the whole Basic image) then there is no point, is there? :)
Well, yes there is. Because maybe if you're going to deploy them all at once, it's still good to partition it.
Let's (yawn) look at Debian again. There is a distinct kernel of packages in Debian that all must be loaded together - there's a bootstrap script to do that so you can build a new system on a mounted disk from scratch. These packages are heavily intertwined and interdependent. Nothing's gonna tick without the C run-time-library, for starters.
So releasing them separately is nonsense.
However, the unstable/testing/stable separations lets developers put their own new versions of their own kernel packages into unstable, play and experiment, quite independently. Sure, things will be messed up at times, but that's why it's called unstable, remember? Then, when some new constellation works, these package versions are most likely to be moved as a whole 'configuration map' into testing, which gets a new core. Bang!. When these versions have proven themselves as being cooperable with all the other stuff, it's time to announce stable old, testing stable, and open a new testing area. At least, that's how I understood it ;).
So being able to maintain them independently is not nonsensical at all.
Cees de Groot cg@cdegroot.com wrote:
On Fri, 25 Feb 2005 10:39:25 +0100, goran.krampe@bluefish.se wrote:
"Rougly" yes, but... if this leads to a single "part" (the whole Basic image) then there is no point, is there? :)
Well, yes there is. Because maybe if you're going to deploy them all at once, it's still good to partition it.
Eh, you just argued on my side - so I am guessing you misunderstood me. So I agree with all you wrote, to the letter. :)
regards, Göran
On Fri, 25 Feb 2005 10:39:25 +0100 , goran.krampe@bluefish.se said:
Hi people!
"Doug Way" dway@mailcan.com wrote:
Just to throw in a concrete example, Goran & I were casually discussing (on a partitioning thread here a month ago) whether Collections should be partitioned as a separate PI package from Kernel. On one hand, you might have a separate group of maintainers taking care of Collections, versus the Kernel, which makes some sense. (At least this was the initial idea with TFNR.) But Collections is not at all separately deployable from Kernel, the two are very dependent on each other. I think the separate deployability aspect is more important, so I'd favor keeping them in the same package.
This is sort of a special case, since the Kernel is not exactly a "package" which can be loaded, it is the base image. (Although it could still be defined as a package or module.)
And thus we should also remember the idea of TFNR - the idea of making sure interested and people willing to take responsibility of a certain part should be able to do that. In this case there are probably several people interested in maintaining Collections - but not the whole kernel!
So even though I generally agree with the "deployable" attribute, for the parts that we typically can't break out yet - staking Collections out as a separate "part" (as in part-itioning) from kernel doesn't seem to be harmful. It will still be maintained using the update stream.
Sure we can lump it all together in a huge "kernel" part - and assign 20 maintainers to it - but will that give the effect we want? The "effect" being responsible motivated maintainers tending carfully for these classes. I am not sure.
Yeah, there's not really an obvious answer here. Both aspects are important. (Deployability and maintainership.) In this particular case, maybe maintainership is the more important aspect.
Also, I'm mixing the simple near-term partitioning project with the longer-term modules project here. (Though it's not *too* long-term as Dan says. :) ) But I think the principles are the same in this case. Our partitioning should still be roughly based on deployable units.
"Rougly" yes, but... if this leads to a single "part" (the whole Basic image) then there is no point, is there? :)
Yeah. :) But with most of the rest of the Basic image, the various parts should be separately deployable (Graphics, MVC, Morphic, etc). You could have a headless Kernel (+Collections) without Graphics loaded, interacting on a command line, and then load Graphics if you wanted. So we should still have a number of parts to our Basic image, even if it's mostly based on deployable units.
- Doug
"Doug Way" dway@mailcan.com wrote:
Just to throw in a concrete example, Goran & I were casually discussing (on a partitioning thread here a month ago) whether Collections should be partitioned as a separate PI package from Kernel. On one hand, you might have a separate group of maintainers taking care of Collections, versus the Kernel, which makes some sense. (At least this was the initial idea with TFNR.) But Collections is not at all separately deployable from Kernel, the two are very dependent on each other. I think the separate deployability aspect is more important, so I'd favor keeping them in the same package.
I snipped the bit where you acknowledged that you're mixing two ideas - partitioning for development and partitioning for deployment. It's good that we make that distinction, particularly at the beginning since we now have lots of stuff that isn't separately deployable, but should be. We don't have to wait for separate deployability to divvy up maintainership of the image.
However, let's not get too used to doing that. The "separately deployable" yardstick is valuable if we stick to it ruthlessly when we make partitioning decisions. Your Kernel vs. Collections example is a great case in point. Conceptually they're really quite different, and I'd think the "feel" of maintaining them would be different as well. On the one hand, you've got really low-level stuff - the machinery of executing code in the image, interfacing with the VM, etc. On the other hand, you've got a much more abstract library, almost a distillation of mathematical concepts. They really *ought* to be separate packages, and maintained by separate groups, if only because they'll attract different kinds of people to work on them.
If we ruthlessly apply the separately-deployable rule, we may find that we'll end up with a Kernel package that includes minimal implementatons of a few core collections, and a higher-level package that embellishes the kernel collections and provides higher-level collections like OrderedCollection, SkipList and so on. There's no need to preserve the class categories as they stand today...
Of course, that kind of partitioning may prove difficult, but it will probably pay off in the end.
Colin
partitioning for development and partitioning for deployment. It's good that we make that distinction, particularly at the beginning since we now have lots of stuff that isn't separately deployable, but should be.
Right.
There's no need to preserve the class categories as they stand today...
Exactly. Let's get rid of class categories. However, it might be difficult to assure backward compatibility. This is on what I was working with the packages as first-class entities...
Of course, that kind of partitioning may prove difficult, but it will probably pay off in the end.
Whatever the strategy of partitioning, we should need tools to do it :-) I am currently working on a new package browser that would perform this step easily...
Alexandre
By the way daniel vainsencher did that with his analysis tool: goruping together packages with circular dependencies.
Stef
On 24 févr. 05, at 17:35, Ned Konz wrote:
On Wednesday 23 February 2005 10:25 pm, Colin Putney wrote:
I quite like Alan Lovejoy's characterization of packages as units of separately deployable code, and I hope that doesn't get overlooked in the all the discussion of versioning, categories, namespaces, dependencies and so on. All those things are necessary to some degree, since they are ways of dealing with the additional complexity that a packaged image brings with it, but they're not end goals in themselves. Separately-deployable packages is what we're after.
Yes, and as Robert Martin points out, that means that if two packages depend on each other, then they may as well be considered parts of the same package, because you can't version them separately.
So much of the refactoring work in complicated systems amounts to removing these circular dependencies so you can develop packages at different rates.
-- Ned Konz http://bike-nomad.com/squeak/
Greetings Florin, Goran, Cees, Steven, Brian, David, and Craig, Ned, Colin! ... and anyone else who is interested but hasn't yet said so.
Sorry, but I spent most of today travelling and getting caught up on diddly stuff. My plan for tomorrow is to start a page on the Swiki, but I can't get to it right now. I want to assemble 1-page summaries of relevant work to consider as we define the project, along the following lines,
What problem is being addressed How it solves the problem What is cool about it What is not Commentary
I'd like various of us to write these summaries, or get someone else to write some, or copy them from somewhere in the next few days. I'm assuming someone knowledgeable will write the first four sections, and then we can all add further commentary in the last section.
I have in mind covering (to build on the earlier list)...
Spoon's Naiads Andreas's Islands Envy E Environments Envy Packages on SqueakMap Packages on Monticello Parcels in VW Namespaces (various models and solutions thereof) ClassBoxes and other stuff from Berne in this regard
... and anything else someone thinks would be useful (for instance I will include Florin's opening statement).
So I'll try to get this in shape tomorrow so that we can start building on it while people are still signing up.
I will take a shot at stating my slant on all of this for people to consider while we read over the background material, and then it will be time to start the serious discussion which, at the top level, will amount to writing a new entry in our list.
Meanwhile, if you haven't already, subscribe to modules-subscribe@discuss.squeakfoundation.org where the serious discussion will take place after we're done with the public rally.
- Dan
A minor correction, lest confusion creep in later...
Spoon's Naiads
At the moment, Spoon's module system is called "Naiad", but there's nothing called a "naiad" in the implementation, so pluralizing the term doesn't make sense. Rather, "Naiad" is an acronym that refers to one of the principle concepts of the system ("Name And Identity Are Distinct").
But you just might have made a typo for all I know. :)
Anyway, I would refer to it in this discussion as "Naiad (Spoon's module system)".
thanks!
-C
-- Craig Latta improvisational musical informaticist craig@netjam.org www.netjam.org [|] Proceed for Truth!
Module Folks -
I have now set up a page on the Swiki for collecting our first context. It is...
http://minnow.cc.gatech.edu/squeak/5608
Please... Clean up the format if you're handy with Swikis (I was in a rush) Fill out the content for any summary page about which you feel knowledgeable
Thanks - Dan
Hello Dan,
Greetings Florin, Goran, Cees, Steven, Brian, David, and Craig, Ned, Colin! ... and anyone else who is interested but hasn't yet said so.
I am definitely. I just found some time now to do it.
What problem is being addressed How it solves the problem What is cool about it What is not Commentary
I filled my part on 'Packages on Monticello' (http://minnow.cc.gatech.edu/squeak/5613)
Cheers, Alexandre
hi dan
since your last visit we have been looking around and this is interesting to see that over the last 4 years a lot of work have been done around a similar idea but not quite the same than our classboxes. Even if I do not believe that classboxes are the solution as is now. I think that they are interesting.
If you want I can send you some OOPSLA and ECOOP papers
For example, virtual classes proposes to introduce kind of classes extensions scope by having inheritance between modules (a hack because they could have a clean import relationship) and they have the problem that they duplicate classes from the refection point of view they have two classes. Nested Inheritance (OOPSLA 04) proposes somehow the same. The idea is that from within a module class name are lookup following some import chain (of course they have types and everything is much more complex then) but the basic principle is to late bound classname lookup so that local redefinitions takes precedence over the ones in other scopes.
mixedjuice (ECOOP 2002) they support classes extensions in Java but everything is static you cannot have different clients seen different behavior as in classboxes. So I have to recompile everything and everybody will see everything.
Again I'm not that this soemthing we want since it can make everything more complex (code navigation....)
Stef
If I become leader of this project, though, my first move will be to strike "future" from the title. While I want it to take us into the future, I want the design done in 3 months, and I want something working in 6 months.
Oh yes!!!!!!!!!!!!!!!! I am for it.
I feel that, once we have something that runs and that some people are happy with, it should be included in the image.
Alexandre
[Well, it got a bit long, but I figured I should try to articulate my own thoughts about Modules. I think it's nice at the beginning of the project to have a few "gestalts" laid out (and thanks especially for the others so far), so that there are some anchors for discussion and so that everyone can start to build up their own vision of what's powerful and what's possible.]
Modules in Squeak I may be wrong, but I feel it is possible to solve several problems at once with a good design for modules. I also feel that it is possible to keep the modularization nearly invisible to the casual user.
What problems should a module system solve? It should support the construction of a large system out of many small ones in such a way that the parts and their relationships can be easily managed.
Partitioning One motivation for this partitioning is to reduce size and with it complexity. For example by putting many Squeak subsystems out on SqueakMap, the kernel system can be kept small and relatively easy for a novice user to comprehend.
I think it is important to keep in mind the difference between logical and physical partitioning here. For instance, we might not be concerned with size, but only with apparent complexity, so as not to overwhelm novice users. In that case, notice that the SqueakMap partitioning could be merged with the browser in such a way that a chock-full demo image could look to the user like a no-frills basic image just by changing a filter setting. But neither the logical nor the physical partitioning is easy without a supporting architecture, nor possible without a basic paradigm.
Export Once it is possible to remove and add parts of a system from and to itself, it is natural to consider moving these parts between different systems (ie, images). Two important examples of this in our current world are the the ability to import a package on SqueakMap into a new release of the system, and the ability to export a project from one system and import it into another. These examples are also important in juxtaposition -- they use completely different mechanisms to deal with boundaries, references across boundaries, and format of the stored modules. Down the road there is reason to hope that these two patterns of use would use the same mechanism.
ImageSegments, Extensions, Install and Uninstall ImageSegments, as many of you know, are a slick piece of squeak technology that allows a well-bounded network of objects to be *very* quickly exported from or imported to, a Squeak image. Well-bounded, here, means not too many in-pointers (ideally only one), and not too many out-pointers. Several years ago, we showed that the entire VM construction category, then about 500k of code, could be imported in a tenth of a second using ImageSegments. However the fact that the system is not already modular has hindered taking real advantage of this technology.
A key issue in partitioning the system is that of "extensions". By these we typically mean, and I specifically mean here, changes to shared classes that are (or should be) local to a package. A typical example would be a String method 'asURL' that makes sure it is a well-formed URL. Such a method would have no place in a system that lacked network support, so it should be a part of the NetworkSupport package rather than part of the base system. This example is fairly compelling, but even seasoned Squeakers are hard-put to say whether some certain utility methods should be included in the base classes or not.
Install and Uninstall are the processes required to install a package into, or remove it from, a host system. Typically, installation registers a number of "entry points" or names in various global tables. Our fileIn process already does much of this through its interning of Symbols and handling of global and pool variables. Uninstalling involves reversing these changes and others more problematical, such as the extensions to classes outside the package.
A Couple of Desiderata Much of my recent thinking about modules has been about how to make install and uninstall trivial. I believe it is possible (and desirable) to make an architecture in which different modules can make conflicting extensions to system classes, and in which the *only* thing required to install a module is to read it in as an imageSegment, and the only thing required to uninstall it is to nil the (only) pointer to it. A related desideratum is that, having imported a squeak project, however complex, the *only* thing required is to remove all references to it, after which it will be reclaimed by GC, and no trace of it will remain in the image. When we build a window out of a rectangle, a border width and a color, we just store three pointers; why should it be any harder to combine three modules?
Isn't this just Packages? So far, this represents my thinking of about three years ago (except I was stumped about extensions at that time), and it really reads more like "Future Packages". So the first thing I want to say is that I am in no way critical of all the great ongoing work on packages, both in package design and the effort to split up the Squeak image. Even if we come up with a better architecture for encapsulating modules in memory (and I hope we do), all of the work done so far will be relevant and useful. Moreover, I believe it is the case that the better things are partitioned, the easier it is to move from one module architecture to another.
But Wait, There's More A big shift in my thinking since that time has come from learning about E (see http://www.ERights.org). E is an architecture for secure distributed computing than can be implemented in a number of languages. Its real strengths can only be realized in pointer-safe systems, though, which makes Squeak (potentially) a very attractive host system for this architecture. The essence of E's security is modularity done right. As Mark Miller says, "security is just extreme modularity".
E in less than a Nutshell [There is a fine paper called "E in a Nutshell" by Mark Stiegler http://www.skyhunter.com/marcs/ewalnut.html ] The E work is well thought out and profound. It surrounds a module that they refer to as a vat, which can only be accessed by capabilities. I am not deeply familiar with their implementation, but I think the best way to think of capabilities in Squeak is as a set of message selectors. If a module has its own name space for selectors, and if it is impossible to forge a pointer, then if you have a module pointer and a set of selectors, then the *only* access you can possibly have to that module is to send exactly those messages.
The profundity of the E work extends to distributed computing on multiple hosts with encryption for inter-host communication, and a process model called "promise pipelining" which is like an efficient lazy publish/subscribe. They deal with race conditions, unreliable links and the whole nine yards. Moreover I believe that their model of computation also covers most of the problems of leveraging multiple processors.
A Point of High Leverage I have no delusions that we can pull off an E system in Squeak in the next six months. However my interpretation of what a module is, namely a rigorous boundary in the runtime architecture, is exactly the point at which an architecture can be made to support the requirements of E (or not). The benefits of carrying out this particular piece of work in a manner that is consistent with the requirements of E could be tremendous. For instance, imagine a 64-bit Squeak with Croquet and an airtight security model. That is why E is required reading for this project.
- Dan
Dan Ingalls wrote on Fri, 25 Feb 2005 10:50:19 -0800
Modules in Squeak I may be wrong, but I feel it is possible to solve several problems at once with a good design for modules.
This is how I like to build systems as well. To cite you (page 22 of the "green book"):
# One way of stating the Smalltalk philosophy is to "choose a small number of # general principals and apply them uniformly." This approach has somewhat # of a recursive thrust, for it implies that once you've built something, you # ought to be using it whenever possible.
The parallel efforts of the various teams just created will tend to go against this, but I will be glad if the modules can be leveraged to help their projects without delaying them.
But back to "several problems at once", a trivial example is virtual memory. Squeak doesn't have it which isn't a problem on most systems on which it runs since they have their own paged virtual memory systems. But if you can quickly load and unload image segments, that can be used as an extremely simple virtual memory system which would come in handy in a PDA which can only access the large Flash memory indirectly, for example.
I also feel that it is possible to keep the modularization nearly invisible to the casual user.
True, and this is what I like about the Traits thing or Göran's namespaces - anyone who didn't have problems with the old system doesn't see them at all. It is only when you try to do something different where things would break down in plain Squeak that you have to learn the new ideas.
It isn't easy, however. The changes tend to leak out. For example: I am building two Smalltalk systems. The 16 bit one is a traditional closed one where things like "ClassX allInstances" make perfect sense. The 36 bit one is modular and as an open system that expression's result is less useful. In particular, you can never be sure that you have seen all the instances "out there". An outdated explanation of my "modules" can be found in http://www.merlintec.com:8080/software/8 if anyone is interested, but a lot of that might not be relevant for Squeak.
While I started seriously thinking about this in 1984 (I thought an image based system was fine in a lab, but wouldn't be as nice in a product) the only stuff in this area which I designed and was actually implemented and tested was an OO OS for the PC AT (286, for you young ones) in 1988.
One thing that I see as a problem in the way of properly partitioning Squeak is that we use a single mechanism to deal with different kinds of coupling between objects. Fixing that would require changes visible to the users, so this is probably not the way to go for this project but I'll explain it anyway. As Alan Cox likes to point out, in the real world you have objects that are soldered together, others that are bolted together, that are wired together with connectors that are easy to snap on/snap off and so on. But when I do
inst := MyHelper new.
and distribute this as text to be recompiled in another user's system, the coupling is looser than it should be. Their environment is different and the text could be ambiguous. I know exactly what object I am talking about and having an intra-module pointer to it would be just right. The other user shouldn't be bothered by the name I used at all, as I think Craig Latta would agree.
On the other hand,
p := HTMLParser new.
is far too tight a coupling for my taste. Replacing the text with an inter-module pointer wouldn't really help. This kind of dependency might allow the module I used while developing to be autoloaded into the user's system when my module is, but is that what we want? As long as p is the right type (understands a given set of messages) the job will get done. If the user's system has a different, but compatible, parser then I would like to use that instead. Some global Protocol (or Interface or whatever) objects that acted as factories would be nice:
p := HTMLParsingProtocol makeInstance.
would check the loaded modules, fetch one according to the local preferences if none of the loaded ones has the needed class and finally would get the class to create a new instance for us. There is no reason why this shouldn't be written exactly as the previous expression (with a Protocol instance simply replacing the class and implementing #new), of course, and then any such change would be far less visible to the users.
In short: my suggestion is that checking all class references in the image (and packages) and separating those that should be tightly coupled from those that should be loose (by making the latter indirect) would make partitioning the image much easier.
About the modules themselves, I like binary ones like imageSegments better because they apply to all objects instead of just sources. My preference is also to refer to modules by some universal ID and then have a separate service to map that to a local file name or URL from where it can be loaded. This allows me to move and rename them without breaking stuff. For immutable modules a good ID is simply the hash of the contents. A sequence of immutable modules can simulate one mutable one (TeaTime?) and again I would rather have a separate service deal with this than hard coding it in the modules themselves.
The iAPX432 (Intel's first 32 bit processor, for you young ones) got me interested in capability based security. Having different people see different interfaces for the same object is the most flexible way to express capabilities. I can imagine more than one way to implement that and for all of them you would get the ability to put extensions to one module inside another and to load incompatible stuff simultanously "for free".
One more thing ;-) (sorry that this is so long already) - in a binary module system I would move the sources from external files into the image (not really - into a module). So I would have a "main" module that would include the class objects, the compiled methods and anything else that is needed at run time. This shouldn't be too big and would be the only thing that most users would ever load. A "source" module would have a set of string objects, which happen to be the sources for the stuff in "main", which would have inter-module pointers to the strings so trying to see the methods in a browser or debugger would automatically load "source". I would include at least a third module, "docs". This would have objects with far nicer documentation than just the sources, and I would include in that a set of test methods. This documentation would be more at a package level than method comments or class comments and could include animations and all kinds of neat stuff, including pointers to related objects in other modules. This would automatically be loaded when my actions in the browser implied "serious" use of the module.
-- Jecel
On 25 févr. 05, at 19:50, Dan Ingalls wrote:
[Well, it got a bit long, but I figured I should try to articulate my own thoughts about Modules. I think it's nice at the beginning of the project to have a few "gestalts" laid out (and thanks especially for the others so far), so that there are some anchors for discussion and so that everyone can start to build up their own vision of what's powerful and what's possible.]
Modules in Squeak I may be wrong, but I feel it is possible to solve several problems at once with a good design for modules. I also feel that it is possible to keep the modularization nearly invisible to the casual user.
yes i think that this requires to change tools but having import at the level of modules and not been able to access names outside explicitly can really give the impression that you code in a box = first class category and you import stuff from other boxes. So box with imageSegments could be fun to have.
What problems should a module system solve? It should support the construction of a large system out of many small ones in such a way that the parts and their relationships can be easily managed.
Partitioning One motivation for this partitioning is to reduce size and with it complexity. For example by putting many Squeak subsystems out on SqueakMap, the kernel system can be kept small and relatively easy for a novice user to comprehend.
I think it is important to keep in mind the difference between logical and physical partitioning here. For instance, we might not be concerned with size, but only with apparent complexity, so as not to overwhelm novice users. In that case, notice that the SqueakMap partitioning could be merged with the browser in such a way that a chock-full demo image could look to the user like a no-frills basic image just by changing a filter setting. But neither the logical nor the physical partitioning is easy without a supporting architecture, nor possible without a basic paradigm.
Export Once it is possible to remove and add parts of a system from and to itself, it is natural to consider moving these parts between different systems (ie, images). Two important examples of this in our current world are the the ability to import a package on SqueakMap into a new release of the system, and the ability to export a project from one system and import it into another. These examples are also important in juxtaposition -- they use completely different mechanisms to deal with boundaries, references across boundaries, and format of the stored modules. Down the road there is reason to hope that these two patterns of use would use the same mechanism.
ImageSegments, Extensions, Install and Uninstall ImageSegments, as many of you know, are a slick piece of squeak technology that allows a well-bounded network of objects to be *very* quickly exported from or imported to, a Squeak image. Well-bounded, here, means not too many in-pointers (ideally only one), and not too many out-pointers. Several years ago, we showed that the entire VM construction category, then about 500k of code, could be imported in a tenth of a second using ImageSegments. However the fact that the system is not already modular has hindered taking real advantage of this technology.
A key issue in partitioning the system is that of "extensions". By these we typically mean, and I specifically mean here, changes to shared classes that are (or should be) local to a package. A typical example would be a String method 'asURL' that makes sure it is a well-formed URL. Such a method would have no place in a system that lacked network support, so it should be a part of the NetworkSupport package rather than part of the base system. This example is fairly compelling, but even seasoned Squeakers are hard-put to say whether some certain utility methods should be included in the base classes or not.
;) Dan I think that at the occasion this would be nice not only to have method extension but state extension.
I recently reread a good pattern from kent beck about asStuff methods. I said that you do not want to have an explosion of the interface and that you should use asStuff when the method returns an object polymorphic to the receiver. Example OrderedCollection>>asSet But you should better use URL readFrom: aString instead of String asURL. And this is was like a ahah effect for me.
Install and Uninstall are the processes required to install a package into, or remove it from, a host system. Typically, installation registers a number of "entry points" or names in various global tables. Our fileIn process already does much of this through its interning of Symbols and handling of global and pool variables. Uninstalling involves reversing these changes and others more problematical, such as the extensions to classes outside the package.
A Couple of Desiderata Much of my recent thinking about modules has been about how to make install and uninstall trivial. I believe it is possible (and desirable) to make an architecture in which different modules can make conflicting extensions to system classes, and in which the *only* thing required to install a module is to read it in as an imageSegment, and the only thing required to uninstall it is to nil the (only) pointer to it. A related desideratum is that, having imported a squeak project, however complex, the *only* thing required is to remove all references to it, after which it will be reclaimed by GC, and no trace of it will remain in the image. When we build a window out of a rectangle, a border width and a color, we just store three pointers; why should it be any harder to combine three modules?
Isn't this just Packages? So far, this represents my thinking of about three years ago (except I was stumped about extensions at that time), and it really reads more like "Future Packages". So the first thing I want to say is that I am in no way critical of all the great ongoing work on packages, both in package design and the effort to split up the Squeak image. Even if we come up with a better architecture for encapsulating modules in memory (and I hope we do), all of the work done so far will be relevant and useful. Moreover, I believe it is the case that the better things are partitioned, the easier it is to move from one module architecture to another.
But Wait, There's More A big shift in my thinking since that time has come from learning about E (see http://www.ERights.org). E is an architecture for secure distributed computing than can be implemented in a number of languages. Its real strengths can only be realized in pointer-safe systems, though, which makes Squeak (potentially) a very attractive host system for this architecture. The essence of E's security is modularity done right. As Mark Miller says, "security is just extreme modularity".
E in less than a Nutshell [There is a fine paper called "E in a Nutshell" by Mark Stiegler http://www.skyhunter.com/marcs/ewalnut.html ] The E work is well thought out and profound. It surrounds a module that they refer to as a vat, which can only be accessed by capabilities. I am not deeply familiar with their implementation, but I think the best way to think of capabilities in Squeak is as a set of message selectors. If a module has its own name space for selectors, and if it is impossible to forge a pointer, then if you have a module pointer and a set of selectors, then the *only* access you can possibly have to that module is to send exactly those messages.
The profundity of the E work extends to distributed computing on multiple hosts with encryption for inter-host communication, and a process model called "promise pipelining" which is like an efficient lazy publish/subscribe. They deal with race conditions, unreliable links and the whole nine yards. Moreover I believe that their model of computation also covers most of the problems of leveraging multiple processors.
I have the impression dan that you should invent a term for these new modules because module is connoted This is why we invented classbox. So islands or safe could be a good candidate.
A Point of High Leverage I have no delusions that we can pull off an E system in Squeak in the next six months. However my interpretation of what a module is, namely a rigorous boundary in the runtime architecture, is exactly the point at which an architecture can be made to support the requirements of E (or not). The benefits of carrying out this particular piece of work in a manner that is consistent with the requirements of E could be tremendous. For instance, imagine a 64-bit Squeak with Croquet and an airtight security model. That is why E is required reading for this project.
- Dan
stéphane ducasse ducasse@iam.unibe.ch wrote...
I have the impression dan that you should invent a term for these new modules because module is connoted This is why we invented classbox. So islands or safe could be a good candidate.
Yes, you are right. However during this early phase, I *want* it to be ambiguous and overlap with all the other connotations. Is this a package mechanism or a security mechanism? Is it an interface to imageSegments, or a name space solution? Yes! I want us to be thinking of all these things because I believe there is a place where they all come together.
When we have finished compiling the little summaries of these related areas, *then* we come to discuss what substrate would give us the most leverage for all these areas. Then we can decide what is the layer for packages, and let the package folks concentrate on that. Then we can decide what is needed for security in the VM and let VM people focus on that. Then we can decide what is the interface to ImageSegments, and let someone work on that. And the kernel design that remains at the center? When we see what that is, then we can give it a good name.
But for now, I want us all to be thinking that this is *everything* anyone ever thought of calling a module because, when we are done, I want it to fit well with every aspect of modularity in Squeak.
- Dan
Dan Ingalls wrote: <lots of good stuff snipped>
Dan, one thing that I find missing in this list is configuration management: code gets changed. So there is the issue of how to coordinate changes that different people make. One way to handle that is to partition the system, in order to distribute "ownership" and reduce conflicts. Also, I want to be sure that when I load a module, I get the same thing I got last time, independent of what is in the image before loading. And I want to be able to keep the instances while loading a different version of a module, so loading a new version is different than uninstall old, install new.
And one thought about the complexity/filtering issue: Even though the tools may be able to filter the view of the system, the code is nonetheless there and may affect the behavior of the system. So filtering may lead to "what you see is not what you get", so to say.
Just my 2c, Thanks Wolfgang
Wolfgang Eder edw@generalmagic.at wrote...
one thing that I find missing in this list is configuration management: code gets changed. So there is the issue of how to coordinate changes that different people make. One way to handle that is to partition the system, in order to distribute "ownership" and reduce conflicts. Also, I want to be sure that when I load a module, I get the same thing I got last time, independent of what is in the image before loading. And I want to be able to keep the instances while loading a different version of a module, so loading a new version is different than uninstall old, install new.
OK. Keep that thought, and we'll discuss it down the road a piece.
And one thought about the complexity/filtering issue: Even though the tools may be able to filter the view of the system, the code is nonetheless there and may affect the behavior of the system. So filtering may lead to "what you see is not what you get", so to say.
This is *exactly* the thought I was trying to inspire by comparing all the packages out on SqueakMap with all of them in memory with filters in the browser. Why do we not have a problem when they are not resident? Why do we have a problem when they are resident? At exactly what point does the problem occur? Heh, heh. Is it possible to work the architecture a bit better so that the problem never occurs? I hope so.
- Dan
Dan Ingalls wrote:
[Well, it got a bit long, but I figured I should try to articulate my own thoughts about Modules. I think it's nice at the beginning of the project to have a few "gestalts" laid out (and thanks especially for the others so far), so that there are some anchors for discussion and so that everyone can start to build up their own vision of what's powerful and what's possible.]
Modules in Squeak I may be wrong, but I feel it is possible to solve several problems at once with a good design for modules. I also feel that it is possible to keep the modularization nearly invisible to the casual user.
What problems should a module system solve? It should support the construction of a large system out of many small ones in such a way that the parts and their relationships can be easily managed.
Partitioning One motivation for this partitioning is to reduce size and with it complexity. For example by putting many Squeak subsystems out on SqueakMap, the kernel system can be kept small and relatively easy for a novice user to comprehend.
I think it is important to keep in mind the difference between logical and physical partitioning here. For instance, we might not be concerned with size, but only with apparent complexity, so as not to overwhelm novice users. In that case, notice that the SqueakMap partitioning could be merged with the browser in such a way that a chock-full demo image could look to the user like a no-frills basic image just by changing a filter setting. But neither the logical nor the physical partitioning is easy without a supporting architecture, nor possible without a basic paradigm.
Export Once it is possible to remove and add parts of a system from and to itself, it is natural to consider moving these parts between different systems (ie, images). Two important examples of this in our current world are the the ability to import a package on SqueakMap into a new release of the system, and the ability to export a project from one system and import it into another. These examples are also important in juxtaposition -- they use completely different mechanisms to deal with boundaries, references across boundaries, and format of the stored modules. Down the road there is reason to hope that these two patterns of use would use the same mechanism.
ImageSegments, Extensions, Install and Uninstall ImageSegments, as many of you know, are a slick piece of squeak technology that allows a well-bounded network of objects to be *very* quickly exported from or imported to, a Squeak image. Well-bounded, here, means not too many in-pointers (ideally only one), and not too many out-pointers. Several years ago, we showed that the entire VM construction category, then about 500k of code, could be imported in a tenth of a second using ImageSegments. However the fact that the system is not already modular has hindered taking real advantage of this technology.
A key issue in partitioning the system is that of "extensions". By these we typically mean, and I specifically mean here, changes to shared classes that are (or should be) local to a package. A typical example would be a String method 'asURL' that makes sure it is a well-formed URL. Such a method would have no place in a system that lacked network support, so it should be a part of the NetworkSupport package rather than part of the base system. This example is fairly compelling, but even seasoned Squeakers are hard-put to say whether some certain utility methods should be included in the base classes or not.
Just a minor nit related to this key issue. It appears to me that you use the word "extension" in its typical meaning: an addition of methods to shared classes, not changes in general. But the same characteristic (changes that are or should be local to a package) applies to a related notion, "overrides" in VW speak. Overrides are needed when you need to modify an already existing method or class definition. Envy for example does not allow them, so when you need a hook in some base class, although you have the notion and support for extensions, and everything else is nicely encapsulated in its own package, you still have to install your hook directly in the other, shared package, modifying it. This has always seemed unclean or incomplete to me, so I welcomed the support for overrides in VW/Store. Plus, if you have overrides, when you unload or uninstall your module, the hook is gone with it. I think it is also important to have this kind of "closure" for our modules from other perspectives, like security: it is a whole module that you trust or not, so if something that conceptually belongs to it lives in a different module, it will have also have different trust characteristics.
Install and Uninstall are the processes required to install a package into, or remove it from, a host system. Typically, installation registers a number of "entry points" or names in various global tables. Our fileIn process already does much of this through its interning of Symbols and handling of global and pool variables. Uninstalling involves reversing these changes and others more problematical, such as the extensions to classes outside the package.
A Couple of Desiderata Much of my recent thinking about modules has been about how to make install and uninstall trivial. I believe it is possible (and desirable) to make an architecture in which different modules can make conflicting extensions to system classes, and in which the *only* thing required to install a module is to read it in as an imageSegment, and the only thing required to uninstall it is to nil the (only) pointer to it. A related desideratum is that, having imported a squeak project, however complex, the *only* thing required is to remove all references to it, after which it will be reclaimed by GC, and no trace of it will remain in the image. When we build a window out of a rectangle, a border width and a color, we just store three pointers; why should it be any harder to combine three modules?
Isn't this just Packages? So far, this represents my thinking of about three years ago (except I was stumped about extensions at that time), and it really reads more like "Future Packages". So the first thing I want to say is that I am in no way critical of all the great ongoing work on packages, both in package design and the effort to split up the Squeak image. Even if we come up with a better architecture for encapsulating modules in memory (and I hope we do), all of the work done so far will be relevant and useful. Moreover, I believe it is the case that the better things are partitioned, the easier it is to move from one module architecture to another.
But Wait, There's More A big shift in my thinking since that time has come from learning about E (see http://www.ERights.org). E is an architecture for secure distributed computing than can be implemented in a number of languages. Its real strengths can only be realized in pointer-safe systems, though, which makes Squeak (potentially) a very attractive host system for this architecture. The essence of E's security is modularity done right. As Mark Miller says, "security is just extreme modularity".
E in less than a Nutshell [There is a fine paper called "E in a Nutshell" by Mark Stiegler http://www.skyhunter.com/marcs/ewalnut.html ] The E work is well thought out and profound. It surrounds a module that they refer to as a vat, which can only be accessed by capabilities. I am not deeply familiar with their implementation, but I think the best way to think of capabilities in Squeak is as a set of message selectors. If a module has its own name space for selectors, and if it is impossible to forge a pointer, then if you have a module pointer and a set of selectors, then the *only* access you can possibly have to that module is to send exactly those messages.
The profundity of the E work extends to distributed computing on multiple hosts with encryption for inter-host communication, and a process model called "promise pipelining" which is like an efficient lazy publish/subscribe. They deal with race conditions, unreliable links and the whole nine yards. Moreover I believe that their model of computation also covers most of the problems of leveraging multiple processors.
A Point of High Leverage I have no delusions that we can pull off an E system in Squeak in the next six months. However my interpretation of what a module is, namely a rigorous boundary in the runtime architecture, is exactly the point at which an architecture can be made to support the requirements of E (or not). The benefits of carrying out this particular piece of work in a manner that is consistent with the requirements of E could be tremendous. For instance, imagine a 64-bit Squeak with Croquet and an airtight security model. That is why E is required reading for this project.
- Dan
On Tue, 2005-02-22 at 14:36 +0100, goran.krampe@bluefish.se wrote:
Hi all!
Yesterday we presented the Team model and now we want to follow that up with proposing two quite important Teams.
As you have seen Ken Causey has already picked up the ball and is proposing a Team for tackling one of our biggest challenges - Harvesting. Both in the short and long term. Perfect.
I just want to clarify something regarding this. At least in the short term the team I'm forming (called 'Janitors' for now, send email to janitors-subscribe@discuss.squeakfoundation.org to express preliminary interest in joining the team) is going to be very narrowly focused and will not be replacing or supplanting any existing groups including the Harvesters. Instead we will be working to manually smooth the harvesting path and keep the flow going. Of course longer term goals are much more nebulous at this point. More about this soon.
Ken
On Tue, 22 Feb 2005 15:34:29 -0600, "Ken Causey" ken@kencausey.com said:
On Tue, 2005-02-22 at 14:36 +0100, goran.krampe@bluefish.se wrote: ... As you have seen Ken Causey has already picked up the ball and is proposing a Team for tackling one of our biggest challenges - Harvesting. Both in the short and long term. Perfect.
I just want to clarify something regarding this. At least in the short term the team I'm forming (called 'Janitors' for now, send email to janitors-subscribe@discuss.squeakfoundation.org to express preliminary interest in joining the team) is going to be very narrowly focused and will not be replacing or supplanting any existing groups including the Harvesters. Instead we will be working to manually smooth the harvesting path and keep the flow going. Of course longer term goals are much more nebulous at this point. More about this soon.
Understood... the Harvesters are more or less whoever has access to the update stream, the new group wouldn't replace them. But I think the new group would have a lot to say about improving the harvesting process in general. And I assume bug/fix/enh tracking is one of the main focuses of the new group.
- Doug
On Tuesday 22 February 2005 5:36 am, goran.krampe@bluefish.se wrote:
The overall theme of this Team is to evolve the technology we already have and to make working solutions for the current and near term Squeak releases. And of course to make it all fit together. Without me knowing if people are interested I can *at least* see the following names in this Team:
Doug Way, Lex Spoon, Göran Krampe, Avi Bryant, Alexandre Bergel, Ned Konz
I am not going to venture into talking about who should be Team leader here - I have a favorite though and its not me ;). But I am definitely signing up as a very active member.
Can I help?
Ned Konz ned@squeakland.org wrote:
On Tuesday 22 February 2005 5:36 am, goran.krampe@bluefish.se wrote:
The overall theme of this Team is to evolve the technology we already have and to make working solutions for the current and near term Squeak releases. And of course to make it all fit together. Without me knowing if people are interested I can *at least* see the following names in this Team:
Doug Way, Lex Spoon, Göran Krampe, Avi Bryant, Alexandre Bergel, Ned Konz
I am not going to venture into talking about who should be Team leader here - I have a favorite though and its not me ;). But I am definitely signing up as a very active member.
Can I help?
Of course. :) We haven't created the list yet - because the moderator should be the Team leader. And the Team leader question is still open. Interested?
In my book you or Doug would fit best.
regards, Göran
squeak-dev@lists.squeakfoundation.org