Colin,
Consider the case where there are dependencies between the modules and more than one version of each module. The current effort of implementing modules is attempting to deal with these issues, and it is a non-trivial problem. It is also possible that because of certain interdependencies between different versions of different modules that some modules may become mutually incompatible. This situation is witnessed with MS Windows .DLL libraries quite often.
One common effect is that some applications have to be installed on a Windows machine in a particular order in order for them to all function properly on the same machine. There can arise instances where because of different sets of .DLL versions required, a set of applications may not all function correctly on one machine.
And regarding SqueakMap and DVS, realize that these are based on the monolithic image model. They are more or less fancy user interfaces to manage filing in the various change sets (I know I'm oversimplifying a bit, but the point is still valid).
Of course, monolithic systems have the issue of shared namespaces which can make it difficult to make different packages play nice together in the same image, so like I said, the problem is non-trivial.
While it would certainly be possible to produce an MPEG player image, with only the items necessary to play MPEGs, and a MIDI image with only the items needed for MIDI processing, and so on, factoring out the common components can be difficult. This is especially true when some "applications" use different versions of a particular component and rely on some characteristic behavior. A hardware example of this is the 4046 phase locked loop IC. It is a fairly standard part, BUT the Motorola Version and the Philips versions are incompatible in that the external timing components require different values to get the same frequency out of the two parts. The same kind of thing can happen in software.
-Dean
-----Original Message----- From: Colin Putney [mailto:cputney@whistler.com] Sent: Wednesday, October 30, 2002 8:03 PM To: squeak-dev@lists.squeakfoundation.org Subject: Re: An uncomfortable question
Hi Dean,
I'm not sure that I understand your concern. Modularizing the image would be a very good thing in my view of the world. In fact, I think it's required in order to build the kind of task-specific system you prefer.
Modules would let us build custom images for specific purposes, rather than attempt to make the standard image flexible enough for any purpose. If I'm using Squeak for a web application, I don't need an MPEG player in the image. If I'm developing a MIDI application, I don't need Comanche.
With the monolithic image, we get bitten coming and going. Not *everything* can be in the standard image, so we frequently have to file in additional classes to get the functionality we need. With the current system this a hassle, much more difficult than it needs to be. On the other hand, the standard image has a bunch of stuff that won't get used most of the time, and removing it from the image is a more than a hassle, it's a nightmare, not to be attempted by mere mortals.
So by all means lets consider our goals carefully. I really like the scenario that's emerging from the interaction of SqueakMap and DVS. I hope that we can arrive at a very basic kernel in the image, with easy-to-install packages available from SqueakMap.
Cheers,
Colin
On Wednesday, October 30, 2002, at 01:46 PM, Swan, Dean wrote:
What 3.3a is trying to do with modules is essentially opening the same can of worms that has made MS Windows such a maintenance nightmare. Modules are (more or less) equivalent to .DLLs and may end up with all the associated issues. I don't know of any "good" solutions to these problems, but as attractive as the "idea" of modules is, I am personally content to stick with monolithic images for the forseeable future.
The whole concept of "software components" is IMO a fantasy propagated by media hype that ignores all the realities of real world engineering. In "real" hardware product designs, one often finds that re-usability is more of a structuring concept than a hard reality. I don't believe I've ever worked on a hardware design that didn't involve some kind of custom connector that is a "slight" variation of a "standard" connector, for example. (Just for context, I'm referring to everything I've worked on in the last 12 years or so).
I think the concept of a monolithic image constrains the reusability myth to realistic proportions, and makes implementation possible. Too much flexibility comes at a cost, which usually rears its ugly head in the form of increased complexity, higher resource requirements, and reduced preformance (relative to a less flexible implementation).
As an example of what I'm talking about, I've worked on a couple of different projects in the last few years where the end product has similar capabilities but one was implemented using a "flexible, reusable platform approach" and the other was implemented using a more task specific approach.
The flexible system has a source tree for the "platform" of about 5 gigabytes, generates a target image of about 170 megabytes, and needs two 300 MHz PowerPC 603e's and 512M of RAM to run. This system has hundreds of developers working on it.
The "task specific" system has a source tree of about 300 megabytes, generates a target image of about 1 megabyte, and runs on one 66 MHz PowerPC with 16M of RAM. This system had 4 developers working on it. The end functionalities of these two systems is *very* similar.
I'm not sure what my point is in all of this, but I think we should consider our goals carefully in relation to the questions that Andrew has raised. My preference is towards "lean and mean" rather than "incredibly flexible, general purpose (and suboptimal)".
Colin Putney Whistler.com
Hi Dean,
Let me drop in a note here. Modules are no DLLs - they are not shared runtime components. E.g., DLL-hell as we know it from Windows is due to the fact that no developer can foresee what DLLs any customer may have installed on his or her system at *runtime*. This is very different from detecting and handling problems at *development* time.
Cheers, - Andreas
-----Original Message----- From: squeak-dev-admin@lists.squeakfoundation.org [mailto:squeak-dev-admin@lists.squeakfoundation.org] On Behalf Of Swan, Dean Sent: Thursday, October 31, 2002 2:33 AM To: 'squeak-dev@lists.squeakfoundation.org' Subject: RE: An uncomfortable question
Colin,
Consider the case where there are dependencies between the modules and more than one version of each module. The current effort of implementing modules is attempting to deal with these issues, and it is a non-trivial problem. It is also possible that because of certain interdependencies between different versions of different modules that some modules may become mutually incompatible. This situation is witnessed with MS Windows .DLL libraries quite often.
One common effect is that some applications have to be installed on a Windows machine in a particular order in order for them to all function properly on the same machine. There can arise instances where because of different sets of .DLL versions required, a set of applications may not all function correctly on one machine.
And regarding SqueakMap and DVS, realize that these are based on the monolithic image model. They are more or less fancy user interfaces to manage filing in the various change sets (I know I'm oversimplifying a bit, but the point is still valid).
Of course, monolithic systems have the issue of shared namespaces which can make it difficult to make different packages play nice together in the same image, so like I said, the problem is non-trivial.
While it would certainly be possible to produce an MPEG player image, with only the items necessary to play MPEGs, and a MIDI image with only the items needed for MIDI processing, and so on, factoring out the common components can be difficult. This is especially true when some "applications" use different versions of a particular component and rely on some characteristic behavior. A hardware example of this is the 4046 phase locked loop IC. It is a fairly standard part, BUT the Motorola Version and the Philips versions are incompatible in that the external timing components require different values to get the same frequency out of the two parts. The same kind of thing can happen in software.
-Dean
-----Original Message----- From: Colin Putney [mailto:cputney@whistler.com] Sent: Wednesday, October 30, 2002 8:03 PM To: squeak-dev@lists.squeakfoundation.org Subject: Re: An uncomfortable question
Hi Dean,
I'm not sure that I understand your concern. Modularizing the image would be a very good thing in my view of the world. In fact, I think it's required in order to build the kind of task-specific system you prefer.
Modules would let us build custom images for specific purposes, rather than attempt to make the standard image flexible enough for any purpose. If I'm using Squeak for a web application, I don't need an MPEG player in the image. If I'm developing a MIDI application, I don't need Comanche.
With the monolithic image, we get bitten coming and going. Not *everything* can be in the standard image, so we frequently have to file in additional classes to get the functionality we need. With the current system this a hassle, much more difficult than it needs to be. On the other hand, the standard image has a bunch of stuff that won't get used most of the time, and removing it from the image is a more than a hassle, it's a nightmare, not to be attempted by mere mortals.
So by all means lets consider our goals carefully. I really like the scenario that's emerging from the interaction of SqueakMap and DVS. I hope that we can arrive at a very basic kernel in the image, with easy-to-install packages available from SqueakMap.
Cheers,
Colin
On Wednesday, October 30, 2002, at 01:46 PM, Swan, Dean wrote:
What 3.3a is trying to do with modules is essentially
opening the same
can of worms that has made MS Windows such a maintenance
nightmare.
Modules are (more or less) equivalent to .DLLs and may end
up with all
the associated issues. I don't know of any "good"
solutions to these
problems, but as attractive as the "idea" of modules is, I am personally content to stick with monolithic images for the
forseeable
future.
The whole concept of "software components" is IMO a fantasy
propagated
by media hype that ignores all the realities of real world engineering. In "real" hardware product designs, one often
finds that
re-usability is more of a structuring concept than a hard
reality. I
don't believe I've ever worked on a hardware design that didn't involve some kind of custom connector that is a "slight"
variation of
a "standard" connector, for example. (Just for context,
I'm referring
to everything I've worked on in the last 12 years or so).
I think the concept of a monolithic image constrains the
reusability
myth to realistic proportions, and makes implementation
possible. Too
much flexibility comes at a cost, which usually rears its
ugly head in
the form of increased complexity, higher resource requirements, and reduced preformance (relative to a less flexible implementation).
As an example of what I'm talking about, I've worked on a couple of different projects in the last few years where the end product has similar capabilities but one was implemented using a "flexible, reusable platform approach" and the other was implemented
using a more
task specific approach.
The flexible system has a source tree for the "platform" of about 5 gigabytes, generates a target image of about 170 megabytes,
and needs
two 300 MHz PowerPC 603e's and 512M of RAM to run. This system has hundreds of developers working on it.
The "task specific" system has a source tree of about 300
megabytes,
generates a target image of about 1 megabyte, and runs on
one 66 MHz
PowerPC with 16M of RAM. This system had 4 developers
working on it.
The end functionalities of these two systems is *very* similar.
I'm not sure what my point is in all of this, but I think we should consider our goals carefully in relation to the questions
that Andrew
has raised. My preference is towards "lean and mean" rather than "incredibly flexible, general purpose (and suboptimal)".
Colin Putney Whistler.com
Andreas Raab wrote:
Hi Dean,
Let me drop in a note here. Modules are no DLLs - they are not shared runtime components. E.g., DLL-hell as we know it from Windows is due to the fact that no developer can foresee what DLLs any customer may have installed on his or her system at *runtime*. This is very different from detecting and handling problems at *development* time.
Cheers,
- Andreas
I think it depends on the point of view:
If you look at the Squeak VM coupled with an image as runtime environment running different apps, then parts of the image - modules - may be viewn as some kind of DLLs...
If you start multiple Squeaks each running just one app (composed from whatever versions of whatever modules), your argument holds; and then the DLL hell is only outside (for Windows)...
Greetings,
Stephan
<...>
Hi Andreas,
Well...considering that the distinction between development time and runtime is a deliberately fuzzy concept in Smalltalk (or at least not a concept enforced by the environment), I think Modules share some similarities with DLLs. Take the example where someone is running Squeak as their operating system...forcing them to run different images (because you haven't solved the modularity issues) is the equivalent of having to dual boot your PC. I don't like solutions that rely on having multiple images available.
I wish we had DLL-hell in Squeak, but we aren't even that evolved. What we have now is Spaghetti-hell, which is even worse. But, I think we can come up with a solution that is much better than the DLL solution.
- Stephen
-----Original Message----- From: squeak-dev-admin@lists.squeakfoundation.org [mailto:squeak-dev-admin@lists.squeakfoundation.org] On Behalf Of Andreas Raab Sent: Wednesday, October 30, 2002 8:47 PM To: squeak-dev@lists.squeakfoundation.org Subject: RE: An uncomfortable question
Hi Dean,
Let me drop in a note here. Modules are no DLLs - they are not shared runtime components. E.g., DLL-hell as we know it from Windows is due to the fact that no developer can foresee what DLLs any customer may have installed on his or her system at *runtime*. This is very different from detecting and handling problems at *development* time.
Cheers,
- Andreas
Hi all!
"Swan, Dean" Dean_Swan@Mitel.COM wrote: [SNIP]
And regarding SqueakMap and DVS, realize that these are based on the monolithic image model. They are more or less fancy user interfaces to manage filing in the various change sets (I know I'm oversimplifying a bit, but the point is still valid).
Well... No, I think you are oversimplifying a bit too much. SqueakMap is a distributed catalog of packages that can be installed into a Squeak image. It is already supporting different package formats:
-.st, .cs, .st.gz, .cs.gz (Simple fileins as you mention) -.pr (Implemented but not released) -.sar (Neds new megaformat capable of anything, probably similar to debs, rpms) -.st DVS (Avi's DVS .st format which both can be filed in conventionally AND filed in using DVS thus making a package more or less upgradeable)
My point here is that SM doesn't just handle changesets but any package format. And when the package formats evolve further we will start seeing new ways to deal with them - like a controlled upgrade for example. Or uninstall.
I am not sure what you mean with "based on the monolithic image model" but IMHO SqueakMap together with DVS etc is changing this. As a small example, very soon Celeste can be cut out of the image and be published on SM - I think Daniel is itching to do that and we already have all the tools that it takes.
In fact SM itself is probably (this is the current proposal) the very first package that is NOT going into the image! Yep, eating my own dogfood. For the curious this means that when SM hits the update stream for 3.2 (we are hopefully talking days now) it will actually only be a little bootstrap script going in - not SM itself. What does this mean in practice?
It means that when you update 3.2 you will get a chance to install SM into it "by remote". As the user you will not see the difference but for us package developers it is a huge difference. The SM code that you now have in your image is NOT maintained using the regular update stream. It is maintained outside of the image. When I (being the maintainer of SM) integrate new fixes etc and decide to release a new version of SM I can do that independently and you will be able to upgrade (from within SM) SM itself when you decide to, also independently of other packages.
In short, packages published in SM (including SM itself) have their own maintainers, their own "updatestreams" and their own place of storage on the net - so they are completely independent of the image.
Of course, monolithic systems have the issue of shared namespaces which can make it difficult to make different packages play nice together in the same image, so like I said, the problem is non-trivial.
Yes, namespaces etc is cool and we will probably end up with it sometime. But we don't need to solve all problems at once - simple class name prefixes go a long way.
SqueakMap does work pretty nicely already - and we all know things are missing but we are working on it. The next big step for SM will probably be versions of packages and handling dependencies (in some way - there are different ways to do it) between them. Before that though I want to wrap up the extensions sent to me from Ned and Avi, smack down on a few silly bugs and get 1.0 out the door.
regards, Göran
GREAT. Continue continue continue...
On jeudi, octobre 31, 2002, at 09:41 am, goran.hultgren@bluefish.se wrote:
Hi all!
"Swan, Dean" Dean_Swan@Mitel.COM wrote: [SNIP]
And regarding SqueakMap and DVS, realize that these are based on the monolithic image model. They are more or less fancy user interfaces to manage filing in the various change sets (I know I'm oversimplifying a bit, but the point is still valid).
Well... No, I think you are oversimplifying a bit too much. SqueakMap is a distributed catalog of packages that can be installed into a Squeak image. It is already supporting different package formats:
-.st, .cs, .st.gz, .cs.gz (Simple fileins as you mention) -.pr (Implemented but not released) -.sar (Neds new megaformat capable of anything, probably similar to debs, rpms) -.st DVS (Avi's DVS .st format which both can be filed in conventionally AND filed in using DVS thus making a package more or less upgradeable)
My point here is that SM doesn't just handle changesets but any package format. And when the package formats evolve further we will start seeing new ways to deal with them - like a controlled upgrade for example. Or uninstall.
I am not sure what you mean with "based on the monolithic image model" but IMHO SqueakMap together with DVS etc is changing this. As a small example, very soon Celeste can be cut out of the image and be published on SM - I think Daniel is itching to do that and we already have all the tools that it takes.
In fact SM itself is probably (this is the current proposal) the very first package that is NOT going into the image! Yep, eating my own dogfood. For the curious this means that when SM hits the update stream for 3.2 (we are hopefully talking days now) it will actually only be a little bootstrap script going in - not SM itself. What does this mean in practice?
It means that when you update 3.2 you will get a chance to install SM into it "by remote". As the user you will not see the difference but for us package developers it is a huge difference. The SM code that you now have in your image is NOT maintained using the regular update stream. It is maintained outside of the image. When I (being the maintainer of SM) integrate new fixes etc and decide to release a new version of SM I can do that independently and you will be able to upgrade (from within SM) SM itself when you decide to, also independently of other packages.
In short, packages published in SM (including SM itself) have their own maintainers, their own "updatestreams" and their own place of storage on the net - so they are completely independent of the image.
Of course, monolithic systems have the issue of shared namespaces which can make it difficult to make different packages play nice together in the same image, so like I said, the problem is non-trivial.
Yes, namespaces etc is cool and we will probably end up with it sometime. But we don't need to solve all problems at once - simple class name prefixes go a long way.
SqueakMap does work pretty nicely already - and we all know things are missing but we are working on it. The next big step for SM will probably be versions of packages and handling dependencies (in some way - there are different ways to do it) between them. Before that though I want to wrap up the extensions sent to me from Ned and Avi, smack down on a few silly bugs and get 1.0 out the door.
regards, Gˆran
Dr. Stéphane DUCASSE (ducasse@iam.unibe.ch) http://www.iam.unibe.ch/~ducasse/ "if you knew today was your last day on earth, what would you do different? ... especially if, by doing something different, today might not be your last day on earth" Calvin&Hobbes
On Wednesday, October 30, 2002, at 05:33 PM, Swan, Dean wrote:
Colin,
Consider the case where there are dependencies between the modules and more than one version of each module. The current effort of implementing modules is attempting to deal with these issues, and it is a non-trivial problem. It is also possible that because of certain interdependencies between different versions of different modules that some modules may become mutually incompatible. This situation is witnessed with MS Windows .DLL libraries quite often.
One common effect is that some applications have to be installed on a Windows machine in a particular order in order for them to all function properly on the same machine. There can arise instances where because of different sets of .DLL versions required, a set of applications may not all function correctly on one machine.
Well sure, incompatibilities between different parts of the system is always a danger. Certain parts of the system expect particular behaviour from other parts of the system. If you make changes you run the risk of violating those expectations. This is true whether those parts are modules, classes, functions or DLLs.
The nice thing about Squeak is that you can have multiple images lying around. So if you need different versions of a module in different situations, you do that without having to say, swap out your hard drive when switching between applications.
Now, this is not to say that dependency tracking, versioning and so on is a non-issue. It is a problem that we're going to have to address. But the ability to build images for specific purposes is a mitigating factor. So when we do have incompatibilities between modules - and it will happen - instead of "hell" we're faced with a mere problem.
And regarding SqueakMap and DVS, realize that these are based on the monolithic image model. They are more or less fancy user interfaces to manage filing in the various change sets (I know I'm oversimplifying a bit, but the point is still valid).
Hmm. It sounds like we have different definitions of "monolithic." When I speak of modularizing the image I mean grouping the code into clusters of classes and methods that a) provide a distinct and coherent piece of functionality, and b) interact with other clusters through well defined interfaces. Once that's accomplished (and yes, it's a non-trivial task) many of the clusters can be removed from the base image and maintained separately.
I place that scenario opposite the "monolithic" system used by Squeak 3.2. Everything is really intertwined and you can't easily remove any functionality from the image. Adding functionality is easier, but still quite a hassle because of the some what artificial distinction between "blessed" code that's maintained by SqC and other stuff, which is floating around in ChangeSets.
So again, I don't think I understand your concern. Are you saying that you prefer the SM/DVS route to 3.3 modules? How do you define "modular" and "monolithic?"
As to SqueakMap and DVS, sure. They really are just tools to make it easier to move code in and out of images. But that's the point! The whole problem here is that it's difficult to manage code.
Of course, monolithic systems have the issue of shared namespaces which can make it difficult to make different packages play nice together in the same image, so like I said, the problem is > non-trivial.
While it would certainly be possible to produce an MPEG player image, with only the items necessary to play MPEGs, and a MIDI image with only the items needed for MIDI processing, and so on, factoring out the common components can be difficult. This is especially true when some "applications" use different versions of a particular component and rely on some characteristic behavior. A hardware example of this is the 4046 phase locked loop IC. It is a fairly standard part, BUT the Motorola Version and the Philips versions are incompatible in that the external timing components require different values to get the same frequency out of the two parts. The same kind of thing can happen in software.
No, it's not trivial, but it can be solved, and it's not even all that difficult.
For one thing, the package model used by DVS makes the problem easier by virtue of its inflexibility. A given package can contain classes and methods. When a package is loaded, it can create classes and it can add methods to existing classes. It cannot modify methods belonging to another package, and it can't delete methods. So it avoids all kinds of problems that can arise when modules can modify each other.
A second factor that makes the problem tractable is that Squeak images offer developers much more control over the execution environment than, say, a desktop operating system. For development, you can build an image that has exactly the configuration you need - all the right packages, of the right versions, loaded in the correct order. For deployment, you just distribute that image. There's no need to have users load your package into whatever randomly configured system they happen to be running. That's the situation that creates DLL hell and it just doesn't apply to Squeak.
And finally, if you do run into a compatibility problem, just fix it! The source code is there, there's a community of people willing to help, and with SqueakMap and DVS, you've got the tools to easily distribute your fix to the rest of the community and to easily benefit from their efforts. As Daniel mentioned, we've got a social advantage over the commercial world.
Cheers,
Colin
squeak-dev@lists.squeakfoundation.org