Hi Dave,
it is now one-two more hours here, until Happy New Year, but this year we are doing the child care and so I have some time for addressing your concerns: the kids went to bed already ;)
On Wed, 31 Dec 2008 18:41:15 +0100, you wrote:
Squeakers, Squeakians, Squeakites, and Squeakods,
Allow me to mark the end of 2008 by sharing with you a little story and a wish list for 2010. [and before anyone asks, yes I have free time and will help] Also allow me to admit that I am a bad Squeak user, in that I abuse the hell out of the system. I come from a school of programming that believes only in the reality of the machine itself, and often feel that computer scientists should never be allowed near a computer, let alone allowed to program one. :)
The (your) "typical" computer scientist is often not a good programmer and the converse also holds often; what's wrong with people having between min and max of talent in their fingers?
That said, last month I was playing around with prototyping an application in Squeak, in a fumbly experimental way, poking around in dark corners most people never bother with. And it struck me that most of the problems that I was having, that Craig Latta tries to address in Spoon, and that causes forks like Pharo, all stem from a fundamental misconception Squeakers have about how their environment should work.
I agree to some degree; too many "mistakes" made it into the "monolithic" .image and people who could clean up, the mess they produced themselves for the long run, are often no longer around.
Sort of, life happens.
And much of the issue stems from my favorite whipping boy, the Smalltalk system dictionary object.
Oh how I love to abuse that object! I just love "polluting" the global namespace of my images with all sorts of often used objects and doodads. Take your favorite object and just jam it in there, and poof! You can start inserting it all over the place, and like magick you stop having to pass along useless arguments.
The protocol for adding new things at the global level is very small and centered precisely: one can easily put an interceptor at work and (at least) record (FBiSmalltalk at: Time now put: evidence) or so.
If you play this game long enough, you quickly get to a place were few if any selectors actually take any arguments!
And classes, now there's where you can have real fun. I like writting methods that programatically generate classes, and then mutate them based upon real world results. Why use a variable that stays largely static for long periods of time, when you can just recompile the method on the off chance it changes? Need a one of a kind one off object? Just abuse the entire notion of class, and make a classy-object and pop it in Smalltalk for good measure.
Let's look at the style of some the code that I've been writing:
myMethod := 'someObject someMethod' compile. myClass addMethod: myMethod named: #goDoSomething. myObject := myClass clone. myObject addMethod: 'someOtherPrototype doSomethingElse' compile name: #goDoSomething.
And as you can see, I'm doing some pretty nasty stuff. I've extended ByteString and ByteArray to invoke the compiler on their contents and produce CompiledMethods, and then wrapped a bit of Behavior's stuff and stuck it in Object because I feel the distinction between classes and objects are counter productive.
No need to go soo deep as if it where a C system. In project NewCompiler it has a BytecodeGenerator "I generate bytecodes in response to 'instructions' messages being sent to me" which also helps to avoid many mistakes.
So why in the world would someone start smashing apart the whole Smalltalk OO world by abusing globals, dumping classes, and start writing self-modifying code in Smalltalk? Well partially because we should be able to, but more importantly because Smalltalk code should be able to. If you're writing Smalltalk that writes Smalltalk, development processes that serve a human well, do not necessarily serve a program well. Especially, when to that software bits are bits, and has no real understanding of a distinction between source, code, and data.
There is a whole project dedicated to "self-modifying code in Smalltalk", over at
- http://code.google.com/p/moebius-st/
I'm sure the Moebius team and you would benefit from each other, towards self-modifying code in Smalltalk.
The experiences and experiments that I've been conducting over the past year, all seem to point to some fundamental weaknesses in the overall system architecture:
1.) Primatives can not be added to the VM on the fly, by just assembling some code and running it.
... that's part of the Moebius project.
2.) The global namespace (aka Smalltalk) is static, but not hyper-static (where in all top level name changes only affect code going forward)
... this seems to be addressed elegantly with NewSpeak (for which we still have to wait until it gets released) but, Igor+my have also worked out concepts which address "local" "globality" at the runtime and no longer at the compile-time level (could work in classic Squeak as well -- a small compiler change and an even smaller bug fix in the stock VM would be all that is needed).
But what's missing is that you talk about the hands which then have to refactor all the until-now existing source code? in order to benefit from such an isolation level?
3.) Class and Trait names are global variables. (which is doubly a problem when you consider #2)
Right. Yes. Damn'it yes (!)
4.) Objects and processes all live within one space, but are often unreasonably hard to reference (anonymous)
This is perhaps a little bit OT but *any* solution to 4) seems to require distributed GC/alloc to work acceptable on *one* CPU, which we haven't seen yet. This is perhaps analogous to the "I" in AI ;)
5.) The operation of one conceptually distinct group of objects (say an application), can easily interfere with the operating of a second conceptually distinct group of object (say the system).
I'm sure you had something in mind, other than this: the app level irrefutably calls the system level, or: the client level irrefutably calls the service level. but what?
6.) It is difficult / impossible to share a single object, between two distinct implementations of the system, without resorting to simulating the system within itself (which is itself a simulation of the system within itself).
Really? and I always thought that, for example a model can be seen+shared by completely different viewers. What would be the problem that you are concerned with here?
Now I admit, these weaknesses may seem a bit odd or even objectionable, and many system level squeakers are going to claim that if you look at the system from a different perspective they may not even exist. But from my POV, what this boils down to is:
1.) I can't test new hardware in a running squeak image
... this must be a mistake, since hardware doesn't run in software (other that in a simulator). if I where a piece of hardware I would object to running in a Smalltalk system ;)
2.) I can't load and unload applications, without affecting the underlying system
Right. In Squeak the developers often mixed up service level and client level, and so produced .images from which the app can no longer be unloaded, unfortunately. I once hoped that the Pharo folks would address this but they do: they throw things out manually (no offense!).
But isn't this a typical "for me it works" scenario? ;)
(OT*0.5) can't we always go back to earlier releases of Squeak in which the not unlodable did not exist?
3.) I can't load and unload multiple versions of a "module" without substantial hackery
Right, and good luck anyways.
4.) I can't run 2 of the same process at once, and be sure that they don't affect eachother's internal state
Why not? what would be wrong with Semaphore and friends?
5.) I can't extend the interface procedurally, without risking breaking my debugger!
... the "interface procedurally" doesn't parse here. sorry.
6.) I can't test multiple versions of my code base against the same set of live objects!
Right. I therefore proposed to have multiple method dictionaries per class in development .images and for production to consolidate the multiple md's again (for example in your case one md per version). sent several emails to several people but they where not interested (perhaps they want "their" "own" solution to that; really, I don't know).
The project that I'm working on does involved embedded systems, robotics, AI, and evolutionary programming. Most of the road blocks I've run into in Squeak, are a product of what is clearly early design trade offs made by people who never imagined one would want to do half the things I'd like to do with their toy.
And if predicting the future where so easy as you seemingly describe it here, why would they care about your needs? did you send them your proposals and requirements?
These problems, however, also effect every other developer out there who is trying to write more conventional applications.
What I really want is a way to treat ImageSegments as their own physically distinct machine, with their own attached Process list and Smalltalk.
Aha. now *you* are proposing a "one thing does it all" solution. doesn't that make you automagically belong to "people who never imagined one would want to do half the things that other people would like to do with that toy" ? and if not, please tell us why not.
The top level Smalltalk System Dictionary, would not be a global dictionary, but a hyper-static dictionary, like the Forth dictionary or the ML top level namespace, where you can multiply redefine globals in effect producing multiple contexts within that global space (hence order of loading mattering). And then, I'd like it if these ImageSegments using either their out pointers mechanism or something else could reference eachother's objects, and send each other messages, but with some ability to easily name those out pointers in the local context. Finally, I'd really love it if the VM had a simple primative that simply let you invoked to content of any ByteArray as native code.
I'm suprised you don't use one of those which available for Squeak. did you check Exupery which precisely does that? or perhaps Huemul? both are based on and do "it" for Squeak.
(This would also have the interesting side effect that you could eventually rewrite the entire low level VM entirely in live objects inside Squeak using Squeak, dumping Slang).
Yes. see the Moebius project which I mentioned above.
Obviously, I'm not saying Squeak sucks, or that the problems require rewriting Squeak in its entirety.
No, right. but incrementally ;)
I think the problem is one of familiarity. We as programmers are far too familiar with the warts in our system, and stop looking at them for what they are. Right now, Squeak is a lot like DOS.
No. it simply isn't. it's perhaps the way and portions of Squeak that you are using that makes you believe the DOS religion. I for example don't use comparable-to-DOS things when I use Squeak as developer and/or user.
You can do one thing at a time, and every project requires a whole new application.
I didn't see anything else than this association during the last 30 years or so. yes, an application is a project.
What we need is a Squeak that is a lot more like a Mach microkernel, capable of keeping lots of balls in the air at once, and easily extensible in "user space".
I'm neutral to that microkernel "one thing does it all" solution. and I know that Huemul did it and has it.
But I believe that Squeak needs even more succesfull applications, "besides" Etoys, Croquet, Plop, CMSbox, Sophie, Seaside+friends, Scratch, and so.
Anywhoo, that's my two last cents for 2008.
Best wishes for 2009,
Thanks, and same to you :-)
/Klaus
Dave