Hi,

This is a *really* strange Smalltalk, but interesting enough to see it.

Thanks you the news,

Diego Gomez Deck

Hey all,

I have just got an email from Gilad Bracha who announced that Animorphic ST is now available to download at:

http://www.cs.ucsb.edu/projects/strongtalk

Animrophic ST (also called Strongtalk) is a high-performance ST implementation with a type system that is both optional and incremental. It has been developed by a small company in the mid 90s before the company got aquired by SUN to work on the Java VM.

Besides the code itself, the webpage also contains the "History" of this ST implmentation, which is pretty interesting.

Cheers, Nathanael

On Monday 01 July 2002 18:41, Diego Gomez Deck wrote:

This is a *really* strange Smalltalk, but interesting enough to see it.

Indeed! And reading their Mix-ins paper it seems pretty close to what Anthony Hannan proposed in his June 14 message "[VI4] Multiple Behaviors/Roles proposal", so he might enjoy it more than the rest of us (though it is only for Windows).

-- Jecel P.S.: how is it that the Sun people are finding the time to get neat old stuff out the door? Am I to suppose that Java is now 100% done? ;-)

Hello, Edmund

On Tue, 2 Jul 2002 09:42:37 +0200 (CEST) Edmund Ronald eronald@cmapx.polytechnique.fr wrote:

If there are some Squeakers in Tokyo, could they contact me by email ? I would love to have a live chat with at least ONE other Squeak user, once in my life :)

Do you call me? :-)

Please check "Who's Where" at minnow. http://minnow.cc.gatech.edu/squeak/1302

There are two major Smalltalk mailing list in Japan.

Smalltalkers' Salon Mailing List http://www.sra.co.jp/smalltalk/SML/

Squeak-J ML for Beginner http://form.easyml.com/easyml/squeak-c8.php3

And you can find many Japanese Squeakers by Google.

Regards, Kazuhiro ABE

Hi Dan,

[snip]

Some questions are:

    Would the system benefit from being cast into StrongTalk?
            and how much work would this be?

Some type of typed system lead us to some sort of multiple inheritance. The types could be:

1) ala C++: Multiple inheritance of implementation and protocol -> no comments. 2) ala Java / Objective C: Multiple inheritance of protocol. 2) ala StrongTalk: Multiple inheritance but based on mixins.

IMHO, the benefits (if any [*]) don't pay the complexity.

[*] I'm not a very old smalltalker, but I have some years of experience and never the type-less system was a problem to me.... on the contrary, I feel that the type-less system helps in the process of development.

    Would anyone care if it ran 10 times faster?
            and how much work would this be?

Yes... of course! I use Squeak (for example) to simulate complex systems and the extra performance is ever welcome.

    Would it be fun to do?

I don't know, you tell me :-)

Comments?

    - Dan

Cheers,

Diego Gomez Deck

Diego Gomez Deck DiegoGomezDeck@ConsultAr.com wrote:

Some type of typed system lead us to some sort of multiple inheritance. The types could be:

1. ala C++: Multiple inheritance of implementation and protocol -> no comments.
2. ala Java / Objective C: Multiple inheritance of protocol.
3. ala StrongTalk: Multiple inheritance but based on mixins.

We can find better starting points than these! Here are the two main ones to consider, IMHO:

- SmallScript. This is a recently-designed Smalltalk with optional typing and is well worth looking at. How much more relevant could you get?

- OCaml. This is an OO language with required typing. While the required typing is a downer, the system is designed by people coming from a functional languages background, and thus a lot of philosophical issues are likely to have been considered and worked through. For example, I know that it tends to use interface types, and that the class hierarchy and the type hierarchy are separate.

Also, Eiffel is certainly worth looking at, especially since piles of code have been developed in it and so there is some practical experience.

If we start with something like Java's or C++'s type system and grow from there, then we will have years of catchup to do.

-Lex

PS -- I don't know about any of these systems first-hand. They just seem relevant.

Thanks for chiming in, David!

This is another important nexus of potential synergy. Thanks for including the summary so those unaware don't have to go rummaging.

- Dan --------------------------

FYI -- to add to your list:

SmallScript, which is an extended dialect of the Smalltalk language, has a full optional type system with dynamic type based binding (multi-methods). This allows optional runtime enforcement of interfaces and contracts.

It also provides multiple-inheritance of behavior and synthetic multiple-inheritance of structure through (under-the-hood) aggregation via concrete interfaces.

... <Great summary snipped>

Hi Alan

I guess you have loked at http://minnow.cc.gatech.edu/squeak/683 . Very cool pictures.

BTW, there are some dead links to Windows, Linux, OS/2 and Mac. Perhaps we should repeair them. Does anybody know what happened to them?

Cheers,

Alan Kay wrote:

Folks --

I'm putting together a brochure about Squeak to whom it may concern (especially potential funders), and I'd like to include some pictures of interesting platforms that Squeak runs on. I already have logos for Mac, Windows, and Linux. But I know that Squeak runs on at least one mainframe, and at least one supercomputer. It runs on the SONY PS II. It runs on various PDAs. It has run on several bare chip sets. And on other oddball and interesting lashups.

Please submit!

Cheers,

Alan

Hi Alan

I've put some PhotoSqueak samples at http://webs.uolsinectis.com.ar/jmvuletich/PhotoSqueakGallery/PhotoSqueakGall...

If you have something specific in mind, just tell me and I'll prepare it for you.

P.S. Good luck with the Squeak brochure (and with the potential funders)!

Cheers,

Alan Kay wrote:

Hi Juan --

Actually, I had forgotten about these. I fetched quite a few of them. Thanks!

P.S. Do you have any neat examples of your picture filtering stuff?

Cheers,

Alan

Thanks Tim!

Hey, by the way, do you have any pix of the good old ActiveBook? I've never forgotten our first meeting and that great demo you showed me.

Cheers,

Alan

------

At 4:11 PM -0700 7/22/02, Tim Rowledge wrote:

Alan Kay Alan.Kay@squeakland.org is claimed by the authorities to have written:

...

Folks --

I'm putting together a brochure about Squeak to whom it may concern (especially potential funders), and I'd like to include some pictures of interesting platforms that Squeak runs on.

Acorn Risc OS picture added on the swiki for you; http://minnow.cc.gatech.edu/squeak/uploads/683/AcornSqueak.jpeg

With luck Craig's old mediapad CD willahve some decent 'pad pictures on it; I just tried to read my CD and OS X simply doesn't recognise it. Sigh.

tim

Tim Rowledge, tim@sumeru.stanford.edu, http://sumeru.stanford.edu/tim "Bother" said Pooh as "Formating Drive C" appeared on the screen...

--

Folks --

It might be of some interest (at least for those with 10 fingers on each hand) to note that Smalltalk was born 30 years ago this Sept, and started working around the end of Oct of the same year.

Cheers,

Alan --

Alan Kay Alan.Kay@squeakland.org is claimed by the authorities to have written:

Folks --

It might be of some interest (at least for those with 10 fingers on each hand) to note that Smalltalk was born 30 years ago this Sept, and started working around the end of Oct of the same year.

It is an amusing coincidence that IBM VM appears to be celebrating its 30th birthday at the same time. Talk about different user experiences...

tim

On Tuesday, July 16, 2002, at 07:49 Uhr, Dan Ingalls wrote:

Ever since talking with Gilad & Co. about it, there has been this little plex in the back of my mind along the lines of "Squeak meets StrongTalk".

Why does that make me thing of the wonderful litttle cartoon "Bambi meets Godzilla"? :-) [And no, I don't know, I don't even think I could say which is which...]

The first component in this gestalt is that I think type annotations are useful as documentation, and I have always felt (and often said) that a Smalltalk with optional types would be an ideal computing environment. It really fills a hole in the metasystem.

Agreed. In my experience, optional typing definitely helps with documenting both system-wide communication patterns and mono- or only slightly polymorphic local/instance variables.

The second component is a sort of BOBW (best of both worlds) notion that with the cool aspects of Squeak and its numerous multimedia facilities, together with what is arguably the fastest execution engine going, we could at least have a lot of fun.

YES! Down with Slang! (with apologies to the creators of that wonderful hack!)

The third component has to do with applying the Squeak philosophy to what otherwise appears to me as a daunting project. The Animorphics VM is (I would suggest) a programming tour de force. I have always been paralyzed when considering such projects (and this goes for, eg, the SELF compiler, too), by the thought that I would burn out simply dealing with so much complexity all in C or worse (if you can imagine that ;-). The whole idea behind Squeak (well, not the whole of it, but the implementation approach) was that we could write it in the language we already knew, and it would be easy to understand and test. I don't see why we couldn't do the same thing for an engine similar to the Animorphics VM (Ian and I have also talked about doing the same kind of thing for Jitter).

And wouldn't we be able to go even better with an Animorphic-style system? Couldn't we eliminate of gnarly, hand-tuned system/slang/plugin code and replace it with *real* Smalltalk and let the optimizer make it good?

Some questions are:

Would the system benefit from being cast into StrongTalk?

Good question. My experience with Objective-C's optional typing has been mixed. While I *think* it would be useful to slowly solidify interfaces as code settles near the core, my practical experience has been that by the time code was settled, there wasn't much of a point in typing it. Also, the type-system seemed to get in the way at times where it wasn't able to express idioms that I had found useful. Of course, Strongtalk is probably more powerful, and a Smalltalk type-system would be able to grow with the system.

and how much work would this be?

Aren't we doing some of this work for modules, making dependency structures visible?

Would anyone care if it ran 10 times faster?

Yes, I think that would make a significant difference. Having more speed to burn enables greater levels of abstraction to be used. Especially with a system that seems to be able to "burn through" layers of abstraction to always arrive at a very similar and nearly optimal executable code.

It seems to me that a lot of code is still implemented in a fairly low-level, gnarly/procedural sort of fashion. I am almost sure this is related to implementors avoiding higher levels of abstraction for performance reasons, maybe even without thinking. I certainly know that coding in Objective-C frees me from some of the performance worries I have when coding in Squeak, a liberating effect.

and how much work would this be?

A lot? I've never done a real VM, so all this seems very daunting to me.

Would it be fun to do?

:-)))

Marcel

-- Marcel Weiher Metaobject Software Technologies marcel@metaobject.com www.metaobject.com Metaprogramming for the Graphic Arts. HOM, IDEAs, MetaAd etc.

Hi Dan and all,

Sorry for this long response, but I'll try to respond in some detail some of the questions you posed, and offer a few suggestions, since I think it would be great if Squeak took off and could take advantage of what we've done.

-----Original Message----- From: Dan Ingalls [mailto:Dan@SqueakLand.org] Sent: Tuesday, July 16, 2002 7:49 PM To: squeak-dev@lists.squeakfoundation.org Cc: David Griswold Subject: Re: Animorphic ST (Strongtalk) released!

<snip>

The second component is a sort of BOBW (best of both worlds) notion that with the cool aspects of Squeak and its numerous multimedia facilities, together with what is arguably the fastest execution engine going, we could at least have a lot of fun.

That would be extremely cool. Many of the things that the Squeak UI does well are things that I never had the time to work on in the Strongtalk UI. One thing that I would urge you to do, however, is to allow the Squeak UI to at least to *appear* to integrate better with host window systems, which I think would help Squeak acceptance quite a bit. For example, breaking out of the world-in-a-window approach so that Squeak windows and host windows were at the same level. I know it makes portability harder, but I think it would help overcome a significant acceptance barrier.

The third component has to do with applying the Squeak philosophy to what otherwise appears to me as a daunting project. The Animorphics VM is (I would suggest) a programming tour de force. I have always been paralyzed when considering such projects (and this goes for, eg, the SELF compiler, too), by the thought that I would burn out simply dealing with so much complexity all in C or worse (if you can imagine that ;-). The whole idea behind Squeak (well, not the whole of it, but the implementation approach) was that we could write it in the language we already knew, and it would be easy to understand and test. I don't see why we couldn't do the same thing for an engine similar to the Animorphics VM (Ian and I have also talked about doing the same kind of thing for Jitter).

When the Strongtalk VM development started, there were a number of discussions about bootstrapping the VM in Smalltalk. We all of course thought it would be very cool to do, but there were a number of reasons we didn't attempt it. We were trying to build a commercial product with a limited amount of startup capital, and it seemed too risky, since there are tight performance constraints on the compiler speed to avoid noticeable pauses, in addition to the obvious problems of trying to avoid infinitely recursive compilation. We also modelled our architecture on the Self VM, which was already in C++. The VM architecture is quite object-oriented; it is not at all like old-fashioned C-style VMs (for example, the collector asks VM objects to enumerate their internal obj references, etc so that it is easy to add new VM data types or change their formats). As you suggest, an OO architecture like this is critical to being able to do it at all- I know that Lars Bak thinks that it would have been practically impossible to do the Animorphic VM in a non-OO language.

I have always thought that the fact that Squeak has a solution to the bootstrapping problem might offer a nice way to do it again in Smalltalk without the risks we tried to avoid. The question then becomes how much of a productivity advantage you would have coding in Smalltalk over the style in which the Animorphic VM was written. I haven't had time yet to look in detail at the Squeak code to see how the bootstrapping translation details work, so I'll ask some questions I think might help illuminate some of the issues and advantages associated with doing it the Squeak way. Of course, one obvious advantage is being able to debug and test in Smalltalk. Other important issues to think about:

1) How fast is the translated C code? The compiler has to be quite fast. Is dispatch as fast as a decent inline-cached dispatch? How much overhead is there for checking/handling the SmallInteger case? How much code space blowup is there in the translated C code (an inlining/deoptimizing compiler is a lot bigger than an interpreter).

2) How much of the Smalltalk coolness is useable in the translateable subset? Are integers still objects? Is there support for some kind of block closure (even if only downward-passed ones) so that control structures other than the hardwired ones like #ifTrue:ifFalse, #whileTrue: etc. can be used? What kind of GC support is available for VM internal data structures that can't be put in the target ObjectMemory, like dynamically allocated GC data structures (if any)? These are the things that would give the most significant advantages over writing the compiler in C++, since the Animorphic VM is already fairly OO.

3) How would compiler portability work? There would obviously have to be a more complex portability infrastructure, since the compiler has to generate native code dynamically.

Some questions are:

Would the system benefit from being cast into StrongTalk? and how much work would this be?

There are a bunch of interrelated issues here, which deserve more in-depth conversations. Here are a few scattered observations. Obviously I think any Smalltalk would benefit from both the type system and inlining compiler, since Strongtalk is my baby. But I think it's fairly clear that the bigger a piece of software is, the more a type system helps for browsing/programming/testing, and the compiler itself is obviously a big piece of code.

Having typed "Blue Book" classes I think helps a lot, but the Animorphic ones have quite a different structure than the Squeak ones, for two reasons: 1) typing requires a different hierarchy and interface structure, and 2) they use far more sends and block closures than other Smalltalk systems because I knew all the inlining would be taking place- they would be too slow as written on Squeak without an inlining compiler. However, I think that the chances of being able to graft them or something like them into Squeak are pretty good, if you are open to wholesale rewriting of a lot of those core classes and changes to code that subclasses them.

One idea to think about: the compiler and VM itself are clearly things that would benefit from running faster while being written in a high-level way, which is what the inlining compiler can help with. While that is not necessary, it would certainly ease some of the performance concerns of writing the compiler and VM in Smalltalk. To take advantage of that would require significant changes to the bootstrapping scheme, like using something like our inlining-database to drive a more restricted kind of static inlining run to generate a faster VM and compiler (however you can't deoptimize in that scenario, which restricts some kinds of inlining, although not all).

This raises an interesting issue: how much benefit could be derived from a much simpler inlining compiler? I have always thought about a much simpler but more restricted inlining scheme that would inline constant sends and use inheritance 'customization' to make self-sends sends constant, and then you could do many of the same block-closure-elimination etc. optimizations on that code. The huge advantage of such a scheme would be that it eliminates the necessity for deoptimization and all of its attendant complexity, and that in turn would also make static translation or compilation of inlined code possible.

This kind of scheme wouldn't inline all the things that Strongtalk can inline, but it might still help a lot. I did some measurements on the VisualWorks image long ago, and determined that statically, about 25% of 'real' sends were self sends (I never did dynamic measurements, though). I suspect my own coding style probably has a much higher fraction of self sends dynamically than that (since leaf sends for instvar accessors etc. are often self sends), and so I suspect that the Strongtalk libraries would run nicely under such a compiler (note that you could even do this kind of optimization on bytecodes, although I think it would require bytecode changes). The upshot is that this kind of optimization would probably tend to disproportionally speed up high-level code rather than hand-optimized code.

Would anyone care if it ran 10 times faster? and how much work would this be?

Yes, yes, yes. I think the Smalltalk community has always underestimated the extent to which performance matters in the real world (and the extent to which proponents of other languages use it as a weapon against Smalltalk). Of course it is true that first you need to write your algorithms correctly, and that *most* applications don't need ultimate speed. However, it can be very hard to predict ahead of time whether your application will need some critical algorithm to run at top speed, and once you are in that situation, it can be incredibly painful to have to write some algorithms in another language, especially if by chance they need to extensively access, create, or modify shared data structures. So this makes it a gamble: are you willing to *bet* your success on the hope that your application will never be compute bound?

And yes, making Smalltalk faster is a huge amount of work. But it is the kind of work that pays off practically forever, especially when you have an open-source system like Squeak that can survive independently of any person or organization. And maybe a scheme like I outlined above could help make it simpler (although a lot more thought would be needed).

Would it be fun to do?

If you like writing cool code ;-) Although building the Animorphic system was an incredibly long, hard, risky process, it was also incredibly exciting. I suspect that you know that feeling.

Cheers, Dave

Sorry for this long response, but I'll try to respond in some detail some of the questions you posed, and offer a few suggestions, since I think it would be great if Squeak took off and could take advantage of what we've done.

Hi, David -

Thanks for responding so fully to my musings. Without spinning off on all branches, I do want to clarify my vision of how the bootstrapping might work...

As you know, the Squeak VM is written in Slang, a subset of Smalltalk that can be translated to C. Its virtues are...

It doesn't require learning C if you're a Squeaker It can be changed and tested as easily as Smalltalk It can be translated to run as fast as C

...and these are its ONLY virtues. The last item is important for Squeak since the Squeak VM is an interpreter, so running it on itself will only make it slower.

So I am NOT suggesting to write the Snort (Strong Squeak, get it ;-) compiler in Slang. It is a compiler, so running it on itself WILL make it faster. The idea is to write it all in Snort (not Slang), except for the runtime required by the compiler (hopefully much of that could still be written in Slang). [I've talked with Ian about applying the same approach to Jitter].

So, what happens when you start it?

It starts to run the first method which is not translated so it runs the interpreter Things lumber along for a while this way until something notices the umpteenth execution of the same method and asks for it to be compiled Now the compiler starts to run but, of course, it's not translated so it starts kind of slow Along the way the compiler hits some inner loops and these need in turn to be translated Gradually the compiler speeds up until it runs as fast as StrongTalk, and what more could we ask for?

This is, of course recursive but, as long as the system supports both interpreted and translated code, I don't think it's a problem as long as you recognize three states of a method:

Interpreted Being translated Translated

The first and last are obvious. You need the middle one so as not to launch circular requests to translate a method that is already being translated. Recursion is OK, but endless looping is not.

There are, of course variants of this approach that are less recursive (like, run it all interpreted, compile the compiler, and then turn on the compiled compiler) but, hey, recursive application is our thing.

My point is that I'm not recommending do write it in Slang like Squeak (an interpreter that you can translate), but to write it in Snort, a compiler that can compile itself. It's still the Squeak philosophy: write everything in the home language, and do what it takes to make it run.

- Dan

Being curios about how this stuff works, and not knowing anything about how JIT VMs work, I have a couple of questions.

First, are you you talking about adding JIT to Squeak? Or something more? What does Jitter do? How does it relate to this discussion?

When you say compile a method, I take that to mean take a normal Smalltalk bytecoded method and compile it into native machine code. What does the compiled method look like? Is it just a sequencing of the machine code that the interpreter would normally run as it processes each bytecode? It must be more than that right? Because a bytecode-threaded interpreter would get to the end or processing one bytecode, then pull the next bytecode and jump directly to the start of the next bytecode's machine code (not even needing to do anything like a case statement). Thus, I wouldn't think that a simple sequencing of the machine code for each bytecode would buy you much (and it would take up a good bit of space too). There must be some more optimization going on in the compilation of bytecodes into machine code right? If that's the case, could someone elaborate on that?

Also, once a method is compiled to machine code, why would there be any need to keep the bytecodes around (as Tim descibes)? Can't you just throw away the old bytecodes at that point? Ah...but perhaps it's so that the image can remain portable to different architectures...is that it?

- Stephen

-----Original Message----- From: squeak-dev-admin@lists.squeakfoundation.org [mailto:squeak-dev-admin@lists.squeakfoundation.org] On Behalf Of Tim Rowledge Sent: Thursday, July 18, 2002 1:52 PM To: squeak-dev@lists.squeakfoundation.org Subject: RE: Animorphic ST (Strongtalk) released!

Dan Ingalls Dan@SqueakLand.org is claimed by the authorities to have written:

...

So, what happens when you start it?

It starts to run the first method which is not translated so it runs the interpreter Things lumber along for a while this way until something notices the umpteenth execution of the same method and asks for it to be compiled Now the compiler starts to run but, of course, it's not translated so it starts kind of slow Along the way the compiler hits some inner loops and these need in turn to be translated Gradually the compiler speeds up until it runs as fast as StrongTalk, and what more could we ask for?

This is, of course recursive but, as long as the system

supports both

...

interpreted and translated code, I don't think it's a

problem as long

...

as you recognize three states of a method:

Interpreted Being translated Translated

Some time ago I suggested a similar possible approach to allow writing of the translator in Smalltalk so that it would be more intelligable, manipulable etc. It's not a particularly original idea of mine, being loosely based on my (probably poor) understanding of Ian P's doctoral thesis and vague discussions before then with LPD, Schiffman, Baker, Eliot M, etc.

The good news is that surprisingly little would need to change in the VM code we already have. An extra oop in compiled methods (or maybe in a lookup cache, who knows) to point to the translated chunk if it exists, some way to queue up an untranslated method for processing, a way to set the translated oop once the translation has finished. The current interpreter would merrily keep going and the translator would run as a suitable priority process. Recognising the middle state of 'in translation' would be a small optimisation to allow for dumping the translation of a method that had been changed since it was queued up.

I think you could even fake up the support framework by making the system allow for the 'translated' methods to be just the normal bytecodes, thus saving having to write the actual translator till later. It would provide nice confirmation of the functionality being reasonable quite quickly.

It would possibly be reasonable to allow the image to be saved with translated methods intact so long as a check were plausible to drop them if the image started on a different machine.

tim

-- Tim Rowledge, tim@sumeru.stanford.edu, http://sumeru.stanford.edu/tim Any given > program will expand to fill available memory.

I am very interested in this thread, but can't write a proper reply today. But here are some brief notes:

1) It seems some people are drooling over the prospect of the sources for the Animorphics VM being released. Given that the Self sources were released in 1995 and a cleaned up version in 2000 and nobody did anything with it (with the exception of Gordon Cichon and Harald Gliebe) I doubt there will be much more than "kicking the tires" if it happens. It is better to focus on the type system (which is out, after all) instead.

2) Machine code has been generated from Squeak in three ways:

- Slang (Smalltalk without objects) to C using a translator in Smalltalk. For the interpreter and plugins

- Bytecodes to PPC and x86 using a translator in C (Jitter 3). For the whole system

- Slang (I think) to ARM using a translator in Smalltalk (by Alan P.)

The original Jitter generated threaded code and doesn't come into this story.

What we want is a Bytecodes to PPC, X86, ARM and MIPS (would those who care about Sparc, Alpha and SH please raise their hands?) using a translator in Smalltalk.

This would even allow the interpreter and plugins to be rewritten in proper Smalltalk. I'll address some concerns with this tomorrow.

3) short Jitter tutorial: by keeping code in bytecode form and translating to native code as needed, you get the best combination of low memory requirements and high performance. By compiling instead of interpreting you can use these tricks:

- generate one native code for each class inheriting a single bytecode method. This lets you move lots of lookups from run time to compile time. All message sends to "self", for example, become static calls. In fact, you might as well inline some of them which opens up lots of other optimizations

- the hardest code to compile is normally rarely (or never) used. Just skip it and put in a call to the compiler to retranslate the method if it turns out we really needed it. You can't do that with a traditional compiler but since we have it during run time anyway...

- trying to extract type information from the bytecodes is a huge time sink. But after you run the code a while, this information can be easily extracted by looking at the caches (PICs - polymorphic inline caches). Most code only runs once anyway, so just do a crude job to get it working and then if it is used a lot (a system hot spot) tell the compiler to try harder, but now with more information obtained from actually running the code

-- Jecel

On Sat, Jul 20, 2002 at 11:51:49AM +0200, Marcus Denker wrote:

All "modern" systems seem to work somewhat like that: They have a *very* fast compiler (very much like J3, maybe even generating worse code) that is called if a method is run the first time. This compiler adds profiling code and uses PICs (Polymorphic Inline Caches) that allow to collect information about what types are actually used at runtime. Then a slow, highly optimizing compiler is called for all hotspots, that is methods that are run very often. This compiler can use the typeinfos collected in the PIC to do very clever optimizations that are even not possible with a static (e.g. C++) compiler.

...

Also, once a method is compiled to machine code, why would there be any need to keep the bytecodes around (as Tim descibes)? Can't you just throw away the old bytecodes at that point? Ah...but perhaps it's so that the image can remain portable to different architectures...is that it?

Bytecodes... I don't like bytecodes. Actually they are a bad idea if you are generating native code: A modern jit-compiler needs a representation of the code that is exactly the one that the smalltalk-compiler had just before it generated the bytecodes. So the first thing a Jit does is decompiling... it has to spend time to regenerate stuff we allready had. So IMHO it would be interesting to look at other representations for compiled methods than bytecodes: Something very simple, no optimizations (like ifTrue: inlining, special sends, eg...) Just a serialised AST.

Bytecode is GREAT for WHAT IT IS GREAT FOR. :) Rather, bytecode is awesome at being memory efficient and fast to interpret. Due to its compactness and the fact that a proper bytecode encapsulates very complex operations into very little space, it can actually end up running faster than non-optimized native code. You really need BOTH syntax trees (or some other graph-like representation) and bytecode to make a great dynamic compilation system. Bytecode will take care of 90% of the code that just needs to be run infrequently and take up inconspicuous amounts of space, while the other representation will allow you to optimize when necessary.

But Squeak of course needs an interpreter: Why not treat the Interpreter as just another target for the JIT-Compiler? Then every new optimization we implement for the compiler would even help to make squeak running faster in interpreted mode (actually, all most optimizations could work for an interpreter, even Inline Caches, Inlining of sends, eg...).

This is a great idea. I've been designing a dynamic compiler for a new language I am currently working on and it incorporates this very idea. Firstly, code is simply adaptively recompiled based on the stored non-bytecode representation, and initially only uses bytecode. After a certain amount of recompilation it will invest the time to switch over to native code, with the already heavily recompiled bytecode/syntax tree available to work from. The net effect is that bytecode is adaptively optimized, which gains signifigant speedups without having to invest all the time to go to native code until it is truely needed.

This would essentially allow us to "late bind" the real-world Interpreter and change the bytecodes used from one release to another.

The clean and simple representation for code in compiledmethods could maybe used for other things, too: Maybe this could be a nice thing for Alan's ideas of a very clean Squeak Kernel. (An interpreter for that would be very simple, but very slow). And the eToy and omniuser tiles could be generated from this representation... maybe we could even think about trowing away the source (:-)).

just some strange ideas...

        Marcus

-- Marcus Denker marcus@ira.uka.de -- Squeak! http://squeakland.org

Lee

On Saturday, July 20, 2002, at 02:51 AM, Marcus Denker wrote:

Tim's approach allows to wait with translation until there is time to do it. He suggested to have a single flag in each method, but you could actually do more: 1) do profiling to find the hotspots and 2) get some typeinformation.

The nice thing about having the compiler in the image is that it is in the image, which means it is very accessible, improvable, able to make full use of decent language facilities, reflection etc. In fact it is my observation that it is always nice to have as much as possible in the image and as little as possible in the vm. I really dislike seeing prims that allocate objects (aside from the ones really needed) since they then tend to get involved in messing with GC policy. This should be deferred to the image, not messed up in the vm. As peripherally mentioned before, anothe nice thing about a deferred translator is that it may very well be able to decide that there is nothing worth doing for many cases, particularly those involved in image startup. Doing nothing is often much much quicker than any alternative. (My favourite TECO instruction, of which I remember nothing but the comment, said - ' this instruction does nothing, but does it very fast')

tim@sumeru.stanford.edu

On Friday, July 19, 2002, at 10:53 Uhr, David Griswold wrote:

The big issue is the speed, which you aptly captured with the word "lumbered". There are two speed issues:

• Startup time: Strongtalk already doesn't start as quickly as Squeak

because of the initial compilation "bulge". Snort would start even more slowly, not only because the compiler is being interpreted, but also because it is being compiled. This might not be to bad a problem, or it might. The critical issue is what the performance locality profile of the compiler is. In other words, does the compiler spend most of its time in some small fraction of its code, or does most of the compiler get exercised pretty heavily. I don't have an answer for that, but the more locality it has, the faster it would speed up.

Just an idea: maybe too much effort is being spent on speeding up the whole system, and doing it both transparently and dynamically during execution. Maybe speeding up the whole system dynamically by some factor isn't all that important. Maybe it is more important to know you can speed up certain portions to within an epsilon of C without going through strange contortions?

<quote> Would anyone care if it ran 10 times faster? and how much work would this be?

Yes, yes, yes. I think the Smalltalk community has always underestimated the extent to which performance matters in the real world (and the extent to which proponents of other languages use it as a weapon against Smalltalk). Of course it is true that first you need to write your algorithms correctly, and that *most* applications don't need ultimate speed. However, it can be very hard to predict ahead of time whether your application will need some critical algorithm to run at top speed, and once you are in that situation, it can be incredibly painful to have to write some algorithms in another language, especially if by chance they need to extensively access, create, or modify shared data structures. So this makes it a gamble: are you willing to *bet* your success on the hope that your application will never be compute bound? </quote>

My experience with Objective-C very much confirms this. Speed really isn't that much of an issue for most of the code, which can actually be written in WebScript (a slow interpreter of a language very close to Objective-C). However, knowing that you can get to the metal without a drastic rewrite (and I do consider Slang + pluginizaation fairly drastic) is both psychologically comforting and tremendously useful in practice.

I also wouldn't mind having to make annotations in order to get the last bit of performance out of specific pieces of code.

• Asymptotic compiler speed: once the compiler is running essentially

full speed, is that fast enough? Even with all the Strongtalk optimizations in place, such code is still a lot slower than fully optimized C++ (in principle it could get fairly close to C++ performance, but the limiting factors are the fact that the best algorithms for more advanced optimizations like scheduling and register allocation are usually too slow to run in realtime, as well as being a giant pain to write efficiently). So I would expect that a compiler similar to the Strongtalk one written in Smalltalk would run about 5-10 times slower than the Strongtalk compiler (although with tuning you might eventually be able to speed it up a lot).

Strongtalk is still 5-10 times slower than C/C++? So doing the compile off-line would probably be a better idea. Could 'C' be generated? That way, there would be a fall-back that would at least allow static compilation.

If you have to recompile all the perf. critical code everytime the system starts up, all this might mean very slow startup, and noticeable pauses for compilations thereafter. But one way to get around all of this might be to keep the compiled code in ObjectMemory and save it with the image, in which case you could start the system in an already optimized state (complicates image portability, but worth it).

Why not store the generated code as a dynamically generated, dynamically loaded shared library, instead of storing within the image? For example, why not have a compiled binary associated with each module? I'd wager that the core modules would not need to be recompiled a lot after they\ve settled. That way, the core classes would really start becoming more part of the VM, or rather, the distinction between VM and objects would start to disappear (with much of the VM also being written 'in objects'). It all becomes more a 'substrate' for running user-defined objects.

If you work a little on the calling-conventions, you might even be able to make the whole thing a lot more interoperable with other compiled code on the system.

In this case, people would generally want to do 'training' runs of their code to get them mostly optimized, and then save the image. Compilation then becomes a fairly rare thing after training is done.

More of a static compiler, very similar to the way Slang is used today, just without the language limitations we currently have with Slang.

Then, if compilation becomes a rare thing, you could even have different compiler modes for fast vs. good compilation, and then run really good optimizations to get closer to C perf. during the training runs, and use a faster compiler (perhaps even really simple, fast non-inlining one) during normal execution, if the really good one is too slow.

Depending on your needs, you might not even need a compiler during execution, for example for a 'packaged' binary...

This would let your good compiler eventually do better optimizations than we could afford, with the sky the limit!

Yeah!

Marcel

-- Marcel Weiher Metaobject Software Technologies marcel@metaobject.com www.metaobject.com Metaprogramming for the Graphic Arts. HOM, IDEAs, MetaAd etc.

"Stephen Pair" spair@advantive.com is claimed by the authorities to have written:

David Griswold wrote:

[snip]

...

But one way to get around all of this might be to keep the compiled code in ObjectMemory and save it with the image, in which case you could start the system in an already optimized state (complicates image portability, but worth it).

[snip] It doesn't complicate much - we already have to check for endianess and compensate for example. We have session ID stuff for file & socket handles so they can be nulled as appropriate. Squeak has some extra possibilities as well why not save any precompiled stuff in an ImageSegment? That way one can check for relevence and only load it if it would be useful/usable. Just think, when Palladium is mandated you would be able to check the certificate to see if you have permission to run as well.

Would it also be possible to simply tell the system to compile everything? Thus, eliminating the need to do 'training' runs and achieve maximum performance?

Depends on how much memory you have available; and you'd need to either do a training run to do the full compile & save or wait during startup for the whole thing to finish.

tim

At 10:14 AM 7/18/2002 -0700, Dan Ingalls wrote:

So I am NOT suggesting to write the Snort (Strong Squeak, get it ;-) compiler in Slang. It is a compiler, so running it on itself WILL make it faster. The idea is to write it all in Snort (not Slang), except for the runtime required by the compiler (hopefully much of that could still be written in Slang). [I've talked with Ian about applying the same approach to Jitter].

So, what happens when you start it?

    It starts to run the first method which is not translated
            so it runs the interpreter
    Things lumber along for a while this way until something
            notices the umpteenth execution of the same method
            and asks for it to be compiled
    Now the compiler starts to run but, of course, it's not
            translated so it starts kind of slow
    Along the way the compiler hits some inner loops and these
            need in turn to be translated
    Gradually the compiler speeds up until it runs as fast
            as StrongTalk, and what more could we ask for?

Dan, An approach very similar to this was taken by the Jalapeño project at IBM T J Watson for building a high performance dynamic compilation Java vm that used Java as it's implementation language. Numerious papers have been published on various aspects of this project and the actual implementation (renamed to "Jikes Research VM") is available under an open source license. I believe the experiences reported in these papers would be highly valuable to anyone attempting to building a Smalltalk implementation of the sort you describe above.

For links to papers, etc. see: http://www.research.ibm.com/jalapeno/ http://www-124.ibm.com/developerworks/oss/jikesrvm/

Also note that there is an upcoming OOPSLA'2002 tutorial titled "The Design and Implementation of the Jikes RVM Optimizing Compiler"

Allen_Wirfs-Brock@Instantiations.com

On Tue, Jul 16, 2002 at 10:49:18AM -0700, Dan Ingalls wrote:

The first component in this gestalt is that I think type annotations are useful as documentation, and I have always felt (and often said) that a Smalltalk with optional types would be an ideal computing environment. It really fills a hole in the metasystem.

After pondering this, as well as Dave Simmons' subsequent summary of SmallScript features, I'd love to play around with this. I'm trying to find a few spare hours to play with SmallScript.

The second component is a sort of BOBW (best of both worlds) notion that with the cool aspects of Squeak and its numerous multimedia facilities, together with what is arguably the fastest execution engine going, we could at least have a lot of fun.

Great fun! (picture me with an ear-to-ear smile :-)

The third component has to do with applying the Squeak philosophy to what otherwise appears to me as a daunting project.

<etc.>

I'm glad that it is no longer daunting to you :-) It does make it seem much more manageable.

Some questions are:

Would the system benefit from being cast into StrongTalk?

Yes! Definitely from a performance perspective. I'm not qualified to comment on the merits of the type system, other then to say that it sounds intriguing.

and how much work would this be?

(drawn blank)

Would anyone care if it ran 10 times faster?

Yes!

and how much work would this be?

(drawn blank)

Would it be fun to do?

If "do" is interpreted literally, then it does sound fun, but I don't have the large amount of time that it would require for me to usefully contribute. However, if I loosely understand "do" to mean "play with it after it's done", then then my answer is unequivocally affirmative.

Joshua

Comments?

• Dan

PS Nathanael: Could you forward this to Gilad? I don't have his email addr, but I'd like to hear his comments (which I or you could forward to the list). Also, David, I'm hoping you will answer with your thoughts.

I only looked into Animorphic ST for a couple of minutes so far, but by first impression was, this is how Smalltalk should look like. Besides the system system, I like the Self-like browers.

I found the type-system the most unnatural part about StrongTalk. It feels displaced and if you look at methods like "->" in Object you can see that there are some rather nasty complexities associated with it (not to mention readability issues).

In Smalltalk (without types) no tool can exactly know the type of an expression and code completion (with is IMHO the best productivity tool of a modern IDE) cannot offer a valid selection of applicable method names.

That has been said many times and yet, I disagree. To the best of my knowledge nobody has ever tried to build a simple type inference system which just goes along with you as you are writing code and tries to discover what the types might be. Name completion and other utilities can even be used to tell such a type inference system about what types of certain objects are and I still think that you should be able to trivially figure out the types for 90% of the system without explicit type annotations. For the remaining 10% you _might_ want to use type annotations but I would hope that some better syntax can be found than is currently in both StrongTalk and SmallScript.

Currently, I'm working with a very large (VisualAge Smalltalk project which was grown over the last five years or so - and most original developers left the company long ago - and it's awful difficult, I want Eclipse (and Java) back. Never thought, that I'd say that but if mediocre programmers hack quick fixes into a system for a couple of years, Smalltalk becomes a mess. This is probably true for other languages, too, but at least types would give you some kind of documentation.

But of course you do realize that *optional* types would hardly ever be used by mediocre programmers do you?! So that an implicit type system would (arguably) be more useful than an explicit one that isn't used.

Cheers, - Andreas

Ian Piumarta wrote:

Why not use message syntax?

You presented an interesting idea to annotation type assert to expressions which can be checked at runtime. The whole idea about a static type system however ist, that you can check for consistency at compile time (aka method accept time). Furthermore, you also what to add type annotation to object containers (variables, instance variables, temporaries and method parameters). This can't be done with your suggested system other than

method: arg | temp | arg type: Integer. temp = arg squared. temp type: Integer.

which IMHO doesn't help to document the variable types as it's not as clear as

method: arg<Integer> | temp<Integer> |

or

method: {Integer}arg | {Integer}tmp |

or

method: arg@integer | tmp@integer |

bye

Hi guys,

...

I only looked into Animorphic ST for a couple of minutes so far, but by first impression was, this is how Smalltalk should look like. Besides the system system, I like the Self-like browers.

I found the type-system the most unnatural part about StrongTalk. It feels displaced and if you look at methods like "->" in Object you can see that there are some rather nasty complexities associated with it (not to mention readability issues).

Before your message I was so depressed because I felt alone :-P

( Andreas, we are on the same side this time =o) )

To say the true, the only thing I like about StrongTalk is the dynamic compiler. The other unnatural thing is the absence of an image.

...

In Smalltalk (without types) no tool can exactly know the type of an expression and code completion (with is IMHO the best productivity tool of a modern IDE) cannot offer a valid selection of applicable method names.

That has been said many times and yet, I disagree. To the best of my knowledge nobody has ever tried to build a simple type inference system which just goes along with you as you are writing code and tries to discover what the types might be.

I feel that tools like code completion are comfortable things, but if you have an object and you don't know what messages you have to send, you are in trouble! and no type system can help you.

Anyway in ST you have a really good option -> the debugger! I you feel a little bit lost, just write in the method "self halt" and the execution context of the method will help you.

Cheers,

Diego Gomez Deck

On Saturday 20 July 2002 09:45 am, Nevin Pratt wrote:

Come to think of it, though, that's exactly what I think of another often touted feature of the static languages computer camp: multiple inheritance. Multiple inheritance is also occasionally useful, but highly overrated, and supporting it introduces a lot of other issues and complexities that I don't care for.

Though I'm finding that mixin style inheritance can be quite useful.

I've been using Ruby quite a bit recently, and it's a comfortable technique.

On Saturday 20 July 2002 02:24 pm, Andreas Raab wrote:

I haven't been around during the clone (err... message-eating nil) wars but after reading your paper it makes me wonder if anyone actually ever _tried_ to use a message eating nil (and not just reason about it) in Smalltalk.

Yes, there was at least one article that I'd read on the web from a Well Known Name In Smalltalk that discussed this as a good idea. Forget who, though. Perhaps it was Alan Wirfs-Brock? Jan Steinman?

I remember making one and using it for a while several years ago. I don't remember how it turned out, though.

You do have to watch out for "== nil" (and #ifNil: #ifNotNil: etc. that end up doing "== nil"). There's a mixture of

isNil ifTrue: (which would work, as it's a message send) == nil ifTrue: (which wouldn't work) ifNil: (which also wouldn't)

etc. in Squeak.

I know that Kent Beck has mentioned in some of his discussions about the Chrysler accounting system that using custom nil objects (i.e. NilEmployee) was helpful. I believe, though, that these were not message-eaters, but just harmless dummy objects.

On Saturday 20 July 2002 03:51 pm, Andreas Raab wrote:

Nah... I meant the "real thing", e.g., make UndefinedObject be "message eating" not some arbitrarily new object. All of the above should work perfectly well if you implement them.

Well, that's easy enough to do...

Hi Nevin,

Thanks for the info. I think I'll just have to try it out :-) I still find it hard to believe that code which was written with exception throwing behavior in mind would fail if you introduce message eating behavior but I guess only experience can really tell the ups and downs.

Cheers, - Andreas

-----Original Message----- From: squeak-dev-admin@lists.squeakfoundation.org [mailto:squeak-dev-admin@lists.squeakfoundation.org] On Behalf Of Nevin Pratt Sent: Sunday, July 21, 2002 4:21 PM To: squeak-dev@lists.squeakfoundation.org Subject: Re: Animorphic ST (Strongtalk) released!

Andreas Raab wrote:

...

...

isNil ifTrue: (which would work, as it's a message send) == nil ifTrue: (which wouldn't work) ifNil: (which also wouldn't)

etc. in Squeak.

Nah... I meant the "real thing", e.g., make UndefinedObject

be "message

...

eating" not some arbitrarily new object. All of the above

should work

...

perfectly well if you implement them.

The main reason I recommended against this in my article is because I found from experience that making UndefinedObject be "message eating" breaks large amounts of existing code. Implement it yourself, and you will discover the same thing very rapidly. For example, streams in VisualWorks deliberately throw and catch exceptions as part of the normal processing. If you eliminate the exception throwing, the code breaks.

So, rather than modifying UndefinedObject, I attempted to mix the paradigms. I immediately encountered difficulty trying to mix existing nil behavior with a message eating Null class within the same software layer. I.e., things would break when I would try mixing it within the Presentation Layer, because existing widgets, etc., in the Presentation Layer again expect alternate nil behavior. Of course, one reason for this is simply a rehash of the same issue above.

I also encountered difficulties trying to mix the paradigms in VisualAge, where #isNil: and #notNil: are optimized away and are not true message sends. Another Smalltalk that doesn't optimize away #isNil: and #notNil: might yield different results, though. But even then, there is still the problem of '== nil' that many programmers like to write (but not me) if you are trying to mix the paradigms.

As mentioned in the article, the entire experience was originally done as an experiment, and was motivated from my Objective-C experience. I was initially apprehensive about it, but have since concluded that most of that apprehension was just excessive paranoia.

I have long concluded that the paradigm is very useful when applied to Smalltalk, and is trouble-free, *if* the entire software layer that it is being used in has been written with that paradigm in mind. And, for practical reasons, I feel that the only layer that you have any chance of achieving that is the domain layer (if at all), since that is the layer you most control. But regardless of the layer, it requires the buy-in of the other team members writing code for that layer, or it won't work.

So, within that narrow usage, I find it to be a very useful paradigm. But I agree it is not suitable for the general case.

Interestingly enough, Dolphin Smalltalk has had a 'DeafObject' class in it's image since about 1996, which predates my article. The 'DeafObject' does essentially what my 'Null' does.

Nevin

On Sunday, July 21, 2002, at 09:24 Uhr, David Simmons wrote:

I think the general pattern of allowing <nil> to silently eat arbitrary messages sent to it is a dangerous (*bad*) practice.

Why do you think so? I have been programming with message-eating <nil> for around 15 years and I do not find it dangerous in practice.

Marcel

-- Marcel Weiher Metaobject Software Technologies marcel@metaobject.com www.metaobject.com Metaprogramming for the Graphic Arts. HOM, IDEAs, MetaAd etc.

On Sunday, July 21, 2002, at 02:40 Uhr, David Simmons wrote:

...

On Sunday, July 21, 2002, at 09:24 Uhr, David Simmons wrote:

...

I think the general pattern of allowing <nil> to silently eat

arbitrary

...

...

messages sent to it is a dangerous (*bad*) practice.

Why do you think so? I have been programming with message-eating

<nil> > for around 15 years and I do not find it dangerous in practice.

Well, in and of itself, that is not a validation of it not being a "dangerous" practice.

I didn't say it was a validation. It is evidence, a *counterexample* to your claim that it is a dangerous practice. Since you are making the claim, it is up to you to validate it.

There are programming patterns, which I've used for many years, which I would characterize as being "dangerous" for general purpose use by arbitrary programmers.

Hmm...like using Smalltalk?

In other words, it is one thing to adopt specific styles and specific techniques encapsulated within ones own code. It is an entirely different thing to standardize such techniques as best-practices, standard system behavior, or encouraged design techniques for all users of a language regardless of their background or the solution area they may apply those techniques/principles to.

You are quite wrong: this is the standard behavior in Objective-C, so it isn't just my personal style of coding.

The case of "black-hole" message receivers, <nil> is an especially troublesome one precisely because it is the default initialization value for all reference-type fields [slots].

Actually, I find the combination to be rather useful. If I don't put anything there, nothing is there, and this nothing does nothing and returns nothing when asked to to something. Which I find to be perfectly acceptable behavior for nothing.

It is quite easy to imagine

Imagination is one thing...experience another. Somehow, this reminds me of all the awful things that people (strongly) advocating mandatory static+manifest typing keep telling Smalltalkers will happen, which Smalltalkers just don't seem to see happening. Show me some evidence that this actually *does* happen in real life, and maybe I will be convinced.

how contractual errors and their resulting failures would go completely undetected because the "faulty" code is silent even at runtime.

You mean like objects of the wrong type that get transported to other parts of the system to lie dormant until they blow up years later? Yes, these things can happen, but the question is wether their frequency outweighs the frequency of errors that could have been ignored but weren't, and the amount of extra code you have to write to constantly check for nil, code that can and will introduce bugs of its own.

This is reminiscent of the really awful and unmaintainable unix code, that relied on zero-address behavior [and applications always loading at the same fixed addresses], that used to plague many of us who worked on unix software over the large span of years prior to the linux era [which obviously has much better coders ;-].

I don't think the analogy holds, and I don't find reminiscences very convincing "validation", much less convincing in fact than a decade plus of actually working with the pattern.

While your code over the last 15 years may not have had problems, I would ask you how you could really be sure?

The only thing I am sure about my code is that it contains bugs.

What is missing (or eliminated by "black-hole" messaging to nil) is the explicit capturing of the programmer's intent and their stated "awareness" that they had a contract to fulfill.

Like Java's mandatory catching of declared exceptions? Sorry, I find that this actually obscures intent more than it clarifies it, because the code you intend to write gets obscured by code you have to write because the language is being anal.

Which, in and of itself, would be acceptable for some problems if well encapsulated where such "black-hole" behavior was restricted to only apply to code for that problem.

I don't see the problem. If you want or need to check for nil, you can still do that. You just don't have to do it at *every step of the way* for longer expressions.

But, since the only "classic Smalltalk" solution is to make "black-hole" messaging system/image wide, then it becomes quite troubling when any thought is given to teams of people integrating code which may come from unrelated parties and their distinct frameworks.

I don't see any validation for your claim that this becomes troubling. None whatsoever. I just have evidence to the contrary.

Marcel

-- Marcel Weiher Metaobject Software Technologies marcel@metaobject.com www.metaobject.com Metaprogramming for the Graphic Arts. HOM, IDEAs, MetaAd etc.

On Sunday, July 21, 2002, at 06:54 Uhr, Andreas Raab wrote:

Marcel, David,

...

...

It is quite easy to imagine

Imagination is one thing...experience another. Somehow, this reminds me of all the awful things that people (strongly) advocating mandatory static+manifest typing keep telling Smalltalkers will happen, which Smalltalkers just don't seem to see happening. Show me some evidence that this actually *does* happen in real life

That was exactly the question I raised to begin with. Nevin's paper quotes a few problems that don't apply to Smalltalk (like auto-coercing nil into different atomic types on varying platforms) and I was really wondering how much of a problem the message eating behavior can cause in practice. I guess there are always a lot of theoretical arguments that can be made here (similarly to the issue of static typing) and only time and experience can tell what works and what doesn't.

Exactly! That was the point I was trying to get across: despite all the horrible things that can happen in theory, it seems to have worked remarkably well in practice, certainly for me and also for many others.

Of course, one has to take Nevin's point into account: there may very well be more problems if you mix with code that doesn't expect a message-eating nil.

Marcel

-- Marcel Weiher Metaobject Software Technologies marcel@metaobject.com www.metaobject.com Metaprogramming for the Graphic Arts. HOM, IDEAs, MetaAd etc.

"David Simmons" David.Simmons@smallscript.com wrote:

...

-----Original Message----- From: squeak-dev-admin@lists.squeakfoundation.org [mailto:squeak-dev- admin@lists.squeakfoundation.org] On Behalf Of Marcel Weiher Sent: Sunday, July 21, 2002 12:51 AM To: squeak-dev@lists.squeakfoundation.org Subject: Re: Animorphic ST (Strongtalk) released!

On Sunday, July 21, 2002, at 09:24 Uhr, David Simmons wrote:

...

I think the general pattern of allowing <nil> to silently eat

arbitrary

...

...

messages sent to it is a dangerous (*bad*) practice.

Why do you think so? I have been programming with message-eating

<nil> > for around 15 years and I do not find it dangerous in practice.

Well, in and of itself, that is not a validation of it not being a "dangerous" practice.

There are programming patterns, which I've used for many years, which I would characterize as being "dangerous" for general purpose use by arbitrary programmers.

In other words, it is one thing to adopt specific styles and specific techniques encapsulated within ones own code. It is an entirely different thing to standardize such techniques as best-practices, standard system behavior, or encouraged design techniques for all users of a language regardless of their background or the solution area they may apply those techniques/principles to.

The case of "black-hole" message receivers, <nil> is an especially troublesome one precisely because it is the default initialization value for all reference-type fields [slots].

It is quite easy to imagine how contractual errors and their resulting failures would go completely undetected because the "faulty" code is silent even at runtime. This is reminiscent of the really awful and unmaintainable unix code, that relied on zero-address behavior [and applications always loading at the same fixed addresses], that used to plague many of us who worked on unix software over the large span of years prior to the linux era [which obviously has much better coders ;-].

While your code over the last 15 years may not have had problems, I would ask you how you could really be sure?

What is missing (or eliminated by "black-hole" messaging to nil) is the explicit capturing of the programmer's intent and their stated "awareness" that they had a contract to fulfill.

Which, in and of itself, would be acceptable for some problems if well encapsulated where such "black-hole" behavior was restricted to only apply to code for that problem.

But, since the only "classic Smalltalk" solution is to make "black-hole" messaging system/image wide, then it becomes quite troubling when any thought is given to teams of people integrating code which may come from unrelated parties and their distinct frameworks.

I.e., modularized, componentized code intended for use in creating, maintaining, and extending complex systems -- which is one area where dynamic languages (and especially Smalltalk) have a flexibility that gives them an edge over other more mainstream and popular languages.

-- Dave S. [www.smallscript.org]

...

Marcel

-- Marcel Weiher Metaobject Software Technologies marcel@metaobject.com www.metaobject.com Metaprogramming for the Graphic Arts. HOM, IDEAs, MetaAd etc.

Andreas Raab wrote:

I found the type-system the most unnatural part about StrongTalk.

For me, Strongtalk *is* the typesystem. It's nice, if the Animorphic ST release is also fast, but IMHO the static type system is the more interesting part.

It feels displaced and if you look at methods like "->" in Object you can see that there are some rather nasty complexities associated with it (not to mention readability issues).

One can always argue about syntax. That's something you have to get used to. All it says is that an Association's type is created from the receiver and the parameter type. I don't understand why that isn't written as

-> other <Y def> ^<Association[Self, Y]>

Instead of <Y def>, I might prefer 'Y. I once saw a Smalltalk dialect which used {} instead of <> to declare types. That could be more readable.

However, types don't become easier just by not writing them down... You have to understand them to program.

...

In Smalltalk (without types) no tool can exactly know the type of an expression and code completion (with is IMHO the best productivity tool of a modern IDE) cannot offer a valid selection of applicable method names.

That has been said many times and yet, I disagree. To the best of my knowledge nobody has ever tried to build a simple type inference system

Perhaps, because it wont't work?

which just goes along with you as you are writing code and tries to discover what the types might be.

Just a little bit of type inferrence doesn't work. Once, I tried it and also had some discussions with somebody who thought about writing is master thesis about this topic and we finally agreed upon that you'd need either a full features type system or stay with no types at all.

Name completion and other utilities can even be used to tell such a type inference system about what types of certain objects are and I still think that you should be able to trivially figure out the types for 90% of the system

someMethod: param param .

Okay, please tell me the methods applicable at the "." ? That's impossile if you don't want to show *all* method names of the system - which are probably 8000-10000.

Code completion is IMHO the most important and best way to learn a class library. VBA is a good example. Using the simple built-in IDE of MS Word, I can script the whole program without much problems, just by some guessing and the code completion.

I would hope that some better syntax can be found than is currently in both StrongTalk and SmallScript.

Here, I agree.

But of course you do realize that *optional* types

Sure, but it wouldn't be Smalltalk if you couldn't simply make the optional mandatory and set the requirement of type in your project.

In addition, I could document my assumption by annotation types which would be then checked by the system and eventually get a better documented, well formed program.

bye

On Saturday 20 July 2002 09:20 am, Diego Gomez Deck wrote:

I had exactly the same type of problems in "sure" typed system (to say: Java). With other problems: (classIsNotAnObject, nullIsNotAnObject, and a large list of etc)

Actually, when you look at the reasons for DNU's or type errors in Smalltalk as well as Java, one particular reason stands out:

The use of nil to represent an optional reference.

Get rid of this (by using a Null Object Pattern, perhaps: http://www.smalltalkchronicles.net/edition2-1/null_object_pattern.htm) and you get rid of many runtime type exceptions.

Or use custom nil objects that provide safe default behavior; i.e. NilCustomer, etc.

Hello,

Diego Gomez Deck wrote:

...

The point here is: You have an object and you don't known what messages you have to send? No tool can help here.

Actually, all a static type system does, is giving you and the browser/editor an idea of what kind of object do you have here and what you can do with it. So, yes, the tool can help because the chances are high that if I don't know, at least the tool does.

As I tried to explain, in Java for example, it's mch easier to undestand new unknown frameworks once you've at least an idea what's going on. Just type

aButton.

and search the presented method set. If you know (in Java) that accessor method are prefixed with get and set and guess, that the buttons label is called text, you can find the setText() method. You immediately learn that it needs a string as argument which as to be filled in.

Let's suppose this is the first time with aButton.

I'll white this code:

self halt.

Then I'll open an Inspector for aButton and play with it until I get enough knowledge of it.

From the inspector I'll try to change the name, send other objects as names (not only Strings) to get the felling of what can I do this it, etc, etc.

Play with objects, don't read code.

Of course, using a second browser, you can also search a Smalltalk system, but that's much more difficult IMHO. You have to interrupt your code writing, change to a second window, remember the type you where looking for, try to find all classes which fulfill that type (a manual step), browse their methods (another manual step), find the method, copy its name (yet another manual step) and switch back to the first browser. Alternatively, you can just guess the method name and globally search for implementors, then choose some method for an interesting looking class (you have to remember the class hierarchy and the type->class mapping as the method browser doesn't show) and continue as above. That is - IMHO - not as comfortable as a modern IDE (like Eclipse) with code completion is.

No... I don't use any of these options. I use the Debugger and Inspectors to play with objects and this is comfortable to me (and really funny!)

...

Could be, but the types generate more "rigid" (dead?) system more difficult to refactor.

Actually, I find refactorings much easier to do in a system with a type system. I wouldn't call that dead but well documented.

The system is documented, not ever well. The documentation shows the point of view of the writer in the moment of writing the code.

And, you'd probably agree, documentation helps refactoring.

That true only when the documentation is up to date. I spent several hours/days/weeks/etc refactoring documentation before the code.

Let's say you want to rename the method at: in your class to something better and also rename all senders of that method. You're lost in Smalltalk. Too many other classes also implement at: and it's very difficult to guess, which senders call your method and which not. So this refactoring is more a trial-and-error operation than a (how is that called) a semantic-conserting operation.

Yes... It's trial-and-error, what is wrong with this? How did you learn to walk, talk, etc?

...

...

They also provide some help for refactoring - there're of course no unit tests so you better don't change what you don't understand because you cannot break a system which is in use by more than 500 in-house customers.

I feel that here is the main point: UnitTest.

If you have a language and a system which requires unit tests, that could be as good as better documentation via types. However, as long as unit tests are voluntary, they don't really help.

If you have programmers trying to write bad code, you'll have bad code anyway. Writing unittest is voluntary just like write any code. In a typed system you can find method using Object in every where and casting all the time.

No tool/language can convert bad programmers in good ones.

Additionally, while it's a working quite well to test your core business model, it's much more difficult to test your UI.

Testing the UI is an open problem.

On the other side, the code in UI with typed languages is extremely more complicated than in ST.

And there's another point with unit tests. As they're written in the same language and are always run together with your code, can you assure that the semantic of your program doesn't change if you don't run them?

I don't follow you. What means "if you don't run them?"

A declarative type system (which isn't run anytime) cannot affect your code.

A declarative type system affect my code too much. The models generated most of the times are more complex than the equivalents without a typed system. Do you want an example? EJB.

[snip]

bye

Stefan Matthias Aust

Cheers,

Diego Gomez Deck

Diego Gomez Deck wrote:

...

As I tried to explain, in Java for example, it's mch easier to undestand new unknown frameworks once you've at least an idea what's going on. Just type

aButton.

and search the presented method set. If you know (in Java) that accessor method are prefixed with get and set and guess, that the buttons label is called text, you can find the setText() method. You immediately learn that it needs a string as argument which as to be filled in.

Let's suppose this is the first time with aButton.

I'll white this code:

self halt.

Then I'll open an Inspector for aButton and play with it until I get enough knowledge of it.

First, there's the problem, how to get that button. Often, a simple Button new is not enough. But once I get my button, sure I can look into an inspector or browse the buttons methods. Still, I've to guess. Trying out methods which might destroy my whole system is dangerous. And I have to look up a method in a browser, remember it, and type it. No help from the system, no tooltips, no code completion.

Play with objects, don't read code.

Okay, I just tried. Opening a workspace, typing

Button new

and pressing Alt+I. ...Hm, doesn't look like a morphic button (no owner, no bounds,...) Let's see

self browse

...ooops, error, drop keyboard, grab mouse, click on abandon. Trying to select browse with keyboard - the old error that you cannot shrink a selection via keyboard is still there (hello old friend) - so I grabbed again the mouse, select the word "browse" open, the popup menu... no browse menu (I moment later while writing this, I remembered, there's a second menu page) So again hand-on-keyboard, pressing Alt+B (I'm better in remembering shortcuts that menus, obviously)

A new browser opens and I learn that Behavior is the class which implements browse. Assuming that I don't know Behavior's place in the class hiearchy, I grab the mouse, open the popup menu and choose "browse hierachy", scrolling up and down in the up-popping browser, I decide that this is a class method, so ...

Diging down to my inspector be closing two browsers, I try

self class browse

and get to my browser I actually tried to open (just by guessing and playing around). Well... From

module: #(Squeak MVC Editors)

I guess that this is an old MVC component (the class comment isn't really helpful) and I close the browser and the inspector.

Let's try

MorphicButton ... Error MorphButton ... Error ButtonMorph ... Error

Grr... I get frustrated. But wait, there was a StringButtonMorph in the correction list so I choose that one. I've a new inspector.

I select self and open a hierachy browser. The class is shown at the top of the class list. Is it the root of the class hierachy? No, can't be. Wait, the pane as a scroll bar which is just too small to should that there's more above. I scroll down. StringButton inherits from StringMorph, I didn't expect that.

How do I get that Button on my screen? Probably a Morph should know...

I'm shocked! Class Morph has 1081 methods and more categories that I ever can look at at once, even if I resize the browser. I select --all-- and try to find something like #open or #show.

Jepp, there's a #show method. Great. I switch back to my inspector (which was totally covered by the browser) and type

self show

nothing happens. Let's try #openInWorld that sounds promissing.

Great there's a word "Flash" in the topleft edge of my screen.

I notice that "contents" contains Flash. I change that to 'Stefan' in the inspector. Nothing happens... I recheck, the variable really has the new value and a

self contents

correctly answers Stefan. Again frustrating. I change the actionSelector to #beep. A quick check

self beep

confirms that everything can beep, even if the "beep" sounds like a pot of coffee falls down. I click my button but nothing happens...

well with the exception that now it shows "Stefa"

Very frustrating. I give up.

That long rant might show that just exploring the system doesn't really help. Disclaimer: Don't try to help me, I know how Morphic works and how I should have proceeded. I just wanted to demonstrate some common mistakes. (Accepting "self" the target slot eventually makes the button to rattle on click)

The big question is whether code completion would have helped me. Not much, I'm afraid, but at least, it could have helped to reduce the large amout of methods I was confronted with.

No... I don't use any of these options. I use the Debugger and Inspectors to play with objects and this is comfortable to me (and really funny!)

I feel less productive in an unknown Smalltalk system (like Squeak has become to me) than an unknown Java system (like the Eclipse source code). Once you got an understanding of most of your system, things might be different.

I once felt much more productive with Smalltalk than with Java.

Yes... It's trial-and-error, what is wrong with this? How did you learn to walk, talk, etc?

I played around with Ned Konz connector morphs (great stuff!) and managed to look up Morphic four times just by following the examples. So trial and error isn't always the best way. It was quite frustrating so see my image (with all stuff unsaved) to become inaccessible, still reacting to my mouse moves, trying to route the connectoring throwing exceptions continuously.

No tool/language can convert bad programmers in good ones.

It can't do this automatically, but it can support programmers in learning to improve there skills. Reading other people's code helps. Smalltalk is great in this respect, as you can learn from the system's code -- if that's written in a clean and easy to read style.

...

And there's another point with unit tests. As they're written in the same language and are always run together with your code, can you assure that the semantic of your program doesn't change if you don't run them?

I don't follow you. What means "if you don't run them?"

I meant, how can you assure that your unit tests don't change the semantics of your code and once, if you remove the unit tests to get your production code, it will fail because something is missing.

I could be as simple as a useful extension #foobar, which the unit test package adds to your system and which you accidentally uses.

A declarative type system affect my code too much. The models generated most of the times are more complex than the equivalents without a typed system. Do you want an example? EJB.

Actually, EJBs are something which weren't made to run with Java. They do all kind of tricks to work around the limitations of Java. Could be so funny, if not anybody would want to use them :-)

bye

Stephan,

That is - IMHO - not as comfortable as a modern IDE (like Eclipse) with code completion is.

I completely agree but one other issue to keep in mind here is that we have a _dynamic_ type system which means that if you can get your hands on a life instance of some object you are perfectly capable to do completions and whatever else. Which in a way relates to what has been written about the "cult of the dead" a while ago and only shows that Smalltalk is not as dynamic as it ought to be ;-)

Incidentally, I did something along these lines a while ago, which basically just went out looking for existing objects and their instances and then figure out what is "typically" used there. And while it wasn't perfect it did allow you to get some rather interesting insights.

Actually, I find refactorings much easier to do in a system with a type system. I wouldn't call that dead but well documented. And, you'd probably agree, documentation helps refactoring.

Absolutely. It's one of the areas where documentation (hm ... actually I wouldn't call it documentation but rather automated consistency checking) is really crucial.

Let's say you want to rename the method at: in your class to something better and also rename all senders of that method. You're lost in Smalltalk.

Yes, but it's not a good example for two reasons. First, try to change the name of [] in Java (hah! got you there ;-) and second there are probably various methods which are so universally used in Java that you would end up with the same problems (like the Comparable interface).

If you have a language and a system which requires unit tests, that could be as good as better documentation via types. However, as long as unit tests are voluntary, they don't really help.

Good point. Which also relates to the point Gilad was making about optional type annotations being used or not. In the end its a management issue.

Cheers, - Andreas

Andreas Raab wrote:

...

If you have a language and a system which requires unit tests, that could be as good as better documentation via types. However, as long as unit tests are voluntary, they don't really help.

Good point. Which also relates to the point Gilad was making about optional type annotations being used or not. In the end its a management issue.

Unit tests could be understood as - examples with - method calls - and assertions.

How about a "strong" class browser, which makes sure, that we - always have a reproducable example of the current class, - initialized so that this method could be run (all variables one needs are initialized) - and (maybe even optional) an assertion holds afterwards.

Otherwise the method won't compile.

The tool I have in mind could be understood as a mixture of a debugger and a class browser and could be compared to Peter van Rooijen's http://www.esug.org/summerschools/2001_Essen/presentations/EnsuDemo/index. htm, though I am thinking more of an "example" than of a test driven development. Tests are then just the insertion of assertions.

(Certainly we always have our old "week" class browsers...)

Opinions?

Markus