Thue-Morse and performance: Squeak v.s. Strongtalk v.s. VisualWorks

Klaus D. Witzel klaus.witzel at cobss.com
Sun Dec 17 12:06:39 UTC 2006


Hi Andreas,

on Sun, 17 Dec 2006 11:52:36 +0100, you wrote:
> Klaus D. Witzel wrote:
>> I'm disappointed, Strongtalk was always advertised as being the fastest  
>> Smalltalk available "...executes Smalltalk much faster than any other  
>> Smalltalk implementation...", and now it shows to be in almost the same  
>> class as Squeak is :) :(
>>  Can somebody reproduce the figures, any other results? Have I done  
>> something wrong?
>
> Yes. First, you are equating the result of a single micro-benchmark with  
> overall system performance. Micro-benchmarks are used to measure  
> specific aspects of a particular implementation. Your benchmark measures  
> highly polymorphic send performance. Which is not typical for Smalltalk  
> code to begin with.
>
> In other words, your claim is based on measuring a single atypical  
> performance characteristic.

Not really. There are at least two motivations:

  - a - being faster means >= and the = part is missing
  - b - the test puts some stress on the call site, any specific suggestion  
 from your side on how to test and compare that on typical situations (this  
is *not* a rhetorical  question)? And, no it's not atypical, see below.

> This has *nothing* to do with "Smalltalk performance". If you want to  
> measure "Smalltalk performance" you should run a number of the standard  
> benchmarks (Richards, Slopstone) that come with Strongtalk and compare  
> those.

Well, how about something new, or are you after stangnation, Andreas (no  
offense, really ;-)

> Quite honestly, I'm surprised to see a person like you who obviously  
> understands enough about dynamic systems to measure PIC effects to make  
> such unsubstantiated claims.

O.K. I understand that as lack of use case. Take this (take that ;-)

  | allCs |
  allCs := Smalltalk allClasses.
  "start timing here"
  1 to: allCs size do: [:i | (allCs at: i) methodDict "just access the  
iVar"]
  "note that #yourself from the previous example is now just #methodDict"

This snippet is performed on behalf of every developer who asks for  
senders and/or implementors. It is, IMHO, the most often used piece of  
code of every Smalltalk, ever.

So I limited the amount of information to be handled by PICs (as noted in  
the comment of Michael), otherwise the comparision would've just been on  
the possible "methodCache" performance (which would've been a nice test as  
well [after eliminating differences with (Smalltalk size)] but I was not  
looking for that).

But, even for the latter I expect to find the = in >= when Strongtalk  
"...executes Smalltalk much faster than any other Smalltalk  
implementation...".

> I would expect that you know how to evaluate the results of a  
> micro-benchmarks, and I would in particular expect that you know that  
> 80-90% of all call-sites in realistic code are mono-morphic to begin  
> with which render your benchmark results absolutely useless for  
> "Smalltalk code".

Absolutely not (yes, I know about these figures. no, I disagree: see  
above).

BTW: does anybody know about recent work in the direction of "Message  
Dispatch on Pipelined Processors", thanks for any pointers.

/Klaus

> Cheers,
>    - Andreas
>
>





More information about the Squeak-dev mailing list