Thue-Morse and performance: Squeak v.s. Strongtalk v.s.
VisualWorks
Klaus D. Witzel
klaus.witzel at cobss.com
Sun Dec 17 12:06:39 UTC 2006
Hi Andreas,
on Sun, 17 Dec 2006 11:52:36 +0100, you wrote:
> Klaus D. Witzel wrote:
>> I'm disappointed, Strongtalk was always advertised as being the fastest
>> Smalltalk available "...executes Smalltalk much faster than any other
>> Smalltalk implementation...", and now it shows to be in almost the same
>> class as Squeak is :) :(
>> Can somebody reproduce the figures, any other results? Have I done
>> something wrong?
>
> Yes. First, you are equating the result of a single micro-benchmark with
> overall system performance. Micro-benchmarks are used to measure
> specific aspects of a particular implementation. Your benchmark measures
> highly polymorphic send performance. Which is not typical for Smalltalk
> code to begin with.
>
> In other words, your claim is based on measuring a single atypical
> performance characteristic.
Not really. There are at least two motivations:
- a - being faster means >= and the = part is missing
- b - the test puts some stress on the call site, any specific suggestion
from your side on how to test and compare that on typical situations (this
is *not* a rhetorical question)? And, no it's not atypical, see below.
> This has *nothing* to do with "Smalltalk performance". If you want to
> measure "Smalltalk performance" you should run a number of the standard
> benchmarks (Richards, Slopstone) that come with Strongtalk and compare
> those.
Well, how about something new, or are you after stangnation, Andreas (no
offense, really ;-)
> Quite honestly, I'm surprised to see a person like you who obviously
> understands enough about dynamic systems to measure PIC effects to make
> such unsubstantiated claims.
O.K. I understand that as lack of use case. Take this (take that ;-)
| allCs |
allCs := Smalltalk allClasses.
"start timing here"
1 to: allCs size do: [:i | (allCs at: i) methodDict "just access the
iVar"]
"note that #yourself from the previous example is now just #methodDict"
This snippet is performed on behalf of every developer who asks for
senders and/or implementors. It is, IMHO, the most often used piece of
code of every Smalltalk, ever.
So I limited the amount of information to be handled by PICs (as noted in
the comment of Michael), otherwise the comparision would've just been on
the possible "methodCache" performance (which would've been a nice test as
well [after eliminating differences with (Smalltalk size)] but I was not
looking for that).
But, even for the latter I expect to find the = in >= when Strongtalk
"...executes Smalltalk much faster than any other Smalltalk
implementation...".
> I would expect that you know how to evaluate the results of a
> micro-benchmarks, and I would in particular expect that you know that
> 80-90% of all call-sites in realistic code are mono-morphic to begin
> with which render your benchmark results absolutely useless for
> "Smalltalk code".
Absolutely not (yes, I know about these figures. no, I disagree: see
above).
BTW: does anybody know about recent work in the direction of "Message
Dispatch on Pipelined Processors", thanks for any pointers.
/Klaus
> Cheers,
> - Andreas
>
>
More information about the Squeak-dev
mailing list
|