>From: Andreas Raab <andreas.raab at gmx.de>
>Geesh. This "bechmark" is as bogus as it gets. Unbelievable. Try it with 
>polymorphic receivers (say more than 3 different ones to kill the cache 
>performance for a change). Seriously, benchmarks should be defined by 
>people who understand what they are measuring.

Wow. What an amazing vitriolic tongue lashing to someone who is just trying 
to help. I think the only unbelievable thing is your response. I think this 
is a perfect opportunity to educate, not tear down. The Squeak community is 
one of the best in the world, but responses like this make people not want 
to participate. It's one of the reasons that I have been more of a lurker. 
It's intimidating. I don't want to feel the wrath of a petulant child on a 
mailing list.

>My comparison was based on the fact that the microbenchmarks show that *on 
>average* the difference between bytecodes and sends is roughly 1:30. On 
>*average* mind you, not on a totally bogus, 
>send-the-same-message-to-the-same-object-with-guarantueed-cache-hit 
>"benchmark".

Again, this is a perfect opportunity to educate, if his understanding was 
wrong, all you had to do was to tell him what the correct way was. The last 
sentence was over the top and unnecessary. Is this not a helpful community?

>And besides, system performance has very little to do with these 
>micro-benchmarks anyway - I don't buy *any* of those points made about how 
>horrible it is that we aren't aware about how much time we "waste" when 
>sending a message instead of utilizing a bytecode.

But, shouldn't we be fast at message sending? Isn't Smalltalk known for 
message sending? It should be fast.  Maybe I'm missing something here and I 
apologize for bursting into this argument. But, I do know that speed on the 
small doesn't matter if the algorithms are bad on the large.

>Let's stop this pointless discussion, shall we?

I think this is a valid discussion, because I would love to know how to make 
valid benchmarks in Squeak. Making Squeak better should always be up for 
discussion. I think you should make benchmarks without internal knowledge of 
the VM. Why? Because you can cook the benchmarks to avoid where the problem 
areas are in the VM. Does Smalltalk have a standard set of VM benchmarks? 
Besides, I do not see the problems with Peace's benchmarks. They seem to be 
valid to me. I would like a humane discussion on why they aren't. Thanks.

>   - Andreas
>
>Peace Jerome wrote:
>>Some comments andreas made got me curious as to how
>>much speed was given up for correctness.
>>
>>So I did a little test.
>>foo isNil vs. foo == nil .
>>
>>My result seem to show that former is half as fast as
>>the latter.
>>
>>
>>After I realized I had to do a control to eliminate
>>the time spent looping from the comparison. And I
>>repeated the test ten times in each loop so the
>>looping time didn't swamp the out come.
>>
>>1000000 timesRepeat: [. . . . . . . . . ]]
>>
>>For a G3 imac running under OS9.1 my results were
>>control: 900
>>isNil     3000
>>== nil   2000
>>
>>Once the control was subtracted out the time factor
>>was about 2::1  regardless of the value of foo.
>>
>>If you want to try this on your system the text of the
>>test was:
>>
>>
>>
>>[ { nil . true  . false . Morph new } collect: [ :foo
>>| {
>>		[1000000 timesRepeat: [. . . . . . . . . ]] .
>>	[1000000 timesRepeat: [foo isNil . foo isNil . foo
>>isNil . foo isNil . foo isNil . foo isNil  . foo isNil
>>. foo isNil . foo isNil . foo isNil]] .
>>[1000000 timesRepeat: [ foo == nil . foo == nil . foo
>>== nil . foo == nil . foo == nil . foo == nil . foo ==
>>nil . foo == nil . foo == nil]] } collect: [
>>:testBlock | testBlock timeToRun ] ] ] value collect:
>>[     :results | results second - results first /
>>(results third - results first ) asFloat ]
>>
>>and my results for this example were:
>>
>>
>>#(2.03679245283019
>>  2.051643192488263
>>  2.058878504672897
>>  2.06578947368421)
>>
>>This is not a conclusive indication of why squeak is
>>now slower.  It is an indication that more examination
>>will be useful.  A lot of  inner loop stuff was
>>changed without regard for the speed impact.
--
Blaine Buxton, Mad Scientist In Training
My Amps: Smalltalk, Lisp, and Ruby
http://www.blainebuxton.com

_________________________________________________________________
On the road to retirement? Check out MSN Life Events for advice on how to 
get there! http://lifeevents.msn.com/category.aspx?cid=Retirement