Such a small benchmark

Andreas Raab squeak-dev at lists.squeakfoundation.org
Thu Oct 17 19:07:11 UTC 2002


Viktor,

> Producers of Dolphin Smalltalk (www.object-arts.com) said 
> theirs Smalltalk is pure interpreter too.

Oh sure it is, but your benchmark is no good. What you are measuring is
a specific optimization - namely whether the compiler inlines
#timesRepeat: messages or not. To prove that point I've copied the
message timesRepeat: in Dolphin, VW, and Squeak and renamed it
#timesRepeat2: (to prevent the compiler recognizing it). Let's have a
look at the results when we use timesRepeat: or timesRepeat2: with your
code (taken on a 1.2GHz P3):

		timesRepeat:		timesRepeat2:
VW			 380ms			 680ms
Dolphin		1311ms			3447ms
Squeak		3351ms			3358ms

So as you can see, both VW and Dolphin double the time if you defeat
this specific compiler optimization. Duh. So much for your benchmark -
in VW and Dolphin it was measuring byte code + array speed, in Squeak it
was measuring byte code + array + block activation speed. Not exactly
the same and if you want to run the benchmark in the mood of its C
version then you have to write it the way Bert wrote it - in which case
again Squeak and Dolphin are on equal grounds again.

Sigh. Methinks we should optimize it too - if only for the sake of
people horribly misinterpreting what it actually is they measure...

Cheers,
  - Andreas




More information about the Squeak-dev mailing list