Benching #Smalltalk vs Squeak
goran.krampe at bluefish.se
goran.krampe at bluefish.se
Tue Nov 25 08:22:42 UTC 2003
"John Brant" <brant at refactory.com> wrote:
> > From: squeak-dev-bounces at lists.squeakfoundation.org [mailto:squeak-dev-
> > bounces at lists.squeakfoundation.org] On Behalf Of goran.krampe at bluefish.se
> >
> > This weekend I installed .Net 1.1 runtime and Mono-0.28 on my XP laptop.
> > Then I downloaded SharpSmalltalk and tried the benchmarks. Also ran the
> > benchmarks on my 3.7alpha image using the latest Win32 VM 3.6.2. I
> > include the results below.
> >
> > By an untrained eye it looks like Mono is much slower than MS runtime
> > (but note that I don't know if I should run mono using some neat
> > switches or anything, I just guessed on "mono" :)) - but I still find it
> > nice that it works at all.
>
> The Mono VM is several times slower than the Microsoft VM running #Smalltalk
> code. I'm not sure, but I'm guessing that some of this time is due to the
> garbage collectors used. I believe that Mono uses the Boehm GC. Since
> #Smalltalk uses boxed SmallIntegers, it really stresses the garbage
> collector.
Ah. Yes, Mono still uses the Boehm according to the homepage. They write
that they are moving over to Intel's ORP (or something similar, might
have gotten the name wrong).
I also noted that by only typing "mono" thus getting help, there are
switches that supposedly turns optimizations on - but I couldn't get
that to work (it barfed on the switch). Searched around on the net about
that but actually came up empty for once.
> You can see some GC problems in Mono if you try to evaluate large
> factorials. For example, if you evaluate "40000 factorial / 39999
> factorial", it will crash mono, but Microsoft's VM runs it just fine.
Ok. Even though, neat that it works, right? :) I wonder if it is as slow
on say Linux.
> > Also - the MS runtime + SharpSmalltalk has pretty decent numbers
> > compared to Squeak. Half the bytecode speed but faster on sends (though
> > I assume tinyBenchmark is only a hint).
>
> The bytecode benchmark is essentially showing the #Smalltalk's SmallIntegers
> are really slow.
Ok.
> > If we look at SlopStone (I just used the "Benchmark" package on SM - not
> > sure if it is the same code that SharpSmalltalk includes in its
> > download, but I guess it is) it seems that .Net shines on adding floats
> > (11 times faster), creating objects (7 times faster) and evaluating
> > blocks (31 times faster).
>
> Yes, both are ports of Bruce Samuelson's benchmarks. I ran the tests a few
> months ago, and #Smalltalk was faster than Squeak on all of the Slopstone
> and Smopstone benchmarks except adding integers and forming sets.
Yes, I only mentioned those where Squeak got really "squashed" since
those might be interesting to understand the reasons why.
> Since
> #Smalltalk uses integer objects instead of tagged integers for
> SmallIntegers, I expect that to be slower. I'm not sure why the forming sets
> test was slower -- it may be due to SmallInteger arithmetic performance.
>
>
> > Of course - these numbers look "suspiciously good". :) Smopstone I can't
> > make heads or tails of - and the printouts aren't the same so perhaps
> > the tests have changed too.
>
> I had to remove a couple tests from the original benchmarks. The sorcerer's
> apprentice was testing rectangle's and #Smalltalk doesn't have any classes
> for rectangles (at least not yet). I also had to remove the generating and
> parsing streams benchmark. Its code was written assuming that a floating
> point number always printed a decimal point, but in #Smalltalk "1.0
> printString = '1'".
Ok - but I also got very confused about the numbers - it didn't seem
like the smopstone number and the execution times are... properly
"related". But I didn't dig into it. :)
For example:
SSt+MS.net
0,6909936 22,8439163546522 generating fractonaccis
0,75108 20,577035735208 generating strings
Squeak
0.499 6.32665330661323 generating fractonaccis
0.187 16.52941176470588 generating strings
So... isn't short time = higher smopstone? For some tests it seems to
be, and not for others.
> > ---------------------------------------------------
> > "Below is running Benchmark using .NET 1.1"
> > C:\SharpSmalltalk\Source\Examples>Benchmarks.exe
> > 49928103 bytecodes/sec; 6900998 sends/sec
> >
> > Starting benchmarks with repetition count = 16000.
> >
> > 1000s time 1000s of
> > itera- sec- iterations slop-
> > tions onds per sec stones explanation
> >
> > 3808 0,200288 19013 2,8807002764504 add integers
> > 544 0,0200288 27161 113,170368003408 add floats
> > 960 0,050072 19172 21,737405618902 access strings
> > 320 0,0300432 10651 30,2594566854033 create objects
> > 160 0,0400576 3994 37,3294232006845 copy objects
> > 480 0,200288 2397 5,63893875172692 perform selectors
>
> Most likely if you ran the benchmarks twice, the perform selectors would be
> much faster the second time. Whenever you perform a selector, the first time
> it is performed, it compiles a block like thing that is directly evaluated
> the next time the selector is performed. I'm guessing that you are seeing
> some bad performance since the classes that .NET uses to generate code
> haven't been jitted yet. I get similar numbers to yours on the other
> benchmarks, but I get 14.1 on the perform selectors.
Ok.
> BTW, these benchmarks should be changed. #Smalltalk's and VisualAge's clocks
> only give 10ms resolution, so most of these benchmarks are close to
> underflowing.
>
>
> John Brant
Thanks for the feedback John. And it sure is nice to have another free
Smalltalk on the block!
Especially one that might keep C# at bay. :)
Btw, how do *you* write SharpSmalltalk programs? I mean, what IDE do you
use?
regards, Göran
More information about the Squeak-dev
mailing list
|