Compiler testing on Mac (was Re: [Vm-dev] The Mac VM (pharo))

[Göran Krampe]

Hey!

I am struggling on with building VMs on Mac.

On 10/01/2013 03:11 PM, Göran Krampe wrote:
> I am now working on a build using GCC 4.9 - not through it yet, but almost.

Done, but unfortunately it gives "Bus error: 10" when I start it. Will 
see if I can get more info from that, I will also test older GCCs.

> For some interesting silly benchmarks:
>
> Stock binary pharo-vm (presume built with GCC 4.2???):
>      133 million sends, 800 mill bytecodes.
>
> Built with clang from Xcode 5:
>      121 million sends, 790 mill bytecodes.

We all know how trustworthy tinyBenchmarks is of course, but still, this 
is interesting. Bytecodes for the above clang compiled VM seem to land 
on around 820 mill no matter what I do.

Now I tried this VM some more, starting it up fresh 3 times and running 
tinyBenchmark a bunch of times giving me these runs:

1: 131, 137, 138, 138
2: 126, 128, 128, 128, 128, 127
3: 123, 138, 136, 138, 138

First run seems often a bit slow as Eliot explained. I have no idea why 
it "got stuck" on 128 on run #2. :)

BUT... hey! Most often around 138 million sends, and this is clang!

> Binary 2776 from Eliot:
>      114 mill sends, but 980 mill bytecodes.

Now I ran this VM several times more and can conclude that it seems to 
be very similar in performance as the clang VM BUT... with slightly more 
bytecodes performance, around 980-1000.

And what I find more interesting: It matters what image you use!

I ran using the Pharo 2.0 image that got sucked down in the VM building 
process, and I also ran using an older Squeak 4.1 based OpenQwaq image 
and... although they both have the same bytecodes in the benchFib method 
the Pharo *image* is consistently slower on sends and also more variant.

I am not dwelving into that, but a little "headsup" here. :)

My personal conclusion: clang is a true option now

regards, Göran