<div dir="ltr">So, after digging a bit I've got some results and conclusions:<div><br></div><div>- The CMake configurations were using gcc to compile, which in mac is llvm-gcc </div><div>- Xcode uses clang compiler, not gcc (and some different compiling flags also)</div>
<div>- I've played changing the configuration to use clang compiler</div><div><br></div><div>setGlobalOptions: maker</div>
<p class=""> super setGlobalOptions: maker.<br> maker set: 'CMAKE_C_COMPILER' to: 'clang'.<br> maker set: 'CMAKE_CXX_COMPILER' to: 'clang'.</p><p class=""><br></p><p class="">And to make it work as in gcc I added the following also (in some plugins such as mp3plugin there are functions with return type and return statements with no values specified).</p>
<p class="">compilerFlagsRelease</p><p class="">
</p><p class=""> ^super compilerFlagsRelease, #( '-Wno-return-type' )</p><p class=""><br></p><p class="">And it compiled with the following results in the tinyBenchmarks:</p><p class="">
</p><p class="">'510723192 bytecodes/sec; -142407 sends/sec'</p><p class="">Which is, in the bytecode part, pretty much close to what we expect, and in the sends, looks buggy :).<br>But the overall performance using the image is far better</p>
<p class=""><br>Cheers,<br>Guille</p></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Jul 26, 2013 at 12:20 PM, Guillermo Polito <span dir="ltr"><<a href="mailto:guillermopolito@gmail.com" target="_blank">guillermopolito@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Ok, following with this. What I can add to the discussion:<div><br></div><div>In linux, latest VMs yield the following results (I added a space every three digits just to enhance readability)</div>
<div><br></div>
<div><div>"Pharo Cog"</div><div>1 tinyBenchmarks </div><div>'887 348 353 bytecodes/sec; 141 150 557 sends/sec'</div><div><br></div><div>"Pharo Stack"</div><div>1 tinyBenchmarks </div><div>'445 217 391 bytecodes/sec; 24 395 999 sends/sec'</div>
</div><div><br></div><div>While in Mac</div><div><br></div><div><div>"Pharo Cog"</div><div>1 tinyBenchmarks </div></div><div>'895 104 895 bytecodes/sec; 138 102 772 sends/sec'<br></div><div><br></div><div>
<div>"Pharo Stack"</div><div>1 tinyBenchmarks</div></div><div>'3 319 502 bytecodes/sec; 217 939 sends/sec'<br></div><div><br></div><div><br></div><div>So, I'd say it's a problem in cmake configuration or just compilation in mac :). Though I didn't test on windowze.</div>
<div><br></div><div>Another thing that I noticed is that when compiling my VM on Mac, since I updated Xcode, I was not longer using gnu gcc but llvm one. I tried to go back using the gnu gcc but couldn't make it work so far, he.</div>
<div><br></div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Feb 21, 2013 at 5:09 AM, Igor Stasenko <span dir="ltr"><<a href="mailto:siguctua@gmail.com" target="_blank">siguctua@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><br>
On 20 February 2013 18:29, Eliot Miranda <<a href="mailto:eliot.miranda@gmail.com" target="_blank">eliot.miranda@gmail.com</a>> wrote:<br>
><br>
> On Tue, Feb 19, 2013 at 11:10 PM, Camillo Bruni <<a href="mailto:camillobruni@gmail.com" target="_blank">camillobruni@gmail.com</a>> wrote:<br>
>><br>
>><br>
>> On 2013-02-20, at 01:25, Eliot Miranda <<a href="mailto:eliot.miranda@gmail.com" target="_blank">eliot.miranda@gmail.com</a>> wrote:<br>
>><br>
>>><br>
>>> On Tue, Feb 19, 2013 at 2:16 PM, Camillo Bruni <<a href="mailto:camillobruni@gmail.com" target="_blank">camillobruni@gmail.com</a>> wrote:<br>
>>>><br>
>>>>>> The most annoying piece is Time machine and its disk access, I<br>
>>>>>> sometimes forget to suspend it, but it was off during the<br>
>>>>>> tinyBenchmark.<br>
>>>>><br>
>>>>> One simple approach is to run the benchmark three times and to discard<br>
>>>>> the best and the worst results.<br>
>>>><br>
>>>> that is as good as taking the first one... if you want decent results<br>
>>>> measure >30 times and do the only scientific correct thing: avg + std deviation?<br>
>>><br>
>>> If the benchmark takes very little time to run and you're trying to<br>
>>> avoid background effects then your approach won't necessarily work<br>
>>> either.<br>
>><br>
>> true, but the deviation will most probably give you exactly that feedback.<br>
>> if you increase the runs but the quality of the result doesn't improve<br>
>> you know that you're dealing with some systematic error source.<br>
>><br>
>> This approach is simply more scientific and less home-brewed.<br>
><br>
> Of course, no argument here. But what's being discussed is using<br>
> tinyBenchmarks as a quick smoke test. A proper CI system can be set<br>
> it up for reliable results, but for IMO for a quick smoke test doing<br>
> three runs manually is fine. IME, what tends to happen is that the<br>
> first run is slow (caches heating up etc) and the second two runs are<br>
> extremely close.<br>
<br>
</div></div>but not in case when you have an order(s) of magnitude speed<br>
degradation. This is too significant to be<br>
considered as measurement error or deviation.<br>
There should be something wrong with VM (cache always fails?).<br>
<span><font color="#888888"><br>
> --<br>
> best,<br>
> Eliot<br>
</font></span><div><div><br>
<br>
<br>
--<br>
Best regards,<br>
Igor Stasenko.<br>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>