[Vm-dev] SpurVM performance "regression"?

[Nicolas Cellier]

+25% is certainly more than noise and a concern

2016-04-03 13:32 GMT+02:00 Holger Freyther <holger at freyther.de>:

>
> Hi Guys,
>
> for one of our GSM components we have a simple decoding benchmark. The
> plot of the benchmark runs across Pharo builds can be seen here[1]. One can
> clearly see the improvement around the 15th of December 2015 when Spur was
> enabled in Pharo5. Around the 16th January we see a reduction in
> performance and the values seem to be more "noisy" as well.
>
> Today I finally had some minutes to try to reproduce this in a more stable
> environment and just by using Eliots VMs. I have downloaded the last build
> of 2015 and the four ones available for this year and executed the mini
> benchmark and wrote the attached script. The results can be seen in
> results.csv.
>
> Now is this a "real" regression or something else? E.g. if I decide to not
> trust the time measurement done by smalltalk itself and use "time" to
> execute the benchmark, I do see a similar ~500ms difference. Is this of
> concern?
>
> If there are more builds I am happy to bisect this further, I can make the
> benchmark image available as well.
>
> kind regards
>         holger
>
>
>
>
>
>
>
>
>
> [1] https://ci.hartl.name/job/TCAP-pharo5-nightly/plot/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20160403/fc033bff/attachment.htm