[Vm-dev] SpurVM performance "regression"?

[Holger Freyther]

Hi Guys,

for one of our GSM components we have a simple decoding benchmark. The plot of the benchmark runs across Pharo builds can be seen here[1]. One can clearly see the improvement around the 15th of December 2015 when Spur was enabled in Pharo5. Around the 16th January we see a reduction in performance and the values seem to be more "noisy" as well.

Today I finally had some minutes to try to reproduce this in a more stable environment and just by using Eliots VMs. I have downloaded the last build of 2015 and the four ones available for this year and executed the mini benchmark and wrote the attached script. The results can be seen in results.csv.

Now is this a "real" regression or something else? E.g. if I decide to not trust the time measurement done by smalltalk itself and use "time" to execute the benchmark, I do see a similar ~500ms difference. Is this of concern?

If there are more builds I am happy to bisect this further, I can make the benchmark image available as well.

kind regards
	holger



-------------- next part --------------
A non-text attachment was scrubbed...
Name: bench.sh
Type: application/octet-stream
Size: 572 bytes
Desc: not available
Url : http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20160403/b093eab2/bench.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: results.csv
Type: text/csv
Size: 610 bytes
Desc: not available
Url : http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20160403/b093eab2/results.bin
-------------- next part --------------





[1] https://ci.hartl.name/job/TCAP-pharo5-nightly/plot/