[Vm-dev] benchmarking and recent results

[Clément Bera]

Hello,

Recently I needed vm benchmarks for different projects. I merged old Squeak
benchmarks that were ported by Stefan Marr to SMark with some other classic
benchmark I found on the web to have a suite. I am now setting up a
benchmarking machine for our continuous integration server (running a bench
on a VM is not accurate). You can find an image with the benchs here:
https://ci.inria.fr/rmod/view/Mate/job/Pharo-bench/ , evaluate
(SMarkVMBench run: 5) to try (takes a few dozens of seconds).

Currently I have:

   - Richards: OS kernel simulation benchmark. The main focus is on
   property access and calling functions and methods.
   - DeltaBlue: One-way constraint solver Benchmark. The main focus is on
   polymorphism and object-oriented programming.
   - Stones: (Slopstone & Smopstone) Smalltalk Low level Operation Stones &
   Smalltalk Medium level Operation Stones, cpu intensive benchmarks
   - Loops: A set of microbenchmarks measuring basic aspects such as
   message send, instance field access, class var bindings, float and int
   arithmetic,and array access cost
   - Compiler: Evaluate the compilation time

Do you have any other benchs implemented in Squeak / Pharo ? If someone can
extract the benchs from strongtalk in the Squeak chunk format I would be
pleased. Seemingly you can only do that on windows and I don't have access
to this kind of computer.

An interesting fact I wanted to mention: I run the benchs twice, once with
cog before the last update (without frameless optimization) and once with
the latest Cog, and I noticed a difference of 7% of performance on
DeltaBlue and Richards. I don't use in this bench an excessive number of
accessors (I mean I use accessors, but I don't use the policy where you use
accessors inside the object's methods to access its instance variable). Is
this an expected result with the frameless optimization ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20131002/213e083a/attachment.htm