[Vm-dev] benchmarking and recent results

[Eliot Miranda]

Hi Clément,

    I have some of the computer benchmarks game benchmarks coded up in
ShootoutTests in the Cog package.  I use these for benchmarking.  I'd love
to see Richards & DeltaBlue added to them.  IMO, Stones should be ignored
totally, if not entirely crushed.  It is a series of irrelevant
microbenchmarks.


On Tue, Oct 1, 2013 at 11:46 PM, Clément Bera <bera.clement at gmail.com>wrote:

>
> Hello,
>
> Recently I needed vm benchmarks for different projects. I merged old
> Squeak benchmarks that were ported by Stefan Marr to SMark with some other
> classic benchmark I found on the web to have a suite. I am now setting up a
> benchmarking machine for our continuous integration server (running a bench
> on a VM is not accurate). You can find an image with the benchs here:
> https://ci.inria.fr/rmod/view/Mate/job/Pharo-bench/ , evaluate
> (SMarkVMBench run: 5) to try (takes a few dozens of seconds).
>
> Currently I have:
>
>    - Richards: OS kernel simulation benchmark. The main focus is on
>    property access and calling functions and methods.
>    - DeltaBlue: One-way constraint solver Benchmark. The main focus is on
>    polymorphism and object-oriented programming.
>    - Stones: (Slopstone & Smopstone) Smalltalk Low level Operation Stones
>    & Smalltalk Medium level Operation Stones, cpu intensive benchmarks
>    - Loops: A set of microbenchmarks measuring basic aspects such as
>    message send, instance field access, class var bindings, float and int
>    arithmetic,and array access cost
>    - Compiler: Evaluate the compilation time
>
> Do you have any other benchs implemented in Squeak / Pharo ? If someone
> can extract the benchs from strongtalk in the Squeak chunk format I would
> be pleased. Seemingly you can only do that on windows and I don't have
> access to this kind of computer.
>
> An interesting fact I wanted to mention: I run the benchs twice, once with
> cog before the last update (without frameless optimization) and once with
> the latest Cog, and I noticed a difference of 7% of performance on
> DeltaBlue and Richards. I don't use in this bench an excessive number of
> accessors (I mean I use accessors, but I don't use the policy where you use
> accessors inside the object's methods to access its instance variable). Is
> this an expected result with the frameless optimization ?
>
>
>
>


-- 
best,
Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20131002/9517be7f/attachment.htm