[Vm-dev] CogVM Benchmarking

[Stefan Marr]

Hi Eliot:

When exactly is the JIT compilation triggered in Cog?

I would like to provide a configuration for the SMark benchmarking framework that ensures that everything got jitted, without doing to much unnecessary warmup.

All the benchmark code is implemented in standard methods and executed very similarly to how SUnit works, thus, they are called like 'suite perform: aBenchmarkSelector'.


How many iterations should I run the code before I can be sure that it is jitted?
From your blog, I assume, it should be jitted after its second execution. (see snippet below)

And I supposed a 'Smalltalk garbageCollect' should not interfere with this, and can be safely performed before a timed run?
Is there anything else, I should cover in the framework? Any particular heuristics/mechanism that should be taken into account when trying to reach a stable state?


What I found on your blog is the following:
<<<
So a simple way to implement the interpret on single-use policy is to only compile to machine code when finding a method in the first-level method lookup cache. We avoid compiling large methods, which are typically initializers, and rarely performance-critical inner loops, by refusing to compile methods whose number of literals exceeds a limit, settable via the command line that defaults to 60 literals, which excludes a mere 0.3% of methods in my image.
>>>

Thanks a lot
Stefan




-- 
Stefan Marr
Software Languages Lab
Vrije Universiteit Brussel
Pleinlaan 2 / B-1050 Brussels / Belgium
http://soft.vub.ac.be/~smarr
Phone: +32 2 629 2974
Fax:   +32 2 629 3525