[Vm-dev] CogVM Benchmarking

[Stefan Marr]

Hi Eliot:


On 16 May 2011, at 17:42, Eliot Miranda wrote:

> So to be sure that you're measuring pure performance, and not any overhead you really need to measure the third evaluation of any benchmark. The first evaluation will load the method lookup caches.  The second will compile everything, but compilation and linking might introduce some overhead.  The third evaluation should be running at full speed.  But you might want to time the successive evaluations.  In my experience compilation and linking overhead is very low so I expect you'll be hard-pressed to see much difference between the second and third runs.

Great, thanks. That's what I wanted to verify.


> Forcing finalization?  Again in my experience having a still machine is very important.  You'll see variations in timing caused by other activities on the machine (Time Machine, your mailer uploading mail from the server etc).
Sure, that's what the framework, a few iterations and some statistics will take care off.
And, a dedicated benchmark machine...

Now let's how many cores I need to overtake the CogVM ;)

Thanks
Stefan




-- 
Stefan Marr
Software Languages Lab
Vrije Universiteit Brussel
Pleinlaan 2 / B-1050 Brussels / Belgium
http://soft.vub.ac.be/~smarr
Phone: +32 2 629 2974
Fax:   +32 2 629 3525