[Vm-dev] Some Performance Numbers: Java vs. CogVM vs. SOM

Stefan Marr smalltalk at stefan-marr.de
Mon Apr 6 20:50:01 UTC 2015


Hi Levente:

My goal with those benchmarks is to compare how well the just-in-time compilers optimize the code.
It is not a goal to write the most efficient version of the code, taking the known implementation details into account. I want those benchmarks to be reasonably high-level, reasonably idiomatic code.

> On 06 Apr 2015, at 22:24, Levente Uzonyi <leves at elte.hu> wrote:
> 
> I've checked the code, and ran some benchmarks in Squeak. When I tried to load the code, I got complaints from the system, because the name of class variables begin with lowercase letters.
> ScriptCollector is also missing from Squeak, though it's easy to work this
> around.
> There are still plenty of #% sends in the code, which I had to rewrite to #\\.

Yes, I did not commit the Squeak compatible code. It’s only in my image.
If someone wants it, I could probably upload it somewhere.


> The PageRank benchmark is so slow that I stopped running it after about 30 minutes. The profiler shows, that it spends over 95% of the time in SomJenkinsRandom class >> #random. I've got a faster (~6x) version of that PRNG, but it's still way to slow. One can consider this as a weakness of the system, but it's also a weak point of the benchmark, that relies so heavily on a fast PRNG implementation. The code is also pretty bad, because it uses only a few bits out of the generated 32, and it has to fight with the signed result. Whoever came up with using that “PRNG" hasn't really put much thought in it...

Yes, the RNG is slow. If you got a faster version, I could surely integrate it thought. As long as it isn’t fundamentally changing things. As mentioned about, for me, the benchmarks are intended to measure how well the optimizers work.

> About porting:
> I don't know what your original goal was, but I don't see why you would keep 0 based indexing in the code. Smalltalk uses 1-based indexing, and this definitely has a negative impact on the Smalltalk results. If you were to port code from Smalltalk to Java, would you keep the 1-based indexes?

The problem here is that not all indexing is pure indexing. Sometimes the indexes are used as values. Getting that right is a pain. So for some benchmarks I gave up after trying to convert to 1-based indexes.


> Another thing is about types:
> [1 - SomPageRank DFactor / n] 

> private static double D_FACTOR = 0.85; // damping factor
> 
> If I were to port this, and I would want to stick to the implementation, then D_FACTOR would be a class variable, and the code would read as: [1.0 - DFactor / n]. But knowing that the constant is not used anywhere else, I see no problem with precalculating the value.

That’s the job of the optimizer/JIT compiler. And, also SOM doesn’t have class variables.
So, I’ll stick with the current code. That’s a puzzle for Sista to solve. Not something I would want to change in the code.

You might be tempted to do such optimizations, because you know the currently limits of the execution mechanisms. But for me, those benchmarks are about measuring how good the VMs optimize and not, how good you know your VMs.

Best regards
Stefan

-- 
Stefan Marr
INRIA Lille - Nord Europe
http://stefan-marr.de/research/





More information about the Vm-dev mailing list