Tobias,

That's excellent to know... I wasn't aware that this was being looked at.  Something that's been in the back of my mind for a while is trying to measure JIT performance vs. JIT code cache size, architecture being run on, and workload.  As I understand it, the JIT code cache is a fixed size (~1meg iirc) which doesn't make a lot of sense to me given the variance in CPU cache size (generally in the range of 512k-4m depending on whether you're running on Intel/AMD/ARM, desktop/server, low/high end, HT enabled/disabled, and oddball/extreme CPU configurations like the Intel 5775C with it's 128meg LLC) and then there's the cost of potentially thrashing the JIT cache vs maintaining a larger cache despite the fact that it might exceed the capacity of the CPU cache. (However to measure this I think a few new VM params would be needed: a r/w JIT cache size, JIT cache code evictions (since last full gc?), JIT cache code additions (since last full gc?), Avg cache code size... the current machine code method/compaction counts are a good start but a bit coarse). Just thought I'd throw this out there since it seemed apropos...

Thanks,
Phil

On Thu, Sep 20, 2018, 5:02 AM Tobias Pape <Das.Linux@gmx.de> wrote:
 
Hi Todd,

> On 20.09.2018, at 10:28, Todd Blanchard <tblanchard@mac.com> wrote:
>
>
> I'm curious if anyone has seen this writeup on VM performance
>
> https://tratt.net/laurie/blog/entries/why_arent_more_users_more_happy_with_our_vms_part_2.html
>
> I found the article very insightful and was curious how we stacked up.
>
> Have written the author to ask about including us in future research.
>
> Cheers.
> -Todd Blanchard
>

I know the author  and share similar concerns.
In fact, for stuff we do here with RSqueak/VM and GraalSqueak, we pay attention to at least some of the concerns Laurie mentions.

Have a look at SMark, which tries to give a reasonable benchmark set set.

http://smalltalkhub.com/#!/~StefanMarr/SMark

also, there's our benchmark runner that builds on SMark

http://www.hpi.uni-potsdam.de/hirschfeld/squeaksource/BenchmarkRunner.html


We also have a codespeed up here: http://speed.squeak.org/

Yea that does not all address the authors issues, but we try :D

Best regards
        -Tobias