[Vm-dev] Call for big benchmarks

[John Dougan]

I don't know if this qualifies, but I ported John Walker's fbench floating
point accuracy benchmark (https://www.fourmilab.ch/fbench/fbench.html) to a
variety of Smalltalk platforms. The numerical code is written in the
standard Numerical Recipes style, which isn't very Smalltalky, but is very
common. Probably lots of opportunities for optimizations. The included code
tries to write to stdout as it was designed to be called from the command
line, but that is pretty trivial to change.

Cheers,
 -  John

On Fri, Mar 24, 2017 at 1:10 AM, Tim Felgentreff <timfelgentreff at gmail.com>
wrote:

>
> Hi Eliot,
>
> the question for me is, how indicative is this workload of real world
> performance? Creating compiled methods may not be something that is highly
> optimized, simply because it doesn't need to be in real applications. One
> would have to be careful about what is being measured, or if the benchmark
> is just measuring how fast we can blow out the caches...
>
> If we're just talking about running parsing and optimizing something, then
> maybe some real world applications are using that, but even then some JSON
> or HTML parsing library that implements e.g. Apache mod_rewrite would be
> more realistic, I think. Dynamically parsing and patching HTML and then
> pretty-printing or minimizing it seems a more common problem.
>
> I know, you're trying to argue that the Opal compiler may show common
> workloads equally well, but we could argue that for some of the Shootout
> benchmarks, too. It's an argument that doesn't seem to convince some people.
>
>
> Eliot Miranda <eliot.miranda at gmail.com> schrieb am Do., 23. März 2017,
> 17:18:
>
>>
>> Hi Tim,
>>
>> On Thu, Mar 23, 2017 at 1:31 AM, Tim Felgentreff <
>> timfelgentreff at gmail.com> wrote:
>>
>>
>>
>> Yes, big benchmarks would be nice. Those on speed.squeak.org or in
>> VMMaker are all somewhat small.
>>
>> Note the Ruby community, for example, has benchmarks such as a NES
>> emulator (optcarrot) that can run for a few thousand frames with predefined
>> input as benchmarks. It's definitely possible.
>>
>> Maybe some of the projects from HPI students could be made to work, there
>> was a Chip8 emulator in Squeak, for example, that seems big enough. Or
>> maybe the DCPU emulator at github.com/fniephaus/BroDCPU without a frame
>> limit would work as a decent CPU bound benchmark.
>>
>>
>> I've discussed with Clément doing something like cloning the Opal
>> compiler, or the Squeak compiler, so that it uses a fixed set of classes
>> that won't change over time, excepting the collections, and using as a
>> benchmark this compiler recompiling all its own methods.  This is a nice
>> mix of string processing (in the tokenizer) and symbolic processing (in the
>> building and optimizing of the parse tree).
>>
>> Cross - dialect could be hard. Pharo and Squeak are fairly easy to do,
>> but with larger programs staying compatible across different dialects is
>> harder.
>>
>>
>> Again, extracting a compiler from its host system would make it possible
>> to maintain a cross-platform version.  It could be left as an exercise to
>> the reader to port it to one's favorite non-Smalltalk dynamic language.
>>
>> tim Rowledge <tim at rowledge.org> schrieb am Mi., 22. März 2017, 21:40:
>>
>>
>>
>> > On 21-03-2017, at 4:53 PM, Javier Pimás <elpochodelagente at gmail.com>
>> wrote:
>> >
>> > Hi everybody! While measuring performance I usually face the problem of
>> assessing performance.
>>
>> Have you tried the benchmarks package - CogBenchmarks - included in the
>> source.squeak.org/VMMaker repository?
>>
>> tim
>> --
>> tim Rowledge; tim at rowledge.org; http://www.rowledge.org/tim
>> Strange OpCodes: BOMB: Burn Out Memory Banks
>>
>>
>>
>>
>>
>>
>> --
>> _,,,^..^,,,_
>> best, Eliot
>>
>
>


-- 
John Dougan
jdougan at acm.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20170324/1e2a24e6/attachment.html>