On 29.04.2016, at 00:02, Levente Uzonyi <leves at caesar.elte.hu> wrote:

> On Thu, 28 Apr 2016, Tobias Pape wrote:
> 
>> 
>> On 28.04.2016, at 22:29, Levente Uzonyi <leves at caesar.elte.hu> wrote:
>> 
>>> On Wed, 27 Apr 2016, Tim Felgentreff wrote:
>>> 
>>>> Hi,
>>>> We have a proposal for a tool that we think might be useful to have in trunk.
>>>> We spent some time pulling together benchmarks from various sources (papers, the mailinglist, projects on squeaksource, ...) and combining them
>>>> with an extended version of Stefan Marr's implementation of a benchmarking framework SMark. The tool and framework are modeled after SUnit, and
>>>> include different execution suites and code to figure out confidence variations over multiple runs and such. Also, it draws graphs over multiple
>>>> runs so you can look at things like warmup and GC behavior, and see how much time is spent doing incremental GCs and full GCs vs plain execution.
>>>> As a part of this I fixed the EPS export so these graphs can be exported in a scalable format.
>>>> Here is a picture of the tool: https://dl.dropboxusercontent.com/u/26242153/screenshot.jpg
>>>> As I said, it's modeled after TestRunner and SUnit, benchmarks subclass from the "Benchmark" class, any method starting with "bench" is a
>>>> benchmark, and you can have setUp and tearDown methods as usual. By default the benchmarks are run under an Autosize runner that re-executes each
>>>> benchmark until the combined runtime reaches 600ms (to smooth out any noise). Beyond that, you can specify a number of iterations that the runner
>>>> will re-do that to see multiple averaged runs. The graph shows the execution times split between running code (gray) incremental GCs (yellow) and
>>>> full GCs (red). There are popups and you can scroll to zoom in and out. There is also a history of benchmark runs stored on the class side of
>>>> benchmark classes for later reference.
>>>> The code currently lives here: http://www.hpi.uni-potsdam.de/hirschfeld/squeaksource/BenchmarkRunner
>>> 
>>> The link seems to be broken.
>>> 
>> 
>> In how far?
>> It works from here :)
> 
> HTTPS Everywhere turns it into an https URL, but that's a 404. It works via http though.

Yeah sorry for that.
But since we're not HTTPS-ready with Monticello, this probably has to wait a tiny bit more...

Best regards
	-Tobias

> 
> Levente
> 
>> 
>>> Levente
>>> 
>>>> Considering we are every so often discussing benchmark results here, I think it might be useful to share an execution framework for those.
>>>> cheers,
>>>> Tim