[Box-Admins] Squeak CI Benchmarking Priority

[Jeff Gonis]

Hi Everyone,

So I initially emailed Frank Shearar and he suggested that I send a
message to this list to get some feedback as well.

Right now, much to my dismay, the Squeak CI benchmarking is pretty
useless, showing huge variances in the runtimes of the benchmarks
between revisions.  I am currently taking a look at the SMark
framework that Stefan Marr suggested for benchmarking, to see if that
will help make the benchmarks be of higher quality.

In the meantime, however, I would like to see what I can do to get
some quick wins.  On my side I am working on running the benchmarks
multiple times to warm up the Cog JIT, but I would also like to
investigate the priority that we run the benchmarks at.

Right now we launch the vm with the default "nice" priority.  I can't
get into the box to see what the default is, but I would suspect that
it is probably 0, as this seems pretty standard. If I bumped the
priority to -20, to see if this would give the benchmarks a bit more
stability would anyone be drastically impacted?  They only run when
the SqueakTrunk project is run, and they shouldn't run for too long.
I am in fact working on cutting down the runtime a bit to have them
complete quicker, so we shouldn't have a super high priority process
running all day every day on the server.

Does anyone have a large objection to this idea?  Is it ok if I go
ahead and implement it on a test basis to see if it has an effect on
the benchmark times?

Thanks for your feedback,
Jeff