[Box-Admins] Re: Squeak Benchmarks Process Priority
Frank Shearar
frank.shearar at gmail.com
Thu Mar 7 08:21:56 UTC 2013
On 07 Mar 2013, at 7:48, Ken Causey <ken at kencausey.com> wrote:
> On 03/07/2013 01:26 AM, Frank Shearar wrote:
>> On 7 March 2013 02:52, Ken Causey<ken at kencausey.com> wrote:
>>> On 03/06/2013 03:57 PM, Frank Shearar wrote:
>>>>
>>>> On 6 March 2013 21:37, Jeff Gonis<jeff.gonis at gmail.com> wrote:
>>>>>
>>>>> Hi Frank,
>>>>>
>>>>> So to my chagrin the performance tests on Squeak-CI are pretty much
>>>>> worse than useless, displaying up to a 100% difference in times
>>>>> between builds. For my part I am looking into the SMark framework
>>>>> that Stefan Marr offered and working to learn that and see how I could
>>>>> use it.
>>>>>
>>>>> But I was curious if there were some other steps I could take in the
>>>>> meantime to try and get the times to be a little more repeatable. The
>>>>> first thing that comes to mind is that we "nice" the vm with I believe
>>>>> the default value before running it for both the tests and the
>>>>> benchmarks, I don't know how to log into the server Jenkins is running
>>>>> on so I can't see what the default value of nice is, but I am assuming
>>>>> it is 0 as that seems fairly standard. Do you think it would be
>>>>> reasonable to try out a high-priority value for at least the
>>>>> benchmarks portion of the SqueakTrunk build. Fire off the benchmarks
>>>>> at a -20 priority and see if we can get some repeatability?
>>>>>
>>>>> I am not sure about the impacts this might have elsewhere, which is
>>>>> why I wanted to run it by you before sending a pull request. If there
>>>>> is some other easy win that I have overlooked that leaps to mind
>>>>> please let me know, otherwise I will keep plugging away with SMark,
>>>>> and also look into running the tests multiple times to warm up Cog.
>>>>>
>>>>> Thanks for your help,
>>>>> Jeff G.
>>>>
>>>>
>>>> Hi Jeff,
>>>>
>>>> The box-admins folk might have further insight, but I think renicing
>>>> to be hardcore might be the sensible thing. The performance tests
>>>> don't run for long, and would only run with the SqueakTrunk build, so
>>>> shouldn't impact too much on things... but I don't recall what other
>>>> services run on that box.
>>>>
>>>> frank
>>>>
>>>>
>>>
>>> Please define clearly "don't run for long". At the moment only Jenkins is
>>> running on that server so in the short run at most, it would impact Jenkins.
>>> However it the not too distant future the mailing lists will be on that
>>> server and while a little delay is no big deal I would like some idea how
>>> much before saying "OK".
>>
>> I don't have a trend, but the last build spent 38 seconds running the
>> performance tests. Compared to the time taken to possibly compile a
>> VM, update an image, and run the tests, the benchmarking's peanuts, at
>> least for now.
>>
>> frank
>
> OK, I guess probably anything up to 5-10 minutes of 'very busy' time is probably OK. I'm assuming this is a once a day task. As it gets longer we may just want to consider scheduling it at otherwise 'quiet' times.
>
> Ken
Levente's script polls every 15 mins so at most this could run something like 24*3 times in a day. But typically the commits occur in chunks, so I reckon 10 times in a day would be a wonderful problem to have.
frank
More information about the Box-Admins
mailing list