[squeak-dev] [Proposal SUnit] Show number of tests and last duration

Marcel Taeumel marcel.taeumel at hpi.de
Wed Sep 4 06:23:42 UTC 2019


Hmm... at the moment, it would be a new entry in the TestCase's history dict besides #passes, #failures, etc.:


How to compute a reliable baseline? You know about SMark and BechmarkRunner?
http://www.squeaksource.com/SMark.html [http://www.squeaksource.com/SMark.html]
http://www.hpi.uni-potsdam.de/hirschfeld/squeaksource/BenchmarkRunner.html [http://www.hpi.uni-potsdam.de/hirschfeld/squeaksource/BenchmarkRunner.html]

A TestCase's history dict is the mechanism to store such kind of data, I suppose. Even though I am not sure why there are no TestResults kept in such a history record. Objects! :-D

Best,
Marcel



Am 04.09.2019 00:45:22 schrieb Chris Muller <asqueaker at gmail.com>:
A great start to detecting "performance failures".  +1

Now imagine if we added a variable to TestCase, say, 'timings', a Dictionary of #testSelector -> lastRunDuration 's.  Then, any test more than ~10% off its baseline would raise an Error (or.. PerformanceFailure?, whatever).

There'd be a method, #resetBaselines to reset this Dictionary under the same condition that we re-ask for developer initials -- when image detects it has moved to a new location.  Since this this could mean possibly different hardware, new baselines would need established to avoid spurious failures.

>From a UI perspective, access to #resetBaselines should be sufficient -- once all are addressed or "okay'ed", the #resetBaselines would allow new baselines to be captured on the next run...

 - Chris



On Tue, Sep 3, 2019 at 3:35 AM Marcel Taeumel <marcel.taeumel at hpi.de [mailto:marcel.taeumel at hpi.de]> wrote:

Can I merge the required changes into Trunk's SUnit package? :-) It's basically a few additions to TestResult and RestTunner.


Best,
Marcel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20190904/f210e0f1/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 46611 bytes
Desc: not available
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20190904/f210e0f1/attachment-0001.png>


More information about the Squeak-dev mailing list