Seems like more of a warning and not a failure.

All the best,

Ron Teitelbaum

On Tue, Jan 5, 2021 at 3:22 AM Marcel Taeumel <marcel.taeumel at hpi.de> wrote:

> Hi Nicolas.
>
> > Do we really want to keep this kind of test?
>
> Such benchmarks (and benchmark-like tests) should at least average over
> several runs and only fail as a test if something actually got slower on
> average. Or something like that. A single misbehaving run should not be the
> reason for such a test failure.
>
> Maybe we can tweak #should:notTakeMoreThan: to evaluate the block several
> times? But then it cannot fail early on as it is doing now ... Hmmm...
>
> Best,
> Marcel
>
> Am 05.01.2021 09:08:46 schrieb Nicolas Cellier <
> nicolas.cellier.aka.nice at gmail.com>:
>
> Hi all,
> sometimes, some build fail for just 1 test...
>
> Here
> https://travis-ci.com/github/OpenSmalltalk/opensmalltalk-vm/jobs/468407844
> a squeak.stack.v3
>
> RenderBugz
> ✗ #testSetForward (7ms)
> TestFailure: Block evaluation took more than the expected 0:00:00:00.004
> RenderBugz(TestCase)>>assert:description:
> RenderBugz(TestCase)>>should:notTakeMoreThan:
> RenderBugz(TestCase)>>should:notTakeMoreThanMilliseconds:
> RenderBugz>>shouldntTakeLong:
> RenderBugz>>testSetForward ...shouldntTakeLong: [ t forwardDirection:
> 180.0 .
> self assert: ( t forwardDirection = 0.0 ) ]
> RenderBugz(TestCase)>>performTest
>
> 4ms, really? On C.I. infrastructure, anything can happen...
> Do we really want to keep this kind of test?
> We eventually could once startup performance is known (see
> isLowerPerformance discussion on squeak-dev), but in the interim, I
> suggest we neutralize this specific test in Smalltalk-CI.
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20210105/8913670b/attachment.html>