[Vm-dev] [squeak-dev] some stupid failures

Nicolas Cellier nicolas.cellier.aka.nice at gmail.com
Tue Jan 12 14:23:48 UTC 2021


Yet another one (stack.v3)

SUnitToolBuilderTests
837fef_b498

 ✗ #testHandlingNotification (18863ms)

Le mar. 12 janv. 2021 à 14:18, Nicolas Cellier
<nicolas.cellier.aka.nice at gmail.com> a écrit :
>
> Here is another source of frequent C.I. failures:
>
> MCMethodDefinitionTest
>
>  ✗ #testLoadAndUnload (20255ms)
>
> TestFailure: Test timed out
>
> Presumably not a lean and mean test...
>
> Le mar. 5 janv. 2021 à 17:59, Ron Teitelbaum <ron at usmedrec.com> a écrit :
> >
> >
> > Seems like more of a warning and not a failure.
> >
> > All the best,
> >
> > Ron Teitelbaum
> >
> > On Tue, Jan 5, 2021 at 3:22 AM Marcel Taeumel <marcel.taeumel at hpi.de> wrote:
> >>
> >> Hi Nicolas.
> >>
> >> > Do we really want to keep this kind of test?
> >>
> >> Such benchmarks (and benchmark-like tests) should at least average over several runs and only fail as a test if something actually got slower on average. Or something like that. A single misbehaving run should not be the reason for such a test failure.
> >>
> >> Maybe we can tweak #should:notTakeMoreThan: to evaluate the block several times? But then it cannot fail early on as it is doing now ... Hmmm...
> >>
> >> Best,
> >> Marcel
> >>
> >> Am 05.01.2021 09:08:46 schrieb Nicolas Cellier <nicolas.cellier.aka.nice at gmail.com>:
> >>
> >>
> >> Hi all,
> >> sometimes, some build fail for just 1 test...
> >>
> >> Here https://travis-ci.com/github/OpenSmalltalk/opensmalltalk-vm/jobs/468407844
> >> a squeak.stack.v3
> >>
> >> RenderBugz
> >> ✗ #testSetForward (7ms)
> >> TestFailure: Block evaluation took more than the expected 0:00:00:00.004
> >> RenderBugz(TestCase)>>assert:description:
> >> RenderBugz(TestCase)>>should:notTakeMoreThan:
> >> RenderBugz(TestCase)>>should:notTakeMoreThanMilliseconds:
> >> RenderBugz>>shouldntTakeLong:
> >> RenderBugz>>testSetForward ...shouldntTakeLong: [ t forwardDirection: 180.0 .
> >> self assert: ( t forwardDirection = 0.0 ) ]
> >> RenderBugz(TestCase)>>performTest
> >>
> >> 4ms, really? On C.I. infrastructure, anything can happen...
> >> Do we really want to keep this kind of test?
> >> We eventually could once startup performance is known (see
> >> isLowerPerformance discussion on squeak-dev), but in the interim, I
> >> suggest we neutralize this specific test in Smalltalk-CI.
> >>
> >>


More information about the Vm-dev mailing list