[Vm-dev] [squeak-dev] some stupid failures

Nicolas Cellier nicolas.cellier.aka.nice at gmail.com
Tue Jan 12 14:41:55 UTC 2021


And the fun of it, each time I retry, I see a different random failure...

#########################

# 1 tests did not pass: #

#########################

CompiledMethodTest
16ccae_ca85

 ✗ #testCopyWithTrailerBytes (11332ms)

Le mar. 12 janv. 2021 à 15:23, Nicolas Cellier
<nicolas.cellier.aka.nice at gmail.com> a écrit :
>
> Yet another one (stack.v3)
>
> SUnitToolBuilderTests
> 837fef_b498
>
>  ✗ #testHandlingNotification (18863ms)
>
> Le mar. 12 janv. 2021 à 14:18, Nicolas Cellier
> <nicolas.cellier.aka.nice at gmail.com> a écrit :
> >
> > Here is another source of frequent C.I. failures:
> >
> > MCMethodDefinitionTest
> >
> >  ✗ #testLoadAndUnload (20255ms)
> >
> > TestFailure: Test timed out
> >
> > Presumably not a lean and mean test...
> >
> > Le mar. 5 janv. 2021 à 17:59, Ron Teitelbaum <ron at usmedrec.com> a écrit :
> > >
> > >
> > > Seems like more of a warning and not a failure.
> > >
> > > All the best,
> > >
> > > Ron Teitelbaum
> > >
> > > On Tue, Jan 5, 2021 at 3:22 AM Marcel Taeumel <marcel.taeumel at hpi.de> wrote:
> > >>
> > >> Hi Nicolas.
> > >>
> > >> > Do we really want to keep this kind of test?
> > >>
> > >> Such benchmarks (and benchmark-like tests) should at least average over several runs and only fail as a test if something actually got slower on average. Or something like that. A single misbehaving run should not be the reason for such a test failure.
> > >>
> > >> Maybe we can tweak #should:notTakeMoreThan: to evaluate the block several times? But then it cannot fail early on as it is doing now ... Hmmm...
> > >>
> > >> Best,
> > >> Marcel
> > >>
> > >> Am 05.01.2021 09:08:46 schrieb Nicolas Cellier <nicolas.cellier.aka.nice at gmail.com>:
> > >>
> > >>
> > >> Hi all,
> > >> sometimes, some build fail for just 1 test...
> > >>
> > >> Here https://travis-ci.com/github/OpenSmalltalk/opensmalltalk-vm/jobs/468407844
> > >> a squeak.stack.v3
> > >>
> > >> RenderBugz
> > >> ✗ #testSetForward (7ms)
> > >> TestFailure: Block evaluation took more than the expected 0:00:00:00.004
> > >> RenderBugz(TestCase)>>assert:description:
> > >> RenderBugz(TestCase)>>should:notTakeMoreThan:
> > >> RenderBugz(TestCase)>>should:notTakeMoreThanMilliseconds:
> > >> RenderBugz>>shouldntTakeLong:
> > >> RenderBugz>>testSetForward ...shouldntTakeLong: [ t forwardDirection: 180.0 .
> > >> self assert: ( t forwardDirection = 0.0 ) ]
> > >> RenderBugz(TestCase)>>performTest
> > >>
> > >> 4ms, really? On C.I. infrastructure, anything can happen...
> > >> Do we really want to keep this kind of test?
> > >> We eventually could once startup performance is known (see
> > >> isLowerPerformance discussion on squeak-dev), but in the interim, I
> > >> suggest we neutralize this specific test in Smalltalk-CI.
> > >>
> > >>


More information about the Vm-dev mailing list