<div dir="ltr"><div class="gmail_default" style="font-size:small">Seems like more of a warning and not a failure. </div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">All the best,</div><div class="gmail_default" style="font-size:small"><br>Ron Teitelbaum  </div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Jan 5, 2021 at 3:22 AM Marcel Taeumel <<a href="mailto:marcel.taeumel@hpi.de">marcel.taeumel@hpi.de</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div id="gmail-m_7619643572878387648__MailbirdStyleContent" style="font-size:10pt;font-family:Arial;color:rgb(0,0,0);text-align:left" dir="ltr">
                                        Hi Nicolas.<div><br></div><div>> <span style="font-family:Arial,Helvetica,sans-serif;font-size:13px">Do we really want to keep this kind of test?</span></div><div><span style="font-family:Arial,Helvetica,sans-serif;font-size:13px"><br></span></div><div><span style="font-family:Arial,Helvetica,sans-serif;font-size:13px">Such benchmarks (and benchmark-like tests) should at least average over several runs and only fail as a test if something actually got slower on average. Or something like that. A single misbehaving run should not be the reason for such a test failure.</span></div><div><span style="font-family:Arial,Helvetica,sans-serif;font-size:13px"><br></span></div><div><span style="font-family:Arial,Helvetica,sans-serif;font-size:13px">Maybe we can tweak #should:notTakeMoreThan: to evaluate the block several times? But then it cannot fail early on as it is doing now ... Hmmm...</span></div><div><span style="font-family:Arial,Helvetica,sans-serif;font-size:13px"><br></span></div><div><span style="font-family:Arial,Helvetica,sans-serif;font-size:13px">Best,</span></div><div><span style="font-family:Arial,Helvetica,sans-serif;font-size:13px">Marcel</span></div><div></div>
                                        <blockquote type="cite" style="border-left-style:solid;border-width:1px;margin-top:20px;margin-left:0px;padding-left:10px;min-width:500px">
                        <p style="color:rgb(170,170,170);margin-top:10px">Am 05.01.2021 09:08:46 schrieb Nicolas Cellier <<a href="mailto:nicolas.cellier.aka.nice@gmail.com" target="_blank">nicolas.cellier.aka.nice@gmail.com</a>>:</p><div style="font-family:Arial,Helvetica,sans-serif"> <br>Hi all,<br>sometimes, some build fail for just 1 test...<br><br>Here <a href="https://travis-ci.com/github/OpenSmalltalk/opensmalltalk-vm/jobs/468407844" target="_blank">https://travis-ci.com/github/OpenSmalltalk/opensmalltalk-vm/jobs/468407844</a><br>a squeak.stack.v3<br><br>RenderBugz<br> ✗ #testSetForward (7ms)<br>TestFailure: Block evaluation took more than the expected 0:00:00:00.004<br>RenderBugz(TestCase)>>assert:description:<br>RenderBugz(TestCase)>>should:notTakeMoreThan:<br>RenderBugz(TestCase)>>should:notTakeMoreThanMilliseconds:<br>RenderBugz>>shouldntTakeLong:<br>RenderBugz>>testSetForward ...shouldntTakeLong: [ t forwardDirection: 180.0 .<br>self assert: ( t forwardDirection = 0.0 )  ]<br>RenderBugz(TestCase)>>performTest<br><br>4ms, really? On C.I. infrastructure, anything can happen...<br>Do we really want to keep this kind of test?<br>We eventually could once startup performance is known (see<br>isLowerPerformance discussion on squeak-dev), but in the interim, I<br>suggest we neutralize this specific test in Smalltalk-CI.<br></div></blockquote></div><br>
</blockquote></div>