[squeak-dev] Squeak Full Test run - some stats

Chris Cunningham cunningham.cb at gmail.com
Sat Jul 14 17:58:27 UTC 2018


Odd. With the latest cog/spur/win64x64

Test: 21:06.89
Slowest:
  05:24.970 PureBehaviorTest
  03:17.736 TraitTest
  03:02.790 TraitMethodDescriptionTest
  02:05.559 TraitCompositionTest
  01:07.887 TraitFileOutTest

So 3 minutes slower, although decompiler test are still significantly
faster.

On the good side, the errors went away:
4494 run, 4442 passes, 29 expected failures, 22 failures, 0 errors, 1
unexpected passes
(probably because there were 134 less tests ran?)

Thanks,
cbc

On Sat, Jul 14, 2018 at 9:40 AM, Chris Cunningham <cunningham.cb at gmail.com>
wrote:

> Yes, stack.  It was the VM that came with the image I downloaded (months
> ago) for dev.
>
> Using a more recent VM (cog/spur):
>   Croquet Closure Cog[Spur] VM [CoInterpreterPrimitives
> VMMaker.oscog-eem.2361]
>   Win32 built on Apr  3 2018 17:24:23 GMT Compiler: 4.2.1 Compatible Clang
> 5.0.1 (tags/RELEASE_501/final)
> (this is the 'stable' VM from https://github.com/
> OpenSmalltalk/opensmalltalk-vm/releases/latest):
>
> Tests: 18:35.123
> Slowest:
>   05:24.681  PureBehaviorTest
>   03:10.968  TraitTest
>   02:32.973  TraitMethodDescriptionTest
>   01:50.439  TraitCompositionTest
> I find it interesting that these tests didn't really improve at all. On
> the other hand, Decompiler test went from 3 minutes to under a minute - it
> is sensitive to the VM, apparently (i.e., faster with newer VM's).
>
> Oh, and the number of failures increased DRAMATICALLY:
> 4528 run, 4311 passes, 32 expected failures, 30 failures, *154 errors*, 1
> unexpected passes
> (the stack VM had  "4479 run, 4426 passes, 29 expected failures, 23
> failures, *0 errors*, 1 unexpected passes".  It seems to have run 150+
> new tests over the last run, and while I added the HelpSystem tests back
> in, those were only 13 total and all passed.  I would guess that the extra
> tests were previous filtered out for Windows, but that stable VM let them
> run, maybe).
>
> Will run with latest VM in a bit, and see if I can find out more about the
> additional errors. That worries me.
>
> -cbc
>
> On Sat, Jul 14, 2018 at 2:58 AM, Levente Uzonyi <leves at caesar.elte.hu>
> wrote:
>
>> Is that a Stack VM? That could explain why it takes more than just a few
>> minutes to run the full test suite.
>>
>> Levente
>>
>>
>> On Fri, 13 Jul 2018, Chris Cunningham wrote:
>>
>> Hi.
>>> I'm trying to run the full test suite in Squeak, and it takes a while.
>>> So, I've timed it and noted which parts are slow.
>>>
>>> Full test suite in Test Runner (except the "Help System-*" tests - more
>>> on that later) took 26:21.798 - just under a 1/2 hour.
>>>
>>> The top 6 test classes (those taking over a minute each):
>>>   05:11.612  PureBehaviorTest
>>>   03:15.047  DecompilerTests
>>>   03:09.779  TraitTest
>>>   03:01.287  TraitMethodDescriptionTest
>>>   02:19.789  TraitCompositionTest
>>>   01:12.013  TraitFileOutTest
>>> So, about 2/3rds of the time on those tests.
>>>
>>> Testing environment:
>>>   Squeak5.2alpha
>>>   latest update: #18145
>>>   Image format 68021 (64 bit)
>>>   Croquet Closure Stack VM [StackInterpreterPrimitives
>>> VMMaker.oscog-eem.2203]
>>>   Win32 built on Apr 27 2017 03:08:19 GMT Compiler: 4.2.1 Compatible
>>> Clang 3.9.1 (tags/RELEASE_391/final)
>>>   platform sources revision VM: 201704270225
>>> hardware:
>>>   Processors: AMD FX(tm)-6300 Six-Core Processor
>>> Identifier: AMD64 Family 21 Model 2 Stepping 0
>>> ~MHZ: 3516
>>>   Operating System: Windows 10 Home (Build 9200 )
>>>
>>> Results:
>>>   4479 run, 4426 passes, 29 expected failures, 23 failures, 0 errors, 1
>>> unexpected passes
>>> Issues:
>>> [IMAGE]
>>>
>>> So, what are the 'normal' failing tests that I should ignore, if any?
>>>
>>> If you are interested in how the timing was done, I loaded this package
>>> (recently created for another purpose):
>>>    MCHttpRepository
>>> location: 'http://www.squeaksource.com/CBCTimeLogging'
>>> user: ''
>>> password: ''
>>> Load "Time Logging" (at least version cbc.2)
>>> Load "Test logging sunit integration" (at least version cbc.1)
>>>
>>> Then, any runs of tests in TestRunner should log the it takes.  The
>>> following commands show the results as noted above:
>>>   (TimeLogging globalTimingLogs category: #testRunner) categoryTime.
>>>   (((TimeLogging globalTimingLogs category: #testRunner)
>>> slowestGroupTimes select: [:sgt| sgt second >= 1 minute]) collect:
>>> #printString) joinSeparatedBy: String cr.
>>>
>>> * About the "Help-System-*" tests, they manage to block the TestRunner
>>> with a failure in showing one of the UI's, which after moving a window or
>>> two locked up the image. Not nice.  Will follow up in another
>>> message.
>>>
>>> Also, why does Test Runner not keep a progress bar showing all of the
>>> time?  It is unnerving now knowing if it is running or not - several times
>>> it just seems locked while it is doing things in the
>>> background.
>>>
>>> Thanks,
>>> cbc
>>>
>>>
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20180714/07e3de72/attachment.html>


More information about the Squeak-dev mailing list