[squeak-dev] Squeak Full Test run - some stats
Chris Cunningham
cunningham.cb at gmail.com
Sat Jul 14 17:58:27 UTC 2018
Odd. With the latest cog/spur/win64x64
Test: 21:06.89
Slowest:
05:24.970 PureBehaviorTest
03:17.736 TraitTest
03:02.790 TraitMethodDescriptionTest
02:05.559 TraitCompositionTest
01:07.887 TraitFileOutTest
So 3 minutes slower, although decompiler test are still significantly
faster.
On the good side, the errors went away:
4494 run, 4442 passes, 29 expected failures, 22 failures, 0 errors, 1
unexpected passes
(probably because there were 134 less tests ran?)
Thanks,
cbc
On Sat, Jul 14, 2018 at 9:40 AM, Chris Cunningham <cunningham.cb at gmail.com>
wrote:
> Yes, stack. It was the VM that came with the image I downloaded (months
> ago) for dev.
>
> Using a more recent VM (cog/spur):
> Croquet Closure Cog[Spur] VM [CoInterpreterPrimitives
> VMMaker.oscog-eem.2361]
> Win32 built on Apr 3 2018 17:24:23 GMT Compiler: 4.2.1 Compatible Clang
> 5.0.1 (tags/RELEASE_501/final)
> (this is the 'stable' VM from https://github.com/
> OpenSmalltalk/opensmalltalk-vm/releases/latest):
>
> Tests: 18:35.123
> Slowest:
> 05:24.681 PureBehaviorTest
> 03:10.968 TraitTest
> 02:32.973 TraitMethodDescriptionTest
> 01:50.439 TraitCompositionTest
> I find it interesting that these tests didn't really improve at all. On
> the other hand, Decompiler test went from 3 minutes to under a minute - it
> is sensitive to the VM, apparently (i.e., faster with newer VM's).
>
> Oh, and the number of failures increased DRAMATICALLY:
> 4528 run, 4311 passes, 32 expected failures, 30 failures, *154 errors*, 1
> unexpected passes
> (the stack VM had "4479 run, 4426 passes, 29 expected failures, 23
> failures, *0 errors*, 1 unexpected passes". It seems to have run 150+
> new tests over the last run, and while I added the HelpSystem tests back
> in, those were only 13 total and all passed. I would guess that the extra
> tests were previous filtered out for Windows, but that stable VM let them
> run, maybe).
>
> Will run with latest VM in a bit, and see if I can find out more about the
> additional errors. That worries me.
>
> -cbc
>
> On Sat, Jul 14, 2018 at 2:58 AM, Levente Uzonyi <leves at caesar.elte.hu>
> wrote:
>
>> Is that a Stack VM? That could explain why it takes more than just a few
>> minutes to run the full test suite.
>>
>> Levente
>>
>>
>> On Fri, 13 Jul 2018, Chris Cunningham wrote:
>>
>> Hi.
>>> I'm trying to run the full test suite in Squeak, and it takes a while.
>>> So, I've timed it and noted which parts are slow.
>>>
>>> Full test suite in Test Runner (except the "Help System-*" tests - more
>>> on that later) took 26:21.798 - just under a 1/2 hour.
>>>
>>> The top 6 test classes (those taking over a minute each):
>>> 05:11.612 PureBehaviorTest
>>> 03:15.047 DecompilerTests
>>> 03:09.779 TraitTest
>>> 03:01.287 TraitMethodDescriptionTest
>>> 02:19.789 TraitCompositionTest
>>> 01:12.013 TraitFileOutTest
>>> So, about 2/3rds of the time on those tests.
>>>
>>> Testing environment:
>>> Squeak5.2alpha
>>> latest update: #18145
>>> Image format 68021 (64 bit)
>>> Croquet Closure Stack VM [StackInterpreterPrimitives
>>> VMMaker.oscog-eem.2203]
>>> Win32 built on Apr 27 2017 03:08:19 GMT Compiler: 4.2.1 Compatible
>>> Clang 3.9.1 (tags/RELEASE_391/final)
>>> platform sources revision VM: 201704270225
>>> hardware:
>>> Processors: AMD FX(tm)-6300 Six-Core Processor
>>> Identifier: AMD64 Family 21 Model 2 Stepping 0
>>> ~MHZ: 3516
>>> Operating System: Windows 10 Home (Build 9200 )
>>>
>>> Results:
>>> 4479 run, 4426 passes, 29 expected failures, 23 failures, 0 errors, 1
>>> unexpected passes
>>> Issues:
>>> [IMAGE]
>>>
>>> So, what are the 'normal' failing tests that I should ignore, if any?
>>>
>>> If you are interested in how the timing was done, I loaded this package
>>> (recently created for another purpose):
>>> MCHttpRepository
>>> location: 'http://www.squeaksource.com/CBCTimeLogging'
>>> user: ''
>>> password: ''
>>> Load "Time Logging" (at least version cbc.2)
>>> Load "Test logging sunit integration" (at least version cbc.1)
>>>
>>> Then, any runs of tests in TestRunner should log the it takes. The
>>> following commands show the results as noted above:
>>> (TimeLogging globalTimingLogs category: #testRunner) categoryTime.
>>> (((TimeLogging globalTimingLogs category: #testRunner)
>>> slowestGroupTimes select: [:sgt| sgt second >= 1 minute]) collect:
>>> #printString) joinSeparatedBy: String cr.
>>>
>>> * About the "Help-System-*" tests, they manage to block the TestRunner
>>> with a failure in showing one of the UI's, which after moving a window or
>>> two locked up the image. Not nice. Will follow up in another
>>> message.
>>>
>>> Also, why does Test Runner not keep a progress bar showing all of the
>>> time? It is unnerving now knowing if it is running or not - several times
>>> it just seems locked while it is doing things in the
>>> background.
>>>
>>> Thanks,
>>> cbc
>>>
>>>
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20180714/07e3de72/attachment.html>
More information about the Squeak-dev
mailing list
|