[squeak-dev] Squeak Full Test run - some stats

Chris Cunningham cunningham.cb at gmail.com
Sat Jul 14 16:40:53 UTC 2018


Yes, stack.  It was the VM that came with the image I downloaded (months
ago) for dev.

Using a more recent VM (cog/spur):
  Croquet Closure Cog[Spur] VM [CoInterpreterPrimitives
VMMaker.oscog-eem.2361]
  Win32 built on Apr  3 2018 17:24:23 GMT Compiler: 4.2.1 Compatible Clang
5.0.1 (tags/RELEASE_501/final)
(this is the 'stable' VM from
https://github.com/OpenSmalltalk/opensmalltalk-vm/releases/latest):

Tests: 18:35.123
Slowest:
  05:24.681  PureBehaviorTest
  03:10.968  TraitTest
  02:32.973  TraitMethodDescriptionTest
  01:50.439  TraitCompositionTest
I find it interesting that these tests didn't really improve at all. On the
other hand, Decompiler test went from 3 minutes to under a minute - it is
sensitive to the VM, apparently (i.e., faster with newer VM's).

Oh, and the number of failures increased DRAMATICALLY:
4528 run, 4311 passes, 32 expected failures, 30 failures, *154 errors*, 1
unexpected passes
(the stack VM had  "4479 run, 4426 passes, 29 expected failures, 23
failures, *0 errors*, 1 unexpected passes".  It seems to have run 150+ new
tests over the last run, and while I added the HelpSystem tests back in,
those were only 13 total and all passed.  I would guess that the extra
tests were previous filtered out for Windows, but that stable VM let them
run, maybe).

Will run with latest VM in a bit, and see if I can find out more about the
additional errors. That worries me.

-cbc

On Sat, Jul 14, 2018 at 2:58 AM, Levente Uzonyi <leves at caesar.elte.hu>
wrote:

> Is that a Stack VM? That could explain why it takes more than just a few
> minutes to run the full test suite.
>
> Levente
>
>
> On Fri, 13 Jul 2018, Chris Cunningham wrote:
>
> Hi.
>> I'm trying to run the full test suite in Squeak, and it takes a while.
>> So, I've timed it and noted which parts are slow.
>>
>> Full test suite in Test Runner (except the "Help System-*" tests - more
>> on that later) took 26:21.798 - just under a 1/2 hour.
>>
>> The top 6 test classes (those taking over a minute each):
>>   05:11.612  PureBehaviorTest
>>   03:15.047  DecompilerTests
>>   03:09.779  TraitTest
>>   03:01.287  TraitMethodDescriptionTest
>>   02:19.789  TraitCompositionTest
>>   01:12.013  TraitFileOutTest
>> So, about 2/3rds of the time on those tests.
>>
>> Testing environment:
>>   Squeak5.2alpha
>>   latest update: #18145
>>   Image format 68021 (64 bit)
>>   Croquet Closure Stack VM [StackInterpreterPrimitives
>> VMMaker.oscog-eem.2203]
>>   Win32 built on Apr 27 2017 03:08:19 GMT Compiler: 4.2.1 Compatible
>> Clang 3.9.1 (tags/RELEASE_391/final)
>>   platform sources revision VM: 201704270225
>> hardware:
>>   Processors: AMD FX(tm)-6300 Six-Core Processor
>> Identifier: AMD64 Family 21 Model 2 Stepping 0
>> ~MHZ: 3516
>>   Operating System: Windows 10 Home (Build 9200 )
>>
>> Results:
>>   4479 run, 4426 passes, 29 expected failures, 23 failures, 0 errors, 1
>> unexpected passes
>> Issues:
>> [IMAGE]
>>
>> So, what are the 'normal' failing tests that I should ignore, if any?
>>
>> If you are interested in how the timing was done, I loaded this package
>> (recently created for another purpose):
>>    MCHttpRepository
>> location: 'http://www.squeaksource.com/CBCTimeLogging'
>> user: ''
>> password: ''
>> Load "Time Logging" (at least version cbc.2)
>> Load "Test logging sunit integration" (at least version cbc.1)
>>
>> Then, any runs of tests in TestRunner should log the it takes.  The
>> following commands show the results as noted above:
>>   (TimeLogging globalTimingLogs category: #testRunner) categoryTime.
>>   (((TimeLogging globalTimingLogs category: #testRunner)
>> slowestGroupTimes select: [:sgt| sgt second >= 1 minute]) collect:
>> #printString) joinSeparatedBy: String cr.
>>
>> * About the "Help-System-*" tests, they manage to block the TestRunner
>> with a failure in showing one of the UI's, which after moving a window or
>> two locked up the image. Not nice.  Will follow up in another
>> message.
>>
>> Also, why does Test Runner not keep a progress bar showing all of the
>> time?  It is unnerving now knowing if it is running or not - several times
>> it just seems locked while it is doing things in the
>> background.
>>
>> Thanks,
>> cbc
>>
>>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20180714/3a02b308/attachment.html>


More information about the Squeak-dev mailing list