On 22 February 2013 02:25, Chris Muller <ma.chris.m at gmail.com> wrote:
> I started to do it but I couldn't gather enough initiative just yet;
> partially because I have to finish some other work first but also
> because I guess one expectation I have of the CI server is that it
> will help me know what part of my code failed (e.g., which test).  The
> Magma test suite works for users who simply install and run it
> manually in 4.5, so I felt like I was about to spend my afternoon
> chasing down something that might be a bug in the CI job and lost my
> steam.
>
> If you could you tell me what HDTestReport does in #runPackages: --
> Magma and Ma Client Server both use special subclasses of TestSuite
> and can only run one way which is:
>
>    MagmaTestCase suite debug
>
> and
>
>   MaClientServerTestCase suite debug
>
> Only running the whole suite that exact way is supported.  I don't
> know anyone who runs individual tests anyway -- if a change is made to
> a package the entire suite needs to be run.

I don't know what you mean by this. The whole point of CI is to run
all tests in a package.

> Can the CI server handle Magma's special needs?

The core of HDTestReport's SUnit usage is this:

HDTestReport >> runClasses: aCollectionOfClasses named: aString
	| suite classes |
	suite := TestSuite named: aString.
	classes := (aCollectionOfClasses
		select: [ :each | (each includesBehavior: TestCase) and: [ each
isAbstract not ] ])
			asSortedCollection: [ :a :b | a name <= b name ].
	classes isEmpty
		ifTrue: [ ^ self ].
	classes
		do: [ :each | each addToSuiteFromSelectors: suite ].
	^ self runSuite: suite

HDTestReport runSuite: aTestSuite
	^ self new
		initializeOn: aTestSuite;
		run

so I think that the issue between CI and Magma is that CI's
constructing a TestSuite rather than using Magma's custom assembled
TestSuite. I'll need to dig into #debug and see what exactly that
does. It _might_ be as simple as saying `HDTestReport new runSuite:
MaClientServerTestCase suite`, but we'll see.

frank

> On Thu, Feb 21, 2013 at 10:41 AM, Frank Shearar <frank.shearar at gmail.com> wrote:
>> Sure. It's probably easiest if you just run the squeak-ci tests, which
>> will set up an environment for you with the right VM and such. Given
>> that CI runs on a Linux, you'll probably want to do the same.
>> * git clone git at github.com:frankshearar/squeak-ci.git
>> * cd squeak-ci
>> * ./run-test.sh Magma
>>
>> HDTestReport is just a thing that runs a test suite consisting of all
>> TestCase subclasses within a particular package, and produces JUnit
>> style XML reports of the test run.
>>
>> frank
>>
>> On 21 February 2013 16:27, Chris Muller <asqueaker at gmail.com> wrote:
>>> I don't know what HDTestReport is -- do you know any way I can
>>> recreate the crash?
>>>
>>> On Thu, Feb 21, 2013 at 8:04 AM, Frank Shearar <frank.shearar at gmail.com> wrote:
>>>> Currently I have this:
>>>>
>>>> Installer squeakmap
>>>>     update;
>>>>     install: 'Magma (head)'.
>>>>
>>>> HDTestReport runPackages: #('Ma-Collections-Dictionarys'
>>>>     'Ma-Collections-Dictionarys-Auto-Tests'
>>>>     'Ma-Collections-Tests'
>>>>     'Ma-Serializer-Tests-SUnit tests'
>>>>     'Ma-Client-Server-Tester-SUnit extensions'
>>>>     'Ma-Client-Server-Tester-Tests'
>>>>     'WriteBarrier-Tests'
>>>>     'Magma-Tester-SUnit tests').
>>>>
>>>> I do need to use HDTestReport because otherwise I can't generate JUnit
>>>> test reports.
>>>>
>>>> I don't see why I should be able to trigger a crash - it looks like an
>>>> infinite recursion leading to stack overflow? - just by running the
>>>> tests though.
>>>>
>>>> But I haven't made any attempt yet to debug the issue, say by running
>>>> only some of the test packages.
>>>>
>>>> frank
>>>>
>>>> On 21 February 2013 02:12, Chris Muller <ma.chris.m at gmail.com> wrote:
>>>>> Hmm, that's strange; the tests work with the Linux 2678 Cog VM if I
>>>>> just install the (head) version from SqueakMap into 4.5.
>>>>>
>>>>> What expression are you using to kick off the test suite?  It should be:
>>>>>
>>>>>   MagmaTestCase suite debug
>>>>>
>>>>> Thanks.
>>>>>
>>>>> On Wed, Feb 20, 2013 at 11:19 AM, Frank Shearar <frank.shearar at gmail.com> wrote:
>>>>>> On 20 February 2013 15:57, Chris Muller <ma.chris.m at gmail.com> wrote:
>>>>>>> Thank you Frank, it's fixed.
>>>>>>>
>>>>>>> If following the guidelines, the (head) release is something that only
>>>>>>> needs to be set up once and then, the only time it will ever need to
>>>>>>> change again is if the package-structure changes.  That's precisely
>>>>>>> what happened with Magma.  Which makes me wonder whether it would be
>>>>>>> nice for the CI server to install and run all packages designated for
>>>>>>> that Squeak release and send out a notification if either the install
>>>>>>> or tests fail.  OR, maybe it could even remove the designation so it
>>>>>>> would no longer appear in the "Safely available" list -- I think a
>>>>>>> short or even empty list could be better for the community than one
>>>>>>> full of stuff that doesn't work..
>>>>>>>
>>>>>>>  - Chris
>>>>>>>
>>>>>>> On Wed, Feb 20, 2013 at 8:19 AM, Frank Shearar <frank.shearar at gmail.com> wrote:
>>>>>>>> Hi Chris,
>>>>>>>>
>>>>>>>> I realise Magma (head) isn't actually marked as working in 4.5, but
>>>>>>>> 4.5's still close enough to 4.4 that I gave it a go.
>>>>>>>>
>>>>>>>> I get a Warning that "This package depends on Medley", caused by
>>>>>>>> WbMedley and WbMedley >> #collectionsClass.
>>>>>>>>
>>>>>>>> What's the install script missing?
>>>>>>>>
>>>>>>>> (Context: I realised I should have written a CI build script for Magma
>>>>>>>> ages ago, and wanted to find out Magma's list of test packages by
>>>>>>>> loading it in via SqueakMap.)
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> frank
>>>>>>
>>>>>> Thanks for the fix, Chris.
>>>>>>
>>>>>> I now have a build script for you, and a VM crash!:
>>>>>> http://build.squeak.org/job/ExternalPackage-Magma/2/console
>>>>>>
>>>>>> frank
>>>>
>>>