[Vm-dev] Running tests for auto-built Cog VMs

[Casey Ransberger]

It'd be really great if you kept more than Pharo in mind: I'd like to be able to shoot someone an email and say "can we add a Cuis image to our shared CI setup?" and have the answer be "yes, not hard, I need you to do X before we can set that up."

If we're running Hudson, I'd sure like to use it to test *more* than just the VM. Squeak has a great big suite of tests and I think we ought to use 'em without making people open the test runner manually. 

Igor! CI! You rock!

On Mar 17, 2011, at 6:21 PM, Igor Stasenko <siguctua at gmail.com> wrote:

> 
> On 18 March 2011 01:30, Eliot Miranda <eliot.miranda at gmail.com> wrote:
>> 
>> 
>> 
>> On Thu, Mar 17, 2011 at 10:29 AM, Igor Stasenko <siguctua at gmail.com> wrote:
>>> 
>>> Hello all.
>>> 
>>> Our next plan is to extend the Hudson build jobs with tests which are
>>> run after VM has built.
>>> 
>>> I'd like to ask, what kind of tests we should run for VMs.
>>> 
>>> So, here's my questions:
>>> 
>>> - do we need a VM specific tests? Or we just use ordinal tests which
>>> are in image(s) by default?
>> 
>> SOme of the standard tests in the Squeak 4.2 suite are specifically for VM stuff.  The entire suite depends on the VM in some way.  It seems straight-forward to include tests for specific pieces of functionality, for example the mirror primitives, cannotInterpret: and objects-as-methods.  Adding tests that test new VM functionality is clearly more than a good idea; it should be de rigueur.
>> 
>>> 
>>> - in case if we need to run VM-specific tests, is they are available
>>> somewhere or not? do we need to create separate package/repository for
>>> holding them?
>> 
>> Check out a 4.2 image and look at the tests.  The organization might not be perfect but it's there.
>> 
> 
> You mean non-VM-specific tests, just those which lying in image(s)?
> 
> Yes, that's the first thing which comes into mind. And sure thing for
> start it is ok.
> 
> For example, Hudson runs tests for PharoCore1.3 image, each time new
> update there.
> https://pharo-ic.lille.inria.fr/hudson/view/Pharo/job/Pharo%20Core%201.3/
> 
> It is easy to tweak this config and run it using freshly built VM(s),
> one per each platform * JIT/Stack/MT configs.
> 
> But i wonder if we may need to be more focused and create some package(s)
> to group tests around VM or its individual plugins.
> 
> 
> 
>>> 
>>> i'd like to brainstorm that a bit. And then we will do that.
>>> 
>>> 
>>> --
>>> Best regards,
>>> Igor Stasenko AKA sig.
>> 
> 
> -- 
> Best regards,
> Igor Stasenko AKA sig.