All tests running green


Ken G. Brown wrote:
> I think that the dev images should ship with all tests running green.

Cool. How?

It is a hard task just to get the basic image to run its tests green.
The dev image introduces a lot of packages to each other often for the first time.

Damien does a mighty service by providing fresh integration every month.
This is probably near the limit of the time service he can give to the community.

Are you willing to recruit someone to post
 the results of the tests runs for a dev image each month?
It seems to me that would be the next step.

The reports would be the feedback that lead to green tests.

*** Ken Wrote: 
>I'd like to have confidence that everything added to 3.10.2 also includes
>tests for everything added, and that all tests run unattended and everything
>ends green.

+1

>For starters, on Mac 10.5.4, fresh sq3.10.2-7179dev08.07.1 on Squeak
>3.8.18beta1U.app vm, pull out the Test Runner, select all standard tests,
>start to ru

n 2788 tests,  and with manual intervention required several
>times, it only gets to 394 run, 374 passes, 2 expected failures, 5 failures
>13 error, before it quits processing.
***


Cool, what test does it quit on? 

What test expect themselves to fail?
And what are the errors and failures?

If you take out those tests how does the rest run?

It would seem to me, just as Damien contributes by producing the images,
 someone needs to contribute by producing the reports
 on the tests from the images.

If someone one is willing, then others will be able to add bug removal.
And the dev images can converge until Ken gets his wish.

Bugs are like litter they can accumulate one at a time
 or be eliminated one at a time.

What the community needs is the accumulated help
 of those who are in the right position to tackle the problems.

I can find bugs and I can puzzle out some good solutions to interesting ones.

I can't also run tests extensively or continuously to discover
 all the possible intergration problems the fixes might lead to.


Sqeuak is 30 years old some of the older parts do not share
 the same assumptions as the newer parts.
There are a lot of integration bugs. 
And the more you integrate the more problems you will find.

 ***And later Ken wrote: 
>I'm suggesting that anything pre-loaded by you into the dev images, come with
>tests and all tests be green for the release dev image.
>Other stuff can however be showcased by listing in the dev image and be easy
>to load, but not be pre-loaded if they cause any tests to fail.
***from http://lists.squeakfoundation.org/pipermail/squeak-dev/2008-July/129970.html

It will probably be important to have distributable images with those properties.
 But the fastest way to get there is to make the mistakes first. 
Damiens dev images are integration betas.
 They are there first and will need to be tested.

Once the tests are made and reported. 
Other integration scripts can be written
 s.t. a stable dev image meeting your critriria could be accomplished.
Either by leaving out packages or failing tests.
Progress is made by walking forward.
Walking forward is made by falling
 and catching yourself before you land on your face.

Damien's images are a necessary step.
Someone stepping forward and reporting test results
 is the next necessary step.


Yours in curiosity and service, --Jerome Peace