[V3dot10] Auto-build progress

Ralph Johnson johnson at cs.uiuc.edu
Tue Jan 30 14:23:46 UTC 2007

Here is what I want from a test server.

We have an image and a set of packages that are supposed to work with
the image.  The 3.9 package universe has over a hundred such packages.

Some packages have tests, some do not.  However, we can, for each
package, run the image, load the package and run both the tests in the
image and the tests in the package.  In general, all tests run for all
packages.  I don't want to be told which tests work, I just want to
know which tests don't work.

The system also tests some combination of packages.  In other words,
there are some combination of packages that are also tested, but
testing them is also just running all the tests in the image and in
the loaded packages.

There are two interesting things that can happen.

One is that someone proposes a change to the image.  We want to make a
new image and then, once again, try loading and testing all the
package combinations.

The second is that someone makes a new version of a package.  We want
to run all the tests again.  I suppose we could just try every package
combination, but most of them do not include the package that is
changing so it seems reasonable to only test combinations that might
have different results from the last time.

How would you do this with your system?


More information about the V3dot10 mailing list