Folks -
I spent a bit of time on the weekend to add various improvements to
ConflictFinder and try them out. Most importantly, CF now has the
ability to run all the tests and detect any newly found failures (i.e.,
it's running all tests that are in the image and blames newly failing
tests on the latest loaded package).
Interestingly, I found that at this point in time this isn't as helpful
as I had assumed - I think that's mostly because test coverage isn't
great at this point and maybe because most blatant failures can be
easily detected by the basic conflict tests. So while I think this will
be helpful in the long term it's turned off for the moment which
dramatically increases the speed of the process.
That said, thanks to the people running ConflictFinder we have some
early feedback about general issues with the current 3.10 universe:
* Missing Dependencies: There seem to be a *lot* of packages that miss
either some or all dependencies. Not sure what to do about them but I've
seen lots of packages with warnings reported from MC about missing
dependencies.
* Package renames: It seems that some packages (Polymorph in particular)
were renamed halfways along the way but the obsolete versions are still
available. This often results in bogus conflicts (different category
names) and also clutters the user-visible space (for Polymorph for
example there seem to be three different names under which it goes:
Polymorph, UI Theme, and Look Enhancements).
* Platform dependencies: Some packages (TimeZoneDB, ExternalWebBrowser)
will only load and work on a particular platform. TimeZoneDB for example
fails to load on Windows boxes since it tries to access some Unix path
directly.
* Overrides: We all know that overrides are evil but occasionally
necessary. I am wondering how we should treat them integration testing.
Any ideas?
Also, there is the question of how we want to go forward from here. For
example, should we try to fix these issues in the current 3.10 universe,
or should we work forward from here and start fresh with a 3.11 universe
at which point we could start running these tests for example weekly and
have feedback about many of the above issues much earlier in the
process. We could then also make a much more well-informed decision
about what the "stable 3.11 universe" really means (i.e., for example
only provide packages that pass the integration tests).
Comments, advice, offers for help are all welcome.
Cheers,
- Andreas