Hi all,

On Monday I noticed that some Squot tests started to fail on Travis CI
due to timeouts (test cases exceeding the 5 sec default timeout). The
problems are not due to changes in Squot itself because if I retrigger
an older build which was already green, it also fails. So I suspect a
change in Squeak Trunk, since the Squeak 5.2 builds are also not
affected.

I did not yet find the cause, but would like to make known what I
found so far. Does anyone have an idea what could have caused this? Of
course I don't expect you to debug my test cases, but maybe you have a
hint for me.

- The Mac builds are much more affected than the Linux builds. This
could be due to differences in Travis CI's setup for the operating
systems. The Linux test runs also got slower, but not by so much.
- Both the 64 bit and 32 bit Mac builds got slower. The 64 bit ones
scratch at the 5 sec mark, but did not exceed it yet.
- If I repeat older Mac builds, they experience the same slowdown.
- The tests on Squeak 5.2 are unaffected on either platform.
- Some tests take about 6x up to 20x more time than before. On Linux
it is only up to x3 times slower and in sum hardly noticeable. For the
rest of this, I will only talk about the Mac Trunk builds.
- testTrackingAPackage has the worst ratio, whereas testTrackingAClass
is virtually unaffected.
- Only tests that involve whole packages, not individual objects or
single classes, seem to be affected. Is the regression in something
that PackageInfo does (enumerating classes, methods, accessing
categories)?
- Surprisingly, the tests with Git are less worse than the tests with
a stub in-memory repository (these are the same test cases, but with a
different backend object). I think the Git tests by chance ran after
the in-memory tests, maybe that affects things, I don't know yet. The
Git tests did not reach the 5 sec mark.
- Tests that involve Monticello (such as the Monticello->Git
conversion) and do not only create MCDefinitions and pass them around,
take about 2x as long now.

For reference:
Good build: https://travis-ci.org/github/hpi-swa/Squot/jobs/698280144
Bad build: https://travis-ci.org/github/hpi-swa/Squot/jobs/700640404
The runtimes are annotated to the test cases when you expand them.
If you do want to look at the code, just load the Git Browser, then
you will also have the test cases.

If I find out more on the weekend, I will post again.

Kind regards,
Jakob