[squeak-dev] Performance regression

Jakob Reschke forums.jakob at resfarm.de
Wed Jun 24 06:58:00 UTC 2020


Forgot the time frame: I am looking for something that changed between
2020-06-14 and 2020-06-22.

The builds run unconditionally every weekend to notice Trunk breakages
and were fine on 2020-06-14.

Am Mi., 24. Juni 2020 um 08:43 Uhr schrieb Jakob Reschke
<forums.jakob at resfarm.de>:
>
> Hi all,
>
> On Monday I noticed that some Squot tests started to fail on Travis CI
> due to timeouts (test cases exceeding the 5 sec default timeout). The
> problems are not due to changes in Squot itself because if I retrigger
> an older build which was already green, it also fails. So I suspect a
> change in Squeak Trunk, since the Squeak 5.2 builds are also not
> affected.
>
> I did not yet find the cause, but would like to make known what I
> found so far. Does anyone have an idea what could have caused this? Of
> course I don't expect you to debug my test cases, but maybe you have a
> hint for me.
>
> - The Mac builds are much more affected than the Linux builds. This
> could be due to differences in Travis CI's setup for the operating
> systems. The Linux test runs also got slower, but not by so much.
> - Both the 64 bit and 32 bit Mac builds got slower. The 64 bit ones
> scratch at the 5 sec mark, but did not exceed it yet.
> - If I repeat older Mac builds, they experience the same slowdown.
> - The tests on Squeak 5.2 are unaffected on either platform.
> - Some tests take about 6x up to 20x more time than before. On Linux
> it is only up to x3 times slower and in sum hardly noticeable. For the
> rest of this, I will only talk about the Mac Trunk builds.
> - testTrackingAPackage has the worst ratio, whereas testTrackingAClass
> is virtually unaffected.
> - Only tests that involve whole packages, not individual objects or
> single classes, seem to be affected. Is the regression in something
> that PackageInfo does (enumerating classes, methods, accessing
> categories)?
> - Surprisingly, the tests with Git are less worse than the tests with
> a stub in-memory repository (these are the same test cases, but with a
> different backend object). I think the Git tests by chance ran after
> the in-memory tests, maybe that affects things, I don't know yet. The
> Git tests did not reach the 5 sec mark.
> - Tests that involve Monticello (such as the Monticello->Git
> conversion) and do not only create MCDefinitions and pass them around,
> take about 2x as long now.
>
> For reference:
> Good build: https://travis-ci.org/github/hpi-swa/Squot/jobs/698280144
> Bad build: https://travis-ci.org/github/hpi-swa/Squot/jobs/700640404
> The runtimes are annotated to the test cases when you expand them.
> If you do want to look at the code, just load the Git Browser, then
> you will also have the test cases.
>
> If I find out more on the weekend, I will post again.
>
> Kind regards,
> Jakob



More information about the Squeak-dev mailing list