[squeak-dev] Performance regression
forums.jakob at resfarm.de
Sun Jun 28 19:01:25 UTC 2020
I just debugged a smalltalkCI run under Windows Subsystem For Linux
and it also fails two of the test cases which got slower. This might
not be the real reason for the Mac failures (because this is WSL...),
but it might give a clue.
On 17th of June, Marcel pushed Monticello-mt.724, which changes
MCWorkingCopy>>uniqueVersionName such that name candidates are checked
against the package cache as well.
The test cases I profiled spent >95% of the time writing backup
Monticello versions to the package cache for each package saved with
Squot during this test (it is a safeguard function from three years
ago when Squot or the Git I/O were new and unreliable). The time was
lost in FileDirectory>>primLookupEntryIn:index:. The package cache is
rather full at this point with what has been loaded during the image
update at the start of the test run and, more importantly, with all
the test package versions saved in other tests before. They are not
cleaned up after each test.
If this is the problem on the Mac as well, it turns out I have to fix
my own code after all. Marcel's change seems sound to me and no disk
I/O vs. repeated directory enumeration makes a difference in test
cases after all. I could try to clean up the test packages from the
cache directory. Or I could just drop that Monticello backup after
all... I have never looked at the backup versions in the cache for
three years, and their ancestry is wrong anyway (ignore switching
Still, may I suggest to optimize includesVersionNamed: for
MCDirectoryRepository? It currently reads all the directory entries
just to check afterwards whether one of them fits the version name.
Instead it could look up files for the sought version name directly:
there is a method allFileNamesForVersionNamed:.
Am Mi., 24. Juni 2020 um 08:58 Uhr schrieb Jakob Reschke
<forums.jakob at resfarm.de>:
> Forgot the time frame: I am looking for something that changed between
> 2020-06-14 and 2020-06-22.
> The builds run unconditionally every weekend to notice Trunk breakages
> and were fine on 2020-06-14.
> Am Mi., 24. Juni 2020 um 08:43 Uhr schrieb Jakob Reschke
> <forums.jakob at resfarm.de>:
> > Hi all,
> > On Monday I noticed that some Squot tests started to fail on Travis CI
> > due to timeouts (test cases exceeding the 5 sec default timeout). The
> > problems are not due to changes in Squot itself because if I retrigger
> > an older build which was already green, it also fails. So I suspect a
> > change in Squeak Trunk, since the Squeak 5.2 builds are also not
> > affected.
> > I did not yet find the cause, but would like to make known what I
> > found so far. Does anyone have an idea what could have caused this? Of
> > course I don't expect you to debug my test cases, but maybe you have a
> > hint for me.
> > - The Mac builds are much more affected than the Linux builds. This
> > could be due to differences in Travis CI's setup for the operating
> > systems. The Linux test runs also got slower, but not by so much.
> > - Both the 64 bit and 32 bit Mac builds got slower. The 64 bit ones
> > scratch at the 5 sec mark, but did not exceed it yet.
> > - If I repeat older Mac builds, they experience the same slowdown.
> > - The tests on Squeak 5.2 are unaffected on either platform.
> > - Some tests take about 6x up to 20x more time than before. On Linux
> > it is only up to x3 times slower and in sum hardly noticeable. For the
> > rest of this, I will only talk about the Mac Trunk builds.
> > - testTrackingAPackage has the worst ratio, whereas testTrackingAClass
> > is virtually unaffected.
> > - Only tests that involve whole packages, not individual objects or
> > single classes, seem to be affected. Is the regression in something
> > that PackageInfo does (enumerating classes, methods, accessing
> > categories)?
> > - Surprisingly, the tests with Git are less worse than the tests with
> > a stub in-memory repository (these are the same test cases, but with a
> > different backend object). I think the Git tests by chance ran after
> > the in-memory tests, maybe that affects things, I don't know yet. The
> > Git tests did not reach the 5 sec mark.
> > - Tests that involve Monticello (such as the Monticello->Git
> > conversion) and do not only create MCDefinitions and pass them around,
> > take about 2x as long now.
> > For reference:
> > Good build: https://travis-ci.org/github/hpi-swa/Squot/jobs/698280144
> > Bad build: https://travis-ci.org/github/hpi-swa/Squot/jobs/700640404
> > The runtimes are annotated to the test cases when you expand them.
> > If you do want to look at the code, just load the Git Browser, then
> > you will also have the test cases.
> > If I find out more on the weekend, I will post again.
> > Kind regards,
> > Jakob
More information about the Squeak-dev