Monticello slowness (was Re: [V3dot10] Prototyping an autobuild
system)
Bert Freudenberg
bert at freudenbergs.de
Thu Jan 25 17:53:17 UTC 2007
On Jan 25, 2007, at 16:30 , Ralph Johnson wrote:
> for building the base image, I intend to use the update stream,
> ScriptLoader, and MC, pretty much like the previous release team did.
Then you might want to port the automatic diff downloading from
MCConfigurations to ScriptLoader. Oor maybe even to MC itself.
One major issue the 3.9 release team faced was the slowness of
loading MC packages. The major cause of this compared to ChangeSets
is that a CS is blindly filed in, whereas an MC package is loaded by
assembling a snapshot of the current package in the image (which
involves getting all sourcecode and timestamp information out of the
sources/changes file), downloading a snapshot of the new version
(might be slow depending on network), taking the difference of these
snapshots (this is actually quite fast), and filing the remaining
changes in. However, if you *know* that the copy in the image is
clean, you can skip the snapshotting and diffing and just apply the
changes.
This is what MCConfigs do - downloading the right MCD (instead of a
full MCZ) which returns a MCDiffyVersion and loading that. I'm pretty
sure I wrote previously about this. The best I can find right now is
in this thread:
http://lists.squeakfoundation.org/pipermail/packages/2005-July/
000238.html
Making ScriptLoader do the same should be fairly trivial - say you
want to load "Kernel-sd.150". Then instead of downloading 600K from
http://source.squeakfoundation.org/39a/Kernel-sd.150.mcz
and snapshotting and diffing etc., you look into your image, see you
have "Kernel-sd.149" loaded so you download only 15 K from
http://source.squeakfoundation.org/39a/Kernel-sd.150(149).mcd
or if you have "Kernel-md.148", then it is 16 K from
http://source.squeakfoundation.org/39a/Kernel-sd.150(md.148).mcd
These are actual working URLs, I just tested. It took a couple of
seconds because the server needed to generate these MCDs first, but
now they are very fast, because the server caches these diffs. You
can construct and download any diff version just by giving the name
in parens - provided both full versions are in the same repository.
- Bert -
More information about the V3dot10
mailing list