Monticello slowness (was Re: [V3dot10] Prototyping an autobuild system)

Bert Freudenberg bert at freudenbergs.de
Thu Jan 25 17:53:17 UTC 2007


On Jan 25, 2007, at 16:30 , Ralph Johnson wrote:

> for building the base image, I intend to use the update stream,  
> ScriptLoader, and MC, pretty much like the previous release team did.

Then you might want to port the automatic diff downloading from  
MCConfigurations to ScriptLoader. Oor maybe even to MC itself.

One major issue the 3.9 release team faced was the slowness of  
loading MC packages. The major cause of this compared to ChangeSets  
is that a CS is blindly filed in, whereas an MC package is loaded by  
assembling a snapshot of the current package in the image (which  
involves getting all sourcecode and timestamp information out of the  
sources/changes file), downloading a snapshot of the new version  
(might be slow depending on network), taking the difference of these  
snapshots (this is actually quite fast), and filing the remaining  
changes in. However, if you *know* that the copy in the image is  
clean, you can skip the snapshotting and diffing and just apply the  
changes.

This is what MCConfigs do - downloading the right MCD (instead of a  
full MCZ) which returns a MCDiffyVersion and loading that. I'm pretty  
sure I wrote previously about this. The best I can find right now is  
in this thread:

	http://lists.squeakfoundation.org/pipermail/packages/2005-July/ 
000238.html

Making ScriptLoader do the same should be fairly trivial - say you  
want to load "Kernel-sd.150". Then instead of downloading 600K from

	http://source.squeakfoundation.org/39a/Kernel-sd.150.mcz

and snapshotting and diffing etc., you look into your image, see you  
have "Kernel-sd.149" loaded so you download only 15 K from

	http://source.squeakfoundation.org/39a/Kernel-sd.150(149).mcd

or if you have "Kernel-md.148", then it is 16 K from

	http://source.squeakfoundation.org/39a/Kernel-sd.150(md.148).mcd

These are actual working URLs, I just tested. It took a couple of  
seconds because the server needed to generate these MCDs first, but  
now they are very fast, because the server caches these diffs. You  
can construct and download any diff version just by giving the name  
in parens - provided both full versions are in the same repository.

- Bert -




More information about the V3dot10 mailing list