On 23.05.2012, at 22:52, Chris Muller wrote:

>>> That's why I said "for example", it's not a silver bullet.  My only
>>> point was that unless we address the inherent scalability issue then
>>> the suffering will be on-going even as we scramble to discuss it and
>>> move files around and then hunt for them later for basic SCM needs.
>>> Seems worthy of a little thoughtful discussion at least..
>> 
>> Agreed. So, as I wrote in the original post: "Other ideas?" :)
> 
> At a minimum please start more conservatively.  Older than two years
> but ensuring to keep a minimum of 10 versions of each package
> regardless of age.  That way we can at least keep SOME history easily
> accessible for all packages and let's see how much speed that gains
> us.

But you are aware that all the history meta data is in each and every MC package? How often do you actually need to look at a package as it was more than a year ago? And, in that case, how much additional effort would it be in that case to select the "attic" repository instead of the "trunk" repository? (that is, if we don't change the tools to automatically look in attic for stuff not found in trunk).

> The older ones should still be accessible in a separate repository.

That would be the proposed "attic", yes.

> ===
> 
> As far as other ideas:  Change SqueakSource's implementation to stop
> using MCFileBasedRepository which I suspect is the root of the
> scalability issue.

Monticello is a purely client-side SCM. You should take a look at the SqueakSource code base. It doesn't use Monticello code for its basic operation at all. It's just a WEBDAV server treating MCZ files as binary blobs. 

Only in very rare circumstances does SqueakSource actually look inside a MCZ file (e.g. to produce diffs or show the history in the web view). Not even the date or author you see at the website is taken from inside the MCZ file, it's the uploading user and date.

>  A Magma-based MC backend performs steadily
> regardless of size, I'd love to test this if someone can get
> SqueakSource going on Squeak 4.3.

I would be surprised if it didn't work on 4.3.

>  I also know there are other
> solutions like ss3 and SmalltalkHub although SH may be too different
> to easily migrate..

Assuming that the HTTP listing of gazillions of versions is the bottleneck, neither of those would help significantly. (of course, we need to profile where the slowness actually comes from).

- Bert -