[squeak-dev] Making trunk repo faster

Bert Freudenberg bert at freudenbergs.de
Fri May 25 20:09:19 UTC 2012


On 25.05.2012, at 14:52, Levente Uzonyi wrote:

> On Fri, 25 May 2012, Bert Freudenberg wrote:
> 
>> 
>> On 24.05.2012, at 00:43, Chris Cunningham wrote:
>> 
>>> On Wed, May 23, 2012 at 10:28 AM, Bert Freudenberg <bert at freudenbergs.de> wrote:
>>>> Accessing the repository gets slower and slower due to the ever increasing number of versions there.
>>>> 
>>>> To make it faster we could remove old package versions. Say, everything older than a year (but keep at least one version of each package). We could move those to an "attic" repository.
>>>> 
>>>> Sounds good? Other ideas?
>>>> 
>>>> - Bert -
>>> 
>>> When you need the configuration file to load specific versions of
>>> packages, I would assume we'd want to keep those versions.
>>> Also, are you considering removing those configuration files as well?
>>> 
>>> -Chris C (not Cunnington)
>> 
>> Good question, and a potential obstacle.
>> 
>> I think the simplest solution would be to not move the update configs over. Instead, rewrite them on the server to have both the trunk and attic repositories. (a config stores the repositories where it is looking for packages).
>> 
>> OTOH, we now have more than 200 configs in trunk, and they, too, slow down the repository listing.
>> 
>> Maybe we need something similar to the MCD generation: If the image lets the server know which updates it already knows about, the server would only have to send newer ones.
> 
> I think the best solution to the problem would be to add a new "view" (a pseudo project/repository) to the SqueakSource server, which would show only a limited amount of packages. E.g.: http://source.squeak.org/trunk/recent would only list recently added packages and configurations, while http://source.squeak.org/trunk would show the full list.
> Saving to the pseudo repository would have the same effect as saving to the main one.
> 
> 
> Levente


That would be one simple way of how the image could let the server know it needs a full or partial file list, yes. Perhaps "/trunk?recent" would work even better than "/trunk/recent"?

One problem is that configs are not time stamped. The server needs to somehow figure out which configs are recent. And for MCZs it still needs to make sure there is at least one version of each package visible even if it is not recent.

Also we need to think about when the image can use the recent and when the regular repo. Or do you think that would be a manual process? Actually adding this as a second trunk repository visible in the Monticello UI?

And we would still need to make the server faster - right now it takes like 20 seconds to open the http://source.squeak.org/trunk.html page. What it does in that time is mainly building the reports of the "all" and "latest" version tabs.

(also, I guess we should move out all packages which are not actually part of trunk but ended up there accidentally)

- Bert -




More information about the Squeak-dev mailing list