[squeak-dev] Too many update configs in trunk?

Fri Jun 10 21:21:50 UTC 2011

On Thu, 9 Jun 2011, Bert Freudenberg wrote:

>
> On 09.06.2011, at 15:18, Levente Uzonyi wrote:
>
>> On Thu, 9 Jun 2011, Bert Freudenberg wrote:
>>
>>> Hi folks,
>>>
>>> I just tried to update an older trunk image to the current one. It took ages.
>>>
>>> I have the impression that people are committing way more update configs than necessary. These are not cheap, but they force an incremental update process. Instead of just loading the latest version, they force me to load many many intermediate versions. I cannot imagine it is necessary to have so many. We are close to 200 now.
>>
>> The intermediate versions for some packages have to be loaded otherwise your image may break.
>>
>>> Normally we only need a new config map when the load order of packages needs to change. But I see many configs where the order does not change at all. Attached below.
>>>
>>> Sometimes it might be necessary to force a specific version to be loaded before a subsequent version. But is that really necessary that often?
>>
>> In most cases it's necessary. Before update-ul.16x.mcm I didn't rely on the package order inside mcms, so some mcms could have been avoided, but I guess their number is less than 10.
>>
>>>
>>> If it is indeed necessary, then we need to figure out a way to only force-loading a specific package, not all packages. That would speed up the updates quite a bit.
>>
>> If we can mark the packages which require a specific version to be loaded, then we'll have the faster update process.
>> One possible way is to only include those packages in the mcms which require a specific version. At the end of the update process update to the latest version all in-image packages which are also present in the repository. But changing the update process by the update process is a PITA as Chris found it out recently.
>
> Loading all packages found is not a good idea. We only want to load the packages mentioned in the config, and do it in the order given.
>
> What should work, IMHO, is when uploading a new config, only update those package versions that need changing. That is, take latest config, change version for the package you need updated, publish. There currently is no UI for this, it only allows "update from image" and "update from repository" which updates all packages. But if individual packages could be updated that should work faster.

I agree.

>
>
>> Levente
>>
>> <rant>
>> Try updating a Pharo image and you'll see how snappy our update process is.
>> </rant>
>
> Never tried. You mean it's even slower? I agree that for day-to-day development the mcm-and-mcd updates are fast enough. We did go to some effort to make it fast, after all.

It feels a lot slower. AFAIK they use an update stream with quite a lot of 
mczs.

>
> However, when updating only once a month or so I think it could be faster.

No, loading an mcz for almost every change is not faster.

Levente

>
> - Bert -
>
>>>
>>> ----
>>> update-ul.118.mcm
>>> ----
>>> update-ul.119.mcm
>>> ----
>>> update-nice.120.mcm
>>> ----
>>> update-nice.121.mcm
>>> ----
>>> update-ul.122.mcm
>>> ----
>>> update-ul.123.mcm
>>> ----
>>> update-ul.124.mcm
>>> ----
>>> update-nice.125.mcm
>>> ----
>>> update-nice.126.mcm
>>> ----
>>> update-nice.127.mcm
>>> ----
>>> update-ar.128.mcm
>>> + Monticello
>>> Compiler
>>> EToys
>>> Exceptions
>>> Files
>>> FlexibleVocabularies
>>> Graphics
>>> GraphicsTests
>>> Installer-Core
>>> Kernel
>>> KernelTests
>>> MonticelloConfigurations
>>> - Monticello
>>> ----
>>> update-ar.129.mcm
>>> ----
>>> update-ar.130.mcm
>>> ----
>>> update-nice.131.mcm
>>> ----
>>> update-nice.132.mcm
>>> ----
>>> update-ul.133.mcm
>>> ----
>>> update-laza.134.mcm
>>> ----
>>> update-nice.135.mcm
>>> ----
>>> update-nice.136.mcm
>>> ----
>>> update-ul.137.mcm
>>> ----
>>> update-ul.138.mcm
>>> + Kernel
>>> Compiler
>>> EToys
>>> Exceptions
>>> Files
>>> FlexibleVocabularies
>>> Graphics
>>> GraphicsTests
>>> Installer-Core
>>> - Kernel
>>> ----
>>> update-ar.139.mcm
>>> + System
>>> Morphic
>>> Multilingual
>>> MorphicExtras
>>> MorphicTests
>>> MultilingualTests
>>> Nebraska
>>> Network
>>> NetworkTests
>>> PackageInfo-Base
>>> PreferenceBrowser
>>> Protocols
>>> SMBase
>>> SMLoader
>>> ST80
>>> SUnit
>>> SUnitGUI
>>> ScriptLoader
>>> Services-Base
>>> Sound
>>> - System
>>> ----
>>> update-nice.140.mcm
>>> ----
>>> update-ul.141.mcm
>>> ----
>>> update-ul.142.mcm
>>> ----
>>> update-nice.143.mcm
>>> ----
>>> update-ul.144.mcm
>>> ----
>>> update-ar.145.mcm
>>> ----
>>> update-ul.146.mcm
>>> ----
>>> update-bf.147.mcm
>>> + HelpSystem-Core
>>> + HelpSystem-Tests
>>> + Help-Squeak-Project
>>> ----
>>> update-ul.148.mcm
>>> ----
>>> update-ar.149.mcm
>>> ----
>>> update-nice.150.mcm
>>> ----
>>> update-ar.151.mcm
>>> + Network
>>> Collections
>>> CollectionsTests
>>> Compression
>>> Monticello
>>> Kernel
>>> Compiler
>>> EToys
>>> Exceptions
>>> Files
>>> FlexibleVocabularies
>>> Graphics
>>> GraphicsTests
>>> Installer-Core
>>> KernelTests
>>> MonticelloConfigurations
>>> System
>>> Morphic
>>> Multilingual
>>> MorphicExtras
>>> MorphicTests
>>> MultilingualTests
>>> Nebraska
>>> - Network
>>> ----
>>> update-ar.152.mcm
>>> + Compression
>>> Collections
>>> CollectionsTests
>>> - Compression
>>> ----
>>> update-ul.153.mcm
>>> ----
>>> update-ul.154.mcm
>>> ----
>>> update-ul.155.mcm
>>> ----
>>> update-ul.156.mcm
>>> ----
>>> update-ul.157.mcm
>>> ----
>>> update-ul.158.mcm
>>> ----
>>> update-ul.159.mcm
>>> ----
>>> update-ul.160.mcm
>>> ----
>>> update-ul.161.mcm
>>> ----
>>> update-ul.162.mcm
>>> ----
>>> update-ul.163.mcm
>>> ----
>>> update-dtl.164.mcm
>>> + Help-Squeak-TerseGuide
>>> ----
>>> update-ul.165.mcm
>>> ----
>>> update-ul.166.mcm
>>> ----
>>> update-ul.167.mcm
>>> ----
>>> update-ul.168.mcm
>>> ----
>>> update-nice.169.mcm
>>> ----
>>> update-dtl.170.mcm
>>> + SystemReporter
>>> ----
>>> update-ul.171.mcm
>>> ----
>>> update-ul.172.mcm
>>> ----
>>> update-ul.173.mcm
>>> ----
>>> update-ul.174.mcm
>>> ----
>>> update-ul.175.mcm
>>> ----
>>> update-dtl.176.mcm
>>> ----
>>> update-ul.177.mcm
>>> + BalloonTests
>>> ----
>>> update-ul.178.mcm
>>> ----
>>> update-cmm.179.mcm
>>> + TrunkScript
>>> ----
>>> update-cmm.180.mcm
>>> - TrunkScript
>>> ----
>>> update-ul.182.mcm
>>> ----
>>> update-cmm.183.mcm
>>> ----
>>> update-nice.184.mcm
>>> ----
>>> update-ul.185.mcm
>>> ----
>>> update-ul.186.mcm
>>> ----
>>> update-ul.187.mcm
>>> ----
>>> update-kb.188.mcm
>>> ----
>>> update-nice.189.mcm
>>> ----
>>> update-ul.190.mcm
>>> ----
>>> update-ul.191.mcm
>>>
>>>
>>> repo := MCRepositoryGroup default repositories detect: [:r | r description = 'http://source.squeak.org/trunk'].
>>> updates := repo allFileNames select: [ :f | (f beginsWith: 'update-') and: [f endsWith: '.mcm'] ].
>>> updates := updates asSortedCollection: [ :a :b | (a allButLast: 4) numericSuffix < (b allButLast: 4) numericSuffix].
>>> Transcript clear.
>>> updates do: [ :update |
>>> 	Transcript show: '----'; cr; show: update; cr.
>>> 	config := repo versionNamed: update.
>>> 	pkgs := String streamContents: [:s | config dependencies do: [:dep | s nextPutAll: dep package name; cr] ].
>>> 	prev ifNotNil: [
>>> 		context := nil.
>>> 		(TextDiffBuilder from: prev to: pkgs)
>>> 			patchSequenceDoIfMatch: [ :string | context ifNotNil: [context := context, string ] ]
>>> 			ifInsert: [ :string | context ifNotNil: [Transcript show: context]. Transcript show: '+ ', string. context := '']
>>> 			ifRemove: [ :string | context ifNotNil: [Transcript show: context]. Transcript show: '- ', string. context := ''].
>>> 		].
>>> 	prev := pkgs].
>>>
>>>
>>>
>>>
>>
>
>
>