I was somewhat hesitant to allow updates which directly load packages 
from SqueakMap.  This was discussed a few months ago, though, and there 
wasn't any significant opposition to the idea.  See:

http://lists.squeakfoundation.org/pipermail/squeak-dev/2004-February/073560.html
http://lists.squeakfoundation.org/pipermail/squeak-dev/2004-February/073588.html

As Goran says, we should only be allowing version-specific package 
loading commands as updates (which are immutable by convention), so we 
should still be following this rule:  "Images produced by updating 
should always be exactly the same, not depending on some external 
changes over time."

I just checked for updates which load SM packages, and it looks like 
there are four of them at the moment.  They all load a specific SM 
version, so we should be safe.  Here they are:

5727SmaCC-RuntimeSM-md  --  http://map1.squeakfoundation.org
5789SUnitFromSM-md  --  http://map1.squeakfoundation.org
5818CompilerFromSM-md  --  http://map1.squeakfoundation.org
5880PackageInfoFromSM-avi  --  http://beta4.com

They should also only load from the SqueakMap server itself so that we 
don't rely on any 3rd-party servers, but it looks like 5880 does not, so 
it looks like we've broken that rule.  The PackageInfo package should 
really be moved over to the SM server (map1.squeakfoundation.org).  
(It's a bit of a pain for me to try to verify that every package-loading 
update is indeed on the SM server.)

Even if we ensure that all loaded packages are on the SM server, I have 
to admit that I'm not crazy about it not having a backup like the update 
server has.  I suppose one thing we could do is temporarily go back to 
not allowing package-load updates until we get an SM mirror in place.  
This would be a stronger incentive to a get a mirror going. :-)  
Actually, a simple read-only updated-once-per-day file mirror would be 
good enough, right?  (Requiring zero work on Goran's part.)  Is there a 
single SM catalog file which could be mirrored?  If so, it could mirror 
the catalog file and then the directory of packages on the SM server.  A 
fancier mirror could be set up later.

As to the issue of whether we should extend the in-image update loader 
to support the SqueakMap installers for various formats (SAR, .cs, .mcz, 
etc...), I don't have a strong opinion on that.  If everyone thinks it's 
a good idea we could try it.  I guess that would mean that the 
changesorter would not really show everything that was loaded as 
updates, though.

- Doug


Ned Konz wrote:

> Recently I had the opportunity to speak in person with Michael Rueger, 
> Scott Wallace, Yoshiki Ohshima, and Diego Gomez Deck about a number of 
> things.
>
> One of the items that we discussed was the nature of the update stream.
>
> There have been several updates in the update stream that were change 
> sets that loaded (or tried to load) packages from SqueakMap.
>
> This has several negative consequences:
>
> * It is possible to have a reliable connection to an update server but 
> not be able to load an update (if either SqueakMap or the server that 
> holds the package is down).
>
> * There is the risk that an external package may be replaced by 
> another, thereby introducing the possibility of there being 
> differences between to images that purport to be the same version. 
> Though we've done this from time to time in the update stream to fix 
> bugs, (I recall that) the replacements have been made quickly after 
> the original was posted, and only when the original update itself was 
> not able to be fixed with another update.
>
> * It is impossible to collect a single updates directory on a disk and 
> then apply the updates from that directory without a network 
> connection (we have "Utilities applyUpdatesFromDisk" which does this 
> easily, and which has been useful to those of us who actually create 
> updates for the update stream).
>
> It also has a positive aspect:
>
> * It is easier to take non-code items like fonts, etc. in SAR files.
>
> By definition, the update stream is for packages that live in the 
> image. There are a few such packages that also are available on 
> SqueakMap (MCInstaller, PackageInfo, SM2, and SARInstaller are the 
> ones that come to mind). All of these packages are just code, and so 
> the packaging advantages of (say) a SAR aren't important.
>
> For my updates to SARInstaller I have published the same code to the 
> update stream as Smalltalk code. We published the Bitstream Vera fonts 
> to the update stream as Smalltalk code, even though it was a fair 
> amount of work to do so (loading a SAR would have been much easier).
>
> However, there have been other updates that have gone to SqueakMap to 
> load packages.
>
> We felt after discussion that it would be a good idea to establish a 
> clear policy for the update stream to avoid the above problems.
>
> Michael's summary of our discussion is:
>
> "General assumption should be that updates can be filed in from disc 
> only, without connection to any server.
> Images produced by updating should always be exactly the same, not 
> depending on some external changes over time as is the case with 
> external packages."
>
> My additional suggestions (not necessarily shared by the above named 
> people):
>
> * We *should* require that all components of an update stream be 
> loaded from the same server. Having to go to more than more server is 
> a really bad idea.
>
> * We *could* require that all the updates be Smalltalk code, as they 
> have been. In the past this has required hacks like MIME-encoded, 
> Gzipped files to load fonts. This does, however, allow easier analysis 
> of code (like Andreas' recent analysis of authorship of methods in the 
> update stream), as well as searching for code (since the files are all 
> text).
>
> * To avoid such hacks, we *could* extend the in-image update loader to 
> use the SqueakMap installers for various formats (SAR, .cs, .mcz, 
> etc...) and thus avoid the need for a change set that then loads 
> another file
>
> If we want to unify SqueakMap with the update stream, the following 
> suggestion from Diego Gomez Deck (who is sitting next to me right now) 
> could be useful:
>
> * We could take items that were downloaded from an update server as 
> part of the update process but are also available on SqueakMap and 
> save them in the local SqueakMap cache to avoid duplicate downloading.
>
> What do you think?
>
> Guides?
>
> Thanks,
> Ned Konz
>
>
>