Hi all,
Sorry for the long silence on this list - I've been negligent in my team leadership duties by not keeping the discussion going (for example, I think Lex posted a status report that should have been forwarded to squeak-dev but was instead completely ignored - sorry about that).
However, it seems that if you ignore a problem long enough, eventually someone else will solve it for you. I thought the recent traffic on the list about Tweak's approach to managing updates and packaging was very relevant: it sounds like Impara have come up with a system that nicely balances Monticello and the update stream, and that actually works. If you didn't see the original message from Andreas, here's the important bit:
"In Tweak, *all* code is in packages[*]. Updates are exclusively used for in-image reshapes, e.g., when part of the system has undergone significant changes that require manual intervention. In order to synchronize these modifications with the package versions that we expect, we typically post a configuration map, e.g., a list of packages where we expect some specific version to be present.
When you update, we always consult the update stream first. This will suck in any intermediate configurations and perform the necessary in-image modifications. Once this is done, we merely upgrade all packages in a well-defined order to their latest version."
Moreover, they've come up with an initial packaging for Squeak 3.8, available at http://source.impara.de/iSqueak.html (see the wiki tab for info and downloads).
I think this is great work, and I'd like to suggest that we adopt it as the basis for the partitioning/packaging efforts in this team. This would mean, probably, these steps:
- We adopt the reorganization changeset from iSqueak as the first packaging changeset to go into the 3.9 stream - We include Monticello and Impara's MonticelloConfigurations as base packages in 3.9 - We set up an initial repository (probably based on SqueakSource) for the resulting Squeak packages, which anyone on this team can commit to, and start committing the results of any further partitioning work to it - We hack the repository as needed to support whatever kind of maintainership model we want for the Squeak packages. Tweak lives in a fairly closed world, where it's reasonable for everyone to update to the latest version everyone has posted, but Squeak (probably) can't do that. So instead of "upgrade all packages in a well-defined order to their latest version", we probably need "to their latest maintainer-approved version", and have a harvesting process that turns submitted versions to approved versions. We may find that the "blessing" mechanism I developed for the UnstableSqueak project is a good step in this direction, but we should talk about it more.
Thoughts?
Avi
Hi guys!
I was just about to pack it up for the weekend, so I will not respond before monday. But in general - *any* way forward is fine with me. :) :)
regards, Göran
I think this makes a lot of sense. The combination of Monticello & the update stream sounds like it would be workable.
Also, I see that their initial packaging is reasonably coarse-grained, which is good. :) (i.e. based on class category prefix, not class category)
When you mentioned updating all packages to the "latest version" versus "latest maintainer-approved version", I was thinking "we could use the 'published' flag for that", but oops, that's SqueakMap, not Monticello. :) But I agree that MC's more rigid/uniform packaging scheme is the way to go for managing the base image. We may need a few little enhacements like that for MC, e.g. marking a version as maintainer-approved. (Well, maybe there's a way for the packages to automatically show up in SqueakMap too.)
The initial maintainership model could err on the side of openness, maybe even having some people with write access to all packages. Then we could restrict it a bit more later.
So, perhaps we should give it a go?
As Stephane said, who wants to set this up?
I confess that I mostly just have time to offer discussion and coordination, and not much time for "real work" here other than perhaps helping handle the update stream side of things.
- Doug
On Fri, 10 Jun 2005 14:38:11 +0200, "Avi Bryant" avi.bryant@gmail.com said:
Hi all,
Sorry for the long silence on this list - I've been negligent in my team leadership duties by not keeping the discussion going (for example, I think Lex posted a status report that should have been forwarded to squeak-dev but was instead completely ignored - sorry about that).
However, it seems that if you ignore a problem long enough, eventually someone else will solve it for you. I thought the recent traffic on the list about Tweak's approach to managing updates and packaging was very relevant: it sounds like Impara have come up with a system that nicely balances Monticello and the update stream, and that actually works. If you didn't see the original message from Andreas, here's the important bit:
"In Tweak, *all* code is in packages[*]. Updates are exclusively used for in-image reshapes, e.g., when part of the system has undergone significant changes that require manual intervention. In order to synchronize these modifications with the package versions that we expect, we typically post a configuration map, e.g., a list of packages where we expect some specific version to be present.
When you update, we always consult the update stream first. This will suck in any intermediate configurations and perform the necessary in-image modifications. Once this is done, we merely upgrade all packages in a well-defined order to their latest version."
Moreover, they've come up with an initial packaging for Squeak 3.8, available at http://source.impara.de/iSqueak.html (see the wiki tab for info and downloads).
I think this is great work, and I'd like to suggest that we adopt it as the basis for the partitioning/packaging efforts in this team. This would mean, probably, these steps:
- We adopt the reorganization changeset from iSqueak as the first
packaging changeset to go into the 3.9 stream
- We include Monticello and Impara's MonticelloConfigurations as base
packages in 3.9
- We set up an initial repository (probably based on SqueakSource) for
the resulting Squeak packages, which anyone on this team can commit to, and start committing the results of any further partitioning work to it
- We hack the repository as needed to support whatever kind of
maintainership model we want for the Squeak packages. Tweak lives in a fairly closed world, where it's reasonable for everyone to update to the latest version everyone has posted, but Squeak (probably) can't do that. So instead of "upgrade all packages in a well-defined order to their latest version", we probably need "to their latest maintainer-approved version", and have a harvesting process that turns submitted versions to approved versions. We may find that the "blessing" mechanism I developed for the UnstableSqueak project is a good step in this direction, but we should talk about it more.
Thoughts?
Avi
Sounds like a plan, Avi!
Ultimately, we probably want the maintenance policies to go like:
1. You have to become an approved developer before you can post updates for official, auto-installable packages. Otherwise, any random guy can post trojans to our repository!
2. We need to think about how to handle releases, so that 3.8 has its own set of auto-installable packages. A big step towards handling releases is having a way to designate sets of packages. As releases get closer and pass by, those designations need to become harder to update. Not just any developer should be able to flip a tag on and cause a change to a released set of packages.
3. We seriously need visible bug trackerss for the packages, so that release managers has the input they need in assembling the package sets of #2.
#3 is dead easy. We just need to install a bug tracker that has a category for each package.
#2 is all we can address in the immediate term. My package universes toolkit addresses the problem directly -- in fact, we could implement #2 just by saying "package universes is official" -- but it's not a huge problem and it could be implemented in many other ways as well. Just watch out for who is making the decisions; developers should submit packages for inclusion, but some smaller group (possibly just a singular release manaer) should give the final approval.
#1 relies on having some sort of membership process, something like Debian's "new maintainer process". Before that will work, we need to set up some sort of organization with bylaws, membership, and elections, i.e. something like Debian's constitution. ;)
A great first step is to get things packagized at all, though, and if Tweak has done it then we can just do what they did.
Lex
On Jun 10, 2005, at 8:38 AM, Avi Bryant wrote:
... If you didn't see the original message from Andreas, here's the important bit:
"In Tweak, *all* code is in packages[*]. Updates are exclusively used for in-image reshapes, e.g., when part of the system has undergone significant changes that require manual intervention. In order to synchronize these modifications with the package versions that we expect, we typically post a configuration map, e.g., a list of packages where we expect some specific version to be present.
When you update, we always consult the update stream first. This will suck in any intermediate configurations and perform the necessary in-image modifications. Once this is done, we merely upgrade all packages in a well-defined order to their latest version."
Is there an example of one of these "config map" changeset updates that we could look at? Just curious as to what these look like.
- Doug
Am 18.06.2005 um 10:12 schrieb Doug Way:
On Jun 10, 2005, at 8:38 AM, Avi Bryant wrote:
... If you didn't see the original message from Andreas, here's the important bit:
"In Tweak, *all* code is in packages[*]. Updates are exclusively used for in-image reshapes, e.g., when part of the system has undergone significant changes that require manual intervention. In order to synchronize these modifications with the package versions that we expect, we typically post a configuration map, e.g., a list of packages where we expect some specific version to be present.
When you update, we always consult the update stream first. This will suck in any intermediate configurations and perform the necessary in-image modifications. Once this is done, we merely upgrade all packages in a well-defined order to their latest version."
Is there an example of one of these "config map" changeset updates that we could look at? Just curious as to what these look like.
This is from the Tweak update stream:
http://squeakalpha.org/tweak/external/updates/0559AddDemoPackage.cs
"upgrading" from a config map loads (*) all the packages in it, but only if there is not a newer version already in the system. So after updating you can be sure to have at least these versions.
- Bert -
(*) or merges, based on a preference
On 6/18/05, Bert Freudenberg bert@impara.de wrote:
"upgrading" from a config map loads (*) all the packages in it, but only if there is not a newer version already in the system. So after updating you can be sure to have at least these versions.
How is "newer" defined? By the ancestry tree, or just by version number? Hopefully the former, ie, you have to have that version or some descendant of it.
Avi
Am 19.06.2005 um 16:17 schrieb Avi Bryant:
On 6/18/05, Bert Freudenberg bert@impara.de wrote:
"upgrading" from a config map loads (*) all the packages in it, but only if there is not a newer version already in the system. So after updating you can be sure to have at least these versions.
How is "newer" defined? By the ancestry tree, or just by version number? Hopefully the former, ie, you have to have that version or some descendant of it.
The former, yes. A config map is just a collection of MCVersionDependencies, and a dependency's version is loaded if the "dependency isFulfilledByAncestors not" (literally from MCConfiguration>>upgrade). (*)
The version number is only used when updating the config map from a repository's file listing, to quickly determine the latest version for each package. That version is retrieved and its UUID put into the config map. When installing, the exact version specified in the config map is loaded, nothing else.
- Bert -
(*) Basically, an MCConfiguration is just the dependencies part of an MCVersion. It's the reification of that "versions without code" concept others have been using. It probably would be a good idea to unify both, or to factor out the dependencies from the versions. Is there anything planned along that direction in MC2?
On 6/19/05, Bert Freudenberg bert@impara.de wrote:
(*) Basically, an MCConfiguration is just the dependencies part of an MCVersion. It's the reification of that "versions without code" concept others have been using. It probably would be a good idea to unify both, or to factor out the dependencies from the versions.
Personally I'd be in favor of simply ripping out the dependencies code from MC proper and having MCConfiguration be the recommended way of managing such things. It feels better to me as a separate layer. If we did that, we should probably roll it into the main MC distribution so people don't have to install both.
Is there anything planned along that direction in MC2?
Well, in MC2, the ancestry metadata is maintained at the definition level rather than the package level. Each version of a method, for example, has a separate ID, can be pulled separately from the repository, etc. There's no longer any requirement to do a load or merge at package granularity: although we expect that especially at first, people will send around some equivalent to MC's package versions (which will just be package name and a list of method IDs), you could also distribute a changeset (still a list of method IDs, but a smaller one, with no associated package information), or a full configuration (a very large list of method IDs + several package names). Or maybe there would be other more structured mechanisms with more indirection around names and version numbers and so on; for now we've mostly been focusing on the lower level mechanisms. Work is moving very, very slowly though, because both Colin and I are too busy with other things. It might be a reasonable time to get some outside help, if anyone's interested...
Avi
packages@lists.squeakfoundation.org
-
Avi Bryant -
Bert Freudenberg -
Doug Way -
goran＠krampe.se -
Lex Spoon