Packages June 2005

packages@lists.squeakfoundation.org

11 participants
10 discussions

Re: source.squeakfoundation.org (was Re: iSqueak)
by Avi Bryant 28 Jul '05

28 Jul '05

On 6/27/05, Alexandre Bergel <bergel(a)iam.unibe.ch> wrote: > But the Project 39a is still empty... Ah, yes, it is. We do need to commit stuff there. I just didn't understand the purpose of a script that created new packages when we already have a changeset from Andreas that does so. I'll populate that project shortly. > I have many changes related to the system change notification that are waiting... Great. Avi

9 97

Feeding the update stream
by Daniel Vainsencher 29 Jun '05

29 Jun '05

First of all, I like everything that's going on. Some good ideas are getting implemented. So was any decision made on how stuff gets submitted (FIX/ENH), then included? One idea that I didn't see mentioned (but I might have just missed it) is to add a ForInclusion project to sources.squeakfoundation.org, with write access for everyone (or Apprentice and up, or whatever). Announce that that is the place for code to be submitted, when (the author thinks) it is ready for inclusion. Then harvesters would simply merge stuff from that repository into the release repository. Then whatever tools get coded for improving awareness of new code in an MC repository automatically apply to the harvesting process. Daniel

3 3

Re: iSqueak
by Doug Way 27 Jun '05

27 Jun '05

On Jun 11, 2005, at 7:16 PM, Andreas Raab wrote: > Actually, instead of having "approval flags" I'd recommend having > "release repositories". E.g., consider that we're setting up a 3.9 > repository which defines the "latest" package versions for everything > that ought to be in 3.9. > > Rather than "approving" a package version in their own repository, a > maintainer (or release master) would publish (or copy) package > versions to the release repository. This has a number of extra > advantages - fixes that turn out to be important for a release can be > put into this repository but may never appear in the "master" > repository for the package (because it might have been a last-minute > workaround). Integration fixes can be done for the release and then > fed back to the package owners. > > Basically, you're defining your own package environment, treating the > repository as a distribution and decide when you want to update the > distribution from the latest package versions developed by the owner > of the package. I think this sounds good. It almost seems like the only way to do it, actually. (For now, at least.) The other thing I like about this is that it handles the case in which you're making a big change in a release which needs to cut across several packages. All the packages are in the same repository, and they can be updated at the same time. It's more cumbersome if they're spread across different repositories, especially if the person making the change only has access to some of the repositories. Certainly if you're doing package-detangling work, you'll need to make changes to multiple packages at once. >> The initial maintainership model could err on the side of openness, >> maybe even having some people with write access to all packages. Then >> we could restrict it a bit more later. > > With the above model you could restrict commit rights for the release > repository to the people actually working on the release, whereas > access to the "master" repository is limited to whatever the package > owner decides. This seems like a good model to me because it leaves > the responsibilities in the right places. Ok, so a group of release people would have commit rights to the (3.9) release repository. I wonder if we might also want the package owners to have commit rights to their packages in the release repository as well. (Although I guess commit rights are repository-specific, not package-specific?) Because the release repository may end up being the location of the de facto "latest/bleeding-edge" version of the package, whereas the "master" repository for the package is more for stable versions. Well, I guess it's up to the package owner whether they want to do major new development in their own master repository, but then they'd have to worry about merging with the release repository version later. In some cases that might make sense. I guess it depends on the package... with something like Kernel, probably all development would happen in the release repository. With something more external like SqueakMap, most development might happen in the owner's master repository, with the latest version being copied over to the release repository occasionally. So, a typical scenario would be that a package, say Network, goes through a number of new versions in the release repository during 3.9 development, from Network.3 to Network.17 or something. (My Monticello ignorance is showing here... I assume that every commit to the package results in a bumped version number? Looks like I will be getting more familiar with MC soon!) So anyway, 3.9 final contains Network.17. The Network package owner may decide to not let any of the interim versions into the master repository for Network, and only add the final Network.17 version. Or he/she may decide that Network.12 is stable enough to be generally useful, and include that one in the master repository as well. But it might be easiest for the package owner to add new enhancements into the one in the release repository, so there's no merging to worry about later. As far as SqueakMap goes, I'd imagine that the final/stable versions in the master repositories would typically all be available on SqueakMap for people to use, but we probably wouldn't bother with making all of the interim/alpha release repository versions (e.g. Network.4, Network.5, etc) available on SqueakMap. Just some thoughts. Are we ready to go ahead with this? :-) - Doug

7 35

Where we are going with partitioning and harvesting etc.
by goran＠krampe.se 23 Jun '05

23 Jun '05

Hi all! (cross posting since Ken's recent post touches on the very same aspects) I have been re-reading the archive (=the packages team archive) to see where we are and what I had promised to do. :) I am a bit unsire about the latest development. I had my sights set on this (brief description): - Get Universes and SM playing together. As promised by me to Lex and the team. - Use PIs to partition the image. The good ol TFNR idea, which Ken now also expresses in his post. - Tie the PIs to corresponding SM entries. This is key to get the full connection to maintainers and their emails etc. - Gradually move to maintaining these "image packages" as MCs "released on SM", instead of using changesets in the stream. This part obviously is different from the recent plans in the Package team. - Add my planned dependency model in parallell with Universes. They in fact complement each other quite nicely when thinking more closely about it. Now it seems like the plan is to move to using something independent of SM, which means I have no idea on how to proceed with the above. Perhaps I am not interpreting the plan correct - then you all may of course explain it to me and slap me. :) I have also some serious doubts about schemes/models that depend on "community work". With that I mean work "by" the community "for" the community, eh, if you know what I mean - instead of work "by" yourself "for" yourself. Hmmm, Ken has a much better english than me, but he is essentially saying the same thing. Harvesting is a typical example, an unselfish act for the good of all - not directly connected to any personal immediate need. Or trying to maintain a Universe, yes, I *do* have doubts about that model - but at the same time I want to enable it - because I may very well be wrong :). Or staking out the image, an act which has stalled numerous times for that specific reason I think. All these activities have that in common - they are "non individual" activities relying on community members to do the work "for the benefit of all". I have come to the personal conclusion that schemes depending on such work have a high risk of failure in the Squeak community. We simply aren't that good at it. We are much more individualistic and we like to scratch our own itches, which means there are two things we are really good at: - Maintaining our *own* packages, typically driven by our own needs in combination with pride and "feeling good" when other people appreciate your work. - Finding and fixing bugs in other Squeakers' packages but almost always driven by our *own* projects/packages, we simply fix stuff we stumble over. So my view is that we need a system which relies on such individual work only, or at least to 95%. Ken asked "What are we waiting for?" and... I am not sure. The PI-partitioning bit - which is key here - should simply "be done" IMHO. I say let's get *a single person* to just do it and push a .cs into the stream which creates the PIs. I was inclined to get Doug to do it (being v3.9 boss and all), but I don't really care as long as someone does it. Again, I am not sure how this plays with the latest plans/developments (read the packages archive for details). Ok, over and out. regards, Göran

6 9

Fwd: iSqueak "config map" changeset
by Doug Way 22 Jun '05

22 Jun '05

FYI, Andreas replied to me and cc-ed the Packages list, but it didn't show up in the list archives because I think he's not subscribed. (This describes an example alpha update which loads an MC config map, which may be needed in conjuction with a do-it update.) ---------------------------------------- Date Sat, 18 Jun 2005 10:44 PM >From "Andreas Raab" <andreas.raab(a)gmx.de> To "Doug Way" <dway(a)mailcan.com> Cc packages(a)discuss.squeakfoundation.org Subject Re: iSqueak > > Is there an example of one of these "config map" changeset updates that > > we could look at? Just curious as to what these look like. > > Like this: > > "Change Set: IslandWriter > Date: 16 June 2005 > Author: Posted by Monticello > > This is a configuration map created by Monticello." > > (MCConfiguration fromArray: #( > repository ('http://source.impara.de/Tweak') > dependency ('Tweak-Maker' 'Tweak-Maker-ar.7' > '0d27ffa7-4ba9-244f-a0f3-3e5876de1b90') > dependency ('Islands' 'Islands-ar.13' > 'e647905a-8781-b749-b0dd-34cce26d6595') > )) upgrade. > > In other words, there is a (set of) repositories (here just one) and a > set of package versions which can be found in the repositories. The > configuration upgrade checks to see if there any newer versions locally > and if so ignores the version requested. If there isn't a newer version > it requests the version listed in the map and loads it. Depending on the > preference updgradeIsMerge it will either load or merge this version > into the image. > > Cheers, > - Andreas

1 0

Re: Where we are going with partitioning and harvesting etc.
by goran＠krampe.se 22 Jun '05

22 Jun '05

Hi all! Ken Causey <ken(a)kencausey.com> wrote: > On Tue, 2005-06-21 at 15:00 +0200, goran(a)krampe.se wrote: > > Ken asked "What are we waiting for?" and... I am not sure. The > > PI-partitioning bit - which is key here - should simply "be done" IMHO. > > I say let's get *a single person* to just do it and push a .cs into the > > stream which creates the PIs. I was inclined to get Doug to do it (being > > v3.9 boss and all), but I don't really care as long as someone does it. > > Again, I am not sure how this plays with the latest plans/developments > > (read the packages archive for details). > > In case you don't know the message from me that Goran is referring to is > at > > http://discuss.squeakfoundation.org/cgi-bin/ezmlm-cgi?8:msp:221:ooeonbbmifc= > eghgaabgc > > I'm assuming you (Goran) mean that 'PI-partitioning' is 'key' to > allowing developers to take responsibility for parts of this system but > I don't think that is true. Even if we stepped back a bit to a > monolithic image with a single update stream there is no technological > barrier to staking out the bits in the image and having people sign up > and take responsibility for them. I'm not saying it is perfect but > there is absolutely no reason I can see that the existing partitioning > into the current categories and classes isn't sufficient. Clearly there > would not be a mapping of one category to one team in every or even most > cases and in some cases it may even be desirable to split a single > category between two teams, but I don't see a big problem with that. Of > course it may be the case that two teams are working on highly > inter-related parts of the system in which case their will need to be > good communication between them, but I think that will be true in any > partitioning system. I think we agree - when I say "PI-partitioning" I don't mean refactoring or recategorization. I am all for the simplest thing - just assigning 1-n categories to one PI (almost, see below). > Each team would then either be given direct access to the update stream > or perhaps more realistically in the short term they would send their > changesets to the Coordinators or other small group that has direct > access to the update stream. It would be understood that the team is > responsible for what enters the update stream and would be expected to > take care that it is likely to be acceptable to the community as a > whole. The 'update stream team' then would not be expected to do much > if any checking of what was submitted only probably keeping track of > what update came from which team and when.=20 Yes, and this is exactly what I was after earlier in TFNR - the idea that you don't need to "split out" packages in order to take responsibility for them - the update stream can work just fine. > In some cases a change to one team's part of the system will affect one > or more other teams. It should be clear to all teams that this is one > of the things they will need to consider and that it is up to them to > coordinate with the other teams and to see that the other teams > understand what needs to be done and are aided in doing that and seeing > that their changes enter the update stream roughly in sync with the > first team's. > > Admittedly this, at least until teams have matured somewhat, will likely > result in a more chaotic update stream. But I believe that it can be > managed and that in any case any other partitioning scheme that I can > see appearing in the near future is almost certainly going to have the > same issues. > > Now let me be clear that I am not by any means against the pursuit of > other and better systems. Please keep working on this. However it's > clear that producing a better system is not simple and is taking time. > Can we afford to stall basic development while we wait? I don't see any > reason why we can't get started with what we have and transition to a > better system when it becomes available. Again, let me explain what *I* meant us to do - I think you and I are all in agreement. 1. A single person sits down and groups the categories into PIs. The current categories are probably just fine, and if that person feels like it he may of course move classes around or rename categories - but *nothing else*. And he mustn't get stuck. The result: A .cs that performs the recategorizations/moves and creates PI instances in the image according to the grouping. 2. Someone (say me) creates packages on SM for each such PI and types the PI name into the field for that. Then we assign the SM packages to an owner and add co-maintainers to that. Tada - we then have our teams with names, emails etc. all neatly described in SM and thus in easy reach for tools. 3. We start by what you describe above - just use the update stream as before. No problem. Ok, now - the above is actually one days work if we just decided to "do it". But there is now a different plan on the playfield - to go with what Impara is doing. Avi is Team leader. I trust him to help us come to a decision on what to do. regards, Göran

2 1

Re: iSqueak
by Doug Way 20 Jun '05

20 Jun '05

On Jun 18, 2005, at 10:37 PM, Andreas Raab wrote: > ... >> Well, I guess it's up to the package owner whether they want to do >> major new development in their own master repository, but then they'd >> have to worry about merging with the release repository version >> later. In some cases that might make sense. I guess it depends on >> the package... with something like Kernel, probably all development >> would happen in the release repository. > > I don't think so. In my understanding there should never be active > development inside the release repository (the one exception being > integration fixes) because the point of packages is to do development > in packages (== the package repository). And if there isn't a master > repository right now for -say- the Kernel package, well, then I guess > whoever wants to hack it better create one. This is one of the ways of > assigning active responsibility for certain areas of Squeak - I am > sure it would spawn an interesting (and useful) discussion if Joe > Random Squeaker would request to create the "master" repository for > the Kernel to hack it. We need this discussion and we need the people > to take on responsibility for these areas. Ok, I misinterpreted that aspect of the proposal. Currently, most of the packages in the Basic image (as partitioned by iSqueak) don't have a master repository yet... only a few such as Monticello and SqueakMap do. But once they're set up, I agree that it makes sense to have the active (non-integration) development in the package's master repository, then. And preferably all or at least most of the packages would have a master repository with some package-specific development taking place. I can imagine there might be a few neglected packages at first which only have integration fixes going on, though... but yeah, if Joe Random squeaker appears to start doing work on the package, that may get things moving as far as determining maintainership, etc. :) >> So, a typical scenario would be that a package, say Network, goes >> through a number of new versions in the release repository during 3.9 >> development, from Network.3 to Network.17 or something. > [...] > > I wouldn't do it that way. I would start from the release plan saying > "we'd like to have some network support in the next release". Based on > this the release master goes to the package maintainer/owner and asks > "what is the latest stable release people ought to use for the next > Squeak release" - to which the package maintainer would answer > "probably Network-foo.bar.mcz" and the release master would copy this > version into the release directory. > > At this point people can start testing the package. If there are > issues, they can be addressed by the package owner (the preferred way) > so that the release master only needs to copy the fixed versions over > to the release repository. If the issues aren't addressed by the > package maintainer (or if they are integration issues that must be > addressed in the release) the fixes would be posted into the release > repository and (hopefully) be merged back into the package master > repository. (and if this doesn't work, then someone might well stand > up and fork off a new "master" repository for the package as it is in > the release right now). > > Notice that at any point the person in control for a release is the > release master, not the package owner/maintainer. It is the release > master who manages the packages included in a release and the package > owner/maintainer supports the release master by addressing any issues > that have come up in the master package repository. Ok, that pretty much follows from your clarification above. Sounds good. >> ... >> Are we ready to go ahead with this? :-) > > Depends on who "we" is ;-) The Packages team, Coordinators, various "SqF" members, people who have access to the update stream, and other organizations du jour. ;-) But seriously, the Packages team, at least. (which is really just whoever is following this list) Of course, this will have a big impact on Squeak 3.9 development, harvesting, etc. One other issue is working on recruiting/assigning maintainers & co-maintainers for all of the packages. Goran has done some work on this in the past. But I don't think we need to wait for that to proceed. In fact, I think it may be easier to find maintainers once the package infrastructure is in place and people see that it is working. - Doug

1 0

iSqueak
by Avi Bryant 19 Jun '05

19 Jun '05

Hi all, Sorry for the long silence on this list - I've been negligent in my team leadership duties by not keeping the discussion going (for example, I think Lex posted a status report that should have been forwarded to squeak-dev but was instead completely ignored - sorry about that). However, it seems that if you ignore a problem long enough, eventually someone else will solve it for you. I thought the recent traffic on the list about Tweak's approach to managing updates and packaging was very relevant: it sounds like Impara have come up with a system that nicely balances Monticello and the update stream, and that actually works. If you didn't see the original message from Andreas, here's the important bit: "In Tweak, *all* code is in packages[*]. Updates are exclusively used for in-image reshapes, e.g., when part of the system has undergone significant changes that require manual intervention. In order to synchronize these modifications with the package versions that we expect, we typically post a configuration map, e.g., a list of packages where we expect some specific version to be present. When you update, we always consult the update stream first. This will suck in any intermediate configurations and perform the necessary in-image modifications. Once this is done, we merely upgrade all packages in a well-defined order to their latest version." Moreover, they've come up with an initial packaging for Squeak 3.8, available at http://source.impara.de/iSqueak.html (see the wiki tab for info and downloads). I think this is great work, and I'd like to suggest that we adopt it as the basis for the partitioning/packaging efforts in this team. This would mean, probably, these steps: - We adopt the reorganization changeset from iSqueak as the first packaging changeset to go into the 3.9 stream - We include Monticello and Impara's MonticelloConfigurations as base packages in 3.9 - We set up an initial repository (probably based on SqueakSource) for the resulting Squeak packages, which anyone on this team can commit to, and start committing the results of any further partitioning work to it - We hack the repository as needed to support whatever kind of maintainership model we want for the Squeak packages. Tweak lives in a fairly closed world, where it's reasonable for everyone to update to the latest version everyone has posted, but Squeak (probably) can't do that. So instead of "upgrade all packages in a well-defined order to their latest version", we probably need "to their latest maintainer-approved version", and have a harvesting process that turns submitted versions to approved versions. We may find that the "blessing" mechanism I developed for the UnstableSqueak project is a good step in this direction, but we should talk about it more. Thoughts? Avi

5 8

Re: iSqueak
by stéphane ducasse 12 Jun '05

12 Jun '05

> > Actually, instead of having "approval flags" I'd recommend having > "release repositories". E.g., consider that we're setting up a 3.9 > repository which defines the "latest" package versions for > everything that ought to be in 3.9. > > Rather than "approving" a package version in their own repository, > a maintainer (or release master) would publish (or copy) package > versions to the release repository. This has a number of extra > advantages - fixes that turn out to be important for a release can > be put into this repository but may never appear in the "master" > repository for the package (because it might have been a last- > minute workaround). Integration fixes can be done for the release > and then fed back to the package owners. > > Basically, you're defining your own package environment, treating > the repository as a distribution and decide when you want to update > the distribution from the latest package versions developed by the > owner of the package. Don't think that this will be a bit difficult to manage after a while. By the I did not have the time to look at the isqueak image but I read the tweak faq and I have a question: do you only load package from a single source? Because I thought that it would be good to be able to load YAXO and MC from their home source to avoid duplicate code. >> The initial maintainership model could err on the side of openness, >> maybe even having some people with write access to all packages. >> Then >> we could restrict it a bit more later. >> > > With the above model you could restrict commit rights for the > release repository to the people actually working on the release, > whereas access to the "master" repository is limited to whatever > the package owner decides. This seems like a good model to me > because it leaves the responsibilities in the right places. My main question is if we have enough man power for that. Stef

2 1

Re: iSqueak
by Avi Bryant 11 Jun '05

11 Jun '05

On 6/12/05, Andreas Raab <andreas.raab(a)gmx.de> wrote: > > :) But I agree that MC's more rigid/uniform packaging scheme is the way > > to go for managing the base image. We may need a few little enhacements > > like that for MC, e.g. marking a version as maintainer-approved. > > Actually, instead of having "approval flags" I'd recommend having > "release repositories". E.g., consider that we're setting up a 3.9 > repository which defines the "latest" package versions for everything > that ought to be in 3.9. Yes, I like this model too. At one point in MC's development I was pushing for the idea of a "tag" file that would list a repository and a set of package names. Loading this tag file would simply load the highest numbered version of each of those packages from that repository. Of course these might be tags themselves, pointing to other packages/repositories. This seems somewhat in line with what MCConfiguration is doing already; maybe it could be extended in that direction? Avi

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Packages June 2005