proposal for starting work on partitioning

Doug Way dway at mailcan.com
Mon Apr 4 18:24:35 UTC 2005


On Mon, 4 Apr 2005 11:25:09 +0200 , goran.krampe at bluefish.se said:
> Hi Avi!
> 
> Avi Bryant <avi.bryant at gmail.com> wrote:
> > Hi all,
> > 
> > Sorry for being so quiet; I've been extremely busy (in fact, if it
> > continues like this, I should probably step down as leader here, since
> > it's silly to have a leader who takes a week to answer emails...).
> 
> We will see. :)

Yes, we will only let Avi step down if someone else eagerly volunteers.
;)

> > If that looks like it will work better, great.  We haven't had such a
> > rush to sign up for pieces that I feel that list is set in stone;
> > Doug, if you have a more granular set of packages in mind, please feel
> > free to just change it.

Ok, will do.  Basically I am just going to consolidate "Network",
"Tools" and "System" into three big packages.  Then the maintainers of
those packages can work on splitting them up inside later.  I see a few
people have already signed up for sub-packages of those, I will list
them as comaintainers of the big packages, with a small note specifying
which sub-package(s) they're interested in.

> ...
> > So let me beg here instead: who is actually interested in doing
> > partitioning work?  Anyone?  Is this just something that we care about
> > more in the abstract than in practice, and nobody really wants it
> > enough to do it?  Are there other ways we should be thinking about
> > this that would get more traction? (like, "how many bits of code can
> > we package up and unload from the image?")
> 
> This is an interesting question. I am interested in doing partitioning
> work - but for me I think it is a priority thing - I would like to do
> the *stake out* first, and "ripping out" later. I explain below.

I generally agree with Avi that I think we can do both at the same time.
 I think the "stake out" (assignment of maintainership) might not be
that easy to finish, so we may want to move forward with *some*
partitioning, which will give us some time to get our process and
package-related tools working.  Then, as people see that the process is
actually moving forward, there may be more interest from people in
taking on maintainership.

Perhaps we should at least have a rule, though, that a particular
package should be staked out by some maintainers before it is
partitioned. (and certainly before it is removed)

Anyway, regarding maintainership, I have three words:  Redundancy,
redundancy, redundancy. :-)  I think it's critical to have multiple
maintainers for each Basic package.  It's inevitable that people will
have varying amounts of time to maintain a package, so we need a
co-maintainer(s) to step in when someone else drops out of sight for a
while.  I think we should have a rule that each Basic package should
have at least three maintainers, including one lead maintainer.  (Well,
maybe bend the rules a bit for a few lesser-used packages such as Speech
if we have to, but I think it's a good rule.)

This is another advantage of consolidating the packages into bigger
ones, btw... it's easier to find multiple maintainers and have this
redundancy of maintainership.

> > > And I say (yet again) don't get held up in refactoring. Let's just start
> > > pumping out 2-3 PIs so that we get this show on the road.
> > 
> > I don't really know what "pumping out PIs" means, or at least what
> > purpose it's supposed to achieve.  It seems to me that something like
> > "registering packages on SqueakMap" is much more meaningful, no? 
> > Which tools (apart from the PackageList that nobody uses for anything,
> > and I think we agreed to drop) actually know or care with PackageInfo
> > instances exist inside the image?
> 
> Ok, let me explain (I know Avi knows this stuff, but perhaps there are
> others on this list).
> 
> SM has a PI field for each package. Using that (as most of us already
> know) we can map an SM package to an in-image PI. Today there are just a
> very few of those mappings - for example for SM itself. What is the
> mapping for? It is the link between the code (classes and extension
> methods) and the SM entry. Given that this link exists we suddenly
> *know* who maintains that code (email) and any co-maintainers of it.
> 
> For example, given the PI-extras package in which Ned has included my
> PI-split-and-email-to-SM-maintainers-mechanism, a user can:
> 
> 1. Fix a bug or add an enh.
> 2. Bring up a changesorter on that cs.
> 3. With a single menu choice split it up into one cs for each affected
> PI and send those (plus the original complete cs) as an email to *all*
> maintainers of those SM packages affected.
> 
> Now... this mechanism is of course a bit crude, but imagine that *all*
> code in the image belong to a PI - then suddenly using the above
> mechanism (or similar) we could actually decentralize the complete
> FIX/ENH process. And we could use MC instead of changesets etc. Suddenly
> we have a structure to work in.
> 
> So in short - I want this to happen:
> 
> 1. Create rough PIs in Basic (using some updates in 3.9a) that cover the
> *complete* image. In whichever granularity. We can always adapt them
> later.
> 
> 2. Create corresponding SM entries for them, typically this is done by
> the maintainers stepping forward. If some are left as orphans, lets
> create some SM packages anyway and let a team of maintainers for the
> orphans sign up on that package. The important thing is that each line
> of code in the image can be mapped to 1-n maintainers.
> 
> 3. Let the maintainers start working. This entails:
> 	1. Adding/removing PIs (using updates) in order to further
> split/combine code.
> 	2. Reorganize, typically classes and methods will have ended up in the
> wrong PI.
> 	3. Refactor, this is modifying code to decouple stuff with the
> intention of making code loadable/unloadable.

This all sounds exactly right... I think we are on the same page here.

A related issue is how soon after PI-partitioning should the package
actually be removed from the base image and have its master version
stored on SqueakMap.  Basically, I don't think we should move *too* fast
with that.  But I'll address that in a separate email.

> > > > Basically, I think there is more danger that the effort will stall if a
> > > > private stream is used, because then there will be a big merging effort
> > > > required at the end.  In fact, if everyone can see progress being made
> > > > in the public stream, that might encourage more people to contribute.
> > > 
> > > Yes, I agree in full. Avi?
> > 
> > Fine with me.
> 
> Good!

Ah, good.  And I think it's not a big deal if we make a few mistakes
along the way and fix them as we're doing this... it is the alpha
release after all.

> > > I would like to hear Avi's thought on how we handle the PIs, as classes
> > > or instances?
> > > There was a discussion around that earlier, and possibly classes is the
> > > way to go since our main tools (ChangeSets, MC) are centered around
> > > that. But I trust Avi to make that decision.
> > 
> > I don't see a concrete place yet where introducing PI subclasses will
> > help with the packaging work, especially given the init script stuff
> > that Miso, Ned and I have been kicking around.  So let's stick with
> > instances for now.
> 
> Ok, instances it is.

Very good.  (I was hoping we were going with instances, actually.)

- Doug



More information about the Packages mailing list