[Squeakfoundation]Harvesting infrastructure (was Re: Order of business ...)

[Doug Way]

Ned Konz wrote:
> 
> On Saturday 16 November 2002 11:00 pm, Doug Way wrote:
> 
> > I haven't thought too much about how the back-end infrastructure
> > might work.  It could be based on SqueakMap or something else.
> >
> > But we definitely want it to be automated so that
> > bugfix/enhancement/refactoring submissions are compiled somewhere
> > with no manual effort on our part.
> 
> I've been considering using Avi's new Repository stuff to grab stuff
> off the Squeak list (like sqfixes does), tag it with proper comments
> (like the message it was attached to), and then provide it to a
> separate UI. From there, it could be fixed by the community, or
> commented on (the UI could store approvals as comments).

Not a bad idea.  I'd eventually like to have a tool to submit fixes/enhancements directly (rather than grabbing them from the list), but perhaps that part could be done later, once everything else is working.

I haven't tried the Repository stuff yet... is it working well enough now that it could be used for this?  If you wanted to put something together on a test server, that might be interesting.

> There's no security structure yet in the Repository stuff, though.
> However, since the Repository keeps all versions, this isn't likely
> to be a problem.

True.

With tweaking, it sounds like either the Repository stuff or SqueakMap might be a usable framework for dealing with harvesting.  The Repository stuff is missing some security stuff which SqueakMap has, but SqueakMap doesn't include actual file storage.  There are probably other trade-offs.  (I haven't really used either one yet enough.)

Perhaps I'll bring up this discussion on squeak-dev.  (Although the people who know the most about SqueakMap & DVS/Repository stuff are already following this list.)

> > One way to do this would be to simply require that people use a
> > tool (either a tool within Squeak, or a web form like with
> > SqueakMap uploads) to submit these submissions.  There would be
> > additional benefits in requiring that certain file formats and
> > information be included with submissions via such a tool (whereas
> > the current submissions are kind of a free-for-all, sometimes the
> > formatting is bad, etc.).
> >
> > I don't think this would be too harsh a requirement for the
> > community. An automatically-generated email could still be sent to
> > the squeak-dev list, so people would still see fixes/etc coming in.
> 
> Or we could have a separate squeak-harvesting (squeak-fixes?) list to
> handle related traffic.

Good idea.

> > Then we would want some sort of upload space to exist, maybe just
> > ftp space on squeakfoundation.org or something, where the
> > submissions would be stored.
> 
> Or a remote Repository.
> 
> > Having both the submission tool and the harvesting/browsing tool
> > within Squeak would provide opportunities for cool functionality...
> > e.g. my ConflictChecker utility could be run on a submission by a
> > simple menu-click.
> >
> > Or there might be other better ways to do this.  I'm just throwing
> > around ideas right now, and it's getting a bit late. :)
> 
> Again, we need some kind of modeling of packages. Right now there are
> two that I have in my image:
> * SqueakMap's card
> * PackageInfo (previously a part of DVS, now separate).
> 
> I see these as being complementary; the SqueakMap deals with releases
> and finding them, and the PackageInfo deals with code.

We do need this in general, although I'm not sure if it relates to harvesting.  I would assume most fix/enhancement submissions would be changesets, not packages.

Or are you saying that a submission should have a way to refer to a package?  That would be good.  Eventually, people will need to be able to submit a fix to a specific package, and the harvesting may become more package-based.  (Although some fixes/enhancements may require changes to more than one package.)

- Doug