[Squeakfoundation]The Harvesting process and the BFAV

[Andreas Raab]

Hi Daniel,

> Ideas to solve this or alternative explanations would be welcome.

Let me throw in a few thoughts on this issue. Speaking only for myself I
find the following issues interesting/relevant in this context:

* External reviews

A key problem with external reviews is that you have to know the area
involved very well to judge the wider implications of changes. It is not
surprising that typically a few people review certain areas since that's
their primary area of expertise. One of the key problems in the current
process is that these people need reviews themselves - even though they are
experts in the area and an external review is likely to just say "sure it
works - what did you expect"? Or even worse, there simply isn't anybody who
feels confident to review that code (who exactly even _could_ review Ned's
and my joint changes for adding/removing morphs?)

Here's an interesting observation about myself: I tend to _not_ look at code
from people I consider "experts" unless I am suspecting something
non-obvious or outright questionable. In all other cases my basic
inclination is "go ahead - if there's a problem we can fix that too" rather
than think ahead and try to foresee what might be affected (which doesn't
work anyway).

Personally, there are various people which I would immediately endorse to
submit fixes and enhancements in various areas. In other words, I trust
those people enough to be willing to live with their changes without having
to explicitly review them. And yes, things will break but they'd break with
or without a review by someone who doesn't really understand it. If so,
what's the point?

* External testing

If you look at the traffic on Squeak-dev, I think you'll find many more
responses that just say "thanks! that did it" than "[er, et]" or whatever
tags. The problem is that feedback from people using a fix is mostly
informal (if at all present). In a perfect world, I would want something
that I can flag as a fix for some problem. When some person installs the fix
the person should (later) get asked "did this solve your problem?" with a
choice of "don't know yet", "yes", "no", "yes but ..." (as in: it fixes the
problem but shows another one), and "no but ..." (as in: it doesn't but I
have package Foo and Bar loaded which might interfere).

One way to solicit this feedback would be to install in particular [FIX]es
in a way that the system _knows_ this is a fix and can ask the user about it
after that fix has been in use for a while. Again, here's an example:
Someone posts a fix for a problem. I load it and keep going. Some time later
(perhaps when I am about to save/quit the image) the system reminds me that
I have a bunch of fixes loaded and whether I'd want to comment on them. If I
choose to, I get the above mentioned options and this feedback is then
automatically sent to the list.

The key point about [et] tags for fixes is that people are individually
bitten by the bugs and that loosing feedback from a single person means
loosing _all_ of the feedback here. You don't run a [fix] right away unless
you're bitten by it.

Cheers,
  - Andreas

> -----Original Message-----
> From: squeakfoundation-bounces at lists.squeakfoundation.org 
> [mailto:squeakfoundation-bounces at lists.squeakfoundation.org] 
> On Behalf Of Daniel Vainsencher
> Sent: Thursday, October 16, 2003 5:57 PM
> To: squeakfoundation at lists.squeakfoundation.org
> Cc: ken at kencausey.com; Brent Vukmer; Cees de Groot; Bert Freudenberg
> Subject: [Squeakfoundation]The Harvesting process and the BFAV
> 
> 
> Hi guys.
> 
> The BFAV has become the technical basis for harvesting 
> progress. In this
> it has improved the rate of harvesting by removing much of the dirty
> work that was involved in the old processes. It has improved the
> efficiency of the approvals many fold.
> 
> It has also allowed us to elaborate the process by adding the quality
> tags, and for the first time having a more or less closed list of
> patches to be dealt with.
> 
> The purpose of the QA tags was to distribute the work of harvesting
> among more people, by allowing casual participants to help by 
> reviewing
> or testing code without accepting the responsibility of the 
> formal role
> of harvester. 
> 
> As the efficiency of approvals went up, and we have an 
> in-squeak list of
> posts and their full status, it has become much more visible that our
> bottleneck is not approvals. It is very clearly in the 
> reviewing/testing
> stage. The reasons are very clear - rather few people help with the
> reviews/testing, and that quite sporadically. 
> 
> This bottleneck is a result of our current structure, and 
> won't go away
> by itself. It causes a situation where most patches meant for 
> the squeak
> image are ignored. I think this is not likely to last - either we
> balance this by removing the bottleneck, or contributors of 
> patches will
> balance it by posting less, an outcome I find regrettable.
> 
> In fact, I think it likely that people are already deciding 
> not to post
> fixes they have available because they're likely to be ignored.
> Therefore, I think that if we can remove the bottleneck, and reach a
> state where patches submitted are all processed in reasonable time, we
> will get far more patches, and be able to make much better 
> progress than
> we are now. 
> 
> I think the following steps are in order -
> 1. We want to lower the barriers to casual participation. So, 
> the BFAV,
> as a prerequisite for participating, should never be absent 
> for anyone.
> It should be included in the base image.
> 2. We should improve and simplify the tools to help 
> reviewers/testers do
> their job. 
> - ClassBoxes, when they're ready, could become a big aid to 
> testing, by
> allowing patches to be installed temporarily and then completely
> removed.
> - The BFAV should be improved, especially for casual users. Brent, I
> recommend to open its development so more people can help 
> with this. The
> Swiki page is already pretty good, just point to it from the 
> help in the
> BFAV. Also it might be good that its future development be carried on
> using Monticello, to make it easier to create and integrate
> improvements. 
> 3. The current flood of BFAV traffic makes people automatically ignore
> the harvesting process. Therefore it MUST be selectively 
> moved away from
> squeak-dev. Cees, or whoever has administrative rights on 
> lists.sqf.org,
> can you please create a new mailing list called "squeak-harvest"? This
> fits Berts existing filters. Having this list would allow some
> selectivity without losing information.
> 4. When all is said and done, many people currently won't 
> participate no
> matter how low the barriers are made, because of different priorities.
> Some of this is obviously reasonable, people need to pay the rent and
> play with whats fun for them, of course. But some of this is IMO not
> reasonable, and comes from a misperception. I think the NewLook code
> showcases this very clearly. The voices for its inclusion are 
> very loud,
> but nobody bothers to review it, therefore, it isn't included. So I
> guess that the people doing the requesting somehow fail to 
> perceive that
> the condition for inclusion is within their grasp. Of course, I could
> just be wrong about this. Ideas to solve this or alternative
> explanations would be welcome.
> 
> Daniel
> _______________________________________________
> Squeakfoundation mailing list
> Squeakfoundation at lists.squeakfoundation.org
> http://lists.squeakfoundation.org/listinfo/squeakfoundation
>