[Squeakfoundation]The Harvesting process and the BFAV

[ducasse]

> * External reviews
>
> A key problem with external reviews is that you have to know the area
> involved very well to judge the wider implications of changes. It is 
> not
> surprising that typically a few people review certain areas since 
> that's
> their primary area of expertise. One of the key problems in the current
> process is that these people need reviews themselves - even though 
> they are
> experts in the area and an external review is likely to just say "sure 
> it
> works - what did you expect"? Or even worse, there simply isn't 
> anybody who
> feels confident to review that code (who exactly even _could_ review 
> Ned's
> and my joint changes for adding/removing morphs?)

Exact! That's why I browse it and approved it.
That's why this would be cool that you too pair a bit to have a look at 
some
enh pending at the morphic level.

> Here's an interesting observation about myself: I tend to _not_ look 
> at code
> from people I consider "experts" unless I am suspecting something
> non-obvious or outright questionable. In all other cases my basic
> inclination is "go ahead - if there's a problem we can fix that too" 
> rather
> than think ahead and try to foresee what might be affected (which 
> doesn't
> work anyway).

Same here, even I read when I have time to learn and check if the code 
is not
completely strange because we can always forget a method somewhere.

> Personally, there are various people which I would immediately endorse 
> to
> submit fixes and enhancements in various areas. In other words, I trust
> those people enough to be willing to live with their changes without 
> having
> to explicitly review them. And yes, things will break but they'd break 
> with
> or without a review by someone who doesn't really understand it.

I agree this is why we should have a fast approved track and a lot of 
alpha testers.

> If so, what's the point?

Andreas I have the impression that having two persons per area at 
minimum would be cool.
because it puts just more pressure on you this is like pair 
programming. I'm lazy while when
I know that somebody else will have a look I pay attention.

>
> * External testing
>
> If you look at the traffic on Squeak-dev, I think you'll find many more
> responses that just say "thanks! that did it" than "[er, et]" or 
> whatever
> tags. The problem is that feedback from people using a fix is mostly
> informal (if at all present). In a perfect world, I would want 
> something
> that I can flag as a fix for some problem. When some person installs 
> the fix
> the person should (later) get asked "did this solve your problem?" 
> with a
> choice of "don't know yet", "yes", "no", "yes but ..." (as in: it 
> fixes the
> problem but shows another one), and "no but ..." (as in: it doesn't 
> but I
> have package Foo and Bar loaded which might interfere).
>
> One way to solicit this feedback would be to install in particular 
> [FIX]es
> in a way that the system _knows_ this is a fix and can ask the user 
> about it
> after that fix has been in use for a while. Again, here's an example:
> Someone posts a fix for a problem. I load it and keep going. Some time 
> later
> (perhaps when I am about to save/quit the image) the system reminds me 
> that
> I have a bunch of fixes loaded and whether I'd want to comment on 
> them. If I
> choose to, I get the above mentioned options and this feedback is then
> automatically sent to the list.

Yes this is a good idea. In particular the fact that the fixes may not 
have  been exercised
requires that the person choose to report. I think that this is a good 
way to involve concrete
feedback.


> The key point about [et] tags for fixes is that people are individually
> bitten by the bugs and that loosing feedback from a single person means
> loosing _all_ of the feedback here. You don't run a [fix] right away 
> unless
> you're bitten by it.
>
> Cheers,
>   - Andreas
>
>> -----Original Message-----
>> From: squeakfoundation-bounces at lists.squeakfoundation.org
>> [mailto:squeakfoundation-bounces at lists.squeakfoundation.org]
>> On Behalf Of Daniel Vainsencher
>> Sent: Thursday, October 16, 2003 5:57 PM
>> To: squeakfoundation at lists.squeakfoundation.org
>> Cc: ken at kencausey.com; Brent Vukmer; Cees de Groot; Bert Freudenberg
>> Subject: [Squeakfoundation]The Harvesting process and the BFAV
>>
>>
>> Hi guys.
>>
>> The BFAV has become the technical basis for harvesting
>> progress. In this
>> it has improved the rate of harvesting by removing much of the dirty
>> work that was involved in the old processes. It has improved the
>> efficiency of the approvals many fold.
>>
>> It has also allowed us to elaborate the process by adding the quality
>> tags, and for the first time having a more or less closed list of
>> patches to be dealt with.
>>
>> The purpose of the QA tags was to distribute the work of harvesting
>> among more people, by allowing casual participants to help by
>> reviewing
>> or testing code without accepting the responsibility of the
>> formal role
>> of harvester.
>>
>> As the efficiency of approvals went up, and we have an
>> in-squeak list of
>> posts and their full status, it has become much more visible that our
>> bottleneck is not approvals. It is very clearly in the
>> reviewing/testing
>> stage. The reasons are very clear - rather few people help with the
>> reviews/testing, and that quite sporadically.
>>
>> This bottleneck is a result of our current structure, and
>> won't go away
>> by itself. It causes a situation where most patches meant for
>> the squeak
>> image are ignored. I think this is not likely to last - either we
>> balance this by removing the bottleneck, or contributors of
>> patches will
>> balance it by posting less, an outcome I find regrettable.
>>
>> In fact, I think it likely that people are already deciding
>> not to post
>> fixes they have available because they're likely to be ignored.
>> Therefore, I think that if we can remove the bottleneck, and reach a
>> state where patches submitted are all processed in reasonable time, we
>> will get far more patches, and be able to make much better
>> progress than
>> we are now.
>>
>> I think the following steps are in order -
>> 1. We want to lower the barriers to casual participation. So,
>> the BFAV,
>> as a prerequisite for participating, should never be absent
>> for anyone.
>> It should be included in the base image.
>> 2. We should improve and simplify the tools to help
>> reviewers/testers do
>> their job.
>> - ClassBoxes, when they're ready, could become a big aid to
>> testing, by
>> allowing patches to be installed temporarily and then completely
>> removed.
>> - The BFAV should be improved, especially for casual users. Brent, I
>> recommend to open its development so more people can help
>> with this. The
>> Swiki page is already pretty good, just point to it from the
>> help in the
>> BFAV. Also it might be good that its future development be carried on
>> using Monticello, to make it easier to create and integrate
>> improvements.
>> 3. The current flood of BFAV traffic makes people automatically ignore
>> the harvesting process. Therefore it MUST be selectively
>> moved away from
>> squeak-dev. Cees, or whoever has administrative rights on
>> lists.sqf.org,
>> can you please create a new mailing list called "squeak-harvest"? This
>> fits Berts existing filters. Having this list would allow some
>> selectivity without losing information.
>> 4. When all is said and done, many people currently won't
>> participate no
>> matter how low the barriers are made, because of different priorities.
>> Some of this is obviously reasonable, people need to pay the rent and
>> play with whats fun for them, of course. But some of this is IMO not
>> reasonable, and comes from a misperception. I think the NewLook code
>> showcases this very clearly. The voices for its inclusion are
>> very loud,
>> but nobody bothers to review it, therefore, it isn't included. So I
>> guess that the people doing the requesting somehow fail to
>> perceive that
>> the condition for inclusion is within their grasp. Of course, I could
>> just be wrong about this. Ideas to solve this or alternative
>> explanations would be welcome.
>>
>> Daniel
>> _______________________________________________
>> Squeakfoundation mailing list
>> Squeakfoundation at lists.squeakfoundation.org
>> http://lists.squeakfoundation.org/listinfo/squeakfoundation
>>
>
> _______________________________________________
> Squeakfoundation mailing list
> Squeakfoundation at lists.squeakfoundation.org
> http://lists.squeakfoundation.org/listinfo/squeakfoundation
>