- External reviews
A key problem with external reviews is that you have to know the area involved very well to judge the wider implications of changes. It is not surprising that typically a few people review certain areas since that's their primary area of expertise. One of the key problems in the current process is that these people need reviews themselves - even though they are experts in the area and an external review is likely to just say "sure it works - what did you expect"? Or even worse, there simply isn't anybody who feels confident to review that code (who exactly even _could_ review Ned's and my joint changes for adding/removing morphs?)
Exact! That's why I browse it and approved it. That's why this would be cool that you too pair a bit to have a look at some enh pending at the morphic level.
Here's an interesting observation about myself: I tend to _not_ look at code from people I consider "experts" unless I am suspecting something non-obvious or outright questionable. In all other cases my basic inclination is "go ahead - if there's a problem we can fix that too" rather than think ahead and try to foresee what might be affected (which doesn't work anyway).
Same here, even I read when I have time to learn and check if the code is not completely strange because we can always forget a method somewhere.
Personally, there are various people which I would immediately endorse to submit fixes and enhancements in various areas. In other words, I trust those people enough to be willing to live with their changes without having to explicitly review them. And yes, things will break but they'd break with or without a review by someone who doesn't really understand it.
I agree this is why we should have a fast approved track and a lot of alpha testers.
If so, what's the point?
Andreas I have the impression that having two persons per area at minimum would be cool. because it puts just more pressure on you this is like pair programming. I'm lazy while when I know that somebody else will have a look I pay attention.
- External testing
If you look at the traffic on Squeak-dev, I think you'll find many more responses that just say "thanks! that did it" than "[er, et]" or whatever tags. The problem is that feedback from people using a fix is mostly informal (if at all present). In a perfect world, I would want something that I can flag as a fix for some problem. When some person installs the fix the person should (later) get asked "did this solve your problem?" with a choice of "don't know yet", "yes", "no", "yes but ..." (as in: it fixes the problem but shows another one), and "no but ..." (as in: it doesn't but I have package Foo and Bar loaded which might interfere).
One way to solicit this feedback would be to install in particular [FIX]es in a way that the system _knows_ this is a fix and can ask the user about it after that fix has been in use for a while. Again, here's an example: Someone posts a fix for a problem. I load it and keep going. Some time later (perhaps when I am about to save/quit the image) the system reminds me that I have a bunch of fixes loaded and whether I'd want to comment on them. If I choose to, I get the above mentioned options and this feedback is then automatically sent to the list.
Yes this is a good idea. In particular the fact that the fixes may not have been exercised requires that the person choose to report. I think that this is a good way to involve concrete feedback.
The key point about [et] tags for fixes is that people are individually bitten by the bugs and that loosing feedback from a single person means loosing _all_ of the feedback here. You don't run a [fix] right away unless you're bitten by it.
Cheers,
- Andreas
-----Original Message----- From: squeakfoundation-bounces@lists.squeakfoundation.org [mailto:squeakfoundation-bounces@lists.squeakfoundation.org] On Behalf Of Daniel Vainsencher Sent: Thursday, October 16, 2003 5:57 PM To: squeakfoundation@lists.squeakfoundation.org Cc: ken@kencausey.com; Brent Vukmer; Cees de Groot; Bert Freudenberg Subject: [Squeakfoundation]The Harvesting process and the BFAV
Hi guys.
The BFAV has become the technical basis for harvesting progress. In this it has improved the rate of harvesting by removing much of the dirty work that was involved in the old processes. It has improved the efficiency of the approvals many fold.
It has also allowed us to elaborate the process by adding the quality tags, and for the first time having a more or less closed list of patches to be dealt with.
The purpose of the QA tags was to distribute the work of harvesting among more people, by allowing casual participants to help by reviewing or testing code without accepting the responsibility of the formal role of harvester.
As the efficiency of approvals went up, and we have an in-squeak list of posts and their full status, it has become much more visible that our bottleneck is not approvals. It is very clearly in the reviewing/testing stage. The reasons are very clear - rather few people help with the reviews/testing, and that quite sporadically.
This bottleneck is a result of our current structure, and won't go away by itself. It causes a situation where most patches meant for the squeak image are ignored. I think this is not likely to last - either we balance this by removing the bottleneck, or contributors of patches will balance it by posting less, an outcome I find regrettable.
In fact, I think it likely that people are already deciding not to post fixes they have available because they're likely to be ignored. Therefore, I think that if we can remove the bottleneck, and reach a state where patches submitted are all processed in reasonable time, we will get far more patches, and be able to make much better progress than we are now.
I think the following steps are in order -
- We want to lower the barriers to casual participation. So,
the BFAV, as a prerequisite for participating, should never be absent for anyone. It should be included in the base image. 2. We should improve and simplify the tools to help reviewers/testers do their job.
- ClassBoxes, when they're ready, could become a big aid to
testing, by allowing patches to be installed temporarily and then completely removed.
- The BFAV should be improved, especially for casual users. Brent, I
recommend to open its development so more people can help with this. The Swiki page is already pretty good, just point to it from the help in the BFAV. Also it might be good that its future development be carried on using Monticello, to make it easier to create and integrate improvements. 3. The current flood of BFAV traffic makes people automatically ignore the harvesting process. Therefore it MUST be selectively moved away from squeak-dev. Cees, or whoever has administrative rights on lists.sqf.org, can you please create a new mailing list called "squeak-harvest"? This fits Berts existing filters. Having this list would allow some selectivity without losing information. 4. When all is said and done, many people currently won't participate no matter how low the barriers are made, because of different priorities. Some of this is obviously reasonable, people need to pay the rent and play with whats fun for them, of course. But some of this is IMO not reasonable, and comes from a misperception. I think the NewLook code showcases this very clearly. The voices for its inclusion are very loud, but nobody bothers to review it, therefore, it isn't included. So I guess that the people doing the requesting somehow fail to perceive that the condition for inclusion is within their grasp. Of course, I could just be wrong about this. Ideas to solve this or alternative explanations would be welcome.
Daniel _______________________________________________ Squeakfoundation mailing list Squeakfoundation@lists.squeakfoundation.org http://lists.squeakfoundation.org/listinfo/squeakfoundation
Squeakfoundation mailing list Squeakfoundation@lists.squeakfoundation.org http://lists.squeakfoundation.org/listinfo/squeakfoundation