First, let me remind people that the general proposal I'm pushing is used extraordinarily widely. The object-list, review-committee approach is likely as old as human memory. It's at least as old as martial arts schools. Further, it's just about the only approach used where there is a non-trivial review at all. All other communities I can think of either base membership on money, or allow anyone to be a member. If you think you see a trivial dismissal, then for the sake of honest argument you should consider again more carefully.
Let's get into some detail, now. Daniel has the following main sticking points: how verification would work, the importance of public archived reviews, and the importance of negative comments. Let me comment on those, and then summarize where things seem to stand on reputation systems in general.
It is certainly true that the objective-list process requires verification. Any process does. I don't see why this is a problem. I, at least, will challenge people who post junk. Further, anyone who does not have any positive verifications is certainly not going to get a vote.
I would suggest that we have some sort of registration committee that is responsible for doing the final judgement calls. We could even use Squeak People ranks for that, since it is not important to get the registration committee exactly representative. They can decide among themselves how much to trust each possible reviewer's verifications.
Now, yes, it is possible for the registration folks to be dishonest. However, they must do everything publicly. If they are dishonest, then the vote is not widely respected and will fail. This situation is common to most any election system. At some point somebody has to count the votes, and if they insist on publicly doing it incorrectly, then the vote doesn't happen. They cannot, however rig the vote in their favor, if everything is done publicly and with carefully drawn rules. So long as the object list of requirements is indeed objective, there should be no wiggle room for registration tsars to affect the vote.
That's the verification issue. The other questions of archiving comments and of negative comments go hand in hand. Negative comments are important in the rare case that someone does try to hack themselves a vote. And archiving negative comments is critical in cases where there is something tricky about the application. For example, suppose someone links to some code that turns out to have mostly been copied from someone else. The dual measures of negative comments and archived application processes means that these situations can be solved, thus improving our general confidence that the official membership designations are accurate.
Now let's review reputation systems in general. There are important issues that need to be answered if we go with one to choose membership.
First, I think we all agree now that we can't just use the current Squeak People rankings, correct? As recently as Jan. 1, Daniel proposed that we do just that. Hopefully that sub-proposal is now dead?
That sub-proposal, however, was the simple option. If we don't follow that, then the question opens of how to fix things up. The most important question here is the choice of roots. To put it politely, it is a bad idea for a subgroup to declare themselves leaders and then also name themselves the as the source of membership for future elections. Can we please agree on this in the abstract and not dig into individual cases?
Leaving aside that option, who should we pick as roots? Answering that question seems just as hard as holding an election, bringing us back to the beginning.
Second, reviews matter, but our current system doesn't give us very good ones. Squeak People reviews are generally "oh, I know that guy, I think he's a so and so". Hopefully there would at least be a lot of these, but Daniel has pointed out that in many cases there are only 2-3 that determine a person's membership. The objective-list approach would mean that there is an objective set of criteria that are used in deciding who is a member. At least 1 person would carefully review each new applicant against that list. Further, the list would be public and could be talked about, instead of every person promoting whoever they feel like.
Third, why do we have reason to expect the current reputation system (or any rep system) to generate accurate membership desginations? It is now widely agreed that many of the people who should get votes are not currently given votes by Squeak People. Why, exactly, should we think that it will work in the future? The objective-list approach is old and provides a lot of positive experience for us. What can give us similar confidence in the reputation-system approach?
Finally, if indeed the system can be modified to answer he above questions (e.g., requiring reviews to be attached to certs), then is the result truly simple any longer? The main appeal of a reputation system for most people appears to be that it is simple. However, it's only simple if we do it poorly. Doing it well appears just at least as complicated, and seems to require just as much raw effort, as a traditional objective-list approach.
Overall, I like reputation systems and hope to see more experiments with them. This is not the time for an experiment, though. Bylaws should be boring.
Lex Spoon