Before we start the point by point phase, something positive:
Lex Spoon wrote:
Issues aside, there is the question of practical experience. Aside from communities where people pay to join, or where membership is wide open to all comers, practically every community I know of uses the defense-before-committee approach. It is not perfect, but it seems to work well, and it is something that people understand. In particular, it handily addresses all of the issues in the previous paragraph. Is there any experience we can rely on for using reputation systems to define membership? Maybe some of the worries from the previous paragraph do not emerge in practice?
I honestly don't know. I just believe that, aside from evidence to the contrary, we ought to keep it simple and use a tried and true solution.
I don't know, I think SqP can save us lots of ugly work by making trust transitive. Its already implemented. It seems to have built in safeguard limiting the amount of damage a single person can do. And I sort of like the fact that its new, heck, we might learn something from it.
It is clear to everyone, I think, that the current set of SqP-certified people does not correspond to the set who should get to vote.
Agreed.
So we can't "just use SqP".
This does not follow - creating new accounts on SqP is almost as easy as adding your name to a wiki page. Do you already have a wiki page with all the voter names?
Converting SqP accounts to voters is much easier than converting names-on-a-wiki into voters - SqP already has the interfaces for certification, query for voter list and so forth. A wiki starts with no structure. If you add it up front, that's work to be done. If you don't, you'll get a mess, and work harder later.
Many people seem interested in somehow improving the system so that it would generate a reasonable list of members.
There is currently no evidence that any code improvements are necessary at all. It is quite possible that the only thing it lacks is use, and when people know that it matters, they'll certify.
Lex, all of the above, I've said about 3 times on this list now. Are you not seeing this?
Before telling you my reservations, please remember that the object-list approach is about as KISS and as reliable as imaginable.
Lets see...
People would sign up on a wiki page if they think they should get a vote,
Simple so far.
and there would be a time period where people can challenge entries on the page.
So by default, everyone gets in, because if I don't know someone, I'm certainly not going to call him a lier, right?
Assuming everyone plays nice, then within a few weeks -- a month tops -- there would be a list of members that could be frozen as the initial set.
Whoa! when did that happen? what usually happens on a wiki page is that any initial structure is used, overloaded and then starts crumbling. In this case, I expect a list of people that put themselves up and are ignored because nobody knows them. Some of them put up excuses that someone chooses to challenge by asking a couple of questions. Half of those questions are answered. Which of those entries are in? who decides that?
This would be a list that everyone should feel good about, because the rules of joining are objective and because the membership applications are all handled publicly.
So far we have a wiki page with persons names, and a list of rules about their accomplishments. How are these matched? do people list their accomplishments? do they provide a link so people can verify it? do they just write the number of the rule that they believe they should be accepted under? I'm not sure what you mean by someone "feeling good" about it. I get the feeling that membership is so open as to be meaningless. Anyone can write themselves in, verification is usually impossible, therefore won't be done. The applications happen publically, but I'm not at all sure in what sense they are handled - seems like they are simply collected.
That said, there are a number of problems with using an automatic reputation system for something as fundamental as membership. People should not go for such a system unless they are comfortable with all of these issues.
A big open question is who are the roots? I asked later but no one has responded. The only objective group I could think of would be to let Alan Kay choose (or, almost equivalently, to make him the only root).
In what sense you call that objective?
Assuming he does not suddenly jump into the fray with the open source community, what else is there?
First - what already exists. True, this is an arbitrary set of roots, but until we find out that someone that has an account, and is a serious user of Squeak can't get certified because the roots are insufficient, there is no problem.
That said...
The question has to be answered, and it needs to be answered in a way that does not leave out entire chunks of the community. What do the pro-repsystem people want to choose, here?
I think that the criterion for adding roots should be "heads of communities" (AK is the prime example, of course). I think this criterion should be judged by votes of the existing voter body.
Second, have people really thought about what it would be like to live under such a system? Membership would be granted and denied based on a dozen or more off-the-cuff reviews instead of 1-3 careful ones.
I don't know where you are taking these assumptions. Numerically, very few certifications seem to be sufficient: only KenC and I have certified Peter, and he got the certification we gave him. As to the quality of the reviews, why exactly should I review more carefully on a wiki page than on SqP?
That's a recipe for superficial decisions -- it will promote populists over quiet doers. Also, reviews would not be archived. People would not have a place to post their *reason* for their certifications.
Of course they do - SqP provides you with some web space, and from a person I can go to the pages of his certifiers. The "reviews" would be archived whenever someone bothers to write them (probably rarely). Note that explicit reviews are really not that important an idea in a web of trust. The transitivity takes care of it. If you trust me enough to read my reviews, just certify me, and the juice will pass on to those I certify.
And they cannot post negative reviews at all, right now.
They can. No negative certifications, that's all. But why do we need those?
(Even if they could, who would put their neck out to do it?)
If I know some new comer is a cheat, why shouldn't I mail his certifiers and mention the problem? assuming I have trust from people that are not dependent on his certifiers, what's to stop me?
On top of these, don't people worry that a reputation system includes people who pester a lot of members but excludes shier folks who do not want to do that?
If I get pestered, I might certify someone who I'd otherwise ignore, but you think I would certify them high? and can you please explain in comparison what would happen in a wiki based system? sounds to me I can pester people to vouch for me in any medium.
Daniel