[Long, and late. Sorry.]
From: [...] Lex Spoon It is clear to everyone, I think, that the current set of SqP-certified people does not correspond to the set who should get to vote.
The set is incomplete. So are the sets on that web-site or wiki over there --> that we're using to run the votes - in fact, they're even less complete, as that list isn't started yet. So SqP is further ahead than any other solution that's been proposed, because it has *some* content.
So we can't "just use SqP".
SqP as it stands at what date, though? We will need to tell our constituency about any voting system, and we will need to give them time to register. I would hope that would generate a spate of registrations, no matter what system we use.
Before telling you my reservations, please remember that the object-list approach is about as KISS and as reliable as imaginable.
Indeed.
Assuming everyone plays nice
Always a dubious assumption in a community of any size.
, then within a few weeks -- a
month tops -- there would be a list of members that could be frozen as the initial set. This would be a list that everyone should feel good about, because the rules of joining are objective and because the membership applications are all handled publicly.
(Having handled online elections on a MUD for many years) How does one deal with the community of punks who decide to object to everything? Who gets to review them and decide that they should be excluded from the list of folks who may object? We ended up moving *away* from an object-list approach for voting to pure positive voting, as the noise in the system was dealt with more effectively by that scheme. We also ended up shunning them as a a larger community, something that an automatic reputation system will tend to note without further intervention.
A big open question is who are the roots?
Indeed. Cees quoted some names to me (not sure whether on-list, I can't find the email). They looked reasonable, and - key point - were not all folks who are themselves up for re-election.
What do the pro-repsystem people want to choose, here?
I'd like a set of careful, gregarious, active people who don't regularly appear on the mailing list with axes to grind. There may be a trade-off there. Tim and Cees would be two of my top choices. Possibly Daniel. As a counter-example, I'm too far on the fringes and too often on one side of an argument to make a good root.
Second, have people really thought about what it would be like to live under such a system? Membership would be granted and denied based on a dozen or more off-the-cuff reviews instead of 1-3 careful ones. That's a recipe for superficial decisions -- it will promote populists over quiet doers.
Not necessarily. The algorithm used here merely determines whether the reputation flows to an individual, not how strongly. Also, don't forget the laws of large numbers. With a large community of reviewers rather than a small community, the peers and admirers of such quiet doers would typically ensure that they were included.
Also, reviews would not be archived. People would not have a place to post their *reason* for their certifications.
I'd distinguish 'would not' from 'can not at present'. If such a system were to be moved into something that could be hacked about more easily, then it would be almost trivial to include a text box alongside the rating drop down, allowing the rater to state why they were rating as they were.
I would in this case want an explicit 'deliberate no rating' value in the drop down, however, so that a person could elect to record why they were making no decision about a person.
And they cannot post negative reviews at all, right now.
Mmm. I'd love to see this, but it's difficult to see how this can be done in a graph-based reputation system. In particular, how do you defend against the punks who rate everyone negatively xcept their peer group? What positive rating does someone require before their negative feedback is noted? And in what order does one evaluate positives and negatives?
(Even if they could, who would put their neck out to do it?)
Me, for one. I'm honest like that.
On top of these, don't people worry that a reputation system includes people who pester a lot of members but excludes shier folks who do not want to do that?
No. See above re law of large numbers. I worry much more that a system with a small number of reviewers will overlook people.
Issues aside, there is the question of practical experience. Aside from communities where people pay to join, or where membership is wide open to all comers, practically every community I know of uses the defense-before-committee approach. It is not perfect, but it seems to work well, and it is something that people understand. In particular, it handily addresses all of the issues in the previous paragraph.
However, it's a lot of work for the committee, and it requires an impartial committee. To turn round the 'who are the roots' question: Who would you choose to form that committee, Lex?
Is there any experience we can rely on for using reputation systems to define membership? Maybe some of the worries from the previous paragraph do not emerge in practice?
I don't have enough experience myself to answer that.
- Peter
elections@lists.squeakfoundation.org