[Elections] reputation systems for membership

[Daniel Vainsencher]

Somewhat long post, but there was a lot of stuff that required responding...

Lex Spoon wrote:
> SqP is a bad source of membership as it stands, if you take the time to
> go look at who would be included and who not.  It's just not capturing
> the right people right now.
I've looked at half the people rated Journeyer and up, and didn't find 
one to whom I would deny the vote. So I think with that filter, it would 
do fine. If you disagree enough you want to discuss names, lets do 
*that* off list :-)

> There are other issues to consider, in general, when trying to use an
> automatic reputation system to handle membership.  Some issues to keep
> in mind:
Note that I was talking about voting rights, not about membership to 
anything in particular. We could make voting rights depend on some 
membership (in SqF?), but don't have to. Extending invitations to people 
whose reputation exceeds some threshold sounds reasonable.
I will consider both (disconnected voting rights vs member voters)

> 	1. What happens when someone's reputation *decreases* past the
> threshold for membership?
Disconnected voting rights can just go away. I don't think membership 
should change. If the organization has some laws for removing members, 
his SqP status may play a role, or not.

> 	2. There is no record of rejections.  So new people can try and try
> again until they find someone to give them a reputation bump.
Certainly the same problem applies to other systems - someone can just 
say "I now posted some code (straight out of /dev/random, he he), 
consider me again".

> 	3. For that matter, there seems to be no consideration of people's
> judgement skills in the current SqP algorithm.  If someone has a history
> of including and rejecting the wrong folks, then their gatekeeping
> abilities ought to decline.
To my eyes this is only a reason to make voting rights disconnected. If 
someone screws up, talk to his certifiers, and the flow algorithm will 
undo all his errors.

> 	4. Garbage in, garbage out.  Are people really going to put the time in
> to enter and maintain reputation designations?  If they don't (and come
> on, they won't), then is the reputation algorithm going to produce
> anything that's useful?
I think that people have practically zero reason to maintain the ratings 
at the moment, seeing as they are not used for anything. Nevertheless, 
the higher ratings seem to be reasonably populated to me. If ratings 
mattered for votes, I'd certainly make a round of certifying everyone I 
believe in before an vote.

> 	5. What about retaliation?  Avoiding #4 probably requires posting lots
> of reputation judgements.  This is a recipe for in-group tension.
I'm not sure what you mean by retaliation, since the worst thing you can 
do to someone is not rate them. I'm not at all sure I addressed this, 
but other systems also have to deal with sour lemons, and its always 
ugly. I saw someone get kicked out of ESUG, and it wasn't pretty, but I 
can't say any membership system would have improved it.

> Finally, I wonder about the reputation system's algorithm.  There is a
> big literature on this topic nowadays, but we appear to have rolled our
> own
Well, adopted advogato's :-)

> without making reference to why ours is better than all the existing
> ones.  We have great people in the group, but it is rather arrogant to
> dismiss the world of CS research without at least explianing why we are
> doing something different.
Call it a first approximation. If anyone ever floats a better algorithm, 
we can put it to a vote :-)

> Overall, I'm in favor of using some list like the one I posted earlier,
> and then working out a different system over time that is more
> traditional.  I would happily revise this if the above concerns are
> addressed 
Let me know if I have, or you see other problems.

> or if someone can point to a group or three that has tried
> using automatic reputation for membership and has found it effective.
Hmm, I don't know of examples yet. Except in the sci fi novels 
"distraction" and "snow crash". But hey, we're squeakers, we're allowed 
to innovate ;-)

> To respond to 2 comments mentioned earlier:
> First, Squeak People being mysterious is not just my idea, but Cees'. 
I haven't seen this comment. Am I missing mails?

> He has explicitly posted that he does not mean for the
> apprentice/journeyman/master designations to designate coding skill or
> level of contribution, and he ha snot said what it *does* designated nor
> why he thinks it shows it to us.  It is as mysterious as it gets.  We
> don't know what it says, and we don't know why we think it says what it
> says.
Like I said, I trust the current meaning (with the Journeyman limit) 
after checking a non-random sample. I agree, though, that we might be 
well advised to document its meaning somewhere as determining voting 
rights in squeak-dev, if we choose to use it so.

Daniel