Somewhat long post, but there was a lot of stuff that required responding...
Lex Spoon wrote:
SqP is a bad source of membership as it stands, if you take the time to go look at who would be included and who not. It's just not capturing the right people right now.
I've looked at half the people rated Journeyer and up, and didn't find one to whom I would deny the vote. So I think with that filter, it would do fine. If you disagree enough you want to discuss names, lets do *that* off list :-)
There are other issues to consider, in general, when trying to use an automatic reputation system to handle membership. Some issues to keep in mind:
Note that I was talking about voting rights, not about membership to anything in particular. We could make voting rights depend on some membership (in SqF?), but don't have to. Extending invitations to people whose reputation exceeds some threshold sounds reasonable. I will consider both (disconnected voting rights vs member voters)
- What happens when someone's reputation *decreases* past the
threshold for membership?
Disconnected voting rights can just go away. I don't think membership should change. If the organization has some laws for removing members, his SqP status may play a role, or not.
- There is no record of rejections. So new people can try and try
again until they find someone to give them a reputation bump.
Certainly the same problem applies to other systems - someone can just say "I now posted some code (straight out of /dev/random, he he), consider me again".
- For that matter, there seems to be no consideration of people's
judgement skills in the current SqP algorithm. If someone has a history of including and rejecting the wrong folks, then their gatekeeping abilities ought to decline.
To my eyes this is only a reason to make voting rights disconnected. If someone screws up, talk to his certifiers, and the flow algorithm will undo all his errors.
- Garbage in, garbage out. Are people really going to put the time in
to enter and maintain reputation designations? If they don't (and come on, they won't), then is the reputation algorithm going to produce anything that's useful?
I think that people have practically zero reason to maintain the ratings at the moment, seeing as they are not used for anything. Nevertheless, the higher ratings seem to be reasonably populated to me. If ratings mattered for votes, I'd certainly make a round of certifying everyone I believe in before an vote.
- What about retaliation? Avoiding #4 probably requires posting lots
of reputation judgements. This is a recipe for in-group tension.
I'm not sure what you mean by retaliation, since the worst thing you can do to someone is not rate them. I'm not at all sure I addressed this, but other systems also have to deal with sour lemons, and its always ugly. I saw someone get kicked out of ESUG, and it wasn't pretty, but I can't say any membership system would have improved it.
Finally, I wonder about the reputation system's algorithm. There is a big literature on this topic nowadays, but we appear to have rolled our own
Well, adopted advogato's :-)
without making reference to why ours is better than all the existing ones. We have great people in the group, but it is rather arrogant to dismiss the world of CS research without at least explianing why we are doing something different.
Call it a first approximation. If anyone ever floats a better algorithm, we can put it to a vote :-)
Overall, I'm in favor of using some list like the one I posted earlier, and then working out a different system over time that is more traditional. I would happily revise this if the above concerns are addressed
Let me know if I have, or you see other problems.
or if someone can point to a group or three that has tried using automatic reputation for membership and has found it effective.
Hmm, I don't know of examples yet. Except in the sci fi novels "distraction" and "snow crash". But hey, we're squeakers, we're allowed to innovate ;-)
To respond to 2 comments mentioned earlier: First, Squeak People being mysterious is not just my idea, but Cees'.
I haven't seen this comment. Am I missing mails?
He has explicitly posted that he does not mean for the apprentice/journeyman/master designations to designate coding skill or level of contribution, and he ha snot said what it *does* designated nor why he thinks it shows it to us. It is as mysterious as it gets. We don't know what it says, and we don't know why we think it says what it says.
Like I said, I trust the current meaning (with the Journeyman limit) after checking a non-random sample. I agree, though, that we might be well advised to document its meaning somewhere as determining voting rights in squeak-dev, if we choose to use it so.
Daniel