Hi all,
Let's talk more directly about the reasons for ranking . . . early today I wrote:
It also gives us more colored dots.
And Yoshiki wrote:
And the explanation of why this is a good thing is not described yet. Kathleen wasn't happy with it and I think your response wasn't answering her concern.
I'll try to be clearer.
As I see things, there are three related issues that are getting combined in the same conversation:
1. showing individual rankings (which is what I think she's most concerned about ... google "Punished by Rewards")
2. showing levels of ongoing community involvement (aka, "top ten contributors")
3. sorting things so the good stuff stays on top and the lesser stuff falls to the bottom
The colored dots next to people's usernames are really about #2. You can get points by voting, by commenting, by submitting projects, etc. It is *not* merely a measure of your project vote points. It's not an average either. One person who submits ten projects that gets 5 points each will earn 50 points, which is the same as another person who submits two projects that get 25 points each. There's no way to tell which person got 25 points apiece and which got 5 points apiece. In a normal grading situation, the 25 pointer would clearly "win out" over the 5 pointer. The Squeakland showcase measures *sustained effort*, not specific performance.
Displaying the colored dots is a motivational thing. When someone sees a comment by "yoshiki with the purple circle", they can see you're an Etoys enthusiast with a history. It's not "Yoshiki who get's straight A's", it's "Yoshiki who's put in a lot of effort over time and has earned that purple circle." Seeing the purple circle is a reward to you for effort, and a signal to others that your opinion might matter more than "puppetAccount31252" with a white circle. The colored circles are a measure of credibility.
Which leads to the most important reason for them .... people with higher levels get asked to rank other projects. They get asked because they've earned the right to have a say in what's valuable.
Some important things to keep in mind:
* no one ever sees your votes
* no one ever knows how their project was ranked, only that it places higher or lower than another project
* there are no displays of "raw points" anywhere ... only relative levels. Today the top points is 362. In a year, it might be 23,252. No one ever sees this.
As for the more general question of why do we even want to rank projects (#3), the answer is quite plain ....
because when there's 250,000 projects on the server, it'll be a big mess that's useful to no one unless there's some ordering that's generally useful.
The Squeakland ranking system is an attempt to sort with a measure of actual worth, not mere popularity. The way to gauge this worth should be tied directly to sustained merit.
Tim
At Tue, 29 Sep 2009 20:24:39 -0400, Timothy Falconer wrote:
Hi all,
Let's talk more directly about the reasons for ranking . . . early today I wrote:
It also gives us more colored dots.
And Yoshiki wrote:
And the explanation of why this is a good thing is not described yet. Kathleen wasn't happy with it and I think your response wasn't answering her concern.
I'll try to be clearer.
There were more in the discussion...
As I see things, there are three related issues that are getting combined in the same conversation:
- showing individual rankings (which is what I think she's most
concerned about ... google "Punished by Rewards")
- showing levels of ongoing community involvement (aka, "top ten
contributors")
- sorting things so the good stuff stays on top and the lesser stuff
falls to the bottom
The colored dots next to people's usernames are really about #2. You can get points by voting, by commenting, by submitting projects, etc. It is *not* merely a measure of your project vote points. It's not an average either. One person who submits ten projects that gets 5 points each will earn 50 points, which is the same as another person who submits two projects that get 25 points each. There's no way to tell which person got 25 points apiece and which got 5 points apiece. In a normal grading situation, the 25 pointer would clearly "win out" over the 5 pointer. The Squeakland showcase measures *sustained effort*, not specific performance.
Displaying the colored dots is a motivational thing. When someone sees a comment by "yoshiki with the purple circle", they can see you're an Etoys enthusiast with a history. It's not "Yoshiki who get's straight A's", it's "Yoshiki who's put in a lot of effort over time and has earned that purple circle." Seeing the purple circle is a reward to you for effort, and a signal to others that your opinion might matter more than "puppetAccount31252" with a white circle. The colored circles are a measure of credibility.
But one of the concerns was that one can lose the colored dot visibly to everybody even when you are doing as much as you have been. When others are saying that "that is de-motivating", you just repeat yourself to say "it is a motivational thing". As Kathleen repeated, coercing people onto a linear scale is not the way to motivate different kind of learners.
The external "reward" for some effort in learning settings should be other people's honest and insightful feedback and getting more ideas (and internally "learn" something), but not a dot.
Your #2 above is not separated from #1, and the 5-6 levels of dots have way more meaning than "top ten contributors". #3 doesn't mean you need to have a visible dots for individual persons, but merely projects should be sorted.
Which leads to the most important reason for them .... people with higher levels get asked to rank other projects. They get asked because they've earned the right to have a say in what's valuable.
But that doesn't mean that person's dot should be visible to everybody.
Some important things to keep in mind:
- no one ever sees your votes
- no one ever knows how their project was ranked, only that it places
higher or lower than another project
- there are no displays of "raw points" anywhere ... only relative
levels. Today the top points is 362. In a year, it might be 23,252. No one ever sees this.
Well, I know that.
As for the more general question of why do we even want to rank projects (#3), the answer is quite plain ....
because when there's 250,000 projects on the server, it'll be a big mess that's useful to no one unless there's some ordering that's generally useful.
The Squeakland ranking system is an attempt to sort with a measure of actual worth, not mere popularity. The way to gauge this worth should be tied directly to sustained merit.
This statement is not an answer to why people should be visibly ranked, and they should lose the rank easily.
-- Yoshiki
On Sep 29, 2009, at 9:08 PM, Yoshiki Ohshima wrote:
At Tue, 29 Sep 2009 20:24:39 -0400, Timothy Falconer wrote:
Hi all,
Let's talk more directly about the reasons for ranking . . . early today I wrote:
It also gives us more colored dots.
And Yoshiki wrote:
And the explanation of why this is a good thing is not described yet. Kathleen wasn't happy with it and I think your response wasn't answering her concern.
I'll try to be clearer.
There were more in the discussion...
As I see things, there are three related issues that are getting combined in the same conversation:
- showing individual rankings (which is what I think she's most
concerned about ... google "Punished by Rewards")
- showing levels of ongoing community involvement (aka, "top ten
contributors")
- sorting things so the good stuff stays on top and the lesser stuff
falls to the bottom
The colored dots next to people's usernames are really about #2. You can get points by voting, by commenting, by submitting projects, etc. It is *not* merely a measure of your project vote points. It's not an average either. One person who submits ten projects that gets 5 points each will earn 50 points, which is the same as another person who submits two projects that get 25 points each. There's no way to tell which person got 25 points apiece and which got 5 points apiece. In a normal grading situation, the 25 pointer would clearly "win out" over the 5 pointer. The Squeakland showcase measures *sustained effort*, not specific performance.
Displaying the colored dots is a motivational thing. When someone sees a comment by "yoshiki with the purple circle", they can see you're an Etoys enthusiast with a history. It's not "Yoshiki who get's straight A's", it's "Yoshiki who's put in a lot of effort over time and has earned that purple circle." Seeing the purple circle is a reward to you for effort, and a signal to others that your opinion might matter more than "puppetAccount31252" with a white circle. The colored circles are a measure of credibility.
But one of the concerns was that one can lose the colored dot visibly to everybody even when you are doing as much as you have been. When others are saying that "that is de-motivating", you just repeat yourself to say "it is a motivational thing". As Kathleen repeated, coercing people onto a linear scale is not the way to motivate different kind of learners.
I think the piece that's missing here is that it will be pretty rare to actually change from one level to another, especially for younger children.
The way it is now will not be the way it is with 1000 users. Color changes won't be a day-to-day goal. My guess it might happen once or twice a year for an individual. Again, I doubt many young children will even make it to level 3.
The external "reward" for some effort in learning settings should be other people's honest and insightful feedback and getting more ideas (and internally "learn" something), but not a dot.
Your #2 above is not separated from #1, and the 5-6 levels of dots have way more meaning than "top ten contributors". #3 doesn't mean you need to have a visible dots for individual persons, but merely projects should be sorted.
This is the central issue .... #1 is completely different than #2. When someone can get just as many points by adding comments as getting "grades", it's entirely about #2.
It's a measure of effort, not excellence. High ranks merely get someone more points more quickly, and rightly so.
Which leads to the most important reason for them .... people with higher levels get asked to rank other projects. They get asked because they've earned the right to have a say in what's valuable.
But that doesn't mean that person's dot should be visible to everybody.
Yes, we could take the colored circles off the website. I think that people are finding them fun, at least the people I'm talking to, and I think there's genuine value in knowing that a commenter is a regular and not a puppet account.
I talked about the whole "Punished by Rewards" thing with a few dozen teachers a year ago when Kathleen first brought up her objections to the Waveplace Awards. She honestly was the only that I talked to that voiced any concern over that event. Quite the opposite, most saw it as a transforming moment for their kids. From what we could tell, the kids who didn't place took it in stride and were proud of their friends. Watch the end of the video to see me address this topic directly, particularly why we thought the awards were a good thing.
I agree that competition goes too far in this society. I agree that there are different kinds of learners that thrive with different types of motivation.
But I humbly disagree with yours and Kathleen's opinion that we should take the colored dots off the website. To me, the benefits outweigh the potential downside. They're useful and fun, with a real purpose.
Tim
At Tue, 29 Sep 2009 22:45:08 -0400, Timothy Falconer wrote:
I think the piece that's missing here is that it will be pretty rare to actually change from one level to another, especially for younger children.
Is this based on the logic described at http://tracker.squeakland.org/browse/SQ-361? It would be a bit surprizing to hear because "maxMojo" change alone affects everybody's rank. Since the owner of maxMojo probably will earn more points quicker than others, and the ranks are linear scale of the point, I would expect to see bigger disparity over time.
Yes, we could take the colored circles off the website. I think that people are finding them fun, at least the people I'm talking to, and I think there's genuine value in knowing that a commenter is a regular and not a puppet account.
Well, people should actually look at the contents to judge the quality of comments. I've been writing fairly stupid comments. They should be treated so.
I talked about the whole "Punished by Rewards" thing with a few dozen teachers a year ago when Kathleen first brought up her objections to the Waveplace Awards. She honestly was the only that I talked to that voiced any concern over that event. Quite the opposite, most saw it as a transforming moment for their kids. From what we could tell, the kids who didn't place took it in stride and were proud of their friends. Watch the end of the video to see me address this topic directly, particularly why we thought the awards were a good thing.
Of course, the question occured to me that how they knew how it is going to work a year ago...
I agree that competition goes too far in this society. I agree that there are different kinds of learners that thrive with different types of motivation.
But I humbly disagree with yours and Kathleen's opinion that we should take the colored dots off the website. To me, the benefits outweigh the potential downside. They're useful and fun, with a real purpose.
Okay.
-- Yoshiki
At Tue, 29 Sep 2009 20:08:46 -0700, Yoshiki Ohshima wrote:
I agree that competition goes too far in this society. I agree that there are different kinds of learners that thrive with different types of motivation.
But I humbly disagree with yours and Kathleen's opinion that we should take the colored dots off the website. To me, the benefits outweigh the potential downside. They're useful and fun, with a real purpose.
Okay.
Oh, but I forgot to mention that one of my suggestion wasn't remove it, but have probably two levels thing and no going back in usual cases. It can even be visible on the site if the rule is clear.
So, it is not "removing the dots or not" question but "how many levels" "how it is calculated", and etc.
-- Yoshiki
On Sep 29, 2009, at 11:08 PM, Yoshiki Ohshima wrote:
At Tue, 29 Sep 2009 22:45:08 -0400, Timothy Falconer wrote:
I think the piece that's missing here is that it will be pretty rare to actually change from one level to another, especially for younger children.
Is this based on the logic described at http://tracker.squeakland.org/browse/SQ-361? It would be a bit surprizing to hear because "maxMojo" change alone affects everybody's rank. Since the owner of maxMojo probably will earn more points quicker than others, and the ranks are linear scale of the point, I would expect to see bigger disparity over time.
Some of the details have changed since that list, but yes, the mojoLevel formula is current.
Over time, some people will continue to contribute and others will stop. The formula rewards sustained effort, not past performance.
Again, this is not a grade ... it's a thank you.
Yes, we could take the colored circles off the website. I think that people are finding them fun, at least the people I'm talking to, and I think there's genuine value in knowing that a commenter is a regular and not a puppet account.
Well, people should actually look at the contents to judge the quality of comments. I've been writing fairly stupid comments. They should be treated so.
Originally we were going to rank comments, but recently I decided that would be just too much for people to have to do.
Comment points are a recognition of effort ... not a grade, but a thank you.
When comment scribbling gets turned on (see the tracker), we'll have a way to prevent bogus comments from gaming the system.
At Wed, 30 Sep 2009 08:44:47 -0400, Timothy Falconer wrote:
On Sep 29, 2009, at 11:08 PM, Yoshiki Ohshima wrote:
At Tue, 29 Sep 2009 22:45:08 -0400, Timothy Falconer wrote:
I think the piece that's missing here is that it will be pretty rare to actually change from one level to another, especially for younger children.
Is this based on the logic described at http://tracker.squeakland.org/browse/SQ-361? It would be a bit surprizing to hear because "maxMojo" change alone affects everybody's rank. Since the owner of maxMojo probably will earn more points quicker than others, and the ranks are linear scale of the point, I would expect to see bigger disparity over time.
Some of the details have changed since that list, but yes, the mojoLevel formula is current.
Ok, so the single person's behavior alone can change the whole system, yes?
Over time, some people will continue to contribute and others will stop. The formula rewards sustained effort, not past performance.
Hmm, are you saying that the top person will eventually stop so it is not going to be a problem?
Again, this is not a grade ... it's a thank you.
You usually don't take "thank you" away after giving it, right?
Yes, we could take the colored circles off the website. I think that people are finding them fun, at least the people I'm talking to, and I think there's genuine value in knowing that a commenter is a regular and not a puppet account.
Well, people should actually look at the contents to judge the quality of comments. I've been writing fairly stupid comments. They should be treated so.
Originally we were going to rank comments, but recently I decided that would be just too much for people to have to do.
Comment points are a recognition of effort ... not a grade, but a thank you.
When comment scribbling gets turned on (see the tracker), we'll have a way to prevent bogus comments from gaming the system.
... I wasn't talking about comment points. I wasn't talking about gaming the system either... What you said above was that people should feel good when somebody with higher level comments on their projects. And I said that people should read the actual comment and feel good when it is good.
-- Yoshiki
squeakland@lists.squeakfoundation.org