[squeakland] ranking system suggestions (from kathleen)

[Timothy Falconer]

Kathleen,

Thanks for your questions and suggestions!  Comments below...

On Sep 24, 2009, at 10:11 AM, <kharness at illinois.edu> wrote:

> Good Morning All,
> I have been trying to wade through all of these ballots and I have  
> several comments and questions.
> 1. Tim's note below said 20 projects but I have 46 and some are  
> duplicates so, can the flow be adjusted to eliminate the duplicates?

This is likely a bug.   I've logged it as http://tracker.squeakland.org/browse/SQ-436

After this initial "big bang", you'll receive ballots one at a time,  
so the "flow" will be lessened (though we still need to prevent  
duplicates).

> 2. The voting page in a project is not clear, those little up/down  
> arrows lead to a page that says page redirect not available.

This is also a bug ... http://tracker.squeakland.org/browse/SQ-432.

For now, click more slowly on the arrows.

> 3. I thought region meant the region where the author lives and it  
> does not seem like something we should vote on or change. And why  
> would I 'approve' with a number the Subject category someone has  
> already selected for their project?

See my comment to Cherry on the squeakland mailing list.  Ultimately  
region ranking may be discarded, but I put it in to be consistent.     
Sometimes people come up with new uses for things that we can't  
anticipate.  At any rate, it governs how things are sorted within a  
region, which may yet be useful when we have sub-regions, like "Brasil".

Approve means "allow people to see the project on the website".   It's  
a filter for inappropriate content, not a voting mechanism.  This will  
be made clearer.

> 4. How will this ranking work over time, the people who have been  
> 'approved' will 'approve' others will similar interests and styles  
> and it will skew the results so that for example, war games creators  
> give higher marks to other war game creators who in turn have more  
> influence. What checks and balances are there? Please explain the  
> algorithm sooner rather than later to all at one time.

Rankers are chosen randomly, with a bias towards people with higher  
"mojo", which is an indication of their contributions to the showcase  
(both projects and taking the time to rank)

> 5. When I was looking at Bert's Farbkunsts and clicked on the up  
> arrow by Region, it took me to Karl's Snake project launch page. Why  
> did this happen and what does it mean? I really do need help to  
> understand how to use the tools. Please explain in detail, as I am  
> unfamiliar with all of this.

You found another of my late night bugs ... http://tracker.squeakland.org/browse/SQ-431

I'll be making a screencast to explain the ranking system soon.  For  
now, this is a family test :)

> 6. What are the criteria for the points awarded? How high many can I  
> award? 10? 100? I am not sure this is useful, nor how the points are  
> used and used by whom?  If the points just accumulated by the  
> 'approve' vote wouldn't that be enough. Will the points show beside  
> the project name or picture or flash inside the project so people  
> know they are looking at a very POPULAR project or one that only a  
> few liked?  How many people need to approve a project? How many need  
> to un-approve and does this mean my vote would cancel that of  
> someone who had already approved. Maybe showing the number of times  
> a project is accessed by visitors could help people choose a project  
> to look at. Even this though denigrates ideas that are worthy, but  
> just not popular. Projects that model bio-medical-engineering topics  
> will have their audience as will the ones that are about the Braatz  
> and how will the rankings account for the range of ages and interests?

Ten rankers are chosen for each new project, based upon previous  
effort, so it's not based on popularity.   Each criteria can be given  
points ranging from 10 to -10.   No evidence of a projects rank is  
shown on the website, except its relative position on "view by rank"  
and within the individual categories and tags.   Authors are never  
even told of their project ranks, though there is an indicator of an  
accounts "relative mojo", though this isn't tied to projects  
entirely ... mojo is also given to rankers and commenters for simply  
contributing.

> 7. I have qualms about popularity contests see #4 above. I have also  
> noticed that when we start ranking winners with children that there  
> are a few who are winners, first place, second place, third place,  
> that's three people and now what about the work of the other  
> thousand children who didn't win. Do we teach them to label  
> themselves Not-a-Winner? I am not a winner, say that enough times to  
> yourself and feel it inside.

I remember your point of view from the Waveplace Awards, and factored  
your thoughts into the ranking system.  This is one of the chief  
reasons that accounts cannot see their own project ranks.  It's a  
filter, not a contest.

> 8. Some of the screens did not let me Approve but only offered the  
> Un-Approve option. So does this mean someone else has already  
> approved this one and no one else can, or does it mean I can  
> unapprove/cancel another person's vote?

It's a toggle button.  If you see "Un-Approve", then someone has  
already approved it for view on the website.  I'll make this clearer.

> 9. The comments so far lead me to think that the comment-ors believe  
> they are offering constructive criticisms that the project designer  
> will then use to revise and 'improve' to meet the suggestions. If I  
> upload a project to the site, it will be one I have finished and it  
> says what I meant it to say. Comments will not be useful. This is  
> not a teaching moment. Besides which, there are giants among us  
> whose skills are far beyond mine and it would be presumptuous to  
> comment on projects I can only marvel at and wish like fury that I  
> knew enough to do something so wonderful.

The commenting system on Scratch is used very effectively for offering  
tips on how to make projects better.  The education team did suggest a  
"don't give me feedback" checkbox, but we didn't have time to put it  
into Etoys. Maybe next time.

> 10. If I click the up/down arrows by Points, I don't see a change  
> just a message Could not find redirect page.

This is the bug I mentioned up in #2.

> 11. Where will the Tags show and is there a search box so all these  
> fields can be accessed?

You can click "view by tags" to see tags, or you can click a tag on a  
project page to see other projects with that tag.

> Is this just a trial run, or is this how it is going out to the  
> general public? This whole process is very complicated and I want to  
> participate but I really do not know what to do. What if nobody  
> approves my projects does that mean I am not part of this anymore?

My apologies, I should have made the screencast before turning the  
system on.   If no one clicks "Approve" on your project, than it won't  
show on the showcase.   At least one human must say, "This is good to  
show."

That was a decision by the ed team, to screen all content before going  
public with it.  We can revisit that decision if it's deemed too  
cumbersome.

Our thinking was that it's more important to prevent inappropriate  
content than to get a quicker response.

Users can still see their projects when logged in, and soon, "groups"  
and "friends" will be able to see un-approved projects as well.    
"Approve" really means "Make Public".

Thank you very much for your detailed comments and willingness to help.

Take care,
Tim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://luna.immuexa.com/pipermail/squeakland/attachments/20090924/8da4f649/attachment.html