Hi Takashi,
This weekend, we ran a usability test on your tutorial. The way I setup the environment is I asked 3 girls (2 of my daughters, both age 15, and their friend, same age) to do, separately: - Start each of the 3 projects of the tutorial (I did the start/stop) - Be in a room with a computer by herself, with no help - Call me in if she gets stuck, BUT only after 2-3 minutes of trying. I helped only the minimum necessary in such case.
All was repeated with each. All 3 finished all 3 tutorials, some help was needed, Jacqueline only needed help once, to "repair the script".
I realize the ages are probably older then the target age. Also my girls did use eToys quite a bit before (up to about 2 years ago, when they used it last .. I noticed they did a _very_ good job forgetting it :) ). Their friend never used eToys; but all used computers to a fairly large degree.
I basically decided to note (and report here) things on which they get stuck too long or surprised by. The report below is sort of a summary of the 3 of them. First, let me say that you would be happy to see (and hear) their comments: - Wow, this is pretty cool - Ah, I can use it - What i cute demon :) - etc
Please do not take the length of this "report" as a negative sign - the girls did love the tutorial, and all got to the end with some help, Jacqueline almost did it all by herself!
So the report: ===========
A. Generic Findings: ================ - Kids are impatient :) they cannot wait 2 minutes without asking for help, but get better over time.
- When they get lost, they make things worse by just clicking around, often trying things they learned in previous lessons.
- The "Try Again" button was valuable, but it would be nice to add a note (bold) "You can always click on the "Try Again" button". Also is it possible for the "Try Again" to completely restore the state, including deleting copies, restoring destroyed scripts etc? For example, sometimes they ended up with many copies of the key (as they decided to try the "duplicate"), and things became confusing. Also, Joelle deleted the scriptor, and that was the end of her lesson until I restarted it. So restoring the page state completely on "Try Again" would be great.
- We had a few MNU, but they did figure to just click the "x" on the left, and kept playing. This is funny, for a developer it is a mojor event. For a child, as long as they know what to do, just get rid of it and keep going...
- Not sure how hard this is - is it possible to "lock up" the instructions, in the sense kids cannot write to it, a few times they got confused and started dragging out images from the instructions
B1. Handles/Hallo Tutorial: =============
- Clicking on the black parts of the "key morph" does not act on in. I realize this is etoys behaviour, but it confused one of them, brought confusion about how the halo is brought up, and I had to explain. I think for the initial tutorial, the key morph should have a black background, so it is sensitive to click anywhere on it or close to it.
- Apart from that, pretty smooth sailing!
B2. Paint Tool Tutorial: =================
- I think the "auto-painter" confused more then helped, overall. I would probably remove it, caused much confusion for everyone. (Also, once painted, the painted morph stays around on all following pages it seems?).
- On the following few "Paint my face" instruction pages: In the instructions, perhaps instead of pasting the icons, better would be to start with hallo up, and put the "Paint Box" close, or inside, of the instructions?. The girls thought that they should click on the icons inside the instructions, instead on the PaintBox all the way right. Eventually they ploughed through it though :)
B3. Scripting Tutorial (hardest, understandably, but excellent): =====================================
- a movie would be ideal (Broken scriptor: hardest to get right)
- The "Try Again" comment from A. Generic Findings: applies.
- The girl who did not use eToys, was confused and kept asking "what is scriptor"?, "what is a tile"? - We should define and describe "scriptor", "tile", "exclamation", and "clock", in more detail, perhaps a more text and arrows would help. It is hard to describe though...
- The arrow "Try the exclamation button" should point to "!" (points to scriptor)
- Page "Is the scriptor broken" page: Perhaps could add more description like: "Once the scriptor is repaired, click the clock as in the previous page". This is where the girl who did not use eToys started to get stuck.
- Page "Let me explain a bit about a scriptor here.": - What do you think about moving this page up front, maybe simplifying it (as noted 3 items above)
5. At the end: ==========
This was funny, where the demon says "I am sorry I captured you. But I just want a human friend. I am a lonely demon.", the girls all noted "why does he ask about a human friend, it should be a cat friend?" I never noticed it at all...
Beautiful stuff - please let me know if we can help. We noticed a few typos and formulation potential changes - if you'd like, we can summarize them.
Thanks Milan, Jacqueline, Joelle and Najiba
On 2006 October 20 11:08, Takashi Yamamiya wrote:
Hi Andreas, Milan,
I have updated handles tutorial. http://tinlizzie.org/olpc/tutorial/DemonCastle1.005.pr
Milan Zimmermann wrote:
Takashi,
I remember one thing I got stuck on (almost) completely, was when I started resizing the "BIG KEY". When you try to make it big and small again in a few directions without letting go (and it goes LARGE/Small/OK"), it becomes almost impossible to hit the OK size. Maybe it should be more generous with the range of OK (it seems to use a relative measure perhaps). I will have more notes over the weekend,
Aha! everybody who tried it complained this issue, but I just forgot... I made it easier, thank you!
On 2006 October 19 23:58, Andreas Raab wrote:
http://tinlizzie.org/olpc/tutorial/DemonCastle1.004.pr (handles)
- On the first page where it says "Click THIS arrow" I'd move the text
closer to the arrow. I was kinda looking for "another" arrow because the one you mean is quite far away from the text.
Oops, you are right. That was bad.
- On the third page (with the first key) I would change the text from
"Go ahead you can move the key" to something like "Go ahead, drag the key on the keyhole" (to emphasize that you mean click and drag)
Make sense,
- On the "resize handle" page it was very hard to find quite the right
size for the key. It seems as if there is only a small range in which it will fit and it was quite tricky to get that "just right". I wonder if there is an easy way to increase that range?
I am going to fix other two projects later. Thank you so much!
Takashi