How will we handle multi-touch in Etoys?
1) Multiple Mice:
When you touch the screen with multiple fingers you get "many mice" So, how can I get the mouseX and mouseY for each "mouse"? Thinking about this for my version of Slice it. So I can imagine one way would be a collection of mice, one could iterate through.
2) No Mouse Fat Fingers problem:
A while back I had a conversation with an expert (some 17 or 18 year old kid who had built a Theremin remote control for his TV) and the idea came to me, I asked him about the power consumption, costs and if there were any interference issues and from the discussion it would seem this may be doable within XO-3 constraints. It would operate similar to the Bamboo Fun tablet which detects the stylus when it hovers over the tablet and you can move the mouse without touching the tablet, then it clicks when you touch the tablet.
One issues I can foresee would be when you "hover" your finger over the screen if the mouse pointer is right under your finger, it would block the view of the pointer, so you would have to position the pointer "above" the finger. But then when you touch you are not "clicking" where you touch.
Of course part of my problem my be I am thinking like a mouse ;)
Stephen
On 24.06.2011, at 10:01, Steve Thomas wrote:
How will we handle multi-touch in Etoys?
Yay! Someone else is thinking about this :)
- Multiple Mice:
When you touch the screen with multiple fingers you get "many mice"
"Multiple hands", we call it in Squeak. I re-purposed the existing multi-user support to support more than one finger for a single user. Now the terminology is not quite right anymore, but it was TSTTCPW.
So, how can I get the mouseX and mouseY for each "mouse"? Thinking about this for my version of Slice it.
I think the best use for multi-touch is to bring up the halo. Single-finger touch would be like clicking the primary mouse button today. When tapping with two fingers, the halo would activate. Not only that, but if you then rotate the fingers or pinch/spread them, it would turn and resize the object. I guess the halo should only be invoked if the two fingers tap at the same time, and close to each other. Otherwise you would just get two separate tap events to manipulate multiple objects (like I showed in my youtube video).
Now, we might add an option to disable that halo behavior inside a playfield. Enabling it would give you all the touch events.
So I can imagine one way would be a collection of mice, one could iterate through.
Iterating isn't well-supported in Etoys. How exactly would you imagine this might work?
A very simple interface would be having a "number of fingers" read out, and "finger 1 x", "finger 1 y", "finger 2 x", "finger 2 y" accessors. Or, maybe have a "second finger down" script trigger. I admit I have not thought much about this yet.
- No Mouse Fat Fingers problem:
A while back I had a conversation with an expert (some 17 or 18 year old kid who had built a Theremin remote control for his TV) and the idea came to me, I asked him about the power consumption, costs and if there were any interference issues and from the discussion it would seem this may be doable within XO-3 constraints. It would operate similar to the Bamboo Fun tablet which detects the stylus when it hovers over the tablet and you can move the mouse without touching the tablet, then it clicks when you touch the tablet.
One issues I can foresee would be when you "hover" your finger over the screen if the mouse pointer is right under your finger, it would block the view of the pointer, so you would have to position the pointer "above" the finger. But then when you touch you are not "clicking" where you touch.
Of course part of my problem my be I am thinking like a mouse ;)
Very few touch screen technologies support "hovering", so I'd rather not rely on that. And, no real touch interface I know even has a visible mouse pointer.
The cheapest touch screens aren't even multi-touch, but offer only a single touch point. It may be a good idea to design for that.
Btw, if you have an iPad, send me your UDID so I can compile a version of Etoys that you can try. Or, if you have jailbroken yours, just install the ipa from
http://etoys.squeak.org/experimental/
Also, not sure if anyone tried my Etoys-To-Go that mimics a touch interface using a huge mouse pointer:
http://etoys.squeak.org/experimental/Etoys-Touch-Emu-100620.zip
- Bert -
On 24/06/11 09:01, Steve Thomas wrote:
How will we handle multi-touch in Etoys?
FYI: there's "touch", then 2-point fake-ish "multi-touch", then true "multi-point" multi-touch. Resistive touch screens are (typical) single touch although 2-point touch can be faked (a useful technique is looking for jump from left/right to middle corresponding to pressing one finger then another on opposite side of the screen). A surprising number of capacitive screens are fake-ish 2-point and shows their limitations in test sw that displays the actual touch points, which jump about under certain conditions. True (imo) multi-touch can cope with more than two fingers, number varies, I have one device with 10-point multi-touch.
- No Mouse Fat Fingers problem:
Two problems: the area in pixels under direct contact with the finger/thumb, which varies depending on resolution and physical screen size and the almost unavoidable roll of the finger/thumb as it comes into contact with the surface. The former complicates how to compensate for small controls (eg, buttons, and assuming UI not scaled-up massively). Multi-sampling around the touch point for nearest/top-most morph helps. The roll-problem results in a small movement which when measured in pixels could be large and may require implementing some dead-band to overcome (possibly reducing the effective resolution for touch).
-D
etoys-dev@lists.squeakfoundation.org