[etoys-dev] Touch Interface thoughts

[Bert Freudenberg]

On 24.06.2011, at 10:01, Steve Thomas wrote:

> How will we handle multi-touch in Etoys? 

Yay! Someone else is thinking about this :)

> 
> 1) Multiple Mice:
> When you touch the screen with multiple fingers you get "many mice"

"Multiple hands", we call it in Squeak. I re-purposed the existing multi-user support to support more than one finger for a single user. Now the terminology is not quite right anymore, but it was TSTTCPW.

>  So, how can I get the mouseX and mouseY for each "mouse"?  Thinking about this for my version of Slice it. 

I think the best use for multi-touch is to bring up the halo. Single-finger touch would be like clicking the primary mouse button today. When tapping with two fingers, the halo would activate. Not only that, but if you then rotate the fingers or pinch/spread them, it would turn and resize the object. I guess the halo should only be invoked if the two fingers tap at the same time, and close to each other. Otherwise you would just get two separate tap events to manipulate multiple objects (like I showed in my youtube video).

Now, we might add an option to disable that halo behavior inside a playfield. Enabling it would give you all the touch events. 

> So I can imagine one way would be a collection of mice, one could iterate through.

Iterating isn't well-supported in Etoys. How exactly would you imagine this might work?

A very simple interface would be having a "number of fingers" read out, and "finger 1 x", "finger 1 y", "finger 2 x", "finger 2 y" accessors. Or, maybe have a "second finger down" script trigger. I admit I have not thought much about this yet.

> 2) No Mouse Fat Fingers problem:
> A while back I had a conversation with an expert (some 17 or 18 year old kid who had built a Theremin remote control for his TV) and the idea came to me, I asked him about the power consumption, costs and if there were any interference issues and from the discussion it would seem this may be doable within XO-3 constraints.  It would operate similar to the Bamboo Fun tablet which detects the stylus when it hovers over the tablet and you can move the mouse without touching the tablet, then it clicks when you touch the tablet.
> 
> One issues I can foresee would be when you "hover" your finger over the screen if the mouse pointer is right under your finger, it would block the view of the pointer, so you would have to position the pointer "above" the finger.  But then when you touch you are not "clicking" where you touch.
> 
> Of course part of my problem my be I am thinking like a mouse ;)

Very few touch screen technologies support "hovering", so I'd rather not rely on that. And, no real touch interface I know even has a visible mouse pointer.

The cheapest touch screens aren't even multi-touch, but offer only a single touch point. It may be a good idea to design for that.

Btw, if you have an iPad, send me your UDID so I can compile a version of Etoys that you can try. Or, if you have jailbroken yours, just install the ipa from

	http://etoys.squeak.org/experimental/

Also, not sure if anyone tried my Etoys-To-Go that mimics a touch interface using a huge mouse pointer:

	http://etoys.squeak.org/experimental/Etoys-Touch-Emu-100620.zip

- Bert -