[squeak-dev] Squeak on a cellphone responds to touchscreen input now (was Re: Squeak on a PostmarketOS cellphone)

Beckmann, Tom Tom.Beckmann at student.hpi.uni-potsdam.de
Sun Sep 13 15:22:04 UTC 2020


Hi everyone,

just wanted to mention that I experimented with bringing XInput2 [4] support to the X11 plugin for Squeak. With this, we get smooth (pixel perfect) scrolling and multitouch support. The code is lacking the device management part, so it may not work correctly when new devices are attached, for example.

Some pointers to code:
- Here are the changes to the X11 event handling loop [1]
- For the touch event tuple [2], I stayed rather close to the info XI2 provides, meaning you get the x,y coordinate, a sequence number that identifies a continuous gesture in the stream of events, and a touch type (begin/end/update/cancel). In terms of the range of information (not its structure) this also mirrors what you get in a browser if we omit experimental features such as radius of the touch area [3].

On the image side, I just simply added new #touch[Begin,Update,Update,Cancel]: callbacks to Morph, to which I forwarded the deconstructed event tuples as TouchEvent objects for demonstration purposes. Nothing fancy there just yet. I agree that a concept of multiple hands will likely provide a better abstraction.

Hope this can be of some interest :)

Best,
Tom

[1] https://github.com/tom95/opensmalltalk-vm/blob/xi-experiment/platforms/unix/vm-display-X11/sqUnixX11.c#L3638
[2] https://github.com/tom95/opensmalltalk-vm/blob/xi-experiment/platforms/unix/vm/sqUnixEvent.c#L188
[3] https://developer.mozilla.org/en-US/docs/Web/API/Touch
[4] https://www.x.org/releases/X11R7.7/doc/inputproto/XI2proto.txt
________________________________________
From: Squeak-dev <squeak-dev-bounces at lists.squeakfoundation.org> on behalf of Vanessa Freudenberg <vanessa at codefrau.net>
Sent: Saturday, September 12, 2020 11:25:24 PM
To: The general-purpose Squeak developers list
Subject: Re: [squeak-dev] Squeak on a cellphone responds to touchscreen input now (was Re: Squeak on a PostmarketOS cellphone)

Ha, talk about a trip down memory lane! I had no idea that still existed :)

- Vanessa -

On Sat, Sep 12, 2020 at 5:42 AM karl ramberg <karlramberg at gmail.com<mailto:karlramberg at gmail.com>> wrote:
There is also the Frank stuff which probably have some of the same multi cursor funktionality:
http://tinlizzie.org/~bert/frank4ipad/

One can download the Frank.ipa and unzip to look at the image.


Best,
Karl



On Fri, Sep 11, 2020 at 5:04 PM Tony Garnock-Jones <tonyg at leastfixedpoint.com<mailto:tonyg at leastfixedpoint.com>> wrote:
Hi Vanessa,

On 9/10/20 10:23 PM, Vanessa Freudenberg wrote:
> Oh, exciting! Please record a multi-hand Etoys demo. Maybe like this:
> https://youtu.be/gYrp31fH-Jk?t=56

Whoa, awesome! I didn't know this existed!

I have questions!

 - How much of that shift-to-get-other-colour-buttons support is still
in the image? So far I haven't touched the default world menu bar, which
is mildly inappropriate for small and/or multitouch devices.

 - What other ideas about mapping multitouch to Morphic interaction are
out there I have missed and am in danger of reinventing?

Also, I am intrigued by the idea of simply having each touch be mapped
to a Hand. It seems like the obvious way to do things, on one level, but
doesn't quite fit my fuzzy intuitions for how (multi-finger) gesture
recognition might work on another level. So:

 - When you were working on this, did you have any thoughts about
deficiencies of the Morphic model wrt multitouch/gestures/etc., and then
did you have any ideas about what to change and how to change it?

I'll try to cook up a little demo like that sometime soon.

Tony




More information about the Squeak-dev mailing list