[squeak-dev] Squeak on a cellphone responds to touchscreen input now (was Re: Squeak on a PostmarketOS cellphone)

Vanessa Freudenberg vanessa at codefrau.net
Sat Sep 12 21:16:18 UTC 2020


On Fri, Sep 11, 2020 at 8:11 AM Tony Garnock-Jones <
tonyg at leastfixedpoint.com> wrote:

> Hi Vanessa,
>
> On 9/10/20 10:23 PM, Vanessa Freudenberg wrote:
> > Oh, exciting! Please record a multi-hand Etoys demo. Maybe like this:
> > https://youtu.be/gYrp31fH-Jk?t=56
>
> Whoa, awesome! I didn't know this existed!
>
> I have questions!
>
>  - How much of that shift-to-get-other-colour-buttons support is still
> in the image? So far I haven't touched the default world menu bar, which
> is mildly inappropriate for small and/or multitouch devices.
>

I never cleaned it up enough to make it into an official release.

I *may* have sent a changeset to some mailing list but my memory is really
hazy. A lot has happened in the past 10 years 😅

... (searching) ...

Oh, found it, here, from 2010:
http://lists.squeakfoundation.org/pipermail/vm-dev/2010-September/005364.html

And from Frank4iPad (2011), see attached two changesets.

 - What other ideas about mapping multitouch to Morphic interaction are
> out there I have missed and am in danger of reinventing?
>
> Also, I am intrigued by the idea of simply having each touch be mapped
> to a Hand. It seems like the obvious way to do things, on one level, but
> doesn't quite fit my fuzzy intuitions for how (multi-finger) gesture
> recognition might work on another level. So:
>

Given that Morphic is one of the very few UI environments that supports
multiple pointers out-of-the-box, mapping each finger to a hand seemed like
the obvious thing to do to me.

Also, I had done it before - probably like 15 years ago I added Wacom
tablet support to the unix vm, and in Morphic that got turned into a
separate hand for each device (stylus, puck, mouse). The stylus "hand" even
showed the stylus' rotation and tilt using a line and its shadow.

So for multi-touch I just did TSTTCPW and it did indeed work. But it never
went anywhere, partly because of Apple's refusal to let Squeak into the app
store, though mostly because my focus shifted elsewhere. Also, nobody else
back in the day seemed interested enough to build on what I had shared.

Adding multi-touch + gestures to SqueakJS is on my idea list but I have
not found the time yet. What I did implement is VM-side gestures for
panning / zooming and right click (all using 2 fingers). Would love to have
morph scale and rotation gestures too but that would require image-side
support. However, with those VM gestures, SqueakJS is now quite usable on
iPhones / iPads.

I do like the big "Cmd" button for two-handed interaction, and that could
be implemented either in the image (see touchCmdMorph attached) or in the
VM. For SqueakJS I'd do it in the VM because one goal for SqueakJS is to be
able to run as many unmodified images as possible. But that means until I
get around to implement it, there is no way to do "Cmd-p" using the
keyboard like you saw in the iPad video. I can only invoke print-it using
the menu.

In SqueakJS there is an explicit button for showing / hiding the keyboard.
I always planned to add IMM plugin support to automate showing/hiding the
keyboard but never got around to do it.

Another thought was to invoke the character recognizer that is still in the
image. At some point its cmd-r key binding got re-used though, so I don't
think there is a way to invoke it now.

 - When you were working on this, did you have any thoughts about
> deficiencies of the Morphic model wrt multitouch/gestures/etc., and then
> did you have any ideas about what to change and how to change it?
>

If I did, I don't recall those thoughts now.

Agreeing on a multi-touch event format for VM events would be a good step
forward—the one John McIntosh did for the iOS VM is pretty much a 1-to-1
mapping of the iOS API. I'd prefer something more platform-neutral, like
mimicking the Web's Pointer Event API
https://developer.mozilla.org/en-US/docs/Web/API/Pointer_events (or the
Touch Events API, but I like the unified API for all kinds of pointing
devices).

I'm excited you're working on this!

- Vanessa -
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20200912/879c03dd/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 2596touchCmdMorph-bf.cs
Type: text/x-csharp
Size: 1633 bytes
Desc: not available
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20200912/879c03dd/attachment-0002.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 2595touchEvents-bf.cs
Type: text/x-csharp
Size: 10889 bytes
Desc: not available
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20200912/879c03dd/attachment-0003.bin>


More information about the Squeak-dev mailing list