Hi, Tim.

The idea of using event filters to actually transform events is interesting.

How would a fake event filter be triggered if not by real events?

Best, Marcel

-- View this message in context: http://forum.world.st/Event-filters-for-user-input-events-i-e-mouse-clicks-e... Sent from the Squeak - Dev mailing list archive at Nabble.com.

tim Rowledge wrote

...

On 18-05-2016, at 9:25 AM, marcel.taeumel <

Marcel.Taeumel@

> wrote:

...

The idea of using event filters to actually transform events is interesting.

Filters can often be useful. We have, for example, the various canvases and scroll-wrapping morphs which filter how input and output is seen by the wrapped bitmaps etc. An obvious event filter (and maybe a stupid one in the real world) would be one that centralises the mouse button swapping; if button swapping is turned on it, err, swaps the button value around. Right now we have a centralised place in EventSensor>processEvent: for something that global (and seriously - for a mouse type event we check the platform name every time?) but having a type of filter involved at lower levels would allow some morphs to *not* get swapped buttons. I dunno, maybe useful for testing how the UI behaves on a different GUI?

If you pass your events *through* a filter instead of just having them on the side then you can do much more.

...

How would a fake event filter be triggered if not by real events?

If some morph/app has a suitable event filter then clearly we can transform any incoming event into any other kind of event (as long as it is one that the morph will not raise an error for) as they are sent by the whole Hand/PasteUp complex. To trigger a faking-filter I guess the best thing would be to throw a FakeEvent at the event handling code at whatever level you need (anywhere from the local world PasteUpMorph on down to your local morph of interest) and let you filter convert that to whatever it wants. Why not simply make the ‘actual’ event type and pass that in? You might have a variety of faking-filters doing different things and want to test them, maybe you want the FakeEvents to be easily assembled by parsing a socket stream or file, I don’t know. It just needs you to bypass the eventsensor and push an event.

I honestly don’t know if it’s an actually useful idea.

tim

tim Rowledge;

tim@

; http://www.rowledge.org/tim Asking whether machines can think is like asking whether submarines can swim.

Hi, there!

Here is another update: event-filters.cs http://forum.world.st/file/n4896366/event-filters.cs

The filters should work now as expected. The change set includes a refactoring of MorphicEventDispatcher and HandMorph >> #handleEvent:. Be sure to update the trunk to the latest version before filing-in the change set.

Then you can do things like "if you type a y, make a z out of it and vice versa":

filter := PluggableEventFilter on: [:evt :target | (evt isKeyboard and: [evt keyCharacter = $z]) ifTrue: [evt instVarNamed: #keyValue put: $y asInteger] ifFalse: [(evt isKeyboard and: [evt keyCharacter = $y]) ifTrue: [evt instVarNamed: #keyValue put: $z asInteger]]. evt].

ActiveHand addEventFilter: filter. ActiveHand removeEventFilter: filter.

Best, Marcel

-- View this message in context: http://forum.world.st/Event-filters-for-user-input-events-i-e-mouse-clicks-e... Sent from the Squeak - Dev mailing list archive at Nabble.com.

Tony Garnock-Jones-3 wrote

On 2016-05-20 9:19 AM, marcel.taeumel wrote:

...

Then you can do things like "if you type a y, make a z out of it and vice versa": [...]

Would it be possible to have "if you type y, pretend you typed abcde"?

That is, instead of having the filter yield zero or one events (for refiltering??), could it yield zero or more events?

Tony

Hi Tony,

yes, I think so. You event create code is kind of awkward but I would work like this:

... [:event :target | ... event hand handleEvent: ((KeyboardEvent newFrom: event) stroke: $a); handleEvent: ((KeyboardEvent newFrom: event) stroke: $b); handleEvent: ((KeyboardEvent newFrom: event) stroke: $c); handleEvent: ((KeyboardEvent newFrom: event) stroke: $d); handleEvent: ((KeyboardEvent newFrom: event) stroke: $e). event wasIgnored: true. event].

Note that this API "(KeyboardEvent newFrom: event) stroke: $a" does not exist at the moment. It is not that convenient to generate new events but "event hand handleEvent: ..." should work. I have not tried this yet, however, the event dispatching and handling code should be reentrant.

I think that I should separate upstream and downstream event filters. When calling "ActiveHand addEventFilter: ..." it is only downstream because this is where event processing starts. When calling "someMorphInTheWorld addEventFilter: ..." this is rather upstream because it starts at Morph >> #handleEvent:. Hmm... downstream event filters for any morph would be possible in Morph >> #processEvent:using:.

Still ... I am struggling with the concept of keyboard focus and mouse focus. There, you directly call Morph >> #handleFocusEvent:, which calls Morph >> #handleEvent:. No bubbling up, no bubbling down. If focused morphs want to bubble down, they have to call Morph >> #processEvent: and watch for the #rejected pattern. See UserDialogBoxMorph >> #handleFocusEvent: for an example. I now that this is a performance thing. If you type in a text field, you do not want to bubble all the way down from the world. Again and again. And again.

Best, Marcel

-- View this message in context: http://forum.world.st/Event-filters-for-user-input-events-i-e-mouse-clicks-e... Sent from the Squeak - Dev mailing list archive at Nabble.com.

tim Rowledge wrote

...

On 20-05-2016, at 12:33 PM, marcel.taeumel <

Marcel.Taeumel@

> wrote:

...

yes, I think so. You event create code is kind of awkward but I would work like this:

... [:event :target | ... event hand handleEvent: ((KeyboardEvent newFrom: event) stroke: $a); handleEvent: ((KeyboardEvent newFrom: event) stroke: $b); handleEvent: ((KeyboardEvent newFrom: event) stroke: $c); handleEvent: ((KeyboardEvent newFrom: event) stroke: $d); handleEvent: ((KeyboardEvent newFrom: event) stroke: $e). event wasIgnored: true. event].

You’d need to be *very* careful here; catching a keyboard event and then sending more keyboard events via the same route…

tim

tim Rowledge;

tim@

; http://www.rowledge.org/tim Abdicate (v.), to give up all hope of ever having a flat stomach.

Hi Tim,

it wouldn't be the same route. Calling HandMorph >> #processEvent: means a fresh start for processing the event argument. With all its glory. :-) Shared state in HandMorph is an issue though:

- keyboard focus - mouse focus - mouse click state - mouse over handler - event listeners (- event filters) (- event stats)

However, you actually can send multiple events along the "same route" if you mean the same target:

[:event :target | ... target "<--- do NOT start event processing over again but use same target" handleEvent: ((KeyboardEvent newFrom: event) stroke: $a); handleEvent: ((KeyboardEvent newFrom: event) stroke: $b); handleEvent: ((KeyboardEvent newFrom: event) stroke: $c); handleEvent: ((KeyboardEvent newFrom: event) stroke: $d); handleEvent: ((KeyboardEvent newFrom: event) stroke: $e). event wasIgnored: true. event].

Given that there is a double-dispatch pattern in UserInputEvent >> #sentTo: and a event type discrimination in MorphicEventDispatcher >> #dispatchEvent:with:, we are fine, even with changing event types and not just multiplicity in a filter.

The return value in an event filter, however, should always be a single event. Ignored if not needed.

Best, Marcel

-- View this message in context: http://forum.world.st/Event-filters-for-user-input-events-i-e-mouse-clicks-e... Sent from the Squeak - Dev mailing list archive at Nabble.com.

marcel.taeumel wrote

Hi, there.

Here is a new version of event filters with support for separating capturing filters from bubbling filters. https://www.w3.org/TR/DOM-Level-2-Events/events.html#Events-flow-capture https://www.w3.org/TR/DOM-Level-2-Events/events.html#Events-flow-bubbling

Here is the change set: event-filters.cs http://forum.world.st/file/n4896486/event-filters.cs

Note that you should either use a fresh trunk image with the latest updates OR unload all the changes from my previous change set because I cleaned up this one. So, there will be some "method removed" calls missing.

Anyway, with support for capturing filters, you can also filter, for example, mouse events for system windows that are always active (see preference "Windows Active Only On Top" disabled).

wnd := ActiveWorld submorphs fourth. mouseFilter := PluggableEventFilter on: [:evt :target | evt ignore]. wnd addMouseCaptureFilter: mouseFilter. wnd removeMouseCaptureFilter: mouseFilter.

See how the behavior is different when you use the filter for the bubbling phase:

wnd addMouseBubbleFilter: mouseFilter. wnd removeMouseBubbleFilter: mouseFilter.

Note that event bubbling is implemented in MorphicEventDispatcher. If there would be no bubbling, those filters would be meaningless. However, I am not aware of an alternative event dispatcher for Squeak's Morphic.

What other dispatch patterns are out there? :-)

Best, Marcel

Hi, there.

I benched the event filters mechanism without having any active filters to reveal the impact of my changes. Here is the benchmark:

0) Close all windows.

1) Open a system browser and position it in the upper-left corner right below the world main docking bar.

2) Open a workspace, not covering the system browser.

3) Execute some code:

event := MouseButtonEvent new setType: #mouseDown position: 100@100 which: 2r000 "no change" buttons: 2r100 "red/left pressed" hand: ActiveHand stamp: 0. "not needed" [ event resetHandlerFields. event hand sendMouseEvent: event. ] bench.

On my Windows 10 machine (MS Surface Pro 3, Core i7 mobile) with Cog/Spur #3663 and Squeak #15987 I got:

Before: 3.73 microseconds per run After: 4.5 microseconds per run

This tests a mouse-down event because it involves full dispatching (resp. capturing) and handling (resp. bubbling) without taking the shortcut via mouse focus (or keyboard focus).

This benchmark combined with MessageTally class >> #spyOn: provided interesting insights about expensive calls.

About 5.9 percent are spend for sending generic and mouse bubble filters. Note that 4.1 percent are spend for accessing the Morph's properties to find out that there are no filters at the moment. This could be optimized by adding instance variables to MorphExtension.

(Interestingly, about 11 percent (before: 12.7 percent) are spent in Morph

...

#handleMouseDown: to call HandMorph >> #removePendingBalloonFor:.)

(On a side note, the current way to provide global keyboard shortcuts eats up 47.1 percent (before: 48.4 precent) for this mouse down handling because of installing/removing event listeners in PasteUpMorph >> #becomeActiveDuring:. But this will be optimized on a few days.)

The following questions remain: - Can I put this in trunk and use it to improve the implementation of global keyboard shortcuts? - Should I add instance variables in MorphExtension to speed-up the look-up? Note that I already did add instance variables to HandMorph to speed-up the look-up of global event capturing filters. - Is it conceptually good to re-evaluate the filters if one of them changes the kind of an event? This affects only mouse event filters and keyboard event filters that convert between mouse and keyboard. Note that an endless loop is possible if such filters do not converge.

Best, Marcel

-- View this message in context: http://forum.world.st/Event-filters-for-user-input-events-i-e-mouse-clicks-e... Sent from the Squeak - Dev mailing list archive at Nabble.com.

On 27.05.2016, at 20:36, Chris Muller asqueaker@gmail.com wrote:

Hi Marcel, global keyboard shortcuts are important, a 1-microsecond slowdown per event does not sound significant. But, yes, for performance critical code like event handling, its probably worth adding an instvar than a dictionary lookup.

I don't know under what case an event-type would ever change. Events are events generated by the input devices. They should basically be created and then immutable...

Unless you want a filter that turns a mouse click into a keypress, like for automation?

Best -Tobias

On Fri, May 27, 2016 at 8:40 AM, marcel.taeumel Marcel.Taeumel@hpi.de wrote:

...

marcel.taeumel wrote

...

Hi, there.

Here is a new version of event filters with support for separating capturing filters from bubbling filters. https://www.w3.org/TR/DOM-Level-2-Events/events.html#Events-flow-capture https://www.w3.org/TR/DOM-Level-2-Events/events.html#Events-flow-bubbling

Here is the change set: event-filters.cs http://forum.world.st/file/n4896486/event-filters.cs

Note that you should either use a fresh trunk image with the latest updates OR unload all the changes from my previous change set because I cleaned up this one. So, there will be some "method removed" calls missing.

Anyway, with support for capturing filters, you can also filter, for example, mouse events for system windows that are always active (see preference "Windows Active Only On Top" disabled).

wnd := ActiveWorld submorphs fourth. mouseFilter := PluggableEventFilter on: [:evt :target | evt ignore]. wnd addMouseCaptureFilter: mouseFilter. wnd removeMouseCaptureFilter: mouseFilter.

See how the behavior is different when you use the filter for the bubbling phase:

wnd addMouseBubbleFilter: mouseFilter. wnd removeMouseBubbleFilter: mouseFilter.

Note that event bubbling is implemented in MorphicEventDispatcher. If there would be no bubbling, those filters would be meaningless. However, I am not aware of an alternative event dispatcher for Squeak's Morphic.

What other dispatch patterns are out there? :-)

Best, Marcel

Hi, there.

I benched the event filters mechanism without having any active filters to reveal the impact of my changes. Here is the benchmark:

0. Close all windows.

1. Open a system browser and position it in the upper-left corner right

below the world main docking bar.

2. Open a workspace, not covering the system browser.

3. Execute some code:

event := MouseButtonEvent new setType: #mouseDown position: 100@100 which: 2r000 "no change" buttons: 2r100 "red/left pressed" hand: ActiveHand stamp: 0. "not needed"

[ event resetHandlerFields. event hand sendMouseEvent: event. ] bench.

On my Windows 10 machine (MS Surface Pro 3, Core i7 mobile) with Cog/Spur #3663 and Squeak #15987 I got:

Before: 3.73 microseconds per run After: 4.5 microseconds per run

This tests a mouse-down event because it involves full dispatching (resp. capturing) and handling (resp. bubbling) without taking the shortcut via mouse focus (or keyboard focus).

This benchmark combined with MessageTally class >> #spyOn: provided interesting insights about expensive calls.

About 5.9 percent are spend for sending generic and mouse bubble filters. Note that 4.1 percent are spend for accessing the Morph's properties to find out that there are no filters at the moment. This could be optimized by adding instance variables to MorphExtension.

(Interestingly, about 11 percent (before: 12.7 percent) are spent in Morph

...

...

#handleMouseDown: to call HandMorph >> #removePendingBalloonFor:.)

(On a side note, the current way to provide global keyboard shortcuts eats up 47.1 percent (before: 48.4 precent) for this mouse down handling because of installing/removing event listeners in PasteUpMorph >> #becomeActiveDuring:. But this will be optimized on a few days.)

The following questions remain:

• Can I put this in trunk and use it to improve the implementation of global

keyboard shortcuts?

• Should I add instance variables in MorphExtension to speed-up the look-up?

Note that I already did add instance variables to HandMorph to speed-up the look-up of global event capturing filters.

• Is it conceptually good to re-evaluate the filters if one of them changes

the kind of an event? This affects only mouse event filters and keyboard event filters that convert between mouse and keyboard. Note that an endless loop is possible if such filters do not converge.

Best, Marcel

-- View this message in context: http://forum.world.st/Event-filters-for-user-input-events-i-e-mouse-clicks-e... Sent from the Squeak - Dev mailing list archive at Nabble.com.

Chris Muller-4 wrote

On Fri, May 27, 2016 at 2:40 PM, Tobias Pape &lt;

Das.Linux@

&gt; wrote:

...

On 27.05.2016, at 20:36, Chris Muller &lt;

asqueaker@

&gt; wrote:

...

...

Hi Marcel, global keyboard shortcuts are important, a 1-microsecond slowdown per event does not sound significant. But, yes, for performance critical code like event handling, its probably worth adding an instvar than a dictionary lookup.

I don't know under what case an event-type would ever change. Events are events generated by the input devices. They should basically be created and then immutable...

Unless you want a filter that turns a mouse click into a keypress, like for automation?

Instead of converting an event, shouldn't the automater-thingy just push a new event?

The responsibility of Events is to interface the UI hardware (keyboard, mouse, etc.) to some UI framework (MVC, Morphic). They represent the low-level, immutable "data" from the hardware, interpreted by the UI framework. IMHO, this "signal" between VM and Morphic should remain a thin, clean, simple and raw encapsulation of the data (i.e., getters and setters). I don't think application-specific behaviors belong way down there.. You mentioned, automation?

If you want to convert a mouse click into a keypress for automation, you would have two options to write a global/hand capture filter:

1st option) Write a mouse filter that ignores the given click event, generates a keyboard event, and processes that keyboard event via "givenMouseEvent hand processEvent: newKeyboardEvent.". Eventually, return that original (but ignored) mouse event.

2nd option) Write a mouse filter that just generates a keyboard event and returns that.

I want to ensure that both options yield the same results to mitigate programming errors. The 1st option would trigger all active filters again, and hence also keyboard filters, because event processing starts again. The 2nd option continues the current event processing and hence should also trigger global keyboard filters (again). Just make option 1 behave like option 2.

@Chris: Yes, such event filters are specific to Morphic. In MVC, such filters do still not exist. The code that I added resides in Morph, HandMorph, and MorphEventDispatcher. However, we should consider moving the UserInputEvent tree out of Morphic into the base system for other frameworks to work with user input events in an object-oriented fashion. :-) One could also implement generic event filters in EventSensor >> #nextEvent but this is not what I had in mind here. Our global keyboard shortcuts are for Morphic only.

Best, Marcel

-- View this message in context: http://forum.world.st/Event-filters-for-user-input-events-i-e-mouse-clicks-e... Sent from the Squeak - Dev mailing list archive at Nabble.com.

Tony Garnock-Jones-3 wrote

On 05/18/2016 12:12 PM, tim Rowledge wrote:

...

Seems to me that a significant part of the potential usefulness would be to have filters transform events [...]

One interesting approach would be to take a leaf from functional programming.

Instead of returning a boolean from a filter, you'd have it expect a continuation block.

filter := PluggableEventFilter new on: [:e :target :k | Transcript showln: e printString. k value: e ].

It would invoke the block with some event, or not, according to the filtering decision.

This gives it an opportunity to pass on either its argument; or a copy of its argument; or its argument, modified; or something unrelated.

Tony

Hi Tony,

I like the idea of filters transforming events. Here, the filter could just return the transformed event instead of a boolean value. (Note that the last statement in a block determines its return value.)

Here is a new version of event filter that does that: event-filters.cs http://forum.world.st/file/n4896027/event-filters.cs

Don't forget to revert the previous change set because I cleaned up this one. Use a fresh trunk image at best.

wnd := ActiveWorld submorphs third. "some window?" filter := PluggableEventFilter on: [:evt :target | evt isMouseOver ifFalse: [ Transcript showln: evt printString, ' ', target printString]. evt wasIgnored: true. evt]. wnd addMouseFilter: filter. wnd removeMouseFilter: filter.

Best, Marcel

-- View this message in context: http://forum.world.st/Event-filters-for-user-input-events-i-e-mouse-clicks-e... Sent from the Squeak - Dev mailing list archive at Nabble.com.