[Vm-dev] [OpenSmalltalk/opensmalltalk-vm] For macOS -metal -core-graphics -opengl backends, fixes several issues concerning performance and high-resolution (PR #620)

Marcel Taeumel notifications at github.com
Fri Apr 1 14:41:54 UTC 2022


Tested with **Squeak6.0alpha** and **Cuis6.0-5069**.

Fixed issues:
- Correct image-side display bounds in high-resolution mode on start-up and via `#setNewScreenSize:` (i.e., HostWindowPlugin primitiveHostWindowSizeSet)
- File drag in high-resolution mode is now matching image-side display bounds
- File drag has now access to file paths in DragEnter, not just DragDrop (i.e., mouseUp)
- `-core-graphics` usable again
- `-opengl` does not slow down VM interpeter loop anymore (e.g., in Morphic's use of primitive 127 and 231)
- `-metal` does not slow down VM interpeter loop anymore (e.g., in Morphic's use of primitive 127 and 231)

Known issues:
- **Rather frequent segfaults in macOS64ARMv8**, maybe related to JIT via `checkForEventsMayContextSwitch:` where `checkCogCompiledCodeCompactionCalledFor` is called before `ioProcessEvents` ... maybe because of new display events queued in `-core-graphics` and `-metal` backends

----

Known issues on macOS 12:
- No OS-supported "Low Resolution" mode; use magnifier
- No `-opengl` support
- High-resolution mode scales up hardware cursor

Known issues on macOS 11.6:
- No `-opengl` support
- High-resolution mode scales up hardware cursor

Known issues on macOS 10.15:
- High-resolution mode scales up hardware cursor

----

Known issues for `-metal` backend:
- Must use image-side buffer for composition to avoid flickering

Known issues for `-core-graphics` backend:
- Must use image-side buffer for composition to avoid flickering
- Blitting on `Display` via `Form reverse` not working bc. alpha value must be FF (255)
- Rather slow bc. only full and no clippy-based updates due to inaccessible double-buffering in CoreGraphics

Known issues for `-opengl` backend:
- No blitting support for Squeak's MVC
- No blitting support for Squeak's Morph's fast window dragging/resizing

-----

Note that Squeak6.0alpha has direct support for that image-side buffer via `#disableDeferredUpdates` preference. The support in Squeak's Morphic has been there since at least 1999. Cuis does not yet have an image-side buffer for composition but flickering is rare.

Note that `-core-graphics` could be made compatible with `Form reverse` blitting rule by using a color with alpha value (almost) 0 such as `Color gray alpha: 1/255` or `Color colorFromPixelValue: 16r01808080`. Try out via `Display fill: (0 at 0 corner: 200 at 200) rule: Form reverse fillColor: Color gray`.

Finally note that both `-metal` and `-core-graphics` now do rather nothing in primitive 231 (i.e., `forceDisplayUpdate`/ `ioForceDisplayUpdate` / `drawThelayers`) but rely on a scheduled display event to be delivered via primitive 94 (i.e., `primitiveGetNextEvent` / `ioProcessEvents` / `vmIOProcessEvents`), which is also called every 20 milliseconds on any other primitive send. See `checkForEventsMayContextSwitch` in VMMaker sources.

-----

**Big thanks to [krono](https://github.com/krono/) for helping out with this high-resolution stuff and other details on macOS!**
You can view, comment on, or merge this pull request online at:

  https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/620

-- Commit Summary --

  * Command-line tools for Xcode have different SDK path
  * Extracting directly from krono/highdpi-v2, fixes high-resolution scaling for Metal backend.
  * For macOS platform, fixes high-resolution scaling when (1) reading prior display extent from image and (2) setting new extent via HostWindowPlugin.
  * Extracting directly from krono/highdpi-v2, fixes high-resolution scaling for both OpenGL and CoreGraphics backends. Note that the CoreGraphics backend is still kind of broken and should not be used on macOS 10.15.x or 11.x.x.
  * Fixes initial screen refresh for macOS OpenGL backend when using high-resolution mode. Only applies to macOS 10.x.x because OpenGL support was dropped starting with macOS 11.x.x.
  * For macOS platform, fixes drag-and-drop for high-resolution mode. Note that this change also makes the dragged file names/urls accessible directly after drag-enter, not just when letting go of the mouse button. Thus, the image can now show the dragged file names early on. The macOS dragging cursor is still visible though.
  * For macOS platform, fixes mouse tracking (entered, exited, moved), which got lost since macOS 10.5 where NSTrackingArea was introduced.
  * For macOS platform, clean up code around mouse tracking. The flag NSTrackingInVisibleRect ensures that macOS will take care of resizing the tracking area when the window is resized.
  * For macOS platform, explicitly allow "Open in low resolution" in case the default gets changed.
  * Fix typo in previous commit. :-/
  * For macOS platform, makes -core-graphics backend kind of usable again. There are still occasional segfaults that should be inspected. Also, the Smalltalk image must not rely on deferred updates in the VM but implement an extra composition buffer in the image. Squeak/Morph has #disableDeferredUpdates for this.
  * For macOS platform, disable vsync for -opengl backend to not slow down the VM interpreter loop.
  * For macOS platform, disable vsync for -metal backend to not slow down the VM interpreter loop.
  * For macOS platform, fixes the sporadic segfaults...
  * For macOS platform using -metal backend, do some code clean-up and add more commentary on where display events are generated to avoid slowing down the VM interpreter loop.
  * For macOS platform using -core-graphics backend, initialize colorMap32 like we do for all supported backends, even though we don't seem to accept display depths other than 32-bit (BGRA) these days, although we could because we the color-mapping tables (1-to-32, 2-to-32, etc.) still present.

-- File Changes --

    M .gitignore (1)
    M building/macos64ARMv8/common/Makefile.flags (1)
    M platforms/iOS/plugins/HostWindowPlugin/sqMacHostWindow.m (4)
    M platforms/iOS/vm/Common/Classes/sqSqueakEventsAPI.m (17)
    M platforms/iOS/vm/Common/Classes/sqSqueakScreenAndWindow.m (10)
    M platforms/iOS/vm/OSX/Newspeak-Info.plist (4)
    M platforms/iOS/vm/OSX/Pharo-Info.plist (4)
    M platforms/iOS/vm/OSX/Squeak-Info.plist (4)
    M platforms/iOS/vm/OSX/SqueakOSXAppDelegate.m (9)
    M platforms/iOS/vm/OSX/plugins/HostWindowPlugin/sqMacHostWindow.m (4)
    M platforms/iOS/vm/OSX/sqSqueakOSXApplication+events.m (10)
    M platforms/iOS/vm/OSX/sqSqueakOSXCGView.h (1)
    M platforms/iOS/vm/OSX/sqSqueakOSXCGView.m (196)
    M platforms/iOS/vm/OSX/sqSqueakOSXHeadlessView.h (1)
    M platforms/iOS/vm/OSX/sqSqueakOSXHeadlessView.m (8)
    M platforms/iOS/vm/OSX/sqSqueakOSXMetalView.h (4)
    M platforms/iOS/vm/OSX/sqSqueakOSXMetalView.m (129)
    M platforms/iOS/vm/OSX/sqSqueakOSXOpenGLView.h (6)
    M platforms/iOS/vm/OSX/sqSqueakOSXOpenGLView.m (74)
    M platforms/iOS/vm/OSX/sqSqueakOSXView.h (7)
    M platforms/iOS/vm/iPhone/Classes/SqueakUIView.h (3)
    M platforms/iOS/vm/iPhone/Classes/SqueakUIView.m (12)

-- Patch Links --

https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/620.patch
https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/620.diff

-- 
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/620
You are receiving this because you are subscribed to this thread.

Message ID: <OpenSmalltalk/opensmalltalk-vm/pull/620 at github.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20220401/a03e09c9/attachment.html>


More information about the Vm-dev mailing list