O2 and O1 compilation produce a segmentation fault due to stack corruption (when debugging we saw some extra pushes from the stack) on callbacks return. This happens on windows 32, when compiling with mingw gcc 7.4.0.
The issue can be reproduced easily by running the Alien qsort example in latest vms in both Pharo and Squeak.
This PR proposes to patch just the thunkEntry function. Not optimizing just that function solves the issue in our environment, though maybe there is a more fine-grained solution. We should still investigate what is the particular optimization that causes the problem. You can view, comment on, or merge this pull request online at:
https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/353
-- Commit Summary --
* Patch callback thunkEntry to not optimize, failing in win32 using gcc 7.4.0
-- File Changes --
M platforms/Cross/plugins/IA32ABI/ia32abicc.c (2)
-- Patch Links --
https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/353.patch https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/353.diff