From noreply at github.com  Mon Mar  1 01:18:22 2021
From: noreply at github.com (Eliot Miranda)
Date: Sun, 28 Feb 2021 17:18:22 -0800
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] a69b4c: On Windows allow
 a zero width in openCamera to sel...
Message-ID: <OpenSmalltalk/opensmalltalk-vm/push/refs/heads/Cog/c53f3e-a69b4c@github.com>

  Branch: refs/heads/Cog
  Home:   https://github.com/OpenSmalltalk/opensmalltalk-vm
  Commit: a69b4c93edbbde759c849aa8e2f9adc97e808694
      https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/a69b4c93edbbde759c849aa8e2f9adc97e808694
  Author: Eliot Miranda <eliot.miranda at gmail.com>
  Date:   2021-02-28 (Sun, 28 Feb 2021)

  Changed paths:
    M platforms/win32/plugins/CameraPlugin/winCameraOps.cpp
    M platforms/win32/plugins/SoundPlugin/sqWin32Sound.c
    M platforms/win32/vm/sqWin32Window.c

  Log Message:
  -----------
  On Windows allow a zero width in openCamera to select the largest frame size
available.  Get rid of an annoying print of the WM_DEVICECHANGE message
(DPRINTF is a misnomer).  Update sound devices on next access after a
WM_DEVICECHANGE message.



From no-reply at appveyor.com  Mon Mar  1 01:37:28 2021
From: no-reply at appveyor.com (AppVeyor)
Date: Mon, 01 Mar 2021 01:37:28 +0000
Subject: [Vm-dev] Build failed: opensmalltalk-vm 1.0.2408
Message-ID: <20210301013728.1.041C2F4A7E14688C@appveyor.com>

An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210301/697ba1e0/attachment.html>

From stes at telenet.be  Mon Mar  1 13:45:38 2021
From: stes at telenet.be (stes@PANDORA.BE)
Date: Mon, 1 Mar 2021 14:45:38 +0100 (CET)
Subject: [Vm-dev] unix SqueakSource socket 'too many open files' problem
Message-ID: <1062729404.50152194.1614606338459.JavaMail.zimbra@telenet.be>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256


One thing to check is also 

	ulimit -n

the numbber of open files that the shell reports.

You could change it with ulimit -n -S <n> where <n> is smaller than the 
current limit.

I'm not sure whether it matters for sockets.

But it may be the case that some software
was installed on your machine that perhaps changed the profile, 
and that change the ulimit -n setting.

David Lewis wrote:

> Tim, one other thing to look out for is the event handling in the VM.
> Since last October it now uses the Linux-specific epoll mechanism.
> If the problem you see now started happening after updating to a
> newer VM, then consider this as a possible suspect. 

Linux epoll originated on Linux but there exist implementations on other
Operating Systems it seems.

OpenIndiana has epoll and other facilities like Berkeley BSD kqueue.

https://illumos.org/man/5/epoll

When running the Squeak opensmalltalk configure script it says for me

$ grep HAVE_EPOLL config.h 

#define HAVE_EPOLL 1
#define HAVE_EPOLL_PWAIT 1

So basically the configure script logic detects on OpenIndiana <sys/epoll.h>
and enabled it.

This is because the ax_have_epoll.m4 M4 file from autoconf was used:

https://www.gnu.org/software/autoconf-archive/ax_have_epoll.html

If I configure ax_cv_have_epoll=no ax_cv_have_epoll_pwait=no

then

$ grep EPOLL config.h 
/* #undef HAVE_EPOLL */
/* #undef HAVE_EPOLL_PWAIT */

So basically when epoll support was introduced,
it automatically switched on OpenIndiana to epoll.

So you could try as an experiment build the VM with

   ax_cv_have_epoll=no ax_cv_have_epoll_pwait=no

The way to do that would be to edit the "mvm" script and add the above
to the configure command line so that git diff reports something like:

- -       CFLAGS="$OPT"
+       CFLAGS="$OPT"\
+       ax_cv_have_epoll=no ax_cv_have_epoll_pwait=no

I am not sure whether it is the right way to disable epoll.

Perhaps a configure --without-epoll exists but I don't see it.

Also an issue is , how to see how the VM was built ?

In my case, by default on OpenIndiana epoll seems to be enabled now.

$ strings squeak | grep aioEnable
aioEnable(%d, %p, %d): IGNORED
aioEnable(%d): fd too large
aioEnable: descriptor %d already enabled
aioEnable(%d, %p, %d): allocating new epoll_event
aioEnable: malloc event
aioEnable(%d, %p, %d): allocating new epollEventData
...
aioEnable(%d, %p, %d): enable non-blocking io and sigio

... but when I compile with ax_cv_have_epoll=no ax_cv_have_epoll_pwait=no

then

$ strings squeak  | grep aioEnable
aioEnable(%d, %p, %d): IGNORED
aioEnable(%d): fd too large
aioEnable: descriptor %d already enabled
aioEnable(%d): external

To return to the Linux case, it could be a good experiment to build the VM
without epoll if that is supported and see if it makes a difference,
either by using the method described or by the recommended method to do it
(as the method I describe may not be the correct one).

Finally as David T. Lewis also indicates the problem may have existed 
in the past before epoll, so there is of course not really an indication
that epoll is involved here.

Basically as I read my interpretation of what David T. Lewis wrote is that
file descriptor socket leaks have always been a problem - with/without epoll.

Regards,
David Stes

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJgPO/GAAoJEAwpOKXMq1MaCUUIAK0d//me/vV4uuICELaOnxEs
zCqHtdb/8J8lI4GhvBW/oniSanyduFyzCxe7powS+5m4L/ons7U/2wmQXS2w3Vcy
gY9pbXm4m5v9xIBUC0oD/REqhTo0epeem8Q/2GN6bvHPBjTt+BWVh3Z42l6HN/TB
LXFfnXPck5buU2Lzo27brXE2XAweZx2JKdjhxB+YGOzsiwjWECqiy+zDXwXi2uN7
KG2ijFdQ6QTd5TaeQWgLhpFT/gnGX1ss8KPlkeKdcE7SH+EM0zPiie50bulUaJ7Q
viLM3pT7+K5U9NZ7FXbkhACg4MR8MPz0llHC4zUznqwiaNv2XEwf4Tk+Zy+rnhs=
=7R9P
-----END PGP SIGNATURE-----

From stes at telenet.be  Mon Mar  1 14:29:44 2021
From: stes at telenet.be (stes@PANDORA.BE)
Date: Mon, 1 Mar 2021 15:29:44 +0100 (CET)
Subject: [Vm-dev] aioDisable: epoll_ctl: Bad file descriptor
Message-ID: <1679457806.50325514.1614608984251.JavaMail.zimbra@telenet.be>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256


Hi,

I tried building opensmalltalk on Linux and for me when I launch the VM,
then quit it, I get:

aioDisable: epoll_ctl: Bad file descriptor

I modified the mvm script build.linux64x64/squeak.cog.spur/build/mvm
and rebuilt with

	ax_cv_have_epoll=no ax_cv_have_epoll_pwait=no

that works and when I then launch and quit the VM the 

aioDisable: epoll_ctl: Bad file descriptor

is not printed.

Both OpenIndiana and Linux have epoll(7) but I'm unsure whether it
is desired to disable epoll or to enable epoll.

Thanks,
David Stes

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJgPPoIAAoJEAwpOKXMq1MaogkH/1hdgrLx0HZ7VObA9J4vemoM
FUNR4MPKxttk0sN2Pwd84QPRv5aEbbQePzg6E/+qziZdrXmdPZOwpyvP14n8CzeK
u44HBkB6/189FCt/wo1RfVgpxkrh39dyQb2kiWQPEJUSNTIU0u8t9G1PQ2CMpkdo
V1njrJX54Xx/KK5msByB/xATbVZUcFRSmWhA9w0UvmJjAsSqtSOIKqdwCUWymsTH
zOuaCDd358lagogS9WmTi9CqEsY/3OKE27CTeXQgY8/r9VbBA1DJ9HiYuxyJ0mmn
YGAK6+SivkkT0bNlNi/R8XMt52a1FgK/8xZ0jF7js9vpgMjGVcLrZKsekqyGMGM=
=cd3I
-----END PGP SIGNATURE-----

From stes at telenet.be  Mon Mar  1 17:20:58 2021
From: stes at telenet.be (stes@PANDORA.BE)
Date: Mon, 1 Mar 2021 18:20:58 +0100 (CET)
Subject: [Vm-dev] regression: TestObjectsAsMethods crash in 4.19.5 (not
 in 4.16.7)
In-Reply-To: <605876499.30501015.1614071179388.JavaMail.zimbra@telenet.be>
References: <949389016.25753821.1613919758428.JavaMail.zimbra@telenet.be>
 <605876499.30501015.1614071179388.JavaMail.zimbra@telenet.be>
Message-ID: <766189717.50931693.1614619258680.JavaMail.zimbra@telenet.be>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256


> Install primitiveArrayBecomeOneWayNoCopyHash as primitive 248
> replacing obsolete primitiveInvokeObjectAsMethod. 

Any news on this Tests-ObjectsAsMethods issue ?

The purpose of those tests are not clear to me;

Perhaps the change for the "primitives" which was made, can be reverted ?

Possibly that creates a new problem for primitiveArrayBecomeOneWayNoCopyHash.

David Stes

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJgPSIjAAoJEAwpOKXMq1MaKBoIALZFV2hJydzkLUcLB1YDIS5D
7M3E/ucLxvfWoCBygz0/r/TLou2Sk57x9WpGTWpBkMwOlQiPfL5q10Zq5iOZ+qDA
Zbdz3iUHEV8FlVTGoOD4HR+eoblfnNhE6KhhkKb5YqmIVkpXr1UNY5ao/MPOFYV0
s12lTGqznhoh5ZI+i2U+sTDJljNiJojxzvWdq5rJ658PISWFPuqw/YJamQZvKiWx
/HFQf+om5MKPt/jtzfpCxskuJ5evU2XBcTb6dFXzNgeaoqJEsLnnUj08vubHMA34
mRYHk+Pp4SxMYV/0CJ3dWOgOirMN7yVi7tTdKAdJeEmoPqP4cMUMbb8C+xOA5Uc=
=5Ffs
-----END PGP SIGNATURE-----

From noreply at github.com  Mon Mar  1 20:38:27 2021
From: noreply at github.com (Eliot Miranda)
Date: Mon, 01 Mar 2021 12:38:27 -0800
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] 61876d: Disable the
 Newspeak CI builds after conversation ...
Message-ID: <OpenSmalltalk/opensmalltalk-vm/push/refs/heads/Cog/a69b4c-61876d@github.com>

  Branch: refs/heads/Cog
  Home:   https://github.com/OpenSmalltalk/opensmalltalk-vm
  Commit: 61876d5cc401968af2524c8d49f95591d0f2c0a5
      https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/61876d5cc401968af2524c8d49f95591d0f2c0a5
  Author: Eliot Miranda <eliot.miranda at gmail.com>
  Date:   2021-03-01 (Mon, 01 Mar 2021)

  Changed paths:
    M .appveyor.yml
    M .travis.yml

  Log Message:
  -----------
  Disable the Newspeak CI builds after conversation with Gilad.



From no-reply at appveyor.com  Mon Mar  1 20:57:00 2021
From: no-reply at appveyor.com (AppVeyor)
Date: Mon, 01 Mar 2021 20:57:00 +0000
Subject: [Vm-dev] Build failed: opensmalltalk-vm 1.0.2409
Message-ID: <20210301205700.1.652B4E1A12255DEC@appveyor.com>

An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210301/2a19a9e4/attachment.html>

From lewis at mail.msen.com  Mon Mar  1 23:39:19 2021
From: lewis at mail.msen.com (David T. Lewis)
Date: Mon, 1 Mar 2021 18:39:19 -0500
Subject: [Vm-dev] aioDisable: epoll_ctl: Bad file descriptor
In-Reply-To: <1679457806.50325514.1614608984251.JavaMail.zimbra@telenet.be>
References: <1679457806.50325514.1614608984251.JavaMail.zimbra@telenet.be>
Message-ID: <20210301233919.GA73711@shell.msen.com>

A fix is here in branch 'dtl/epoll-forksqueak':

  https://github.com/OpenSmalltalk/opensmalltalk-vm/tree/dtl/epoll-forksqueak


I'm fairly sure these fixes are correct, but I don't want to merge to
the Cog branch without review.

I can't build the tip of Cog on my Ubuntu box right now due to an
unrelated issue with configuring header files, but if someone else
could try merging 'dtl/epoll-forksqueak' into branch 'Cog' to confirm
no issues, that would be great.

Dave



On Mon, Mar 01, 2021 at 03:29:44PM +0100, stes at PANDORA.BE wrote:
>  
> 
> Hi,
> 
> I tried building opensmalltalk on Linux and for me when I launch the VM,
> then quit it, I get:
> 
> aioDisable: epoll_ctl: Bad file descriptor
> 
> I modified the mvm script build.linux64x64/squeak.cog.spur/build/mvm
> and rebuilt with
> 
> 	ax_cv_have_epoll=no ax_cv_have_epoll_pwait=no
> 
> that works and when I then launch and quit the VM the 
> 
> aioDisable: epoll_ctl: Bad file descriptor
> 
> is not printed.
> 
> Both OpenIndiana and Linux have epoll(7) but I'm unsure whether it
> is desired to disable epoll or to enable epoll.
> 
> Thanks,
> David Stes
> 

From lewis at mail.msen.com  Tue Mar  2 00:04:10 2021
From: lewis at mail.msen.com (David T. Lewis)
Date: Mon, 1 Mar 2021 19:04:10 -0500
Subject: [Vm-dev] regression: TestObjectsAsMethods crash in 4.19.5 (not
	in 4.16.7)
In-Reply-To: <766189717.50931693.1614619258680.JavaMail.zimbra@telenet.be>
References: <949389016.25753821.1613919758428.JavaMail.zimbra@telenet.be>
 <605876499.30501015.1614071179388.JavaMail.zimbra@telenet.be>
 <766189717.50931693.1614619258680.JavaMail.zimbra@telenet.be>
Message-ID: <20210302000410.GA79682@shell.msen.com>

On Mon, Mar 01, 2021 at 06:20:58PM +0100, stes at PANDORA.BE wrote:
>  
> > Install primitiveArrayBecomeOneWayNoCopyHash as primitive 248
> > replacing obsolete primitiveInvokeObjectAsMethod. 
> 
> Any news on this Tests-ObjectsAsMethods issue ?
> 
> The purpose of those tests are not clear to me;
> 
> Perhaps the change for the "primitives" which was made, can be reverted ?
> 
> Possibly that creates a new problem for primitiveArrayBecomeOneWayNoCopyHash.
> 
> David Stes
>

You can consider this a bug in the most recent interpreter VMs. I had
failed to notice that there is a hard coded reference to that primitive
number within the VM itself, and this reference is now resulting in
invocation of the wrong function.

A trivial fix is to rearrange some primtive number assignments, but the
correct thing to do is adopt the logic that Eliot uses in oscog so that
the function is not invoked as a "primitive" at all.

I'll try to follow up on this next week.

Dave

From noreply at github.com  Tue Mar  2 00:31:34 2021
From: noreply at github.com (Eliot Miranda)
Date: Mon, 01 Mar 2021 16:31:34 -0800
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] 15fdea: Update from c++11
 to c++14 on MacOS builds. Affec...
Message-ID: <OpenSmalltalk/opensmalltalk-vm/push/refs/heads/Cog/61876d-15fdea@github.com>

  Branch: refs/heads/Cog
  Home:   https://github.com/OpenSmalltalk/opensmalltalk-vm
  Commit: 15fdeaa142257eeb6bff3b41014f2f407420a70d
      https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/15fdeaa142257eeb6bff3b41014f2f407420a70d
  Author: Eliot Miranda <eliot.miranda at gmail.com>
  Date:   2021-03-01 (Mon, 01 Mar 2021)

  Changed paths:
    M build.macos32x86/common/Makefile.rules
    M build.macos64ARMv8/common/Makefile.rules
    M build.macos64x64/common/Makefile.rules

  Log Message:
  -----------
  Update from c++11 to c++14 on MacOS builds.  Affects only Bochs simulators
and Terf plugins, hence [ci skip]



From tim at rowledge.org  Tue Mar  2 04:08:31 2021
From: tim at rowledge.org (tim Rowledge)
Date: Mon, 1 Mar 2021 20:08:31 -0800
Subject: [Vm-dev] unix SqueakSource socket 'too many open files' problem
In-Reply-To: <1afb3b3d-82b7-413e-5f5c-2fa155cbc3b9@gmail.com>
References: <0E53D947-69DB-46D1-906F-E173B1FD3661@rowledge.org>
 <20210225211202.GA55616@shell.msen.com>
 <237F031E-D0DC-4CAE-ADB9-1D3BF5A93CB0@rowledge.org>
 <20210225231714.GA78767@shell.msen.com>
 <B6E27A9D-0C4E-4167-83D7-61003B407EFA@rowledge.org>
 <41B85629-A9EF-48E7-807B-3EA8CDD6652B@rowledge.org>
 <20210226211244.GA85415@shell.msen.com>
 <1afb3b3d-82b7-413e-5f5c-2fa155cbc3b9@gmail.com>
Message-ID: <9899BB21-3B3F-4315-963A-0BD4A2E939CC@rowledge.org>



> On 2021-02-26, at 10:32 PM, K K Subbu <kksubbu.ml at gmail.com> wrote:
> 
> On 27/02/21 2:42 am, David T. Lewis wrote:
>> Here is what I saw on the squeaksource.com server a few minutes ago:
>>   squeaksourcecom at dan:~$ ls -l /proc/16147/fd
> 
> The socket count can be generated directly with:
> 
> $ ls -l /proc/$(pgrep squeak)/fd | grep -c socket:

That's pretty neat, thanks.

> 
> This can then be visually graphed every, say, 3600s with:
> 
> bars='======================================================+';
> while true; do
> 	echo $bars | cut -c 1-$(ls -l /proc/$(pgrep squeak)/fd | grep -c socket:)
> 	sleep 3600
> done
> 

Goodness; shell script bargraphs. Whatever next?


> If this shows a monotonic growth, then a leak can be suspected.

I'm pretty sure there is a slow leak; but slow. It's climbed from 19 to 30 over three days


tim
--
tim Rowledge; tim at rowledge.org; 
"You're a ghost driving a meat-coated skeleton made from stardust, 
riding a rock, hurtling through space. Fear nothing"


From kksubbu.ml at gmail.com  Tue Mar  2 12:18:25 2021
From: kksubbu.ml at gmail.com (K K Subbu)
Date: Tue, 2 Mar 2021 17:48:25 +0530
Subject: [Vm-dev] unix SqueakSource socket 'too many open files' problem
In-Reply-To: <9899BB21-3B3F-4315-963A-0BD4A2E939CC@rowledge.org>
References: <0E53D947-69DB-46D1-906F-E173B1FD3661@rowledge.org>
 <20210225211202.GA55616@shell.msen.com>
 <237F031E-D0DC-4CAE-ADB9-1D3BF5A93CB0@rowledge.org>
 <20210225231714.GA78767@shell.msen.com>
 <B6E27A9D-0C4E-4167-83D7-61003B407EFA@rowledge.org>
 <41B85629-A9EF-48E7-807B-3EA8CDD6652B@rowledge.org>
 <20210226211244.GA85415@shell.msen.com>
 <1afb3b3d-82b7-413e-5f5c-2fa155cbc3b9@gmail.com>
 <9899BB21-3B3F-4315-963A-0BD4A2E939CC@rowledge.org>
Message-ID: <96b07b8d-ed38-5343-2a71-4d7b79243033@gmail.com>

On 02/03/21 9:38 am, tim Rowledge wrote:
>> If this shows a monotonic growth, then a leak can be suspected.
> I'm pretty sure there is a slow leak; but slow. It's climbed from 19 to 30 over three days
Tim,

Ugh! slow leaks can be really annoying to diagnose. You may narrow down 
the specific leak with ss (socket stats)

$ ss -s

to get a breakdown by tcp/udp ipv4/ipv6 etc. or

$ ss state close-wait

to catch all sockets in close-wait state. The arguments is a filter 
condition and the man page gives examples of various filter expressions.

You may also use the watch command to sample it every day (-n 86400) and 
highlight the difference (-d):

$ watch -n 86400 -d ss state close-wait

BTW, an early trick I learned is to keep a small terminal running such 
commands in a corner (or second monitor these days). This lets me use 
the center of the screen for all active work. Any gross changes in shape 
(char graphs ;-)) or highlight will catch the corner of the eye.

HTH .. Subbu

From stes at telenet.be  Wed Mar  3 15:50:29 2021
From: stes at telenet.be (stes@PANDORA.BE)
Date: Wed, 3 Mar 2021 16:50:29 +0100 (CET)
Subject: [Vm-dev] aioDisable: epoll_ctl: Bad file descriptor
In-Reply-To: <1679457806.50325514.1614608984251.JavaMail.zimbra@telenet.be>
References: <1679457806.50325514.1614608984251.JavaMail.zimbra@telenet.be>
Message-ID: <1232427951.2281475.1614786629222.JavaMail.zimbra@telenet.be>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256


Hi,

Good news; I tried your changes related to epoll(7) support,
and in to my limited testing so far, it seems an improvement.

I tried on my Linux test server:

$ git branch
* (HEAD detached at origin/dtl/epoll-forksqueak)
  Cog

I built on Linux from that epoll-forksqueak branch, 

$ grep EPOLL config.h 
#define HAVE_EPOLL 1
#define HAVE_EPOLL_PWAIT 1

now for both the Cog and Stack Spur VM, when I launch it on Linux,
then quit it, I do *not* get the :

aioDisable: epoll_ctl: Bad file descriptor

However the odd thing now is that if I try the main Cog branch on Linux,
I'm not getting the error any longer, so it seems to be gone.

I may by coincidence have seen the message, while it may be
perhaps not 100% reproducible but happening only occasionaly.

Anyway when on OpenIndiana (not Linux), I tried:

 configure

configure decides for me that it wants to enable epoll on OpenIndiana:

$ uname
SunOS

$ grep EPOLL config.h 
#define HAVE_EPOLL 1
#define HAVE_EPOLL_PWAIT 1

so also on OpenIndiana (which is in the SunOS family) configure enables epoll.

$ git branch
* (HEAD detached at origin/dtl/epoll-forksqueak)
  Cog

so I compiled that version on OpenIndiana (with epoll as set by configure).

When I launch it, do something (open some browser), then quit it,
no error.

I tried sqstkspur64 and sqcogspur64.

Actually on OpenIndiana I have so far never seen the

aioDisable: epoll_ctl: Bad file descriptor

for the time epoll was enabled (a few months ago now).

But I have a feeling that these issues can be quite complex and subtle,
and that the issue may only be happening on some occasions.

In any case as far as I can see, your 'branch' seems an improvement.

David Stes

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJgP7AKAAoJEAwpOKXMq1MaJXYH/ijJkk5Qg5K6gfC8C5F36kZp
EYr4yCfpibaYUgjg206MV9yN/RHUPnl+cnZtKzyv9xPgkdfDjZfA+V1IDahXnH/O
BmJNR3Y3bpJasDzlaTp/cCS9wJFrM54v1o92ZTERe9xIfGufTZSTLxRos/nh0gNy
a6jwj7GYnhORykXJAIrMCDA3Mnu9ziYPN5O/ycYfp2AfoLvQZHb8qsX5W4AD8JB+
0kIsnrLfShyTC8M8m3tWy6r2fKeMQc5FH8IK0hxZCqe19RSRnrBo+RhhP9GBV7sX
B6ZUNNGsoyE/CDgUK3l5yn5E7d7/Q2LixmqYcmAlbE96q79+hKXYVivvBTTM+2I=
=hhf5
-----END PGP SIGNATURE-----

From noreply at github.com  Thu Mar  4 20:42:18 2021
From: noreply at github.com (Eliot Miranda)
Date: Thu, 04 Mar 2021 12:42:18 -0800
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] 04af39: C reserved words
 if, while, switch are not function ...
Message-ID: <OpenSmalltalk/opensmalltalk-vm/push/refs/heads/Cog/15fdea-04af39@github.com>

  Branch: refs/heads/Cog
  Home:   https://github.com/OpenSmalltalk/opensmalltalk-vm
  Commit: 04af39a36e2bea3ffa20234d2ba5c983205bc759
      https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/04af39a36e2bea3ffa20234d2ba5c983205bc759
  Author: Eliot Miranda <eliot.miranda at gmail.com>
  Date:   2021-03-04 (Thu, 04 Mar 2021)

  Changed paths:
    M platforms/win32/vm/sqWin32.h
    M platforms/win32/vm/sqWin32Alloc.c
    M platforms/win32/vm/sqWin32DirectInput.c
    M platforms/win32/vm/sqWin32DnsInfo.c
    M platforms/win32/vm/sqWin32HandleTable.h
    M platforms/win32/vm/sqWin32Heartbeat.c
    M platforms/win32/vm/sqWin32Main.c
    M platforms/win32/vm/sqWin32Prefs.c
    M platforms/win32/vm/sqWin32Service.c
    M platforms/win32/vm/sqWin32Time.c
    M platforms/win32/vm/sqWin32Window.c

  Log Message:
  -----------
  C reserved words if,while,switch are not function calls. Hence always put a space
before the following parenthesis.



From no-reply at appveyor.com  Thu Mar  4 21:01:32 2021
From: no-reply at appveyor.com (AppVeyor)
Date: Thu, 04 Mar 2021 21:01:32 +0000
Subject: [Vm-dev] Build failed: opensmalltalk-vm 1.0.2410
Message-ID: <20210304210132.1.4841081B85528211@appveyor.com>

An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210304/59a05b10/attachment.html>

From eliot.miranda at gmail.com  Fri Mar  5 03:00:57 2021
From: eliot.miranda at gmail.com (Eliot Miranda)
Date: Thu, 4 Mar 2021 19:00:57 -0800
Subject: [Vm-dev] Removing the old pre-event-driven input polling input
	primitives
Message-ID: <CAC20JE0dxGH_dfnghOdCGa6P9Dw2bBKAtiiTKt0gYZjS4hNqzg@mail.gmail.com>

Hi all,

    I'm just checking.  It seems to me that primitives 90, 107, &
108, oldPrimMousePt, oldPrimMouseButtons, & oldPrimKbdNext, have been
obsolete for a very long time.  Am I correct? Is there any possibility
these could be relevant to a Cog VM, that is any image from 4.0 on?  WOuld
removing the support for these primitives form the VM potentially impact
porting old Etoys projects?  Any other potential impacts?

I'd like to nuke the code to ease extending the VM to support modern mice,
which have many more buttons available, because it makes things
unnecessarily complicated.
_,,,^..^,,,_
best, Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210304/ea974248/attachment.html>

From commits at source.squeak.org  Fri Mar  5 20:30:59 2021
From: commits at source.squeak.org (commits at source.squeak.org)
Date: Fri, 5 Mar 2021 20:30:59 0000
Subject: [Vm-dev] VM Maker: VMMaker.oscog-eem.2946.mcz
Message-ID: <E1lIH6f-0002SW-So@andreas>

Eliot Miranda uploaded a new version of VMMaker to project VM Maker:
http://source.squeak.org/VMMaker/VMMaker.oscog-eem.2946.mcz

==================== Summary ====================

Name: VMMaker.oscog-eem.2946
Author: eem
Time: 5 March 2021, 12:30:49.883238 pm
UUID: 461911e1-d450-4f05-8a40-7b0470487041
Ancestors: VMMaker.oscog-eem.2945

Nuke the obsolete polling input primitives primitiveMouseButtons, primitiveKbdNext et al.  These really are irrelevant to all images that run on Cog VMs.
Fix a couple of compiler warnings.

=============== Diff against VMMaker.oscog-eem.2945 ===============

Item was changed:
  ----- Method: CogMethodZone>>cogMethodContaining: (in category 'jit - api') -----
  cogMethodContaining: mcpc
  	"Answer the method containing mcpc for the purposes of code zone compaction,
  	 where mcpc is actually the value of instructionPointer at the time of a compaction."
  	<var: 'mcpc' type: #usqInt>
  	<api>
  	| cogMethod prevMethod |
  	mcpc > limitAddress ifTrue:
  		[^nil].
  	mcpc < baseAddress ifTrue:
  		[cogit assertMcpcIsPrimReturn: mcpc.
  		 ^nil].
+ 	self assert: mcpc < self freeStart.
- 	self assert: mcpc < self limitZony.
  	cogMethod := coInterpreter cCoerceSimple: baseAddress to: #'CogMethod *'.
  	[cogMethod asUnsignedInteger < mcpc] whileTrue:
  		[prevMethod := cogMethod.
  		 cogMethod := self methodAfter: cogMethod].
  
  	"Since mcpc is actually instructionPointer we expect that it is either at the stack check
  	 (normal code zone reclamation invoked through checkForEventsMayContextSwitch:)
  	 or is in a primitive, immediately following the call of the C primitive routine."
  	self assert: (prevMethod notNil
  				and: [mcpc = (prevMethod asUnsignedInteger + prevMethod stackCheckOffset)
  					or: [(cogit backEnd isCallPrecedingReturnPC: mcpc)
  						and: [(coInterpreter
  									primitiveIndexOfMethod: prevMethod methodObject
  									header: prevMethod methodHeader) > 0
  							or: [(cogit backEnd callTargetFromReturnAddress: mcpc) = cogit ceCheckForInterruptTrampoline]]]]).
  	 ^prevMethod!

Item was changed:
  ----- Method: InterpreterPrimitives>>primitiveKbdNext (in category 'I/O primitives') -----
  primitiveKbdNext
  	"Obsolete on virtually all platforms; old style input polling code.
  	Return the next keycode and remove it from the input buffer. The low byte is the 8-bit ISO character. The next four bits are the Smalltalk modifier bits <cmd><option><ctrl><shift>."
+ 	<option: #Legacy>
- 
  	| keystrokeWord |
  	self pop: 1.
  	keystrokeWord := self ioGetKeystroke.
  	keystrokeWord >= 0
  		ifTrue: [self pushInteger: keystrokeWord]
  		ifFalse: [self push: objectMemory nilObject].!

Item was changed:
  ----- Method: InterpreterPrimitives>>primitiveKbdPeek (in category 'I/O primitives') -----
  primitiveKbdPeek
  	"Obsolete on virtually all platforms; old style input polling code.
  	Return the next keycode and without removing it from the input buffer. The low byte is the 8-bit ISO character. The next four bits are the Smalltalk modifier bits <cmd><option><ctrl><shift>."
+ 	<option: #Legacy>
- 
  	| keystrokeWord |
  	self pop: 1.
  	keystrokeWord := self ioPeekKeystroke.
  	keystrokeWord >= 0
  		ifTrue: [self pushInteger: keystrokeWord]
  		ifFalse: [self push: objectMemory nilObject].!

Item was changed:
  ----- Method: InterpreterPrimitives>>primitiveMouseButtons (in category 'I/O primitives') -----
  primitiveMouseButtons
  	"Obsolete on virtually all platforms; old style input polling code.
  	Return the mouse button state. The low three bits encode the state of the <red><yellow><blue> mouse buttons. The next four bits encode the Smalltalk modifier bits <cmd><option><ctrl><shift>."
+ 	<option: #Legacy>
- 
  	| buttonWord |
  	self pop: 1.
  	buttonWord := self ioGetButtonState.
  	self pushInteger: buttonWord.!

Item was changed:
  ----- Method: InterpreterPrimitives>>primitiveMousePoint (in category 'I/O primitives') -----
  primitiveMousePoint
  	"Obsolete on virtually all platforms; old style input polling code.
  	Return a Point indicating current position of the mouse. Note that mouse coordinates may be negative if the mouse moves above or to the left of the top-left corner of the Smalltalk window."
+ 	<option: #Legacy>
- 
  	| pointWord x y |
  	pointWord := self ioMousePoint.
  	x := self signExtend16: ((pointWord >> 16) bitAnd: 16rFFFF).
  	y := self signExtend16: (pointWord bitAnd: 16rFFFF).
  	self pop: 1 thenPush: (self makePointwithxValue: x  yValue: y)!

Item was changed:
  ----- Method: InterpreterPrimitives>>traceInputEvent: (in category 'I/O primitive support') -----
  traceInputEvent: evtBuf
  	<var: #evtBuf declareC: 'sqIntptr_t evtBuf[8]'>
  	| eventTypeNames |
  	eventTypeNames := self
  							cCoerce: #('None' 'Mouse' 'Keyboard' 'DragDropFiles' 'Menu' 'Window' 'Complex' 'MouseWheel' 'Plugin')
  							to: #'char **'.
  	'Event%s/%d @ %u\t\t%d/%x %d/%x\n\t%d/%x %d/%x\t %d/%x %p\n'
  		f: #stderr
  		printf: {	((evtBuf at: 0) between: 0 and: 8)
  						ifTrue: [eventTypeNames at: (evtBuf at: 0)]
  						ifFalse: ['?'].
  					evtBuf at: 0.
  					evtBuf at: 1. "timestamp"
  					evtBuf at: 2. evtBuf at: 2. evtBuf at: 3. evtBuf at: 3.
  					evtBuf at: 4. evtBuf at: 4. evtBuf at: 5. evtBuf at: 5.
  					evtBuf at: 6. evtBuf at: 6.
+ 					(evtBuf at: 7) asVoidPointer } "windowIndex"!
- 					evtBuf at: 7 } "windowIndex"!

Item was changed:
  ----- Method: StackInterpreter class>>initializePrimitiveTable (in category 'initialization') -----
(excessive size, no diff calculated)


From noreply at github.com  Fri Mar  5 21:25:09 2021
From: noreply at github.com (Eliot Miranda)
Date: Fri, 05 Mar 2021 13:25:09 -0800
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] 924a24: CogVM source as
 per VMMaker.oscog-eem.2946
Message-ID: <OpenSmalltalk/opensmalltalk-vm/push/refs/heads/Cog/04af39-924a24@github.com>

  Branch: refs/heads/Cog
  Home:   https://github.com/OpenSmalltalk/opensmalltalk-vm
  Commit: 924a24bb54870a621c5283f23631261d5a792000
      https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/924a24bb54870a621c5283f23631261d5a792000
  Author: Eliot Miranda <eliot.miranda at gmail.com>
  Date:   2021-03-05 (Fri, 05 Mar 2021)

  Changed paths:
    M nsspur64src/vm/cogit.h
    M nsspur64src/vm/cogitARMv8.c
    M nsspur64src/vm/cogitX64SysV.c
    M nsspur64src/vm/cogitX64WIN64.c
    M nsspur64src/vm/cointerp.c
    M nsspur64src/vm/cointerp.h
    M nsspur64src/vm/gcc3x-cointerp.c
    M nsspursrc/vm/cogit.h
    M nsspursrc/vm/cogitARMv5.c
    M nsspursrc/vm/cogitIA32.c
    M nsspursrc/vm/cogitMIPSEL.c
    M nsspursrc/vm/cointerp.c
    M nsspursrc/vm/cointerp.h
    M nsspursrc/vm/gcc3x-cointerp.c
    M nsspurstack64src/vm/gcc3x-interp.c
    M nsspurstack64src/vm/interp.c
    M nsspurstacksrc/vm/gcc3x-interp.c
    M nsspurstacksrc/vm/interp.c
    M platforms/Cross/vm/sq.h
    M platforms/Mac OS/vm/Developer/sqMacMinimal.c
    M platforms/Mac OS/vm/sqMacNSPluginUILogic.c
    M platforms/Mac OS/vm/sqMacUIEvents.c
    M platforms/Mac OS/vm/sqMacUIEventsUniversal.c
    M platforms/Plan9/vm/sqPlan9io.c
    M platforms/RiscOS/vm/sqRPCEvents.c
    M platforms/iOS/vm/OSX/macintoshosxextra.c
    M platforms/iOS/vm/iPhone/macintoshextra.c
    M platforms/minheadless/common/sqWindow-Dispatch.c
    M platforms/minheadless/common/sqWindow-Null.c
    M platforms/minheadless/common/sqWindow.h
    M platforms/minheadless/sdl2-window/sqWindow-SDL2.c
    M platforms/unix/vm-display-null/sqUnixDisplayNull.c
    M platforms/unix/vm/SqDisplay.h
    R platforms/unix/vm/sqUnixEvent.c
    M platforms/unix/vm/sqUnixMain.c
    M platforms/win32/vm/sqWin32Window.c
    M spur64src/vm/cogit.h
    M spur64src/vm/cogitARMv8.c
    M spur64src/vm/cogitX64SysV.c
    M spur64src/vm/cogitX64WIN64.c
    M spur64src/vm/cointerp.c
    M spur64src/vm/cointerp.h
    M spur64src/vm/cointerpmt.c
    M spur64src/vm/cointerpmt.h
    M spur64src/vm/gcc3x-cointerp.c
    M spur64src/vm/gcc3x-cointerpmt.c
    M spurlowcode64src/vm/cogit.h
    M spurlowcode64src/vm/cogitARMv8.c
    M spurlowcode64src/vm/cogitX64SysV.c
    M spurlowcode64src/vm/cogitX64WIN64.c
    M spurlowcode64src/vm/cointerp.c
    M spurlowcode64src/vm/cointerp.h
    M spurlowcode64src/vm/gcc3x-cointerp.c
    M spurlowcodesrc/vm/cogit.h
    M spurlowcodesrc/vm/cogitARMv5.c
    M spurlowcodesrc/vm/cogitIA32.c
    M spurlowcodesrc/vm/cogitMIPSEL.c
    M spurlowcodesrc/vm/cointerp.c
    M spurlowcodesrc/vm/cointerp.h
    M spurlowcodesrc/vm/gcc3x-cointerp.c
    M spurlowcodestack64src/vm/gcc3x-interp.c
    M spurlowcodestack64src/vm/interp.c
    M spurlowcodestacksrc/vm/gcc3x-interp.c
    M spurlowcodestacksrc/vm/interp.c
    M spursista64src/vm/cogit.h
    M spursista64src/vm/cogitARMv8.c
    M spursista64src/vm/cogitX64SysV.c
    M spursista64src/vm/cogitX64WIN64.c
    M spursista64src/vm/cointerp.c
    M spursista64src/vm/cointerp.h
    M spursista64src/vm/gcc3x-cointerp.c
    M spursistasrc/vm/cogit.h
    M spursistasrc/vm/cogitARMv5.c
    M spursistasrc/vm/cogitIA32.c
    M spursistasrc/vm/cogitMIPSEL.c
    M spursistasrc/vm/cointerp.c
    M spursistasrc/vm/cointerp.h
    M spursistasrc/vm/gcc3x-cointerp.c
    M spursrc/vm/cogit.h
    M spursrc/vm/cogitARMv5.c
    M spursrc/vm/cogitIA32.c
    M spursrc/vm/cogitMIPSEL.c
    M spursrc/vm/cointerp.c
    M spursrc/vm/cointerp.h
    M spursrc/vm/cointerpmt.c
    M spursrc/vm/cointerpmt.h
    M spursrc/vm/gcc3x-cointerp.c
    M spursrc/vm/gcc3x-cointerpmt.c
    M spurstack64src/vm/gcc3x-interp.c
    M spurstack64src/vm/interp.c
    M spurstack64src/vm/validImage.c
    M spurstacksrc/vm/gcc3x-interp.c
    M spurstacksrc/vm/interp.c
    M spurstacksrc/vm/validImage.c
    M src/vm/cogit.h
    M src/vm/cogitARMv5.c
    M src/vm/cogitIA32.c
    M src/vm/cogitMIPSEL.c
    M src/vm/cointerp.c
    M src/vm/cointerp.h
    M src/vm/gcc3x-cointerp.c
    M stacksrc/vm/gcc3x-interp.c
    M stacksrc/vm/interp.c

  Log Message:
  -----------
  CogVM source as per VMMaker.oscog-eem.2946

Nuke the obsolete polling input primitives primitiveMouseButtons,
primitiveKbdNext et al.  These really are irrelevant to all images
that run on Cog VMs (unused since about Squeak 3.6).

Fix a couple of compiler warnings.



From no-reply at appveyor.com  Fri Mar  5 21:45:11 2021
From: no-reply at appveyor.com (AppVeyor)
Date: Fri, 05 Mar 2021 21:45:11 +0000
Subject: [Vm-dev] Build failed: opensmalltalk-vm 1.0.2411
Message-ID: <20210305214511.1.5C2AD00090816989@appveyor.com>

An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210305/c09d43d1/attachment.html>

From notifications at github.com  Mon Mar  8 04:08:53 2021
From: notifications at github.com (Ronie Salgado)
Date: Sun, 07 Mar 2021 20:08:53 -0800
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] OS X custom URL scheme
 and document open event injection fixup (#553)
Message-ID: <OpenSmalltalk/opensmalltalk-vm/pull/553@github.com>

These change fixup the events for handling custom url schemes, and the support for opening document types registered in the plist that are received as events before the image is even opened by the VM. These event are injected as drop event into the image for further processing.
You can view, comment on, or merge this pull request online at:

  https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/553

-- Commit Summary --

  * I these changes perform minor refactoring on the place where the dragItems list  is store with the purpose of handling url schema and document open events that are raised before the application is even started.

-- File Changes --

    M platforms/iOS/vm/OSX/SqueakOSXAppDelegate.h (6)
    M platforms/iOS/vm/OSX/SqueakOSXAppDelegate.m (73)
    M platforms/iOS/vm/OSX/sqSqueakOSXApplication+events.m (4)
    M platforms/iOS/vm/OSX/sqSqueakOSXApplication.m (21)
    M platforms/iOS/vm/OSX/sqSqueakOSXCGView.h (1)
    M platforms/iOS/vm/OSX/sqSqueakOSXCGView.m (30)
    M platforms/iOS/vm/OSX/sqSqueakOSXDropAPI.m (8)
    M platforms/iOS/vm/OSX/sqSqueakOSXHeadlessView.h (1)
    M platforms/iOS/vm/OSX/sqSqueakOSXHeadlessView.m (3)
    M platforms/iOS/vm/OSX/sqSqueakOSXMetalView.h (1)
    M platforms/iOS/vm/OSX/sqSqueakOSXMetalView.m (29)
    M platforms/iOS/vm/OSX/sqSqueakOSXOpenGLView.h (1)
    M platforms/iOS/vm/OSX/sqSqueakOSXOpenGLView.m (30)
    M platforms/iOS/vm/OSX/sqSqueakOSXView.h (1)

-- Patch Links --

https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/553.patch
https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/553.diff

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/553
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210307/6e1688fa/attachment.html>

From vanessa at codefrau.net  Mon Mar  8 04:16:27 2021
From: vanessa at codefrau.net (Vanessa Freudenberg)
Date: Sun, 7 Mar 2021 20:16:27 -0800
Subject: [Vm-dev] [SqueakJS] Faster JIT ideas
Message-ID: <CAOWGXNCmyE1N9K87+jKiXM7awb7KbsQws7HKO3uAXAKV9PBc2Q@mail.gmail.com>

Hi all,

ideas for a faster SqueakJS JIT have been swirling around in my head for
many years. This weekend I took the time to write some of them down:

https://squeak.js.org/docs/jit.md.html

Feedback welcome!

Vanessa
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210307/47116af1/attachment.html>

From no-reply at appveyor.com  Mon Mar  8 04:26:47 2021
From: no-reply at appveyor.com (AppVeyor)
Date: Mon, 08 Mar 2021 04:26:47 +0000
Subject: [Vm-dev] Build failed: opensmalltalk-vm 1.0.2412
Message-ID: <20210308042647.1.005244C1D615C573@appveyor.com>

An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210308/dac0d775/attachment.html>

From notifications at github.com  Mon Mar  8 16:40:21 2021
From: notifications at github.com (Eliot Miranda)
Date: Mon, 08 Mar 2021 08:40:21 -0800
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] OS X custom URL
 scheme and document open event injection fixup (#553)
In-Reply-To: <OpenSmalltalk/opensmalltalk-vm/pull/553@github.com>
References: <OpenSmalltalk/opensmalltalk-vm/pull/553@github.com>
Message-ID: <OpenSmalltalk/opensmalltalk-vm/pull/553/review/606484543@github.com>

@eliotmiranda approved this pull request.

Best to get this in and fix any problems in situ.  I want this :-)



-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/553#pullrequestreview-606484543
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210308/85b00301/attachment.html>

From notifications at github.com  Mon Mar  8 16:41:37 2021
From: notifications at github.com (Eliot Miranda)
Date: Mon, 08 Mar 2021 08:41:37 -0800
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] OS X custom URL
 scheme and document open event injection fixup (#553)
In-Reply-To: <OpenSmalltalk/opensmalltalk-vm/pull/553@github.com>
References: <OpenSmalltalk/opensmalltalk-vm/pull/553@github.com>
Message-ID: <OpenSmalltalk/opensmalltalk-vm/pull/553/issue_event/4422276894@github.com>

Merged #553 into Cog.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/553#event-4422276894
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210308/d351f723/attachment-0001.html>

From noreply at github.com  Mon Mar  8 16:41:37 2021
From: noreply at github.com (Eliot Miranda)
Date: Mon, 08 Mar 2021 08:41:37 -0800
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] d10e5e: I these changes
 perform minor refactoring on the p...
Message-ID: <OpenSmalltalk/opensmalltalk-vm/push/refs/heads/Cog/924a24-119e76@github.com>

  Branch: refs/heads/Cog
  Home:   https://github.com/OpenSmalltalk/opensmalltalk-vm
  Commit: d10e5ea9732eacbf9261ebe1ec48627357a898c6
      https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/d10e5ea9732eacbf9261ebe1ec48627357a898c6
  Author: Ronie Salgado <roniesalg at gmail.com>
  Date:   2021-03-08 (Mon, 08 Mar 2021)

  Changed paths:
    M platforms/iOS/vm/OSX/SqueakOSXAppDelegate.h
    M platforms/iOS/vm/OSX/SqueakOSXAppDelegate.m
    M platforms/iOS/vm/OSX/sqSqueakOSXApplication+events.m
    M platforms/iOS/vm/OSX/sqSqueakOSXApplication.m
    M platforms/iOS/vm/OSX/sqSqueakOSXCGView.h
    M platforms/iOS/vm/OSX/sqSqueakOSXCGView.m
    M platforms/iOS/vm/OSX/sqSqueakOSXDropAPI.m
    M platforms/iOS/vm/OSX/sqSqueakOSXHeadlessView.h
    M platforms/iOS/vm/OSX/sqSqueakOSXHeadlessView.m
    M platforms/iOS/vm/OSX/sqSqueakOSXMetalView.h
    M platforms/iOS/vm/OSX/sqSqueakOSXMetalView.m
    M platforms/iOS/vm/OSX/sqSqueakOSXOpenGLView.h
    M platforms/iOS/vm/OSX/sqSqueakOSXOpenGLView.m
    M platforms/iOS/vm/OSX/sqSqueakOSXView.h

  Log Message:
  -----------
  I these changes perform minor refactoring on the place where the dragItems list  is store with the purpose of handling url schema and document open events that are raised before the application is even started.


  Commit: 119e76c5c467f7e61fef4f87f3b622923ac52151
      https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/119e76c5c467f7e61fef4f87f3b622923ac52151
  Author: Eliot Miranda <eliot.miranda at gmail.com>
  Date:   2021-03-08 (Mon, 08 Mar 2021)

  Changed paths:
    M platforms/iOS/vm/OSX/SqueakOSXAppDelegate.h
    M platforms/iOS/vm/OSX/SqueakOSXAppDelegate.m
    M platforms/iOS/vm/OSX/sqSqueakOSXApplication+events.m
    M platforms/iOS/vm/OSX/sqSqueakOSXApplication.m
    M platforms/iOS/vm/OSX/sqSqueakOSXCGView.h
    M platforms/iOS/vm/OSX/sqSqueakOSXCGView.m
    M platforms/iOS/vm/OSX/sqSqueakOSXDropAPI.m
    M platforms/iOS/vm/OSX/sqSqueakOSXHeadlessView.h
    M platforms/iOS/vm/OSX/sqSqueakOSXHeadlessView.m
    M platforms/iOS/vm/OSX/sqSqueakOSXMetalView.h
    M platforms/iOS/vm/OSX/sqSqueakOSXMetalView.m
    M platforms/iOS/vm/OSX/sqSqueakOSXOpenGLView.h
    M platforms/iOS/vm/OSX/sqSqueakOSXOpenGLView.m
    M platforms/iOS/vm/OSX/sqSqueakOSXView.h

  Log Message:
  -----------
  Merge pull request #553 from ronsaldo/feature-osx-uri-document-type-event-fixup

OS X custom URL scheme and document open event injection fixup


Compare: https://github.com/OpenSmalltalk/opensmalltalk-vm/compare/924a24bb5487...119e76c5c467

From no-reply at appveyor.com  Mon Mar  8 17:00:55 2021
From: no-reply at appveyor.com (AppVeyor)
Date: Mon, 08 Mar 2021 17:00:55 +0000
Subject: [Vm-dev] Build failed: opensmalltalk-vm 1.0.2413
Message-ID: <20210308170055.1.33F8B840AF6543F6@appveyor.com>

An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210308/8a2e6db8/attachment.html>

From notifications at github.com  Fri Mar 12 15:44:20 2021
From: notifications at github.com (Christoph Thiede)
Date: Fri, 12 Mar 2021 07:44:20 -0800
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] primitiveSSLCreate fails
 sporadically on Windows (#554)
Message-ID: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>

Hi all,

I think this is not a new issue, but I am experiencing it with increasing frequency on my Win 10 machine: When instantiating SqueakSSL, primitiveCreate fails. I don't see an error in the console so I cannot tell more details about the error. Sometimes restarting my image helps, but right now, it does not. Other images running with the same VM (202010232046) can still access the internet via SSL.

Maybe it plays a role that the affected image puts the plugin much more under stress by always holding at least one open connection to a server.

Is this a known problem? Are there any workarounds?

Looking forward to a fix,
Christoph

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/554
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210312/f2620627/attachment.html>

From notifications at github.com  Sat Mar 13 10:04:18 2021
From: notifications at github.com (alexshendi)
Date: Sat, 13 Mar 2021 02:04:18 -0800
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] sqUnixEvent.c missing
 from platforms/unix/vm (#555)
Message-ID: <OpenSmalltalk/opensmalltalk-vm/issues/555@github.com>

Hello OpenSmalltalk,

Apparently the file "sqUnixEvent.c" is missing from the directory platforms/unix/vm.
It is still needed for the X11 plugin. 
My quick fix was to checkout tag "202003021730", and to copy the file to the folder. 



-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/555
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210313/cc62926c/attachment.html>

From notifications at github.com  Sat Mar 13 14:23:57 2021
From: notifications at github.com (Christoph Thiede)
Date: Sat, 13 Mar 2021 06:23:57 -0800
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] primitiveSSLCreate
 fails sporadically on Windows (#554)
In-Reply-To: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
References: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
Message-ID: <OpenSmalltalk/opensmalltalk-vm/issues/554/798436609@github.com>

Oh no, this time the issue even persists after rebooting my operating system. `SqueakSSL new` just always fails. Exchanging the VM did not help; exchanging the image helped. Also, the failure occurs independently of whether my device is connected to any network or whether it isn't. So I guess my image has some kind of corrupted state? Is there some network/SSL configuration on the image side that I can reset in any way?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/554#issuecomment-798436609
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210313/b4a2f21b/attachment.html>

From notifications at github.com  Sat Mar 13 14:35:33 2021
From: notifications at github.com (Christoph Thiede)
Date: Sat, 13 Mar 2021 06:35:33 -0800
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] primitiveSSLCreate
 fails sporadically on Windows (#554)
In-Reply-To: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
References: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
Message-ID: <OpenSmalltalk/opensmalltalk-vm/issues/554/798452714@github.com>

Well, I just found out that neither `Smalltalk listBuiltinModules` nor `Smalltalk listLoadedModules` contains the `SqueakSSL` plugin that is used by `#primitiveSSLCreate`. But it is also not loaded in a fresh trunk image where SSL works without any problems. Is something wrong with this? How could I reload the plugin?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/554#issuecomment-798452714
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210313/353059b2/attachment-0001.html>

From tim at rowledge.org  Sat Mar 13 18:29:37 2021
From: tim at rowledge.org (tim Rowledge)
Date: Sat, 13 Mar 2021 10:29:37 -0800
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] primitiveSSLCreate
 fails sporadically on Windows (#554)
In-Reply-To: <OpenSmalltalk/opensmalltalk-vm/issues/554/798452714@github.com>
References: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
 <OpenSmalltalk/opensmalltalk-vm/issues/554/798452714@github.com>
Message-ID: <2D8F6E37-03FD-4D06-B9CE-1778B007E7B4@rowledge.org>



> On 2021-03-13, at 6:35 AM, Christoph Thiede <notifications at github.com> wrote:
> 
> 
> Well, I just found out that neither Smalltalk listBuiltinModules nor Smalltalk listLoadedModules contains the SqueakSSL plugin that is used by #primitiveSSLCreate. But it is also not loaded in a fresh trunk image where SSL works without any problems. Is something wrong with this? How could I reload the plugin?

It's an external plugin on all my systems - Mac & linux - so you shouldn't see it in the listBuiltinModules list. If I start a fresh image, the SSL plugin is not loaded. If I run the SqueakSSLTest tests in the TestRunner, they all pass and the plugin is in the listLoadedModules list.


tim
--
tim Rowledge; tim at rowledge.org; http://www.rowledge.org/tim
Strange OpCodes: WDS: Warp Drive, Scotty!



From notifications at github.com  Sat Mar 13 18:34:39 2021
From: notifications at github.com (Tobias Pape)
Date: Sat, 13 Mar 2021 10:34:39 -0800
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] primitiveSSLCreate
 fails sporadically on Windows (#554)
In-Reply-To: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
References: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
Message-ID: <OpenSmalltalk/opensmalltalk-vm/issues/554/798754750@github.com>

Christoph, where does your SqueakSSL.dll live? Is it the right architecture (32/64-bit)?

-- 
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/554#issuecomment-798754750
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210313/d061112e/attachment.html>

From notifications at github.com  Sat Mar 13 19:20:56 2021
From: notifications at github.com (Christoph Thiede)
Date: Sat, 13 Mar 2021 11:20:56 -0800
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] primitiveSSLCreate
 fails sporadically on Windows (#554)
In-Reply-To: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
References: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
Message-ID: <OpenSmalltalk/opensmalltalk-vm/issues/554/798771136@github.com>

> It's an external plugin on all my systems - Mac & linux - so you shouldn't see it in the listBuiltinModules list. If I start a fresh image, the SSL plugin is not loaded. If I run the SqueakSSLTest tests in the TestRunner, they all pass and the plugin is in the listLoadedModules list.

Thanks, Tim, this description makes sense and it looks the same in my fresh image! So the problem is that my problematic image does not automatically load the SSL plugin when invoking the primitive?

> Christoph, where does your SqueakSSL.dll live? Is it the right architecture (32/64-bit)?

It's both 64-bit.

I have just made two more observations:

1. I patched `primitiveSSLCreate` and added an `error: ec` argument to the primitive pragma. When the primitive fails, `ec` is set to `#'not found'`.
2. I used another VM (202003021730 instead of 202010232046) to open my image and it shows two interesting observations: First, just during the start-up phase during `ThisOSProcess` is started up, and second, when evaluating `SqueakSSL new`, a number of access violation errors pop out in the debug console of the VM:
   ```
   LoadLibrary(Win32OSProcessPlugin) (998: Unzuläÿssiger Zugriff auf einen Speicherbereich.
   
   )
   LoadLibrary(Win32OSProcessPlugin.dll) (998: Unzuläÿssiger Zugriff auf einen Speicherbereich.
   
   )
   # about 12 repetitions of the above ...
   LoadLibrary(SqueakSSL) (998: Unzuläÿssiger Zugriff auf einen Speicherbereich.
   
   )
   LoadLibrary(SqueakSSL.dll) (998: Unzuläÿssiger Zugriff auf einen Speicherbereich.
   
   )
   ```
   "Unzuläÿssiger Zugriff auf einen Speicherbereich" is German for "Invalid access to memory location" (by the way, the encoding is wrong, `äÿ` should be only `ä`).

There has already been a similar issue reported on the mailing-list, but apparently, it has not yet been solved: http://lists.squeakfoundation.org/pipermail/vm-dev/2017-June/025516.html

-- 
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/554#issuecomment-798771136
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210313/c3442ead/attachment.html>

From notifications at github.com  Sat Mar 13 19:55:22 2021
From: notifications at github.com (Tobias Pape)
Date: Sat, 13 Mar 2021 11:55:22 -0800
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] primitiveSSLCreate
 fails sporadically on Windows (#554)
In-Reply-To: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
References: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
Message-ID: <OpenSmalltalk/opensmalltalk-vm/issues/554/798775987@github.com>

Can you please tell where the SqueakSSL.dll is located? and maybe find out which version you have?

-- 
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/554#issuecomment-798775987
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210313/3d421506/attachment.html>

From notifications at github.com  Sun Mar 14 13:35:48 2021
From: notifications at github.com (Christoph Thiede)
Date: Sun, 14 Mar 2021 06:35:48 -0700
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] primitiveSSLCreate
 fails sporadically on Windows (#554)
In-Reply-To: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
References: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
Message-ID: <OpenSmalltalk/opensmalltalk-vm/issues/554/798907900@github.com>

> Can you please tell where the SqueakSSL.dll is located?

In my default configuration, the locations are as follows:

```
C:\Program Files (x86)\Squeak\SqueakSSL.dll
C:\Program Files (x86)\Squeak\Squeak.exe
C:\Users\Christoph\OneDrive\Dokumente\Squeak\ChristophTrunk.image
```

However, the problem still exists when I change the paths as follows:

```
C:\sq\SqueakSSL.dll
C:\sq\Squeak.exe
C:\ChristophTrunk.image
```

> and maybe find out which version you have?

If I run a fresh trunk image with Squeak.exe, after evaluating `SqueakSSL new`, the SystemReporter says `SqueakSSL VMMaker.oscog-eem.2805 (e)` resp. `SqueakSSL VMMaker.oscog-eem.2673 (e)` (I tried two VM versions).

In the affected image, the module does not appear in the list.

-- 
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/554#issuecomment-798907900
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210314/26487b39/attachment.html>

From notifications at github.com  Sun Mar 14 14:00:28 2021
From: notifications at github.com (Christoph Thiede)
Date: Sun, 14 Mar 2021 07:00:28 -0700
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] primitiveSSLCreate
 fails sporadically on Windows (#554)
In-Reply-To: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
References: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
Message-ID: <OpenSmalltalk/opensmalltalk-vm/issues/554/798912822@github.com>

Here is a stack trace from Visual Studio (collected by attaching the debugger to running process):

```
0xC0000005: Access violating when writing at position 0xFFFFFFFFA05CB32C.

ntdll.dll!LdrpAllocateTlsEntry()
ntdll.dll!LdrpHandleTlsData()
ntdll.dll!LdrpDoPostSnapWork()
ntdll.dll!LdrpSnapModule()
ntdll.dll!LdrpMapAndSnapDependency()
ntdll.dll!LdrpMapDllWithSectionHandle()
ntdll.dll!LdrpMapDllNtFileName()
ntdll.dll!LdrpMapDllRetry()
ntdll.dll!LdrpProcessWork()
ntdll.dll!LdrpDrainWorkQueue()
ntdll.dll!LdrpLoadDllInternal()
ntdll.dll!LdrpLoadDll()
ntdll.dll!LdrLoadDll()
KernelBase.dll!LoadLibraryExW()
KernelBase.dll!LoadLibraryExA()
KernelBase.dll!LoadLibraryA()
Squeak.exe!00000000004aa129()
Squeak.exe!00000000004b8cd7()
Squeak.exe!00000000004b9062()
Squeak.exe!000000000041c642()
Squeak.exe!0000000000407926()
Squeak.exe!0000000000411c9e()
Squeak.exe!0000000000401d3c()
Squeak.exe!00000000004aea6b()
Squeak.exe!00000000004aed8f()
Squeak.exe!00000000004013c7()
Squeak.exe!00000000004014cb()
kernel32.dll!BaseThreadInitThunk()
ntdll.dll!RtlUserThreadStart()
```

Is this helpful in any way? :-)

-- 
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/554#issuecomment-798912822
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210314/0306fa40/attachment-0001.html>

From stes at telenet.be  Sun Mar 14 18:30:25 2021
From: stes at telenet.be (stes@PANDORA.BE)
Date: Sun, 14 Mar 2021 19:30:25 +0100 (CET)
Subject: [Vm-dev] regression: TestObjectsAsMethods crash in 4.19.5 (not
 in 4.16.7)
In-Reply-To: <605876499.30501015.1614071179388.JavaMail.zimbra@telenet.be>
References: <949389016.25753821.1613919758428.JavaMail.zimbra@telenet.be>
 <605876499.30501015.1614071179388.JavaMail.zimbra@telenet.be>
Message-ID: <857058996.42907317.1615746625509.JavaMail.zimbra@telenet.be>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256


> A trivial fix is to rearrange some primtive number assignments, but the
> correct thing to do is adopt the logic that Eliot uses in oscog so that
> the function is not invoked as a "primitive" at all.
>
> I'll try to follow up on this next week. 

Would it please be possible to commit to subversion the trivial fix,
while waiting for the correct fix.

As I understand it, this means to undo the "primitives" change.

David Stes

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJgTlXxAAoJEAwpOKXMq1MaG0kH/1aQ1MNWaMkbErs/ce3VcFhN
a/97ONAJ4uMkXT6xHQbt1Pkapx7hpXd7dvLXjFoWQ+lv7dPv7y2ac5mQyjUJs0nY
HrnTh12C8VqP2zwAX1q8AYog7ZSLGqyo8SIRcoM8Lzqbw01SPkN2FuXGz9dqKCwe
+6/D4m+HnfwdeqtazYYNr4dGiJWrkXf/uGiCMWogbJAphxKzokeiKpUD21lBFzyG
2NGzkWncn51z1lZPfLoRfqO/vNQ1jv3q5mOJ8iFJCwsq84k6h300+ZSfO4BvRDFh
zRkoDrNotuNn5VzWpiwNjNrkth2XzLM/MU+m5SKoTKBwQUaNkTa97NVT4U6hzkc=
=EGUN
-----END PGP SIGNATURE-----

From stes at telenet.be  Sun Mar 14 20:15:26 2021
From: stes at telenet.be (stes@PANDORA.BE)
Date: Sun, 14 Mar 2021 21:15:26 +0100 (CET)
Subject: [Vm-dev] regression: TestObjectsAsMethods crash in 4.19.5 (not
 in 4.16.7)
In-Reply-To: <857058996.42907317.1615746625509.JavaMail.zimbra@telenet.be>
References: <949389016.25753821.1613919758428.JavaMail.zimbra@telenet.be>
 <605876499.30501015.1614071179388.JavaMail.zimbra@telenet.be>
 <857058996.42907317.1615746625509.JavaMail.zimbra@telenet.be>
Message-ID: <1310783440.43126857.1615752926405.JavaMail.zimbra@telenet.be>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256


If there exists a solution that simply makes the ObjectsAsMethodsExample
fail (instead of crash) that would also be OK.

For example, I suppose it could be a choice to "abandon"
primitiveInvokeObjectAsMethod() and run:with:in
and to declare it as "obsolete".

It can be a deliberate choice to make the TestObjectsAsMethods *fail*
in the future ... but preferably not crash.

By fail I mean that when running the TestRunner it marks the test as failed,
and continues to run the other tests.  The current situation is that the VM
crashes and all tests are aborted.

TestObjectsAsMethods can be made some sort of an expected failure, I suppose.

Such a solution would be fine for the moment, while waiting for the 
definitive fix.

David Stes

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJgTm59AAoJEAwpOKXMq1MajcUH/1hxKnoJf+rbp/86+yASjxdq
fOlkl48X3iVFK+lqUcMGclpmqjqzShGUryTq9d9S1/8hnVZAesOeB9Y2DCNZcGVS
vO8A33sw1w1cptggb9mvQ1Bg6bZ4kcEMj7znfpnLSDRd7h1NUZluzsBpWG4YEDI9
McBUXTFlHtZ2jjZ4DFLm7B/DkENje9uT696GgFdyR1YChBVclWBJ+x7nYsW98467
NQ7OtMDBjxhSAwiZIzpeuelOST2akXAzPZImg7MU4UTCXpAk7Fst+Ua74Z+lZb6L
JvnjdnSPibpShwcTYo4Woc5LwrFmvwefCdUt5Mh2l9SpZSjE2NQCr7Fo/tdcG3Y=
=VeC2
-----END PGP SIGNATURE-----

From florin.mateoc at gmail.com  Mon Mar 15 01:18:32 2021
From: florin.mateoc at gmail.com (Florin Mateoc)
Date: Sun, 14 Mar 2021 21:18:32 -0400
Subject: [Vm-dev] [SqueakJS] Faster JIT ideas
In-Reply-To: <CAOWGXNCmyE1N9K87+jKiXM7awb7KbsQws7HKO3uAXAKV9PBc2Q@mail.gmail.com>
References: <CAOWGXNCmyE1N9K87+jKiXM7awb7KbsQws7HKO3uAXAKV9PBc2Q@mail.gmail.com>
Message-ID: <CA+7DxaY=EC_p2n=rhVyxXfm75v-qASwRu6av=fQj6MSNUJvn3g@mail.gmail.com>

Hi Vanessa,

Sorry for the delay in responding - as somebody who has been inspired by
your SqueakJS project, I think I should mention that I am working on a
related project, for now tentatively called JsSqueak.
In addition to the inspiration provided by SqueakJS, it also scratches my
longstanding itch about compiling (transpiling) Squeak.

I hesitated to talk about it, as it is still a work in progress - after
small bits and pieces that I worked on over a long period, I had the
opportunity to spend a significant and uninterrupted chunk of time on it
last summer, when I was unemployed for 3 months, and I was able to make
good progress. I was optimistically thinking of releasing a first version
before the end of last year, but after I started working on my new job,
progress on JsSqueak has slowed down significantly. I must confess that I
(and especially my wife) hesitate in recreating that productive unemployed
situation :)

I started with Squeak 4.5 - I already had code transforming Smalltalk code
to a form more suitable for translation - and I also started with
VMMakerJS-dtl.18 for the plugin generation part. Of course, I had to
heavily modify it, since I have to get rid of the stack usage for
arguments/receiver and returns.
Both of these big parts are working. I also implemented most numbered
primitives by hand - they are inlined at generation time in the methods
calling them.
I am also taking advantage of the latest and greatest additions to
JavaScript. I am, of course, using classes, and the parallel class-side
hierarchy is implemented using statics. To implement green threads/process
switching, all translated methods are implemented as generator functions,
and all calls are through yield* expressions. The preemption/interrupt
check points are inlined. With this, a process switch is achieved by simply
yield-ing (in the process/semaphore primitives).
With this, the Integer>>#benchFib method is translated (as a method in
Number.prototype, there is one more, simpler, implementation in BigInt) as:

*_benchFib() {
   if (Number.isSafeInteger(this.valueOf())) { // Effective (inherited
or local) source for #benchFib in SmallInteger
      /*Handy send-heavy benchmark*/
   /*(result // seconds to run) = approx calls per second*/
   /* | r t |
     t := Time millisecondsToRun: [r := 26 benchFib].
     (r // 1000) // t*/
   /*138000 on a Mac 8100//100*/
   if (GlobalActivationCounter-- < 0) yield* GlobalCheckForInterrupts();

   return (yield* this._lt( 2)).booleanValueOf("questionMark:colon:")
? (1) : (yield* (yield* (yield* (yield* this._sub(
1))._benchFib())._add( yield* (yield* this._sub(
2))._benchFib()))._add( 1));
} else // No implementation for #benchFib in Float hierarchy, trigger a DNU
      return yield* super._benchFib()
}

The top-level check for smallIntegers is because both SmallInteger and
Float are mapped to Number.

The booleanValueOf call is for implementing the mustBeBoolean
machinery (it actually translates directly to DNU, like it is done
nowadays in Squeak).

Of course, in Boolean, booleanValueOf is just an alias for valueOf

As you can see, though, this is not terribly efficient, but there is
room for improvement/optimizations. With more work, in this case, the
_lt call could be replaced by the < operator, and even the _sub and
_add calls could be optimized,
although not completely, since their result can morph into
LargeInteger (mapped to BigInt).

As hinted above, SmallInteger is mapped to Number (in the safeInteger
range), Float is mapped to Number as well, and LargeInteger is mapped
to BigInt.

BlockClosure is mapped to Function, Boolean is mapped to Boolean,
Character is mapped to String, weak references are implemented via
WeakRef.
I have briefly considered also doing slightly higher-level mappings,
for IdentitySet to Set and IdentityDictionary to Map, but this is not
a priority.

The image is serialized sort of like a JavaScript storeString. No
processes or contexts though, or rather they are not brought back in
on the JavaScript side. Blocks are both stored and loaded.

Non-local returns, unwind blocks, resumable and restartable exceptions
are implemented via JavaScript exception handling plus explicit
handler block chains associated with the processes.

The "image" starts with the global state loaded, but all processes are
started from scratch instead of resumed. A non-UI image is thus
trivially started.

One major todo left is hooking up the UI/browser. I did take
vm.display.browser.js from SqueakJS and adapted the code in order to
implement its numbered primitives, but I still have to work through
squeak.js from the same to initialize
and hook up the display.

Florin




On Sun, Mar 7, 2021 at 11:17 PM Vanessa Freudenberg <vanessa at codefrau.net>
wrote:

>
> Hi all,
>
> ideas for a faster SqueakJS JIT have been swirling around in my head for
> many years. This weekend I took the time to write some of them down:
>
> https://squeak.js.org/docs/jit.md.html
>
> Feedback welcome!
>
> Vanessa
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210314/2b4e78e9/attachment.html>

From florin.mateoc at gmail.com  Mon Mar 15 01:27:56 2021
From: florin.mateoc at gmail.com (Florin Mateoc)
Date: Sun, 14 Mar 2021 21:27:56 -0400
Subject: [Vm-dev] [SqueakJS] Faster JIT ideas
In-Reply-To: <CA+7DxaY=EC_p2n=rhVyxXfm75v-qASwRu6av=fQj6MSNUJvn3g@mail.gmail.com>
References: <CAOWGXNCmyE1N9K87+jKiXM7awb7KbsQws7HKO3uAXAKV9PBc2Q@mail.gmail.com>
 <CA+7DxaY=EC_p2n=rhVyxXfm75v-qASwRu6av=fQj6MSNUJvn3g@mail.gmail.com>
Message-ID: <CA+7DxabT6-aAYT5cQ9ABoB7RxMsoF77qUbyzp148sLN_VvJRyw@mail.gmail.com>

I almost forgot: this is how my console looks like when trying to start via
Chrome


   - MemoryInfo {totalJSHeapSize: 2685131, usedJSHeapSize: 1667135,
   jsHeapSizeLimit: 4294705152}

startupBrowser.mjs:23
Loading initial glue code
startupBrowser.mjs:26
Loading translated Squeak image 4.5
startupBrowser.mjs:29
Loading generated classes (all the code, plus minimal reflection data)
startupBrowser.mjs:34
Elapsed 42074ms
startupBrowser.mjs:36
Loading manual overrides for the generated code
startupBrowser.mjs:39
Loading more glue code
startupBrowser.mjs:43

   - MemoryInfo {totalJSHeapSize: 195878416, usedJSHeapSize: 176798312,
   jsHeapSizeLimit: 4294705152}

startupBrowser.mjs:44
Loading serialized image state
SmalltalkGlobals.js:254
reified compiledMethod for method _notEqEq of class _ProtoObject has not
been set up, the method might be a manual JavaScript override
SmalltalkGlobals.js:254
reified compiledMethod for method _eqEq of class _ProtoObject has not been
set up, the method might be a manual JavaScript override
SmalltalkGlobals.js:254
reified compiledMethod for method _class of class _Object has not been set
up, the method might be a manual JavaScript override
SmalltalkGlobals.js:254
reified compiledMethod for method _lookup_ of class _Symbol class has not
been set up, the method might be a manual JavaScript override
SmalltalkGlobals.js:254
reified compiledMethod for method _rehash of class _Symbol class has not
been set up, the method might be a manual JavaScript override
SmalltalkGlobals.js:254
reified compiledMethod for method _allSymbolTablesDo_ of class _Symbol
class has not been set up, the method might be a manual JavaScript override
SmalltalkGlobals.js:254
reified compiledMethod for method _compactSymbolTable of class _Symbol
class has not been set up, the method might be a manual JavaScript override
SmalltalkGlobals.js:254
reified compiledMethod for method _allSymbols of class _Symbol class has
not been set up, the method might be a manual JavaScript override
SmalltalkGlobals.js:254
reified compiledMethod for method _allSymbolTablesDo_after_ of class
_Symbol class has not been set up, the method might be a manual JavaScript
override
SmalltalkGlobals.js:254
reified compiledMethod for method _intern_ of class _Symbol class has not
been set up, the method might be a manual JavaScript override
SmalltalkGlobals.js:254
reified compiledMethod for method _minVal of class _SmallInteger class has
not been set up, the method might be a manual JavaScript override
SmalltalkGlobals.js:254
reified compiledMethod for method _maxVal of class _SmallInteger class has
not been set up, the method might be a manual JavaScript override
SmalltalkGlobals.js:254
reified compiledMethod for method _installLowSpaceWatcher of class
_SmalltalkImage has not been set up, the method might be a manual
JavaScript override
SmalltalkGlobals.js:254
reified compiledMethod for method _signal of class _Exception has not been
set up, the method might be a manual JavaScript override
SmalltalkGlobals.js:254
reified compiledMethod for method _return_ of class _Exception has not been
set up, the method might be a manual JavaScript override
SmalltalkGlobals.js:254
reified compiledMethod for method _resumeUnchecked_ of class _Exception has
not been set up, the method might be a manual JavaScript override
SmalltalkGlobals.js:254
reified compiledMethod for method _pass of class _Exception has not been
set up, the method might be a manual JavaScript override
SmalltalkGlobals.js:254
reified compiledMethod for method _retry of class _Exception has not been
set up, the method might be a manual JavaScript override
SmalltalkGlobals.js:254
reified compiledMethod for method _outer of class _Exception has not been
set up, the method might be a manual JavaScript override
startupBrowser.mjs:47
Elapsed 105913ms
startupBrowser.mjs:49

   - MemoryInfo {totalJSHeapSize: 462667346, usedJSHeapSize: 438684470,
   jsHeapSizeLimit: 4294705152}

startupBrowser.mjs:51

   - MemoryInfo {totalJSHeapSize: 453213778, usedJSHeapSize: 373908346,
   jsHeapSizeLimit: 4294705152}

startupBrowser.mjs:52
Loading VM/scheduler
SmalltalkVM.js:81
cleaning up
SmalltalkVM.js:141

   - MemoryInfo {totalJSHeapSize: 409552989, usedJSHeapSize: 371929977,
   jsHeapSizeLimit: 4294705152}

SmalltalkVM.js:146

   - MemoryInfo {totalJSHeapSize: 409815133, usedJSHeapSize: 371860153,
   jsHeapSizeLimit: 4294705152}

SmalltalkVM.js:177
setting up the scheduler loop
SmalltalkVM.js:195
start known processes
SmalltalkVM.js:196
_Delay._startTimerEventLoop
SmalltalkVM.js:199
_Delay._startUp_(true)
SmalltalkVM.js:201
_ProcessorScheduler._startUp_(true)
SmalltalkVM.js:203
_WeakArray._startUp_(true)
SmalltalkVM.js:205
finished starting non-UI known processes
startUI_browser.mjs:25
starting EventSensor
startUI_browser.mjs:27
spawning primordial UI process
SmalltalkVM.js:180
entering new active process with priority 80
SmalltalkVM.js:184
primitive 86 semaphore wait
SmalltalkVM.js:180
entering new active process with priority 60
SmalltalkVM.js:184
primitive 86 semaphore wait
SmalltalkVM.js:180
entering new active process with priority 60
SmalltalkVM.js:184
primitive 85 semaphore signal
SmalltalkVM.js:180
entering new active process with priority 80
SmalltalkVM.js:184
primitive 86 semaphore wait
SmalltalkVM.js:180
entering new active process with priority 60
SmalltalkVM.js:184
primitive 86 semaphore wait
SmalltalkVM.js:180
entering new active process with priority 50
SmalltalkVM.js:184
primitive 86 semaphore wait
SmalltalkVM.js:180
entering new active process with priority 40
_DisplayScreen.js:843
Uncaught TypeError: Cannot read property 'width' of undefined

On Sun, Mar 14, 2021 at 9:18 PM Florin Mateoc <florin.mateoc at gmail.com>
wrote:

> Hi Vanessa,
>
> Sorry for the delay in responding - as somebody who has been inspired by
> your SqueakJS project, I think I should mention that I am working on a
> related project, for now tentatively called JsSqueak.
> In addition to the inspiration provided by SqueakJS, it also scratches my
> longstanding itch about compiling (transpiling) Squeak.
>
> I hesitated to talk about it, as it is still a work in progress - after
> small bits and pieces that I worked on over a long period, I had the
> opportunity to spend a significant and uninterrupted chunk of time on it
> last summer, when I was unemployed for 3 months, and I was able to make
> good progress. I was optimistically thinking of releasing a first version
> before the end of last year, but after I started working on my new job,
> progress on JsSqueak has slowed down significantly. I must confess that I
> (and especially my wife) hesitate in recreating that productive unemployed
> situation :)
>
> I started with Squeak 4.5 - I already had code transforming Smalltalk code
> to a form more suitable for translation - and I also started with
> VMMakerJS-dtl.18 for the plugin generation part. Of course, I had to
> heavily modify it, since I have to get rid of the stack usage for
> arguments/receiver and returns.
> Both of these big parts are working. I also implemented most numbered
> primitives by hand - they are inlined at generation time in the methods
> calling them.
> I am also taking advantage of the latest and greatest additions to
> JavaScript. I am, of course, using classes, and the parallel class-side
> hierarchy is implemented using statics. To implement green threads/process
> switching, all translated methods are implemented as generator functions,
> and all calls are through yield* expressions. The preemption/interrupt
> check points are inlined. With this, a process switch is achieved by simply
> yield-ing (in the process/semaphore primitives).
> With this, the Integer>>#benchFib method is translated (as a method in
> Number.prototype, there is one more, simpler, implementation in BigInt) as:
>
> *_benchFib() {
>    if (Number.isSafeInteger(this.valueOf())) { // Effective (inherited or local) source for #benchFib in SmallInteger
>       /*Handy send-heavy benchmark*/
>    /*(result // seconds to run) = approx calls per second*/
>    /* | r t |
>      t := Time millisecondsToRun: [r := 26 benchFib].
>      (r // 1000) // t*/
>    /*138000 on a Mac 8100//100*/
>    if (GlobalActivationCounter-- < 0) yield* GlobalCheckForInterrupts();
>
>    return (yield* this._lt( 2)).booleanValueOf("questionMark:colon:") ? (1) : (yield* (yield* (yield* (yield* this._sub( 1))._benchFib())._add( yield* (yield* this._sub( 2))._benchFib()))._add( 1));
> } else // No implementation for #benchFib in Float hierarchy, trigger a DNU
>       return yield* super._benchFib()
> }
>
> The top-level check for smallIntegers is because both SmallInteger and Float are mapped to Number.
>
> The booleanValueOf call is for implementing the mustBeBoolean machinery (it actually translates directly to DNU, like it is done nowadays in Squeak).
>
> Of course, in Boolean, booleanValueOf is just an alias for valueOf
>
> As you can see, though, this is not terribly efficient, but there is room for improvement/optimizations. With more work, in this case, the _lt call could be replaced by the < operator, and even the _sub and _add calls could be optimized,
> although not completely, since their result can morph into LargeInteger (mapped to BigInt).
>
> As hinted above, SmallInteger is mapped to Number (in the safeInteger range), Float is mapped to Number as well, and LargeInteger is mapped to BigInt.
>
> BlockClosure is mapped to Function, Boolean is mapped to Boolean, Character is mapped to String, weak references are implemented via WeakRef.
> I have briefly considered also doing slightly higher-level mappings, for IdentitySet to Set and IdentityDictionary to Map, but this is not a priority.
>
> The image is serialized sort of like a JavaScript storeString. No processes or contexts though, or rather they are not brought back in on the JavaScript side. Blocks are both stored and loaded.
>
> Non-local returns, unwind blocks, resumable and restartable exceptions are implemented via JavaScript exception handling plus explicit handler block chains associated with the processes.
>
> The "image" starts with the global state loaded, but all processes are started from scratch instead of resumed. A non-UI image is thus trivially started.
>
> One major todo left is hooking up the UI/browser. I did take vm.display.browser.js from SqueakJS and adapted the code in order to implement its numbered primitives, but I still have to work through squeak.js from the same to initialize
> and hook up the display.
>
> Florin
>
>
>
>
> On Sun, Mar 7, 2021 at 11:17 PM Vanessa Freudenberg <vanessa at codefrau.net>
> wrote:
>
>>
>> Hi all,
>>
>> ideas for a faster SqueakJS JIT have been swirling around in my head for
>> many years. This weekend I took the time to write some of them down:
>>
>> https://squeak.js.org/docs/jit.md.html
>>
>> Feedback welcome!
>>
>> Vanessa
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210314/38c3e262/attachment-0001.html>

From notifications at github.com  Mon Mar 15 11:02:43 2021
From: notifications at github.com (Tom Beckmann)
Date: Mon, 15 Mar 2021 04:02:43 -0700
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] sqUnixEvent.c missing
 from platforms/unix/vm (#555)
In-Reply-To: <OpenSmalltalk/opensmalltalk-vm/issues/555@github.com>
References: <OpenSmalltalk/opensmalltalk-vm/issues/555@github.com>
Message-ID: <OpenSmalltalk/opensmalltalk-vm/issues/555/799327721@github.com>

The commit that removed it was this one: https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/924a24bb54870a621c5283f23631261d5a792000

I saw that you did some cleanup of old event primitives, as documented in the commit message, @eliotmiranda. Was sqUnixEvent.c deleted by accident as part of this?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/555#issuecomment-799327721
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210315/1d1a2e3e/attachment.html>

From notifications at github.com  Mon Mar 15 11:22:41 2021
From: notifications at github.com (Tobias Pape)
Date: Mon, 15 Mar 2021 04:22:41 -0700
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] primitiveSSLCreate
 fails sporadically on Windows (#554)
In-Reply-To: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
References: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
Message-ID: <OpenSmalltalk/opensmalltalk-vm/issues/554/799340893@github.com>

can you do `dumpbin /exports` on the DLL?

-- 
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/554#issuecomment-799340893
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210315/4f0b29ff/attachment.html>

From notifications at github.com  Mon Mar 15 11:34:51 2021
From: notifications at github.com (Tobias Pape)
Date: Mon, 15 Mar 2021 04:34:51 -0700
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] primitiveSSLCreate
 fails sporadically on Windows (#554)
In-Reply-To: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
References: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
Message-ID: <OpenSmalltalk/opensmalltalk-vm/issues/554/799348147@github.com>

Also, do you get such a backtrace when you invoke the method of UUID that contains `primitiveMakeUUID` ?

-- 
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/554#issuecomment-799348147
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210315/907c5280/attachment.html>

From notifications at github.com  Mon Mar 15 14:59:21 2021
From: notifications at github.com (Christoph Thiede)
Date: Mon, 15 Mar 2021 07:59:21 -0700
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] primitiveSSLCreate
 fails sporadically on Windows (#554)
In-Reply-To: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
References: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
Message-ID: <OpenSmalltalk/opensmalltalk-vm/issues/554/799488434@github.com>

> can you do `dumpbin /exports` on the DLL?

202010232046:
<details><pre><code>
        0.00 version
           1 ordinal base
          12 number of functions
          12 number of names

    ordinal hint RVA      name

         12    0 000013A0 getModuleName
          2    1 000013B0 primitiveAccept = printf
          3    2 000014F0 primitiveConnect
          4    3 00001630 primitiveCreate
          5    4 00001680 primitiveDecrypt
          6    5 000017C0 primitiveDestroy
          7    6 00001820 primitiveEncrypt
          8    7 00001960 primitiveGetIntProperty
          9    8 000019F0 primitiveGetStringProperty
         10    9 00001B40 primitiveSetIntProperty
         11    A 00001BE0 primitiveSetStringProperty
          1    B 00001CA0 setInterpreter

  Summary

        1000 .CRT
        1000 .bss
        1000 .data
        4000 .debug_abbrev
        1000 .debug_aranges
        2000 .debug_frame
       44000 .debug_info
        9000 .debug_line
       13000 .debug_loc
        1000 .debug_macinfo
        1000 .debug_ranges
        3000 .debug_str
        1000 .edata
        1000 .idata
        1000 .pdata
        2000 .rdata
        1000 .reloc
        1000 .rsrc
        9000 .text
        1000 .tls
        1000 .xdata
</code></pre></details>

202003021730:
<details><pre><code>
File Type: DLL

  Section contains the following exports for SqueakSSL.dll

    00000000 characteristics
    5E5E1711 time date stamp Tue Mar  3 09:36:33 2020
        0.00 version
           1 ordinal base
          12 number of functions
          12 number of names

    ordinal hint RVA      name

         12    0 000013B0 getModuleName
          2    1 000013C0 primitiveAccept = sqSetupCert
          3    2 00001500 primitiveConnect
          4    3 00001640 primitiveCreate
          5    4 00001690 primitiveDecrypt
          6    5 000017D0 primitiveDestroy
          7    6 00001830 primitiveEncrypt
          8    7 00001970 primitiveGetIntProperty
          9    8 00001A00 primitiveGetStringProperty
         10    9 00001B50 primitiveSetIntProperty
         11    A 00001BF0 primitiveSetStringProperty
          1    B 00001CB0 setInterpreter

  Summary

        1000 .CRT
        1000 .bss
        1000 .data
        3000 .debug_abbrev
        1000 .debug_aranges
        1000 .debug_frame
       39000 .debug_info
        4000 .debug_line
        6000 .debug_loc
        1000 .debug_macinfo
        1000 .debug_ranges
        3000 .debug_str
        1000 .edata
        1000 .idata
        1000 .pdata
        1000 .rdata
        1000 .reloc
        1000 .rsrc
        4000 .text
        1000 .tls
        1000 .xdata
</code></pre></details>

> Also, do you get such a backtrace when you invoke the method of UUID that contains `primitiveMakeUUID`?

In the 202010232046 VM, `UUID new` does not raise an error but returns a plausible result. In the 202003021730 VM, another `LoadLibrary` warning appears in the console but the expressions answers a plausible result, too. The trace is:

```
0xC0000005: Access violation when writing at position 0xFFFFFFFF9DAC75EC.

ntdll.dll!LdrpAllocateTlsEntry()
ntdll.dll!LdrpHandleTlsData()
ntdll.dll!LdrpDoPostSnapWork()
ntdll.dll!LdrpSnapModule()
ntdll.dll!LdrpMapAndSnapDependency()
ntdll.dll!LdrpMapDllWithSectionHandle()
ntdll.dll!LdrpMapDllNtFileName()
ntdll.dll!LdrpMapDllRetry()
ntdll.dll!LdrpProcessWork()
ntdll.dll!LdrpDrainWorkQueue()
ntdll.dll!LdrpLoadDllInternal()
ntdll.dll!LdrpLoadDll()
ntdll.dll!LdrLoadDll()
KernelBase.dll!LoadLibraryExW()
KernelBase.dll!LoadLibraryExA()
KernelBase.dll!LoadLibraryA()
Squeak.exe!00000000004aa129()
Squeak.exe!00000000004b8cd7()
Squeak.exe!00000000004b9062()
Squeak.exe!000000000041c642()
Squeak.exe!0000000000407926()
Squeak.exe!0000000000411c9e()
Squeak.exe!0000000000401d3c()
Squeak.exe!00000000004aea6b()
Squeak.exe!00000000004aed8f()
Squeak.exe!00000000004013c7()
Squeak.exe!00000000004014cb()
kernel32.dll!BaseThreadInitThunk()
ntdll.dll!RtlUserThreadStart()
```

So yes, the stack appears to be indentical.

-- 
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/554#issuecomment-799488434
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210315/ec15d982/attachment.html>

From notifications at github.com  Mon Mar 15 15:06:14 2021
From: notifications at github.com (Tobias Pape)
Date: Mon, 15 Mar 2021 08:06:14 -0700
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] primitiveSSLCreate
 fails sporadically on Windows (#554)
In-Reply-To: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
References: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
Message-ID: <OpenSmalltalk/opensmalltalk-vm/issues/554/799493894@github.com>

so, this hints towars the general Module loading. 
can you build a debug vm and stop in https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/win32/vm/sqWin32ExternalPrims.c#L29? It would be interesting to see what `libName` is.

-- 
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/554#issuecomment-799493894
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210315/bffaafe0/attachment-0001.html>

From vanessa at codefrau.net  Mon Mar 15 19:40:38 2021
From: vanessa at codefrau.net (Vanessa Freudenberg)
Date: Mon, 15 Mar 2021 12:40:38 -0700
Subject: [Vm-dev] [SqueakJS] Faster JIT ideas
In-Reply-To: <CA+7DxaY=EC_p2n=rhVyxXfm75v-qASwRu6av=fQj6MSNUJvn3g@mail.gmail.com>
References: <CAOWGXNCmyE1N9K87+jKiXM7awb7KbsQws7HKO3uAXAKV9PBc2Q@mail.gmail.com>
 <CA+7DxaY=EC_p2n=rhVyxXfm75v-qASwRu6av=fQj6MSNUJvn3g@mail.gmail.com>
Message-ID: <CAOWGXNAtjzX3jfYf1rcBoy=_UxjgQzFwdnHch93L9WeXGy=dAQ@mail.gmail.com>

Hi Florin,

wow, that looks exciting!

This is indeed a much more thorough Squeak-to-JS mapping, where mine is
more of a "traditional" VM. I love it!

Since my original post I implemented a mockup of "my" new JIT scheme:

https://squeak.js.org/docs/jit.md.html#sketch:contextproxieswithintrospectionandinlinecaching/newjitsketch/performanceestimate
[image: image.png]
You can play with it here: https://codepen.io/codefrau/pen/JjbmVGw

Could you share the performance numbers you are seeing for your benchFib,
in comparison to SqueakJS or my mockup? I am curious if yield* is the way
to go.

Thanks for sharing! And congrats on your new job. My progress is slow too,
I only work on it some weekends. But then, I'm not in a hurry :)

Vanessa

On Sun, Mar 14, 2021 at 6:18 PM Florin Mateoc <florin.mateoc at gmail.com>
wrote:

>
> Hi Vanessa,
>
> Sorry for the delay in responding - as somebody who has been inspired by
> your SqueakJS project, I think I should mention that I am working on a
> related project, for now tentatively called JsSqueak.
> In addition to the inspiration provided by SqueakJS, it also scratches my
> longstanding itch about compiling (transpiling) Squeak.
>
> I hesitated to talk about it, as it is still a work in progress - after
> small bits and pieces that I worked on over a long period, I had the
> opportunity to spend a significant and uninterrupted chunk of time on it
> last summer, when I was unemployed for 3 months, and I was able to make
> good progress. I was optimistically thinking of releasing a first version
> before the end of last year, but after I started working on my new job,
> progress on JsSqueak has slowed down significantly. I must confess that I
> (and especially my wife) hesitate in recreating that productive unemployed
> situation :)
>
> I started with Squeak 4.5 - I already had code transforming Smalltalk code
> to a form more suitable for translation - and I also started with
> VMMakerJS-dtl.18 for the plugin generation part. Of course, I had to
> heavily modify it, since I have to get rid of the stack usage for
> arguments/receiver and returns.
> Both of these big parts are working. I also implemented most numbered
> primitives by hand - they are inlined at generation time in the methods
> calling them.
> I am also taking advantage of the latest and greatest additions to
> JavaScript. I am, of course, using classes, and the parallel class-side
> hierarchy is implemented using statics. To implement green threads/process
> switching, all translated methods are implemented as generator functions,
> and all calls are through yield* expressions. The preemption/interrupt
> check points are inlined. With this, a process switch is achieved by simply
> yield-ing (in the process/semaphore primitives).
> With this, the Integer>>#benchFib method is translated (as a method in
> Number.prototype, there is one more, simpler, implementation in BigInt) as:
>
> *_benchFib() {
>    if (Number.isSafeInteger(this.valueOf())) { // Effective (inherited or local) source for #benchFib in SmallInteger
>       /*Handy send-heavy benchmark*/
>    /*(result // seconds to run) = approx calls per second*/
>    /* | r t |
>      t := Time millisecondsToRun: [r := 26 benchFib].
>      (r // 1000) // t*/
>    /*138000 on a Mac 8100//100*/
>    if (GlobalActivationCounter-- < 0) yield* GlobalCheckForInterrupts();
>
>    return (yield* this._lt( 2)).booleanValueOf("questionMark:colon:") ? (1) : (yield* (yield* (yield* (yield* this._sub( 1))._benchFib())._add( yield* (yield* this._sub( 2))._benchFib()))._add( 1));
> } else // No implementation for #benchFib in Float hierarchy, trigger a DNU
>       return yield* super._benchFib()
> }
>
> The top-level check for smallIntegers is because both SmallInteger and Float are mapped to Number.
>
> The booleanValueOf call is for implementing the mustBeBoolean machinery (it actually translates directly to DNU, like it is done nowadays in Squeak).
>
> Of course, in Boolean, booleanValueOf is just an alias for valueOf
>
> As you can see, though, this is not terribly efficient, but there is room for improvement/optimizations. With more work, in this case, the _lt call could be replaced by the < operator, and even the _sub and _add calls could be optimized,
> although not completely, since their result can morph into LargeInteger (mapped to BigInt).
>
> As hinted above, SmallInteger is mapped to Number (in the safeInteger range), Float is mapped to Number as well, and LargeInteger is mapped to BigInt.
>
> BlockClosure is mapped to Function, Boolean is mapped to Boolean, Character is mapped to String, weak references are implemented via WeakRef.
> I have briefly considered also doing slightly higher-level mappings, for IdentitySet to Set and IdentityDictionary to Map, but this is not a priority.
>
> The image is serialized sort of like a JavaScript storeString. No processes or contexts though, or rather they are not brought back in on the JavaScript side. Blocks are both stored and loaded.
>
> Non-local returns, unwind blocks, resumable and restartable exceptions are implemented via JavaScript exception handling plus explicit handler block chains associated with the processes.
>
> The "image" starts with the global state loaded, but all processes are started from scratch instead of resumed. A non-UI image is thus trivially started.
>
> One major todo left is hooking up the UI/browser. I did take vm.display.browser.js from SqueakJS and adapted the code in order to implement its numbered primitives, but I still have to work through squeak.js from the same to initialize
> and hook up the display.
>
> Florin
>
>
>
>
> On Sun, Mar 7, 2021 at 11:17 PM Vanessa Freudenberg <vanessa at codefrau.net>
> wrote:
>
>>
>> Hi all,
>>
>> ideas for a faster SqueakJS JIT have been swirling around in my head for
>> many years. This weekend I took the time to write some of them down:
>>
>> https://squeak.js.org/docs/jit.md.html
>>
>> Feedback welcome!
>>
>> Vanessa
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210315/3d27ee21/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 42124 bytes
Desc: not available
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210315/3d27ee21/attachment-0001.png>

From florin.mateoc at gmail.com  Tue Mar 16 01:19:40 2021
From: florin.mateoc at gmail.com (Florin Mateoc)
Date: Mon, 15 Mar 2021 21:19:40 -0400
Subject: [Vm-dev] [SqueakJS] Faster JIT ideas
In-Reply-To: <CAOWGXNAtjzX3jfYf1rcBoy=_UxjgQzFwdnHch93L9WeXGy=dAQ@mail.gmail.com>
References: <CAOWGXNCmyE1N9K87+jKiXM7awb7KbsQws7HKO3uAXAKV9PBc2Q@mail.gmail.com>
 <CA+7DxaY=EC_p2n=rhVyxXfm75v-qASwRu6av=fQj6MSNUJvn3g@mail.gmail.com>
 <CAOWGXNAtjzX3jfYf1rcBoy=_UxjgQzFwdnHch93L9WeXGy=dAQ@mail.gmail.com>
Message-ID: <CA+7DxaZz1k2wnV4DEKaQFX5u2icdoZa1c5Sd-2F-7E-Z-B2ezA@mail.gmail.com>

I don't think the numbers could be meaningfully compared. The whole purpose
of the yield* invocations is to enable process switching, which I don't
think your jitted methods allow for. But then are you merely comparing
yield* invocations against direct invocations? For sure, the yield* ones
will be  much slower.
Let alone my implementation, which also uses yield* for #< , #- and #+, so
it would be much slower than below as operators are surely well optimized
by JS, we can just measure direct vs yield* for the main recursive
invocation:

Number.prototype.benchFib = function benchFib() {
  return this < 2 ? 1 : (this - 1).benchFib() + (this - 2).benchFib() + 1
}

var t = performance.now();
(30).benchFib();
performance.now() - t

gives on my laptop in Firefox 920, 911, 919, 898

Versus

Number.prototype.benchFiby = function* benchFiby() {
   return this < 2 ? 1 : (yield* (this - 1).benchFiby()) + (yield* (this - 2
).benchFiby()) + 1
}

var t = performance.now();
(30).benchFiby().next();
performance.now() - t

gives 2998, 3125, 3116, 3140


On Mon, Mar 15, 2021 at 3:41 PM Vanessa Freudenberg <vanessa at codefrau.net>
wrote:

>
> Hi Florin,
>
> wow, that looks exciting!
>
> This is indeed a much more thorough Squeak-to-JS mapping, where mine is
> more of a "traditional" VM. I love it!
>
> Since my original post I implemented a mockup of "my" new JIT scheme:
>
>
> https://squeak.js.org/docs/jit.md.html#sketch:contextproxieswithintrospectionandinlinecaching/newjitsketch/performanceestimate
> [image: image.png]
> You can play with it here: https://codepen.io/codefrau/pen/JjbmVGw
>
> Could you share the performance numbers you are seeing for your benchFib,
> in comparison to SqueakJS or my mockup? I am curious if yield* is the way
> to go.
>
> Thanks for sharing! And congrats on your new job. My progress is slow too,
> I only work on it some weekends. But then, I'm not in a hurry :)
>
> Vanessa
>
> On Sun, Mar 14, 2021 at 6:18 PM Florin Mateoc <florin.mateoc at gmail.com>
> wrote:
>
>>
>> Hi Vanessa,
>>
>> Sorry for the delay in responding - as somebody who has been inspired by
>> your SqueakJS project, I think I should mention that I am working on a
>> related project, for now tentatively called JsSqueak.
>> In addition to the inspiration provided by SqueakJS, it also scratches my
>> longstanding itch about compiling (transpiling) Squeak.
>>
>> I hesitated to talk about it, as it is still a work in progress - after
>> small bits and pieces that I worked on over a long period, I had the
>> opportunity to spend a significant and uninterrupted chunk of time on it
>> last summer, when I was unemployed for 3 months, and I was able to make
>> good progress. I was optimistically thinking of releasing a first version
>> before the end of last year, but after I started working on my new job,
>> progress on JsSqueak has slowed down significantly. I must confess that I
>> (and especially my wife) hesitate in recreating that productive unemployed
>> situation :)
>>
>> I started with Squeak 4.5 - I already had code transforming Smalltalk
>> code to a form more suitable for translation - and I also started with
>> VMMakerJS-dtl.18 for the plugin generation part. Of course, I had to
>> heavily modify it, since I have to get rid of the stack usage for
>> arguments/receiver and returns.
>> Both of these big parts are working. I also implemented most numbered
>> primitives by hand - they are inlined at generation time in the methods
>> calling them.
>> I am also taking advantage of the latest and greatest additions to
>> JavaScript. I am, of course, using classes, and the parallel class-side
>> hierarchy is implemented using statics. To implement green threads/process
>> switching, all translated methods are implemented as generator functions,
>> and all calls are through yield* expressions. The preemption/interrupt
>> check points are inlined. With this, a process switch is achieved by simply
>> yield-ing (in the process/semaphore primitives).
>> With this, the Integer>>#benchFib method is translated (as a method in
>> Number.prototype, there is one more, simpler, implementation in BigInt) as:
>>
>> *_benchFib() {
>>    if (Number.isSafeInteger(this.valueOf())) { // Effective (inherited or local) source for #benchFib in SmallInteger
>>       /*Handy send-heavy benchmark*/
>>    /*(result // seconds to run) = approx calls per second*/
>>    /* | r t |
>>      t := Time millisecondsToRun: [r := 26 benchFib].
>>      (r // 1000) // t*/
>>    /*138000 on a Mac 8100//100*/
>>    if (GlobalActivationCounter-- < 0) yield* GlobalCheckForInterrupts();
>>
>>    return (yield* this._lt( 2)).booleanValueOf("questionMark:colon:") ? (1) : (yield* (yield* (yield* (yield* this._sub( 1))._benchFib())._add( yield* (yield* this._sub( 2))._benchFib()))._add( 1));
>> } else // No implementation for #benchFib in Float hierarchy, trigger a DNU
>>       return yield* super._benchFib()
>> }
>>
>> The top-level check for smallIntegers is because both SmallInteger and Float are mapped to Number.
>>
>> The booleanValueOf call is for implementing the mustBeBoolean machinery (it actually translates directly to DNU, like it is done nowadays in Squeak).
>>
>> Of course, in Boolean, booleanValueOf is just an alias for valueOf
>>
>> As you can see, though, this is not terribly efficient, but there is room for improvement/optimizations. With more work, in this case, the _lt call could be replaced by the < operator, and even the _sub and _add calls could be optimized,
>> although not completely, since their result can morph into LargeInteger (mapped to BigInt).
>>
>> As hinted above, SmallInteger is mapped to Number (in the safeInteger range), Float is mapped to Number as well, and LargeInteger is mapped to BigInt.
>>
>> BlockClosure is mapped to Function, Boolean is mapped to Boolean, Character is mapped to String, weak references are implemented via WeakRef.
>> I have briefly considered also doing slightly higher-level mappings, for IdentitySet to Set and IdentityDictionary to Map, but this is not a priority.
>>
>> The image is serialized sort of like a JavaScript storeString. No processes or contexts though, or rather they are not brought back in on the JavaScript side. Blocks are both stored and loaded.
>>
>> Non-local returns, unwind blocks, resumable and restartable exceptions are implemented via JavaScript exception handling plus explicit handler block chains associated with the processes.
>>
>> The "image" starts with the global state loaded, but all processes are started from scratch instead of resumed. A non-UI image is thus trivially started.
>>
>> One major todo left is hooking up the UI/browser. I did take vm.display.browser.js from SqueakJS and adapted the code in order to implement its numbered primitives, but I still have to work through squeak.js from the same to initialize
>> and hook up the display.
>>
>> Florin
>>
>>
>>
>>
>> On Sun, Mar 7, 2021 at 11:17 PM Vanessa Freudenberg <vanessa at codefrau.net>
>> wrote:
>>
>>>
>>> Hi all,
>>>
>>> ideas for a faster SqueakJS JIT have been swirling around in my head for
>>> many years. This weekend I took the time to write some of them down:
>>>
>>> https://squeak.js.org/docs/jit.md.html
>>>
>>> Feedback welcome!
>>>
>>> Vanessa
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210315/73e5ff9c/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 42124 bytes
Desc: not available
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210315/73e5ff9c/attachment-0001.png>

From vanessa at codefrau.net  Tue Mar 16 02:28:01 2021
From: vanessa at codefrau.net (Vanessa Freudenberg)
Date: Mon, 15 Mar 2021 19:28:01 -0700
Subject: [Vm-dev] [SqueakJS] Faster JIT ideas
In-Reply-To: <CA+7DxaZz1k2wnV4DEKaQFX5u2icdoZa1c5Sd-2F-7E-Z-B2ezA@mail.gmail.com>
References: <CAOWGXNCmyE1N9K87+jKiXM7awb7KbsQws7HKO3uAXAKV9PBc2Q@mail.gmail.com>
 <CA+7DxaY=EC_p2n=rhVyxXfm75v-qASwRu6av=fQj6MSNUJvn3g@mail.gmail.com>
 <CAOWGXNAtjzX3jfYf1rcBoy=_UxjgQzFwdnHch93L9WeXGy=dAQ@mail.gmail.com>
 <CA+7DxaZz1k2wnV4DEKaQFX5u2icdoZa1c5Sd-2F-7E-Z-B2ezA@mail.gmail.com>
Message-ID: <CAOWGXND059RWN8woaDZrhGRocf=qG8gX8AtxDZdUrnYJmZd8nw@mail.gmail.com>

Thanks for that Florin, very helpful!

I'm curious why you need to make every send into a generator call, and not
just rely on the one in your GlobalCheckForInterrupts()?

My implementation is intended to allow context switching that way:

if (--vm.interruptCheckCounter <= 0 &&
vm.handleDepthAndInterrupts(depth, thisProxy) === true) return false;


This is the same technique as in other VMs, where the actual check for
context switch is not done for every send, but only when
the interruptCheckCounter goes below zero. In the first mockup on codepen,
vm.handleDepthAndInterrupts prints the contents of all the contexts,
proving that the information is accessible in case we need to context
switch or reify the stack. My hope was that inside of
handleDepthAndInterrupts() I could use *yield to do the actual context
switching.

My second, exception based mockup on codepen uses the same approach, except
that when a context switch is needed it would throw, unwinding the stack
fully, and creating actual context objects along the way. I have not mocked
up that part yet (because it would need a mockup interpreter too, to
continue after unwind), but you can see the unwind working correctly by
uncommenting the line with
// throw Error("unwind")

I tried your benchFib vs benchFiby on Chrome, which seems to
optimize generators a lot better than Firefox. On Chrome the overhead is
just 50% or so, vs 300% on Firefox. Safari appears to be between the two.

I will need to make more complete mockups before deciding on a design.
Especially how closures would be handled. Does anyone have a tiny
one-method benchmark that would highlight closure performance?

Vanessa

On Mon, Mar 15, 2021 at 6:20 PM Florin Mateoc <florin.mateoc at gmail.com>
wrote:

>
> I don't think the numbers could be meaningfully compared. The whole
> purpose of the yield* invocations is to enable process switching, which I
> don't think your jitted methods allow for. But then are you merely
> comparing yield* invocations against direct invocations? For sure, the
> yield* ones will be  much slower.
> Let alone my implementation, which also uses yield* for #< , #- and #+, so
> it would be much slower than below as operators are surely well optimized
> by JS, we can just measure direct vs yield* for the main recursive
> invocation:
>
> Number.prototype.benchFib = function benchFib() {
>   return this < 2 ? 1 : (this - 1).benchFib() + (this - 2).benchFib() + 1
> }
>
> var t = performance.now();
> (30).benchFib();
> performance.now() - t
>
> gives on my laptop in Firefox 920, 911, 919, 898
>
> Versus
>
> Number.prototype.benchFiby = function* benchFiby() {
>    return this < 2 ? 1 : (yield* (this - 1).benchFiby()) + (yield* (this -
> 2).benchFiby()) + 1
> }
>
> var t = performance.now();
> (30).benchFiby().next();
> performance.now() - t
>
> gives 2998, 3125, 3116, 3140
>
>
> On Mon, Mar 15, 2021 at 3:41 PM Vanessa Freudenberg <vanessa at codefrau.net>
> wrote:
>
>>
>> Hi Florin,
>>
>> wow, that looks exciting!
>>
>> This is indeed a much more thorough Squeak-to-JS mapping, where mine is
>> more of a "traditional" VM. I love it!
>>
>> Since my original post I implemented a mockup of "my" new JIT scheme:
>>
>>
>> https://squeak.js.org/docs/jit.md.html#sketch:contextproxieswithintrospectionandinlinecaching/newjitsketch/performanceestimate
>> [image: image.png]
>> You can play with it here: https://codepen.io/codefrau/pen/JjbmVGw
>>
>> Could you share the performance numbers you are seeing for your benchFib,
>> in comparison to SqueakJS or my mockup? I am curious if yield* is the
>> way to go.
>>
>> Thanks for sharing! And congrats on your new job. My progress is slow
>> too, I only work on it some weekends. But then, I'm not in a hurry :)
>>
>> Vanessa
>>
>> On Sun, Mar 14, 2021 at 6:18 PM Florin Mateoc <florin.mateoc at gmail.com>
>> wrote:
>>
>>>
>>> Hi Vanessa,
>>>
>>> Sorry for the delay in responding - as somebody who has been inspired by
>>> your SqueakJS project, I think I should mention that I am working on a
>>> related project, for now tentatively called JsSqueak.
>>> In addition to the inspiration provided by SqueakJS, it also scratches
>>> my longstanding itch about compiling (transpiling) Squeak.
>>>
>>> I hesitated to talk about it, as it is still a work in progress - after
>>> small bits and pieces that I worked on over a long period, I had the
>>> opportunity to spend a significant and uninterrupted chunk of time on it
>>> last summer, when I was unemployed for 3 months, and I was able to make
>>> good progress. I was optimistically thinking of releasing a first version
>>> before the end of last year, but after I started working on my new job,
>>> progress on JsSqueak has slowed down significantly. I must confess that I
>>> (and especially my wife) hesitate in recreating that productive unemployed
>>> situation :)
>>>
>>> I started with Squeak 4.5 - I already had code transforming Smalltalk
>>> code to a form more suitable for translation - and I also started with
>>> VMMakerJS-dtl.18 for the plugin generation part. Of course, I had to
>>> heavily modify it, since I have to get rid of the stack usage for
>>> arguments/receiver and returns.
>>> Both of these big parts are working. I also implemented most numbered
>>> primitives by hand - they are inlined at generation time in the methods
>>> calling them.
>>> I am also taking advantage of the latest and greatest additions to
>>> JavaScript. I am, of course, using classes, and the parallel class-side
>>> hierarchy is implemented using statics. To implement green threads/process
>>> switching, all translated methods are implemented as generator functions,
>>> and all calls are through yield* expressions. The preemption/interrupt
>>> check points are inlined. With this, a process switch is achieved by simply
>>> yield-ing (in the process/semaphore primitives).
>>> With this, the Integer>>#benchFib method is translated (as a method in
>>> Number.prototype, there is one more, simpler, implementation in BigInt) as:
>>>
>>> *_benchFib() {
>>>    if (Number.isSafeInteger(this.valueOf())) { // Effective (inherited or local) source for #benchFib in SmallInteger
>>>       /*Handy send-heavy benchmark*/
>>>    /*(result // seconds to run) = approx calls per second*/
>>>    /* | r t |
>>>      t := Time millisecondsToRun: [r := 26 benchFib].
>>>      (r // 1000) // t*/
>>>    /*138000 on a Mac 8100//100*/
>>>    if (GlobalActivationCounter-- < 0) yield* GlobalCheckForInterrupts();
>>>
>>>    return (yield* this._lt( 2)).booleanValueOf("questionMark:colon:") ? (1) : (yield* (yield* (yield* (yield* this._sub( 1))._benchFib())._add( yield* (yield* this._sub( 2))._benchFib()))._add( 1));
>>> } else // No implementation for #benchFib in Float hierarchy, trigger a DNU
>>>       return yield* super._benchFib()
>>> }
>>>
>>> The top-level check for smallIntegers is because both SmallInteger and Float are mapped to Number.
>>>
>>> The booleanValueOf call is for implementing the mustBeBoolean machinery (it actually translates directly to DNU, like it is done nowadays in Squeak).
>>>
>>> Of course, in Boolean, booleanValueOf is just an alias for valueOf
>>>
>>> As you can see, though, this is not terribly efficient, but there is room for improvement/optimizations. With more work, in this case, the _lt call could be replaced by the < operator, and even the _sub and _add calls could be optimized,
>>> although not completely, since their result can morph into LargeInteger (mapped to BigInt).
>>>
>>> As hinted above, SmallInteger is mapped to Number (in the safeInteger range), Float is mapped to Number as well, and LargeInteger is mapped to BigInt.
>>>
>>> BlockClosure is mapped to Function, Boolean is mapped to Boolean, Character is mapped to String, weak references are implemented via WeakRef.
>>> I have briefly considered also doing slightly higher-level mappings, for IdentitySet to Set and IdentityDictionary to Map, but this is not a priority.
>>>
>>> The image is serialized sort of like a JavaScript storeString. No processes or contexts though, or rather they are not brought back in on the JavaScript side. Blocks are both stored and loaded.
>>>
>>> Non-local returns, unwind blocks, resumable and restartable exceptions are implemented via JavaScript exception handling plus explicit handler block chains associated with the processes.
>>>
>>> The "image" starts with the global state loaded, but all processes are started from scratch instead of resumed. A non-UI image is thus trivially started.
>>>
>>> One major todo left is hooking up the UI/browser. I did take vm.display.browser.js from SqueakJS and adapted the code in order to implement its numbered primitives, but I still have to work through squeak.js from the same to initialize
>>> and hook up the display.
>>>
>>> Florin
>>>
>>>
>>>
>>>
>>> On Sun, Mar 7, 2021 at 11:17 PM Vanessa Freudenberg <
>>> vanessa at codefrau.net> wrote:
>>>
>>>>
>>>> Hi all,
>>>>
>>>> ideas for a faster SqueakJS JIT have been swirling around in my head
>>>> for many years. This weekend I took the time to write some of them down:
>>>>
>>>> https://squeak.js.org/docs/jit.md.html
>>>>
>>>> Feedback welcome!
>>>>
>>>> Vanessa
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210315/eae7c912/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 42124 bytes
Desc: not available
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210315/eae7c912/attachment-0001.png>

From florin.mateoc at gmail.com  Tue Mar 16 02:56:47 2021
From: florin.mateoc at gmail.com (Florin Mateoc)
Date: Mon, 15 Mar 2021 22:56:47 -0400
Subject: [Vm-dev] [SqueakJS] Faster JIT ideas
In-Reply-To: <CAOWGXND059RWN8woaDZrhGRocf=qG8gX8AtxDZdUrnYJmZd8nw@mail.gmail.com>
References: <CAOWGXNCmyE1N9K87+jKiXM7awb7KbsQws7HKO3uAXAKV9PBc2Q@mail.gmail.com>
 <CA+7DxaY=EC_p2n=rhVyxXfm75v-qASwRu6av=fQj6MSNUJvn3g@mail.gmail.com>
 <CAOWGXNAtjzX3jfYf1rcBoy=_UxjgQzFwdnHch93L9WeXGy=dAQ@mail.gmail.com>
 <CA+7DxaZz1k2wnV4DEKaQFX5u2icdoZa1c5Sd-2F-7E-Z-B2ezA@mail.gmail.com>
 <CAOWGXND059RWN8woaDZrhGRocf=qG8gX8AtxDZdUrnYJmZd8nw@mail.gmail.com>
Message-ID: <CA+7DxaZcmfcj8AEfTxg6hHBigiTmq=CjRmi9tQrHLmaMsjiokQ@mail.gmail.com>

Well, who makes the generator call inside GlobalCheckForInterrupts if the
method hosting the GlobalCheckForInterrupts call was not called via yield*?
yield* needs to traverse the whole execution stack. If you do a top-level
next(), no inner invocations will occur if they are not generator calls.

Sure, you can print the stack, but have you checked that you can switch and
then come back and resume an interrupted one?

Anyway, thank you for the good news with Chrome. I will also check with Node

On Mon, Mar 15, 2021 at 10:28 PM Vanessa Freudenberg <vanessa at codefrau.net>
wrote:

>
> Thanks for that Florin, very helpful!
>
> I'm curious why you need to make every send into a generator call, and not
> just rely on the one in your GlobalCheckForInterrupts()?
>
> My implementation is intended to allow context switching that way:
>
> if (--vm.interruptCheckCounter <= 0 && vm.handleDepthAndInterrupts(depth, thisProxy) === true) return false;
>
>
> This is the same technique as in other VMs, where the actual check for
> context switch is not done for every send, but only when
> the interruptCheckCounter goes below zero. In the first mockup on codepen,
> vm.handleDepthAndInterrupts prints the contents of all the contexts,
> proving that the information is accessible in case we need to context
> switch or reify the stack. My hope was that inside of
> handleDepthAndInterrupts() I could use *yield to do the actual context
> switching.
>
> My second, exception based mockup on codepen uses the same approach,
> except that when a context switch is needed it would throw, unwinding the
> stack fully, and creating actual context objects along the way. I have not
> mocked up that part yet (because it would need a mockup interpreter too, to
> continue after unwind), but you can see the unwind working correctly by
> uncommenting the line with
> // throw Error("unwind")
>
> I tried your benchFib vs benchFiby on Chrome, which seems to
> optimize generators a lot better than Firefox. On Chrome the overhead is
> just 50% or so, vs 300% on Firefox. Safari appears to be between the two.
>
> I will need to make more complete mockups before deciding on a design.
> Especially how closures would be handled. Does anyone have a tiny
> one-method benchmark that would highlight closure performance?
>
> Vanessa
>
> On Mon, Mar 15, 2021 at 6:20 PM Florin Mateoc <florin.mateoc at gmail.com>
> wrote:
>
>>
>> I don't think the numbers could be meaningfully compared. The whole
>> purpose of the yield* invocations is to enable process switching, which I
>> don't think your jitted methods allow for. But then are you merely
>> comparing yield* invocations against direct invocations? For sure, the
>> yield* ones will be  much slower.
>> Let alone my implementation, which also uses yield* for #< , #- and #+,
>> so it would be much slower than below as operators are surely well
>> optimized by JS, we can just measure direct vs yield* for the main
>> recursive invocation:
>>
>> Number.prototype.benchFib = function benchFib() {
>>   return this < 2 ? 1 : (this - 1).benchFib() + (this - 2).benchFib() + 1
>> }
>>
>> var t = performance.now();
>> (30).benchFib();
>> performance.now() - t
>>
>> gives on my laptop in Firefox 920, 911, 919, 898
>>
>> Versus
>>
>> Number.prototype.benchFiby = function* benchFiby() {
>>    return this < 2 ? 1 : (yield* (this - 1).benchFiby()) + (yield* (this
>> - 2).benchFiby()) + 1
>> }
>>
>> var t = performance.now();
>> (30).benchFiby().next();
>> performance.now() - t
>>
>> gives 2998, 3125, 3116, 3140
>>
>>
>> On Mon, Mar 15, 2021 at 3:41 PM Vanessa Freudenberg <vanessa at codefrau.net>
>> wrote:
>>
>>>
>>> Hi Florin,
>>>
>>> wow, that looks exciting!
>>>
>>> This is indeed a much more thorough Squeak-to-JS mapping, where mine is
>>> more of a "traditional" VM. I love it!
>>>
>>> Since my original post I implemented a mockup of "my" new JIT scheme:
>>>
>>>
>>> https://squeak.js.org/docs/jit.md.html#sketch:contextproxieswithintrospectionandinlinecaching/newjitsketch/performanceestimate
>>> [image: image.png]
>>> You can play with it here: https://codepen.io/codefrau/pen/JjbmVGw
>>>
>>> Could you share the performance numbers you are seeing for your
>>> benchFib, in comparison to SqueakJS or my mockup? I am curious if yield*
>>> is the way to go.
>>>
>>> Thanks for sharing! And congrats on your new job. My progress is slow
>>> too, I only work on it some weekends. But then, I'm not in a hurry :)
>>>
>>> Vanessa
>>>
>>> On Sun, Mar 14, 2021 at 6:18 PM Florin Mateoc <florin.mateoc at gmail.com>
>>> wrote:
>>>
>>>>
>>>> Hi Vanessa,
>>>>
>>>> Sorry for the delay in responding - as somebody who has been inspired
>>>> by your SqueakJS project, I think I should mention that I am working on a
>>>> related project, for now tentatively called JsSqueak.
>>>> In addition to the inspiration provided by SqueakJS, it also scratches
>>>> my longstanding itch about compiling (transpiling) Squeak.
>>>>
>>>> I hesitated to talk about it, as it is still a work in progress - after
>>>> small bits and pieces that I worked on over a long period, I had the
>>>> opportunity to spend a significant and uninterrupted chunk of time on it
>>>> last summer, when I was unemployed for 3 months, and I was able to make
>>>> good progress. I was optimistically thinking of releasing a first version
>>>> before the end of last year, but after I started working on my new job,
>>>> progress on JsSqueak has slowed down significantly. I must confess that I
>>>> (and especially my wife) hesitate in recreating that productive unemployed
>>>> situation :)
>>>>
>>>> I started with Squeak 4.5 - I already had code transforming Smalltalk
>>>> code to a form more suitable for translation - and I also started with
>>>> VMMakerJS-dtl.18 for the plugin generation part. Of course, I had to
>>>> heavily modify it, since I have to get rid of the stack usage for
>>>> arguments/receiver and returns.
>>>> Both of these big parts are working. I also implemented most numbered
>>>> primitives by hand - they are inlined at generation time in the methods
>>>> calling them.
>>>> I am also taking advantage of the latest and greatest additions to
>>>> JavaScript. I am, of course, using classes, and the parallel class-side
>>>> hierarchy is implemented using statics. To implement green threads/process
>>>> switching, all translated methods are implemented as generator functions,
>>>> and all calls are through yield* expressions. The preemption/interrupt
>>>> check points are inlined. With this, a process switch is achieved by simply
>>>> yield-ing (in the process/semaphore primitives).
>>>> With this, the Integer>>#benchFib method is translated (as a method in
>>>> Number.prototype, there is one more, simpler, implementation in BigInt) as:
>>>>
>>>> *_benchFib() {
>>>>    if (Number.isSafeInteger(this.valueOf())) { // Effective (inherited or local) source for #benchFib in SmallInteger
>>>>       /*Handy send-heavy benchmark*/
>>>>    /*(result // seconds to run) = approx calls per second*/
>>>>    /* | r t |
>>>>      t := Time millisecondsToRun: [r := 26 benchFib].
>>>>      (r // 1000) // t*/
>>>>    /*138000 on a Mac 8100//100*/
>>>>    if (GlobalActivationCounter-- < 0) yield* GlobalCheckForInterrupts();
>>>>
>>>>    return (yield* this._lt( 2)).booleanValueOf("questionMark:colon:") ? (1) : (yield* (yield* (yield* (yield* this._sub( 1))._benchFib())._add( yield* (yield* this._sub( 2))._benchFib()))._add( 1));
>>>> } else // No implementation for #benchFib in Float hierarchy, trigger a DNU
>>>>       return yield* super._benchFib()
>>>> }
>>>>
>>>> The top-level check for smallIntegers is because both SmallInteger and Float are mapped to Number.
>>>>
>>>> The booleanValueOf call is for implementing the mustBeBoolean machinery (it actually translates directly to DNU, like it is done nowadays in Squeak).
>>>>
>>>> Of course, in Boolean, booleanValueOf is just an alias for valueOf
>>>>
>>>> As you can see, though, this is not terribly efficient, but there is room for improvement/optimizations. With more work, in this case, the _lt call could be replaced by the < operator, and even the _sub and _add calls could be optimized,
>>>> although not completely, since their result can morph into LargeInteger (mapped to BigInt).
>>>>
>>>> As hinted above, SmallInteger is mapped to Number (in the safeInteger range), Float is mapped to Number as well, and LargeInteger is mapped to BigInt.
>>>>
>>>> BlockClosure is mapped to Function, Boolean is mapped to Boolean, Character is mapped to String, weak references are implemented via WeakRef.
>>>> I have briefly considered also doing slightly higher-level mappings, for IdentitySet to Set and IdentityDictionary to Map, but this is not a priority.
>>>>
>>>> The image is serialized sort of like a JavaScript storeString. No processes or contexts though, or rather they are not brought back in on the JavaScript side. Blocks are both stored and loaded.
>>>>
>>>> Non-local returns, unwind blocks, resumable and restartable exceptions are implemented via JavaScript exception handling plus explicit handler block chains associated with the processes.
>>>>
>>>> The "image" starts with the global state loaded, but all processes are started from scratch instead of resumed. A non-UI image is thus trivially started.
>>>>
>>>> One major todo left is hooking up the UI/browser. I did take vm.display.browser.js from SqueakJS and adapted the code in order to implement its numbered primitives, but I still have to work through squeak.js from the same to initialize
>>>> and hook up the display.
>>>>
>>>> Florin
>>>>
>>>>
>>>>
>>>>
>>>> On Sun, Mar 7, 2021 at 11:17 PM Vanessa Freudenberg <
>>>> vanessa at codefrau.net> wrote:
>>>>
>>>>>
>>>>> Hi all,
>>>>>
>>>>> ideas for a faster SqueakJS JIT have been swirling around in my head
>>>>> for many years. This weekend I took the time to write some of them down:
>>>>>
>>>>> https://squeak.js.org/docs/jit.md.html
>>>>>
>>>>> Feedback welcome!
>>>>>
>>>>> Vanessa
>>>>>
>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210315/befd8c29/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 42124 bytes
Desc: not available
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210315/befd8c29/attachment-0001.png>

From florin.mateoc at gmail.com  Tue Mar 16 03:12:56 2021
From: florin.mateoc at gmail.com (Florin Mateoc)
Date: Mon, 15 Mar 2021 23:12:56 -0400
Subject: [Vm-dev] [SqueakJS] Faster JIT ideas
In-Reply-To: <CA+7DxaZcmfcj8AEfTxg6hHBigiTmq=CjRmi9tQrHLmaMsjiokQ@mail.gmail.com>
References: <CAOWGXNCmyE1N9K87+jKiXM7awb7KbsQws7HKO3uAXAKV9PBc2Q@mail.gmail.com>
 <CA+7DxaY=EC_p2n=rhVyxXfm75v-qASwRu6av=fQj6MSNUJvn3g@mail.gmail.com>
 <CAOWGXNAtjzX3jfYf1rcBoy=_UxjgQzFwdnHch93L9WeXGy=dAQ@mail.gmail.com>
 <CA+7DxaZz1k2wnV4DEKaQFX5u2icdoZa1c5Sd-2F-7E-Z-B2ezA@mail.gmail.com>
 <CAOWGXND059RWN8woaDZrhGRocf=qG8gX8AtxDZdUrnYJmZd8nw@mail.gmail.com>
 <CA+7DxaZcmfcj8AEfTxg6hHBigiTmq=CjRmi9tQrHLmaMsjiokQ@mail.gmail.com>
Message-ID: <CA+7DxaZpD0sn=agj7brmY-C0MbV6iAkRqT=oPyfoPu+9hi1xBw@mail.gmail.com>

Perhaps better expressed: since benchFiby is a generator function, the call
benchFiby() does not execute its contents (so it cannot reach any potential
yields inside it), it just returns a generator, so we have to make all the
calls generator calls

On Mon, Mar 15, 2021 at 10:56 PM Florin Mateoc <florin.mateoc at gmail.com>
wrote:

> Well, who makes the generator call inside GlobalCheckForInterrupts if the
> method hosting the GlobalCheckForInterrupts call was not called via yield*?
> yield* needs to traverse the whole execution stack. If you do a top-level
> next(), no inner invocations will occur if they are not generator calls.
>
> Sure, you can print the stack, but have you checked that you can switch
> and then come back and resume an interrupted one?
>
> Anyway, thank you for the good news with Chrome. I will also check with
> Node
>
> On Mon, Mar 15, 2021 at 10:28 PM Vanessa Freudenberg <vanessa at codefrau.net>
> wrote:
>
>>
>> Thanks for that Florin, very helpful!
>>
>> I'm curious why you need to make every send into a generator call, and
>> not just rely on the one in your GlobalCheckForInterrupts()?
>>
>> My implementation is intended to allow context switching that way:
>>
>> if (--vm.interruptCheckCounter <= 0 && vm.handleDepthAndInterrupts(depth, thisProxy) === true) return false;
>>
>>
>> This is the same technique as in other VMs, where the actual check for
>> context switch is not done for every send, but only when
>> the interruptCheckCounter goes below zero. In the first mockup on codepen,
>> vm.handleDepthAndInterrupts prints the contents of all the contexts,
>> proving that the information is accessible in case we need to context
>> switch or reify the stack. My hope was that inside of
>> handleDepthAndInterrupts() I could use *yield to do the actual context
>> switching.
>>
>> My second, exception based mockup on codepen uses the same approach,
>> except that when a context switch is needed it would throw, unwinding the
>> stack fully, and creating actual context objects along the way. I have not
>> mocked up that part yet (because it would need a mockup interpreter too, to
>> continue after unwind), but you can see the unwind working correctly by
>> uncommenting the line with
>> // throw Error("unwind")
>>
>> I tried your benchFib vs benchFiby on Chrome, which seems to
>> optimize generators a lot better than Firefox. On Chrome the overhead is
>> just 50% or so, vs 300% on Firefox. Safari appears to be between the two.
>>
>> I will need to make more complete mockups before deciding on a design.
>> Especially how closures would be handled. Does anyone have a tiny
>> one-method benchmark that would highlight closure performance?
>>
>> Vanessa
>>
>> On Mon, Mar 15, 2021 at 6:20 PM Florin Mateoc <florin.mateoc at gmail.com>
>> wrote:
>>
>>>
>>> I don't think the numbers could be meaningfully compared. The whole
>>> purpose of the yield* invocations is to enable process switching, which I
>>> don't think your jitted methods allow for. But then are you merely
>>> comparing yield* invocations against direct invocations? For sure, the
>>> yield* ones will be  much slower.
>>> Let alone my implementation, which also uses yield* for #< , #- and #+,
>>> so it would be much slower than below as operators are surely well
>>> optimized by JS, we can just measure direct vs yield* for the main
>>> recursive invocation:
>>>
>>> Number.prototype.benchFib = function benchFib() {
>>>   return this < 2 ? 1 : (this - 1).benchFib() + (this - 2).benchFib() + 1
>>> }
>>>
>>> var t = performance.now();
>>> (30).benchFib();
>>> performance.now() - t
>>>
>>> gives on my laptop in Firefox 920, 911, 919, 898
>>>
>>> Versus
>>>
>>> Number.prototype.benchFiby = function* benchFiby() {
>>>    return this < 2 ? 1 : (yield* (this - 1).benchFiby()) + (yield* (this
>>> - 2).benchFiby()) + 1
>>> }
>>>
>>> var t = performance.now();
>>> (30).benchFiby().next();
>>> performance.now() - t
>>>
>>> gives 2998, 3125, 3116, 3140
>>>
>>>
>>> On Mon, Mar 15, 2021 at 3:41 PM Vanessa Freudenberg <
>>> vanessa at codefrau.net> wrote:
>>>
>>>>
>>>> Hi Florin,
>>>>
>>>> wow, that looks exciting!
>>>>
>>>> This is indeed a much more thorough Squeak-to-JS mapping, where mine is
>>>> more of a "traditional" VM. I love it!
>>>>
>>>> Since my original post I implemented a mockup of "my" new JIT scheme:
>>>>
>>>>
>>>> https://squeak.js.org/docs/jit.md.html#sketch:contextproxieswithintrospectionandinlinecaching/newjitsketch/performanceestimate
>>>> [image: image.png]
>>>> You can play with it here: https://codepen.io/codefrau/pen/JjbmVGw
>>>>
>>>> Could you share the performance numbers you are seeing for your
>>>> benchFib, in comparison to SqueakJS or my mockup? I am curious if
>>>> yield* is the way to go.
>>>>
>>>> Thanks for sharing! And congrats on your new job. My progress is slow
>>>> too, I only work on it some weekends. But then, I'm not in a hurry :)
>>>>
>>>> Vanessa
>>>>
>>>> On Sun, Mar 14, 2021 at 6:18 PM Florin Mateoc <florin.mateoc at gmail.com>
>>>> wrote:
>>>>
>>>>>
>>>>> Hi Vanessa,
>>>>>
>>>>> Sorry for the delay in responding - as somebody who has been inspired
>>>>> by your SqueakJS project, I think I should mention that I am working on a
>>>>> related project, for now tentatively called JsSqueak.
>>>>> In addition to the inspiration provided by SqueakJS, it also scratches
>>>>> my longstanding itch about compiling (transpiling) Squeak.
>>>>>
>>>>> I hesitated to talk about it, as it is still a work in progress -
>>>>> after small bits and pieces that I worked on over a long period, I had the
>>>>> opportunity to spend a significant and uninterrupted chunk of time on it
>>>>> last summer, when I was unemployed for 3 months, and I was able to make
>>>>> good progress. I was optimistically thinking of releasing a first version
>>>>> before the end of last year, but after I started working on my new job,
>>>>> progress on JsSqueak has slowed down significantly. I must confess that I
>>>>> (and especially my wife) hesitate in recreating that productive unemployed
>>>>> situation :)
>>>>>
>>>>> I started with Squeak 4.5 - I already had code transforming Smalltalk
>>>>> code to a form more suitable for translation - and I also started with
>>>>> VMMakerJS-dtl.18 for the plugin generation part. Of course, I had to
>>>>> heavily modify it, since I have to get rid of the stack usage for
>>>>> arguments/receiver and returns.
>>>>> Both of these big parts are working. I also implemented most numbered
>>>>> primitives by hand - they are inlined at generation time in the methods
>>>>> calling them.
>>>>> I am also taking advantage of the latest and greatest additions to
>>>>> JavaScript. I am, of course, using classes, and the parallel class-side
>>>>> hierarchy is implemented using statics. To implement green threads/process
>>>>> switching, all translated methods are implemented as generator functions,
>>>>> and all calls are through yield* expressions. The preemption/interrupt
>>>>> check points are inlined. With this, a process switch is achieved by simply
>>>>> yield-ing (in the process/semaphore primitives).
>>>>> With this, the Integer>>#benchFib method is translated (as a method in
>>>>> Number.prototype, there is one more, simpler, implementation in BigInt) as:
>>>>>
>>>>> *_benchFib() {
>>>>>    if (Number.isSafeInteger(this.valueOf())) { // Effective (inherited or local) source for #benchFib in SmallInteger
>>>>>       /*Handy send-heavy benchmark*/
>>>>>    /*(result // seconds to run) = approx calls per second*/
>>>>>    /* | r t |
>>>>>      t := Time millisecondsToRun: [r := 26 benchFib].
>>>>>      (r // 1000) // t*/
>>>>>    /*138000 on a Mac 8100//100*/
>>>>>    if (GlobalActivationCounter-- < 0) yield* GlobalCheckForInterrupts();
>>>>>
>>>>>    return (yield* this._lt( 2)).booleanValueOf("questionMark:colon:") ? (1) : (yield* (yield* (yield* (yield* this._sub( 1))._benchFib())._add( yield* (yield* this._sub( 2))._benchFib()))._add( 1));
>>>>> } else // No implementation for #benchFib in Float hierarchy, trigger a DNU
>>>>>       return yield* super._benchFib()
>>>>> }
>>>>>
>>>>> The top-level check for smallIntegers is because both SmallInteger and Float are mapped to Number.
>>>>>
>>>>> The booleanValueOf call is for implementing the mustBeBoolean machinery (it actually translates directly to DNU, like it is done nowadays in Squeak).
>>>>>
>>>>> Of course, in Boolean, booleanValueOf is just an alias for valueOf
>>>>>
>>>>> As you can see, though, this is not terribly efficient, but there is room for improvement/optimizations. With more work, in this case, the _lt call could be replaced by the < operator, and even the _sub and _add calls could be optimized,
>>>>> although not completely, since their result can morph into LargeInteger (mapped to BigInt).
>>>>>
>>>>> As hinted above, SmallInteger is mapped to Number (in the safeInteger range), Float is mapped to Number as well, and LargeInteger is mapped to BigInt.
>>>>>
>>>>> BlockClosure is mapped to Function, Boolean is mapped to Boolean, Character is mapped to String, weak references are implemented via WeakRef.
>>>>> I have briefly considered also doing slightly higher-level mappings, for IdentitySet to Set and IdentityDictionary to Map, but this is not a priority.
>>>>>
>>>>> The image is serialized sort of like a JavaScript storeString. No processes or contexts though, or rather they are not brought back in on the JavaScript side. Blocks are both stored and loaded.
>>>>>
>>>>> Non-local returns, unwind blocks, resumable and restartable exceptions are implemented via JavaScript exception handling plus explicit handler block chains associated with the processes.
>>>>>
>>>>> The "image" starts with the global state loaded, but all processes are started from scratch instead of resumed. A non-UI image is thus trivially started.
>>>>>
>>>>> One major todo left is hooking up the UI/browser. I did take vm.display.browser.js from SqueakJS and adapted the code in order to implement its numbered primitives, but I still have to work through squeak.js from the same to initialize
>>>>> and hook up the display.
>>>>>
>>>>> Florin
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Sun, Mar 7, 2021 at 11:17 PM Vanessa Freudenberg <
>>>>> vanessa at codefrau.net> wrote:
>>>>>
>>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> ideas for a faster SqueakJS JIT have been swirling around in my head
>>>>>> for many years. This weekend I took the time to write some of them down:
>>>>>>
>>>>>> https://squeak.js.org/docs/jit.md.html
>>>>>>
>>>>>> Feedback welcome!
>>>>>>
>>>>>> Vanessa
>>>>>>
>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210315/86c83329/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 42124 bytes
Desc: not available
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210315/86c83329/attachment-0001.png>

From vanessa at codefrau.net  Tue Mar 16 08:38:33 2021
From: vanessa at codefrau.net (Vanessa Freudenberg)
Date: Tue, 16 Mar 2021 01:38:33 -0700
Subject: [Vm-dev] [SqueakJS] Faster JIT ideas
In-Reply-To: <CA+7DxaZcmfcj8AEfTxg6hHBigiTmq=CjRmi9tQrHLmaMsjiokQ@mail.gmail.com>
References: <CAOWGXNCmyE1N9K87+jKiXM7awb7KbsQws7HKO3uAXAKV9PBc2Q@mail.gmail.com>
 <CA+7DxaY=EC_p2n=rhVyxXfm75v-qASwRu6av=fQj6MSNUJvn3g@mail.gmail.com>
 <CAOWGXNAtjzX3jfYf1rcBoy=_UxjgQzFwdnHch93L9WeXGy=dAQ@mail.gmail.com>
 <CA+7DxaZz1k2wnV4DEKaQFX5u2icdoZa1c5Sd-2F-7E-Z-B2ezA@mail.gmail.com>
 <CAOWGXND059RWN8woaDZrhGRocf=qG8gX8AtxDZdUrnYJmZd8nw@mail.gmail.com>
 <CA+7DxaZcmfcj8AEfTxg6hHBigiTmq=CjRmi9tQrHLmaMsjiokQ@mail.gmail.com>
Message-ID: <CAOWGXNCrHKzDGZddU-zZ4YFHnuKEkOwsu1jbeHsH17XLnTnAtw@mail.gmail.com>

On Mon, Mar 15, 2021 at 7:57 PM Florin Mateoc <florin.mateoc at gmail.com>
wrote:

Perhaps better expressed: since benchFiby is a generator function, the call
> benchFiby() does not execute its contents (so it cannot reach any potential
> yields inside it), it just returns a generator, so we have to make all the
> calls generator calls


Ah, that's a JS thing I had not understood yet, that all calls to generator
functions need to be made using yield*, and you can only use yield* inside
of a generator function.


> Well, who makes the generator call inside GlobalCheckForInterrupts if the
> method hosting the GlobalCheckForInterrupts call was not called via yield*?
>

I think that would be the trampoline function that goes from interpreted
code to compiled code. That same function needs to handle grabbing function
args off the context's stack, passing them as arguments to the jitted
method, and once that returns, popping the stack + pushing the result.

yield* needs to traverse the whole execution stack. If you do a top-level
> next(), no inner invocations will occur if they are not generator calls.
>

Okay. I'll have to think about that. My hope was that all the regular
execution can happen using regular function invocations and only process
switching would use yield. But you're right, we need to use yield* all the
way down. Hope that's not too expensive ... or maybe unwinding to process
switch is the better tradeoff after all.

Sure, you can print the stack, but have you checked that you can switch and
> then come back and resume an interrupted one?
>

Not yet, I'm still designing it and trying things.


> Anyway, thank you for the good news with Chrome. I will also check with
> Node
>

Meanwhile, I made a little scheduler just to learn how to use yield*. Works
nicely:
https://codepen.io/codefrau/pen/jOVRmxm

Vanessa


> On Mon, Mar 15, 2021 at 10:28 PM Vanessa Freudenberg <vanessa at codefrau.net>
> wrote:


>> Thanks for that Florin, very helpful!
>>
>> I'm curious why you need to make every send into a generator call, and
>> not just rely on the one in your GlobalCheckForInterrupts()?
>>
>> My implementation is intended to allow context switching that way:
>>
>> if (--vm.interruptCheckCounter <= 0 && vm.handleDepthAndInterrupts(depth, thisProxy) === true) return false;
>>
>>
>> This is the same technique as in other VMs, where the actual check for
>> context switch is not done for every send, but only when
>> the interruptCheckCounter goes below zero. In the first mockup on codepen,
>> vm.handleDepthAndInterrupts prints the contents of all the contexts,
>> proving that the information is accessible in case we need to context
>> switch or reify the stack. My hope was that inside of
>> handleDepthAndInterrupts() I could use *yield to do the actual context
>> switching.
>>
>> My second, exception based mockup on codepen uses the same approach,
>> except that when a context switch is needed it would throw, unwinding the
>> stack fully, and creating actual context objects along the way. I have not
>> mocked up that part yet (because it would need a mockup interpreter too, to
>> continue after unwind), but you can see the unwind working correctly by
>> uncommenting the line with
>> // throw Error("unwind")
>>
>> I tried your benchFib vs benchFiby on Chrome, which seems to
>> optimize generators a lot better than Firefox. On Chrome the overhead is
>> just 50% or so, vs 300% on Firefox. Safari appears to be between the two.
>>
>> I will need to make more complete mockups before deciding on a design.
>> Especially how closures would be handled. Does anyone have a tiny
>> one-method benchmark that would highlight closure performance?
>>
>> Vanessa
>>
>> On Mon, Mar 15, 2021 at 6:20 PM Florin Mateoc <florin.mateoc at gmail.com>
>> wrote:
>>
>>>
>>> I don't think the numbers could be meaningfully compared. The whole
>>> purpose of the yield* invocations is to enable process switching, which I
>>> don't think your jitted methods allow for. But then are you merely
>>> comparing yield* invocations against direct invocations? For sure, the
>>> yield* ones will be  much slower.
>>> Let alone my implementation, which also uses yield* for #< , #- and #+,
>>> so it would be much slower than below as operators are surely well
>>> optimized by JS, we can just measure direct vs yield* for the main
>>> recursive invocation:
>>>
>>> Number.prototype.benchFib = function benchFib() {
>>>   return this < 2 ? 1 : (this - 1).benchFib() + (this - 2).benchFib() + 1
>>> }
>>>
>>> var t = performance.now();
>>> (30).benchFib();
>>> performance.now() - t
>>>
>>> gives on my laptop in Firefox 920, 911, 919, 898
>>>
>>> Versus
>>>
>>> Number.prototype.benchFiby = function* benchFiby() {
>>>    return this < 2 ? 1 : (yield* (this - 1).benchFiby()) + (yield* (this
>>> - 2).benchFiby()) + 1
>>> }
>>>
>>> var t = performance.now();
>>> (30).benchFiby().next();
>>> performance.now() - t
>>>
>>> gives 2998, 3125, 3116, 3140
>>>
>>>
>>> On Mon, Mar 15, 2021 at 3:41 PM Vanessa Freudenberg <
>>> vanessa at codefrau.net> wrote:
>>>
>>>>
>>>> Hi Florin,
>>>>
>>>> wow, that looks exciting!
>>>>
>>>> This is indeed a much more thorough Squeak-to-JS mapping, where mine is
>>>> more of a "traditional" VM. I love it!
>>>>
>>>> Since my original post I implemented a mockup of "my" new JIT scheme:
>>>>
>>>>
>>>> https://squeak.js.org/docs/jit.md.html#sketch:contextproxieswithintrospectionandinlinecaching/newjitsketch/performanceestimate
>>>> [image: image.png]
>>>> You can play with it here: https://codepen.io/codefrau/pen/JjbmVGw
>>>>
>>>> Could you share the performance numbers you are seeing for your
>>>> benchFib, in comparison to SqueakJS or my mockup? I am curious if
>>>> yield* is the way to go.
>>>>
>>>> Thanks for sharing! And congrats on your new job. My progress is slow
>>>> too, I only work on it some weekends. But then, I'm not in a hurry :)
>>>>
>>>> Vanessa
>>>>
>>>> On Sun, Mar 14, 2021 at 6:18 PM Florin Mateoc <florin.mateoc at gmail.com>
>>>> wrote:
>>>>
>>>>>
>>>>> Hi Vanessa,
>>>>>
>>>>> Sorry for the delay in responding - as somebody who has been inspired
>>>>> by your SqueakJS project, I think I should mention that I am working on a
>>>>> related project, for now tentatively called JsSqueak.
>>>>> In addition to the inspiration provided by SqueakJS, it also scratches
>>>>> my longstanding itch about compiling (transpiling) Squeak.
>>>>>
>>>>> I hesitated to talk about it, as it is still a work in progress -
>>>>> after small bits and pieces that I worked on over a long period, I had the
>>>>> opportunity to spend a significant and uninterrupted chunk of time on it
>>>>> last summer, when I was unemployed for 3 months, and I was able to make
>>>>> good progress. I was optimistically thinking of releasing a first version
>>>>> before the end of last year, but after I started working on my new job,
>>>>> progress on JsSqueak has slowed down significantly. I must confess that I
>>>>> (and especially my wife) hesitate in recreating that productive unemployed
>>>>> situation :)
>>>>>
>>>>> I started with Squeak 4.5 - I already had code transforming Smalltalk
>>>>> code to a form more suitable for translation - and I also started with
>>>>> VMMakerJS-dtl.18 for the plugin generation part. Of course, I had to
>>>>> heavily modify it, since I have to get rid of the stack usage for
>>>>> arguments/receiver and returns.
>>>>> Both of these big parts are working. I also implemented most numbered
>>>>> primitives by hand - they are inlined at generation time in the methods
>>>>> calling them.
>>>>> I am also taking advantage of the latest and greatest additions to
>>>>> JavaScript. I am, of course, using classes, and the parallel class-side
>>>>> hierarchy is implemented using statics. To implement green threads/process
>>>>> switching, all translated methods are implemented as generator functions,
>>>>> and all calls are through yield* expressions. The preemption/interrupt
>>>>> check points are inlined. With this, a process switch is achieved by simply
>>>>> yield-ing (in the process/semaphore primitives).
>>>>> With this, the Integer>>#benchFib method is translated (as a method in
>>>>> Number.prototype, there is one more, simpler, implementation in BigInt) as:
>>>>>
>>>>> *_benchFib() {
>>>>>    if (Number.isSafeInteger(this.valueOf())) { // Effective (inherited or local) source for #benchFib in SmallInteger
>>>>>       /*Handy send-heavy benchmark*/
>>>>>    /*(result // seconds to run) = approx calls per second*/
>>>>>    /* | r t |
>>>>>      t := Time millisecondsToRun: [r := 26 benchFib].
>>>>>      (r // 1000) // t*/
>>>>>    /*138000 on a Mac 8100//100*/
>>>>>    if (GlobalActivationCounter-- < 0) yield* GlobalCheckForInterrupts();
>>>>>
>>>>>    return (yield* this._lt( 2)).booleanValueOf("questionMark:colon:") ? (1) : (yield* (yield* (yield* (yield* this._sub( 1))._benchFib())._add( yield* (yield* this._sub( 2))._benchFib()))._add( 1));
>>>>> } else // No implementation for #benchFib in Float hierarchy, trigger a DNU
>>>>>       return yield* super._benchFib()
>>>>> }
>>>>>
>>>>> The top-level check for smallIntegers is because both SmallInteger and Float are mapped to Number.
>>>>>
>>>>> The booleanValueOf call is for implementing the mustBeBoolean machinery (it actually translates directly to DNU, like it is done nowadays in Squeak).
>>>>>
>>>>> Of course, in Boolean, booleanValueOf is just an alias for valueOf
>>>>>
>>>>> As you can see, though, this is not terribly efficient, but there is room for improvement/optimizations. With more work, in this case, the _lt call could be replaced by the < operator, and even the _sub and _add calls could be optimized,
>>>>> although not completely, since their result can morph into LargeInteger (mapped to BigInt).
>>>>>
>>>>> As hinted above, SmallInteger is mapped to Number (in the safeInteger range), Float is mapped to Number as well, and LargeInteger is mapped to BigInt.
>>>>>
>>>>> BlockClosure is mapped to Function, Boolean is mapped to Boolean, Character is mapped to String, weak references are implemented via WeakRef.
>>>>> I have briefly considered also doing slightly higher-level mappings, for IdentitySet to Set and IdentityDictionary to Map, but this is not a priority.
>>>>>
>>>>> The image is serialized sort of like a JavaScript storeString. No processes or contexts though, or rather they are not brought back in on the JavaScript side. Blocks are both stored and loaded.
>>>>>
>>>>> Non-local returns, unwind blocks, resumable and restartable exceptions are implemented via JavaScript exception handling plus explicit handler block chains associated with the processes.
>>>>>
>>>>> The "image" starts with the global state loaded, but all processes are started from scratch instead of resumed. A non-UI image is thus trivially started.
>>>>>
>>>>> One major todo left is hooking up the UI/browser. I did take vm.display.browser.js from SqueakJS and adapted the code in order to implement its numbered primitives, but I still have to work through squeak.js from the same to initialize
>>>>> and hook up the display.
>>>>>
>>>>> Florin
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Sun, Mar 7, 2021 at 11:17 PM Vanessa Freudenberg <
>>>>> vanessa at codefrau.net> wrote:
>>>>>
>>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> ideas for a faster SqueakJS JIT have been swirling around in my head
>>>>>> for many years. This weekend I took the time to write some of them down:
>>>>>>
>>>>>> https://squeak.js.org/docs/jit.md.html
>>>>>>
>>>>>> Feedback welcome!
>>>>>>
>>>>>> Vanessa
>>>>>>
>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210316/b5475096/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 42124 bytes
Desc: not available
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210316/b5475096/attachment-0001.png>

From commits at source.squeak.org  Tue Mar 16 19:56:21 2021
From: commits at source.squeak.org (commits at source.squeak.org)
Date: Tue, 16 Mar 2021 19:56:21 0000
Subject: [Vm-dev] VM Maker: VMMaker.oscog-eem.2947.mcz
Message-ID: <E1lMFo5-0001mN-6X@andreas>

Eliot Miranda uploaded a new version of VMMaker to project VM Maker:
http://source.squeak.org/VMMaker/VMMaker.oscog-eem.2947.mcz

==================== Summary ====================

Name: VMMaker.oscog-eem.2947
Author: eem
Time: 16 March 2021, 12:55:58.857415 pm
UUID: 009d2db3-059e-4852-8a76-567d6ee0e2d2
Ancestors: VMMaker.oscog-eem.2946

Fix a dreadful slip in InterpreterPrimitives>>primitiveIdentical.  In the one argument case the old implementation checked the receiver for being forwarded rather than the argument.  This means that the interpreter and/or the debugger will answer the wrong value for e.g. ObjectTest testBecomeForward pt3 == pt2.

=============== Diff against VMMaker.oscog-eem.2946 ===============

Item was changed:
  ----- Method: InterpreterPrimitives>>primitiveIdentical (in category 'object access primitives') -----
  primitiveIdentical
  	"is the receiver/first argument the same object as the (last) argument?.
  	 pop argumentCount because this can be used as a mirror primitive."
  	| thisObject otherObject |
+ 	thisObject := self stackValue: 1.
+ 	otherObject := self stackTop.
- 	otherObject := self stackValue: 1.
- 	thisObject := self stackTop.
  	((objectMemory isOopForwarded: otherObject)
  	 or: [argumentCount > 1
  		 and: [objectMemory isOopForwarded: thisObject]])
  		ifTrue:
  			[self primitiveFailFor: PrimErrBadArgument]
  		ifFalse:
  			[self pop: argumentCount + 1 thenPushBool: thisObject = otherObject]!

Item was changed:
  ----- Method: StackInterpreterSimulator>>primitiveDoPrimitiveWithArgs (in category 'debugging traps') -----
  primitiveDoPrimitiveWithArgs
+ 	| primIndex |
+ 	primIndex := objectMemory integerValueOf: (self stackValue: 1).
+ 	NewspeakVM ifFalse:
+ 		[transcript nextPutAll: 'DO PRIMITIVE: '; print: (self functionPointerFor: primIndex inClass: nil); cr; flush].
+ 	(#(110) includes: primIndex) ifTrue:
+ 		[self halt].
- 	NewspeakVM ifFalse: [self halt].
  	^super primitiveDoPrimitiveWithArgs!


From noreply at github.com  Tue Mar 16 20:09:17 2021
From: noreply at github.com (Eliot Miranda)
Date: Tue, 16 Mar 2021 13:09:17 -0700
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] 607b87: CogVM source as
 per VMMaker.oscog-eem.2947
Message-ID: <OpenSmalltalk/opensmalltalk-vm/push/refs/heads/Cog/119e76-607b87@github.com>

  Branch: refs/heads/Cog
  Home:   https://github.com/OpenSmalltalk/opensmalltalk-vm
  Commit: 607b872095dc34d8d8473bb87014feb9c9c25412
      https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/607b872095dc34d8d8473bb87014feb9c9c25412
  Author: Eliot Miranda <eliot.miranda at gmail.com>
  Date:   2021-03-16 (Tue, 16 Mar 2021)

  Changed paths:
    M nsspur64src/vm/cointerp.c
    M nsspur64src/vm/cointerp.h
    M nsspur64src/vm/gcc3x-cointerp.c
    M nsspursrc/vm/cointerp.c
    M nsspursrc/vm/cointerp.h
    M nsspursrc/vm/gcc3x-cointerp.c
    M nsspurstack64src/vm/gcc3x-interp.c
    M nsspurstack64src/vm/interp.c
    M nsspurstacksrc/vm/gcc3x-interp.c
    M nsspurstacksrc/vm/interp.c
    M spur64src/vm/cointerp.c
    M spur64src/vm/cointerp.h
    M spur64src/vm/cointerpmt.c
    M spur64src/vm/cointerpmt.h
    M spur64src/vm/gcc3x-cointerp.c
    M spur64src/vm/gcc3x-cointerpmt.c
    M spurlowcode64src/vm/cointerp.c
    M spurlowcode64src/vm/cointerp.h
    M spurlowcode64src/vm/gcc3x-cointerp.c
    M spurlowcodesrc/vm/cointerp.c
    M spurlowcodesrc/vm/cointerp.h
    M spurlowcodesrc/vm/gcc3x-cointerp.c
    M spurlowcodestack64src/vm/gcc3x-interp.c
    M spurlowcodestack64src/vm/interp.c
    M spurlowcodestacksrc/vm/gcc3x-interp.c
    M spurlowcodestacksrc/vm/interp.c
    M spursista64src/vm/cointerp.c
    M spursista64src/vm/cointerp.h
    M spursista64src/vm/gcc3x-cointerp.c
    M spursistasrc/vm/cointerp.c
    M spursistasrc/vm/cointerp.h
    M spursistasrc/vm/gcc3x-cointerp.c
    M spursrc/vm/cointerp.c
    M spursrc/vm/cointerp.h
    M spursrc/vm/cointerpmt.c
    M spursrc/vm/cointerpmt.h
    M spursrc/vm/gcc3x-cointerp.c
    M spursrc/vm/gcc3x-cointerpmt.c
    M spurstack64src/vm/gcc3x-interp.c
    M spurstack64src/vm/interp.c
    M spurstack64src/vm/validImage.c
    M spurstacksrc/vm/gcc3x-interp.c
    M spurstacksrc/vm/interp.c
    M spurstacksrc/vm/validImage.c
    M src/vm/cointerp.c
    M src/vm/cointerp.h
    M src/vm/gcc3x-cointerp.c
    M stacksrc/vm/gcc3x-interp.c
    M stacksrc/vm/interp.c

  Log Message:
  -----------
  CogVM source as per VMMaker.oscog-eem.2947

Fix a dreadful slip in InterpreterPrimitives>>primitiveIdentical.  In the one
argument case the old implementation checked the receiver for being forwarded
rather than the argument.  This means that the interpreter and/or the debugger
will answer the wrong value for e.g. ObjectTest testBecomeForward pt3 == pt2.



From commits at source.squeak.org  Tue Mar 16 20:25:11 2021
From: commits at source.squeak.org (commits at source.squeak.org)
Date: Tue, 16 Mar 2021 20:25:11 0000
Subject: [Vm-dev] VM Maker: VMMaker.oscog-eem.2948.mcz
Message-ID: <E1lMGFo-0002BH-JZ@andreas>

Eliot Miranda uploaded a new version of VMMaker to project VM Maker:
http://source.squeak.org/VMMaker/VMMaker.oscog-eem.2948.mcz

==================== Summary ====================

Name: VMMaker.oscog-eem.2948
Author: eem
Time: 16 March 2021, 1:24:48.66302 pm
UUID: 935d1ee7-0e61-47a5-92e2-953224c7ffcf
Ancestors: VMMaker.oscog-eem.2947

...and primitiveNotIdentical contains the same mistake.

=============== Diff against VMMaker.oscog-eem.2947 ===============

Item was changed:
  ----- Method: InterpreterPrimitives>>primitiveNotIdentical (in category 'object access primitives') -----
  primitiveNotIdentical
  	"is the receiver/first argument not the same object as the (last) argument?.
  	 pop argumentCount because this can be used as a mirror primitive."
+ 	"is the receiver/first argument the same object as the (last) argument?.
+ 	 pop argumentCount because this can be used as a mirror primitive."
  	| thisObject otherObject |
+ 	thisObject := self stackValue: 1.
+ 	otherObject := self stackTop.
+ 	((objectMemory isOopForwarded: otherObject)
+ 	 or: [argumentCount > 1
+ 		 and: [objectMemory isOopForwarded: thisObject]])
+ 		ifTrue:
+ 			[self primitiveFailFor: PrimErrBadArgument]
+ 		ifFalse:
+ 			[self pop: argumentCount + 1 thenPushBool: thisObject ~= otherObject]!
- 	otherObject := self stackValue: 1.
- 	thisObject := self stackTop.
- 	(objectMemory isOopForwarded: otherObject) ifTrue:
- 		[self assert: argumentCount > 1.
- 		 otherObject := objectMemory followForwarded: thisObject].
- 	(objectMemory isOopForwarded: thisObject) ifTrue:
- 		[thisObject := objectMemory followForwarded: thisObject].
- 	self pop: argumentCount + 1 thenPushBool: thisObject ~= otherObject!


From no-reply at appveyor.com  Tue Mar 16 20:28:07 2021
From: no-reply at appveyor.com (AppVeyor)
Date: Tue, 16 Mar 2021 20:28:07 +0000
Subject: [Vm-dev] Build failed: opensmalltalk-vm 1.0.2414
Message-ID: <20210316202807.1.F8E24CA3B2B0B485@appveyor.com>

An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210316/31dcc6e0/attachment.html>

From noreply at github.com  Tue Mar 16 20:34:24 2021
From: noreply at github.com (Eliot Miranda)
Date: Tue, 16 Mar 2021 13:34:24 -0700
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] ca9794: CogVM source as
 per VMMaker.oscog-eem.2948
Message-ID: <OpenSmalltalk/opensmalltalk-vm/push/refs/heads/Cog/607b87-ca9794@github.com>

  Branch: refs/heads/Cog
  Home:   https://github.com/OpenSmalltalk/opensmalltalk-vm
  Commit: ca9794e2ffd60fe40292f88fc4ff0114d7591c18
      https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/ca9794e2ffd60fe40292f88fc4ff0114d7591c18
  Author: Eliot Miranda <eliot.miranda at gmail.com>
  Date:   2021-03-16 (Tue, 16 Mar 2021)

  Changed paths:
    M nsspur64src/vm/cointerp.c
    M nsspur64src/vm/cointerp.h
    M nsspur64src/vm/gcc3x-cointerp.c
    M nsspursrc/vm/cointerp.c
    M nsspursrc/vm/cointerp.h
    M nsspursrc/vm/gcc3x-cointerp.c
    M nsspurstack64src/vm/gcc3x-interp.c
    M nsspurstack64src/vm/interp.c
    M nsspurstacksrc/vm/gcc3x-interp.c
    M nsspurstacksrc/vm/interp.c
    M spur64src/vm/cointerp.c
    M spur64src/vm/cointerp.h
    M spur64src/vm/cointerpmt.c
    M spur64src/vm/cointerpmt.h
    M spur64src/vm/gcc3x-cointerp.c
    M spur64src/vm/gcc3x-cointerpmt.c
    M spurlowcode64src/vm/cointerp.c
    M spurlowcode64src/vm/cointerp.h
    M spurlowcode64src/vm/gcc3x-cointerp.c
    M spurlowcodesrc/vm/cointerp.c
    M spurlowcodesrc/vm/cointerp.h
    M spurlowcodesrc/vm/gcc3x-cointerp.c
    M spurlowcodestack64src/vm/gcc3x-interp.c
    M spurlowcodestack64src/vm/interp.c
    M spurlowcodestacksrc/vm/gcc3x-interp.c
    M spurlowcodestacksrc/vm/interp.c
    M spursista64src/vm/cointerp.c
    M spursista64src/vm/cointerp.h
    M spursista64src/vm/gcc3x-cointerp.c
    M spursistasrc/vm/cointerp.c
    M spursistasrc/vm/cointerp.h
    M spursistasrc/vm/gcc3x-cointerp.c
    M spursrc/vm/cointerp.c
    M spursrc/vm/cointerp.h
    M spursrc/vm/cointerpmt.c
    M spursrc/vm/cointerpmt.h
    M spursrc/vm/gcc3x-cointerp.c
    M spursrc/vm/gcc3x-cointerpmt.c
    M spurstack64src/vm/gcc3x-interp.c
    M spurstack64src/vm/interp.c
    M spurstack64src/vm/validImage.c
    M spurstacksrc/vm/gcc3x-interp.c
    M spurstacksrc/vm/interp.c
    M spurstacksrc/vm/validImage.c
    M src/vm/cointerp.c
    M src/vm/cointerp.h
    M src/vm/gcc3x-cointerp.c
    M stacksrc/vm/gcc3x-interp.c
    M stacksrc/vm/interp.c

  Log Message:
  -----------
  CogVM source as per VMMaker.oscog-eem.2948

...and primitiveNotIdentical contains the same mistake.



From no-reply at appveyor.com  Tue Mar 16 20:53:18 2021
From: no-reply at appveyor.com (AppVeyor)
Date: Tue, 16 Mar 2021 20:53:18 +0000
Subject: [Vm-dev] Build failed: opensmalltalk-vm 1.0.2415
Message-ID: <20210316205318.1.E7D38F384DEEC201@appveyor.com>

An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210316/d4758d53/attachment.html>

From vanessa at codefrau.net  Wed Mar 17 09:01:09 2021
From: vanessa at codefrau.net (Vanessa Freudenberg)
Date: Wed, 17 Mar 2021 02:01:09 -0700
Subject: [Vm-dev] [SqueakJS] Faster JIT ideas
In-Reply-To: <CAOWGXNCrHKzDGZddU-zZ4YFHnuKEkOwsu1jbeHsH17XLnTnAtw@mail.gmail.com>
References: <CAOWGXNCmyE1N9K87+jKiXM7awb7KbsQws7HKO3uAXAKV9PBc2Q@mail.gmail.com>
 <CA+7DxaY=EC_p2n=rhVyxXfm75v-qASwRu6av=fQj6MSNUJvn3g@mail.gmail.com>
 <CAOWGXNAtjzX3jfYf1rcBoy=_UxjgQzFwdnHch93L9WeXGy=dAQ@mail.gmail.com>
 <CA+7DxaZz1k2wnV4DEKaQFX5u2icdoZa1c5Sd-2F-7E-Z-B2ezA@mail.gmail.com>
 <CAOWGXND059RWN8woaDZrhGRocf=qG8gX8AtxDZdUrnYJmZd8nw@mail.gmail.com>
 <CA+7DxaZcmfcj8AEfTxg6hHBigiTmq=CjRmi9tQrHLmaMsjiokQ@mail.gmail.com>
 <CAOWGXNCrHKzDGZddU-zZ4YFHnuKEkOwsu1jbeHsH17XLnTnAtw@mail.gmail.com>
Message-ID: <CAOWGXNBSvj3jV1kp=4sgDvy0o9_zZ6CEmVp_-W81a1q8U9kHDg@mail.gmail.com>

On Tue, Mar 16, 2021 at 1:38 AM Vanessa Freudenberg <vanessa at codefrau.net>
wrote:

> Meanwhile, I made a little scheduler just to learn how to use yield*.
> Works nicely:
> https://codepen.io/codefrau/pen/jOVRmxm
>

And here is a JIT mockup using that scheduler:
https://codepen.io/codefrau/pen/LYbopBV

It's executing "5 benchFib", "4 benchFib", and "3 benchFib" in three
separate processes, switching every 3 sends just for testing, and printing
all contexts.

No benchmarks yet.

Vanessa


> Vanessa
>
>
>> On Mon, Mar 15, 2021 at 10:28 PM Vanessa Freudenberg <
>> vanessa at codefrau.net> wrote:
>
>
>>> Thanks for that Florin, very helpful!
>>>
>>> I'm curious why you need to make every send into a generator call, and
>>> not just rely on the one in your GlobalCheckForInterrupts()?
>>>
>>> My implementation is intended to allow context switching that way:
>>>
>>> if (--vm.interruptCheckCounter <= 0 && vm.handleDepthAndInterrupts(depth, thisProxy) === true) return false;
>>>
>>>
>>> This is the same technique as in other VMs, where the actual check for
>>> context switch is not done for every send, but only when
>>> the interruptCheckCounter goes below zero. In the first mockup on codepen,
>>> vm.handleDepthAndInterrupts prints the contents of all the contexts,
>>> proving that the information is accessible in case we need to context
>>> switch or reify the stack. My hope was that inside of
>>> handleDepthAndInterrupts() I could use *yield to do the actual context
>>> switching.
>>>
>>> My second, exception based mockup on codepen uses the same approach,
>>> except that when a context switch is needed it would throw, unwinding the
>>> stack fully, and creating actual context objects along the way. I have not
>>> mocked up that part yet (because it would need a mockup interpreter too, to
>>> continue after unwind), but you can see the unwind working correctly by
>>> uncommenting the line with
>>> // throw Error("unwind")
>>>
>>> I tried your benchFib vs benchFiby on Chrome, which seems to
>>> optimize generators a lot better than Firefox. On Chrome the overhead is
>>> just 50% or so, vs 300% on Firefox. Safari appears to be between the two.
>>>
>>> I will need to make more complete mockups before deciding on a design.
>>> Especially how closures would be handled. Does anyone have a tiny
>>> one-method benchmark that would highlight closure performance?
>>>
>>> Vanessa
>>>
>>> On Mon, Mar 15, 2021 at 6:20 PM Florin Mateoc <florin.mateoc at gmail.com>
>>> wrote:
>>>
>>>>
>>>> I don't think the numbers could be meaningfully compared. The whole
>>>> purpose of the yield* invocations is to enable process switching, which I
>>>> don't think your jitted methods allow for. But then are you merely
>>>> comparing yield* invocations against direct invocations? For sure, the
>>>> yield* ones will be  much slower.
>>>> Let alone my implementation, which also uses yield* for #< , #- and #+,
>>>> so it would be much slower than below as operators are surely well
>>>> optimized by JS, we can just measure direct vs yield* for the main
>>>> recursive invocation:
>>>>
>>>> Number.prototype.benchFib = function benchFib() {
>>>>   return this < 2 ? 1 : (this - 1).benchFib() + (this - 2).benchFib() +
>>>> 1
>>>> }
>>>>
>>>> var t = performance.now();
>>>> (30).benchFib();
>>>> performance.now() - t
>>>>
>>>> gives on my laptop in Firefox 920, 911, 919, 898
>>>>
>>>> Versus
>>>>
>>>> Number.prototype.benchFiby = function* benchFiby() {
>>>>    return this < 2 ? 1 : (yield* (this - 1).benchFiby()) + (yield* (
>>>> this - 2).benchFiby()) + 1
>>>> }
>>>>
>>>> var t = performance.now();
>>>> (30).benchFiby().next();
>>>> performance.now() - t
>>>>
>>>> gives 2998, 3125, 3116, 3140
>>>>
>>>>
>>>> On Mon, Mar 15, 2021 at 3:41 PM Vanessa Freudenberg <
>>>> vanessa at codefrau.net> wrote:
>>>>
>>>>>
>>>>> Hi Florin,
>>>>>
>>>>> wow, that looks exciting!
>>>>>
>>>>> This is indeed a much more thorough Squeak-to-JS mapping, where mine
>>>>> is more of a "traditional" VM. I love it!
>>>>>
>>>>> Since my original post I implemented a mockup of "my" new JIT scheme:
>>>>>
>>>>>
>>>>> https://squeak.js.org/docs/jit.md.html#sketch:contextproxieswithintrospectionandinlinecaching/newjitsketch/performanceestimate
>>>>> [image: image.png]
>>>>> You can play with it here: https://codepen.io/codefrau/pen/JjbmVGw
>>>>>
>>>>> Could you share the performance numbers you are seeing for your
>>>>> benchFib, in comparison to SqueakJS or my mockup? I am curious if
>>>>> yield* is the way to go.
>>>>>
>>>>> Thanks for sharing! And congrats on your new job. My progress is slow
>>>>> too, I only work on it some weekends. But then, I'm not in a hurry :)
>>>>>
>>>>> Vanessa
>>>>>
>>>>> On Sun, Mar 14, 2021 at 6:18 PM Florin Mateoc <florin.mateoc at gmail.com>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>> Hi Vanessa,
>>>>>>
>>>>>> Sorry for the delay in responding - as somebody who has been inspired
>>>>>> by your SqueakJS project, I think I should mention that I am working on a
>>>>>> related project, for now tentatively called JsSqueak.
>>>>>> In addition to the inspiration provided by SqueakJS, it also
>>>>>> scratches my longstanding itch about compiling (transpiling) Squeak.
>>>>>>
>>>>>> I hesitated to talk about it, as it is still a work in progress -
>>>>>> after small bits and pieces that I worked on over a long period, I had the
>>>>>> opportunity to spend a significant and uninterrupted chunk of time on it
>>>>>> last summer, when I was unemployed for 3 months, and I was able to make
>>>>>> good progress. I was optimistically thinking of releasing a first version
>>>>>> before the end of last year, but after I started working on my new job,
>>>>>> progress on JsSqueak has slowed down significantly. I must confess that I
>>>>>> (and especially my wife) hesitate in recreating that productive unemployed
>>>>>> situation :)
>>>>>>
>>>>>> I started with Squeak 4.5 - I already had code transforming Smalltalk
>>>>>> code to a form more suitable for translation - and I also started with
>>>>>> VMMakerJS-dtl.18 for the plugin generation part. Of course, I had to
>>>>>> heavily modify it, since I have to get rid of the stack usage for
>>>>>> arguments/receiver and returns.
>>>>>> Both of these big parts are working. I also implemented most numbered
>>>>>> primitives by hand - they are inlined at generation time in the methods
>>>>>> calling them.
>>>>>> I am also taking advantage of the latest and greatest additions to
>>>>>> JavaScript. I am, of course, using classes, and the parallel class-side
>>>>>> hierarchy is implemented using statics. To implement green threads/process
>>>>>> switching, all translated methods are implemented as generator functions,
>>>>>> and all calls are through yield* expressions. The preemption/interrupt
>>>>>> check points are inlined. With this, a process switch is achieved by simply
>>>>>> yield-ing (in the process/semaphore primitives).
>>>>>> With this, the Integer>>#benchFib method is translated (as a method
>>>>>> in Number.prototype, there is one more, simpler, implementation in BigInt)
>>>>>> as:
>>>>>>
>>>>>> *_benchFib() {
>>>>>>    if (Number.isSafeInteger(this.valueOf())) { // Effective (inherited or local) source for #benchFib in SmallInteger
>>>>>>       /*Handy send-heavy benchmark*/
>>>>>>    /*(result // seconds to run) = approx calls per second*/
>>>>>>    /* | r t |
>>>>>>      t := Time millisecondsToRun: [r := 26 benchFib].
>>>>>>      (r // 1000) // t*/
>>>>>>    /*138000 on a Mac 8100//100*/
>>>>>>    if (GlobalActivationCounter-- < 0) yield* GlobalCheckForInterrupts();
>>>>>>
>>>>>>    return (yield* this._lt( 2)).booleanValueOf("questionMark:colon:") ? (1) : (yield* (yield* (yield* (yield* this._sub( 1))._benchFib())._add( yield* (yield* this._sub( 2))._benchFib()))._add( 1));
>>>>>> } else // No implementation for #benchFib in Float hierarchy, trigger a DNU
>>>>>>       return yield* super._benchFib()
>>>>>> }
>>>>>>
>>>>>> The top-level check for smallIntegers is because both SmallInteger and Float are mapped to Number.
>>>>>>
>>>>>> The booleanValueOf call is for implementing the mustBeBoolean machinery (it actually translates directly to DNU, like it is done nowadays in Squeak).
>>>>>>
>>>>>> Of course, in Boolean, booleanValueOf is just an alias for valueOf
>>>>>>
>>>>>> As you can see, though, this is not terribly efficient, but there is room for improvement/optimizations. With more work, in this case, the _lt call could be replaced by the < operator, and even the _sub and _add calls could be optimized,
>>>>>> although not completely, since their result can morph into LargeInteger (mapped to BigInt).
>>>>>>
>>>>>> As hinted above, SmallInteger is mapped to Number (in the safeInteger range), Float is mapped to Number as well, and LargeInteger is mapped to BigInt.
>>>>>>
>>>>>> BlockClosure is mapped to Function, Boolean is mapped to Boolean, Character is mapped to String, weak references are implemented via WeakRef.
>>>>>> I have briefly considered also doing slightly higher-level mappings, for IdentitySet to Set and IdentityDictionary to Map, but this is not a priority.
>>>>>>
>>>>>> The image is serialized sort of like a JavaScript storeString. No processes or contexts though, or rather they are not brought back in on the JavaScript side. Blocks are both stored and loaded.
>>>>>>
>>>>>> Non-local returns, unwind blocks, resumable and restartable exceptions are implemented via JavaScript exception handling plus explicit handler block chains associated with the processes.
>>>>>>
>>>>>> The "image" starts with the global state loaded, but all processes are started from scratch instead of resumed. A non-UI image is thus trivially started.
>>>>>>
>>>>>> One major todo left is hooking up the UI/browser. I did take vm.display.browser.js from SqueakJS and adapted the code in order to implement its numbered primitives, but I still have to work through squeak.js from the same to initialize
>>>>>> and hook up the display.
>>>>>>
>>>>>> Florin
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Sun, Mar 7, 2021 at 11:17 PM Vanessa Freudenberg <
>>>>>> vanessa at codefrau.net> wrote:
>>>>>>
>>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> ideas for a faster SqueakJS JIT have been swirling around in my head
>>>>>>> for many years. This weekend I took the time to write some of them down:
>>>>>>>
>>>>>>> https://squeak.js.org/docs/jit.md.html
>>>>>>>
>>>>>>> Feedback welcome!
>>>>>>>
>>>>>>> Vanessa
>>>>>>>
>>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210317/50202d27/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 42124 bytes
Desc: not available
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210317/50202d27/attachment-0001.png>

From ken.dickey at whidbey.com  Wed Mar 17 15:13:13 2021
From: ken.dickey at whidbey.com (ken.dickey at whidbey.com)
Date: Wed, 17 Mar 2021 08:13:13 -0700
Subject: [Vm-dev] Copy Down to avoid Looking Up
Message-ID: <ed4fd791dfcc2ac54cacdd69f0fe99f8@whidbey.com>

You can help me out.

I keep getting this wacky idea which does not seem to go away.  You can 
tell me what is wrong with it.

OK.  Here it is:

To get rid of PICs and dynamic method lookups, each Class knows 
selectors of methods it defines.  Each successful method definition is 
added not only to the method dictionary of the class, but copied down 
transitively to all subclasses, and their sub-subclasses, who have not 
defined a method with the same selector.

At #DeepRehash time, _each_ method dictionary knows all of its selectors 
and creates a "perfect hash" function, or something close to it.

So lookup is just into the specific method dictionary of the instance's 
class which yields the proper method (backstop method is DNU).  Never a 
miss.  No class hierarchy search for inherited methods.

Memory is cheap these days, but selectors and compiled methods are 
mostly shared and the memory cost of method dictionaries is probably 
balanced out by elimination of PICS at the call sites.

The UI tools need to keep some simple rules (e,g, transitively deleting 
methods from subclass method dictionaries), but these should be simple 
and obvious.

The above seems simple enough, but nobody seems to be doing it.

Why not?

Thanks much for insights!
-KenD

From nabble.42 at klix.ch  Wed Mar 17 18:31:19 2021
From: nabble.42 at klix.ch (nabble.42 at klix.ch)
Date: Wed, 17 Mar 2021 19:31:19 +0100
Subject: [Vm-dev] Copy Down to avoid Looking Up
In-Reply-To: <ed4fd791dfcc2ac54cacdd69f0fe99f8@whidbey.com>
References: <ed4fd791dfcc2ac54cacdd69f0fe99f8@whidbey.com>
Message-ID: <7c0465b0-dfb9-6300-ce81-f29b19d1aa57@klix.ch>

Hi Ken,

I am by no means a VM expert, but this idea can
be enhanced further:

Let's generate consecutive
numbers for all the symbols used as message selectors then this "perfect 
hash" degenerates
to an array with pointers to the compiled
method. Let's also ignore
-- for the sake of simplicity --
symbols that are garbage collected;
the holes occurring can simply be filled
by some sort "free-list-algorithm.

Now you have something the C++ programmer knows
really well: A virtual function table.

I have no clue whether those VFTs will help.


Just my 0.01€.


Best Regards,

Gerald



On 2021-03-17 16:13, ken.dickey at whidbey.com wrote:
> 
> You can help me out.
> 
> I keep getting this wacky idea which does not seem to go away.  You can 
> tell me what is wrong with it.
> 
> OK.  Here it is:
> 
> To get rid of PICs and dynamic method lookups, each Class knows 
> selectors of methods it defines.  Each successful method definition is 
> added not only to the method dictionary of the class, but copied down 
> transitively to all subclasses, and their sub-subclasses, who have not 
> defined a method with the same selector.
> 
> At #DeepRehash time, _each_ method dictionary knows all of its selectors 
> and creates a "perfect hash" function, or something close to it.
> 
> So lookup is just into the specific method dictionary of the instance's 
> class which yields the proper method (backstop method is DNU).  Never a 
> miss.  No class hierarchy search for inherited methods.
> 
> Memory is cheap these days, but selectors and compiled methods are 
> mostly shared and the memory cost of method dictionaries is probably 
> balanced out by elimination of PICS at the call sites.
> 
> The UI tools need to keep some simple rules (e,g, transitively deleting 
> methods from subclass method dictionaries), but these should be simple 
> and obvious.
> 
> The above seems simple enough, but nobody seems to be doing it.
> 
> Why not?
> 
> Thanks much for insights!
> -KenD

From jecel at merlintec.com  Wed Mar 17 20:08:21 2021
From: jecel at merlintec.com (Jecel Assumpcao Jr)
Date: Wed, 17 Mar 2021 17:08:21 -0300 
Subject: [Vm-dev] Copy Down to avoid Looking Up
In-Reply-To: <ed4fd791dfcc2ac54cacdd69f0fe99f8@whidbey.com>
References: <ed4fd791dfcc2ac54cacdd69f0fe99f8@whidbey.com>
Message-ID: <20210317200826.C696EEE026C@proxy.email-ssl.com.br>

Ken,

I presented a similar design at OOPSLA 2003. It was for a unique
hardware (a low cost terminal for truck drivers) and had several
features I would not have normally put in a VM. The main memory was 4MB
of SDRAM since that was the lowest cost option at the time while it only
had 512KB of Flash. Even after expanding the compressed image from the
Flash to the SDRAM there was a lot of memory left over.

This was a 16 bit VM with 32K SmallIntegers and only 32K object
pointers. I made SmallIntegers do double duty as Symbols. So the number
3 and the symbol #between:and: might be the exact same object. You could
use it as one or the other by sending different messages as there is
very little overlap. You can find the "symbol string" associated with 3
by looking at the third entry in SymbolTable.

The equivalent of Class was the CodeObject. It includes all of the
compiled code for the class, both local and inhericated (copied down, as
you said). The processor had a bytecode style machine language so the
source texts from the compressed image was translated to that when
copied to the CodeObject.

The code objects had two parts: an initial set of entries with one entry
for every possible symbol and one huge chunk of code. Each entry had
enough room for 4 instructions, which is enough to implement most
methods. For those that need more space you just include a jump to
somewhere in the second part of the code object. The point was to
eliminate every extra clock cycle possible from a message send. Given a
pointer to some object, you could get its corresponding code object by
just zeroing the bottom few bits of the pointer (no memory access to get
the class from the oop). So if you are trying to send message 3 to oop
you just jump to:

call (oop & CodeObjectMask) + 3 * EntrySizeInBytes

Single clock cycle message sends for both local and inherited methods,
at a rather absurd cost in memory. If it were a 32 bit VM the wasted
memory would just be too silly.

A nice overview of the different ways to compress the class vs selector
tables can be found in the ECOOP 95 paper "Message Dispatch on Pipelined
Processors" by Karel Driesen, Urs Hölzle and Jan Vitek:

> https://www.researchgate.net/publication/221496253_Message_Dispatch_on_Pipelined_Processors

-- Jecel

From lewis at mail.msen.com  Thu Mar 18 00:02:52 2021
From: lewis at mail.msen.com (David T. Lewis)
Date: Wed, 17 Mar 2021 20:02:52 -0400
Subject: [Vm-dev] regression: TestObjectsAsMethods crash in 4.19.5 (not
	in 4.16.7)
In-Reply-To: <857058996.42907317.1615746625509.JavaMail.zimbra@telenet.be>
References: <949389016.25753821.1613919758428.JavaMail.zimbra@telenet.be>
 <605876499.30501015.1614071179388.JavaMail.zimbra@telenet.be>
 <857058996.42907317.1615746625509.JavaMail.zimbra@telenet.be>
Message-ID: <20210318000252.GA36831@shell.msen.com>

Hi David,

Sorry I did not reply earlier,

On Sun, Mar 14, 2021 at 07:30:25PM +0100, stes at PANDORA.BE wrote:
>  
> > A trivial fix is to rearrange some primtive number assignments, but the
> > correct thing to do is adopt the logic that Eliot uses in oscog so that
> > the function is not invoked as a "primitive" at all.
> >
> > I'll try to follow up on this next week. 
> 
> Would it please be possible to commit to subversion the trivial fix,
> while waiting for the correct fix.

Unfortunately no. I tried reassigning primitiveInvokeObjectAsMethod to
an unused primitive number. That worked fine, but I quickly found that
more recent versions of Squeak trunk were using that "unused" primitive
number for some other purpose, so this is a no-go.

A quick and dirty fix is not feasible, it needs to be done right. That
means adapting or adopting Eliot's implementation.


<OT>
  <RANT>
      Numbered primitives are a plague and a pestilence. Please just stop doing that.
  </RANT>
</OT>

Dave


From stes at telenet.be  Thu Mar 18 14:05:29 2021
From: stes at telenet.be (stes@PANDORA.BE)
Date: Thu, 18 Mar 2021 15:05:29 +0100 (CET)
Subject: [Vm-dev] regression: TestObjectsAsMethods crash in 4.19.5 (not
 in 4.16.7)
In-Reply-To: <1310783440.43126857.1615752926405.JavaMail.zimbra@telenet.be>
References: <949389016.25753821.1613919758428.JavaMail.zimbra@telenet.be>
 <605876499.30501015.1614071179388.JavaMail.zimbra@telenet.be>
 <857058996.42907317.1615746625509.JavaMail.zimbra@telenet.be>
 <1310783440.43126857.1615752926405.JavaMail.zimbra@telenet.be>
Message-ID: <576861635.57225754.1616076329186.JavaMail.zimbra@telenet.be>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256


If a good solution exists, it's better of course than a quick 
(and possibly) wrong short term solution ...

Thanks for having looked into this.

David Stes

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJgU14JAAoJEAwpOKXMq1MaexAH/2Yrs3XcC5oqvVXtt1e3n1HI
xHn9As+/BO6d2yq7M8Xd86ZP3JuSo/MtvvCAqLX7GlhQDeCG567MBBdX+7g1Cgqt
0U+OTTaaVrzH1YOEeKEXINLyOKhtzgv7mSMzNhvh56Mbo96ww24288P7srG0ozv5
HB+jvLuxiCHPIdQaZcnw4GGI3jQXgZDqADAQ/AYOfX5AvSTda++N+DMxd0o9q5O/
EwbdyOWMMOVNX85q+sQ1VISnm8nzafSmxr6IPW8TuH0c70wRIDOcdpfRHR0Sz1CH
GKE5JJIneIiSsVGL74BU7qVu8fmUviuvw9sX/t94Op0abh/xV0i7RE4nlvVapgk=
=W5VI
-----END PGP SIGNATURE-----

From stes at telenet.be  Thu Mar 18 14:10:14 2021
From: stes at telenet.be (stes@PANDORA.BE)
Date: Thu, 18 Mar 2021 15:10:14 +0100 (CET)
Subject: [Vm-dev] patch for allocateBuffer in AsyncFilePlugin
Message-ID: <669270513.57271174.1616076614691.JavaMail.zimbra@telenet.be>


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256


Hi,

Is it please possible to fix a small issue in Squeak 4.19 (Classical VM).

On my system the compiler is complaining that a function has no return type:

file: platforms/unix/plugins/AsynchFilePlugin/sqUnixAsynchFile.c

"platforms/unix/plugins/AsynchFilePlugin/sqUnixAsynchFile.c", line 145: warning: no explicit type given
 
INLINE static allocateBuffer(struct FileBuf *buf, int size)

The function is used only once in that file, as in :

  if (!allocateBuffer(&fp->buf, count))

Please apply my patch (in attach) to the classical VM ...

Regards,
David Stes

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJgU1sLAAoJEAwpOKXMq1Ma4gwH/Rwte3vJraeRSh7kj/EkXBL+
+Ctus32dJQR5echOLrVwzouifIEFZj8lSn64DGQiQaE1ZNczizn6HDaCeGvSB7K2
GNbmg8Fxq1Ghts5WX9QaeeGEotptcHDGGkZf9oaQnr+qrx2CbLLz1MFk9Lg/B+In
sV2qL99vhFnS6kOjbpA926qj7Lb+bdsVg8ap7NO//t8UH6Irm7b4UMoRebJ68QpB
YpuqtvtFaWmNr03Hsh5pcgGzOAoC+MITIL5umaIcs6nla1so0J9GJ/gnxfPKEety
uo2kat03Onvjjn9FrwPi+qmmpq9hCDYMKFUqiV+OsYHWdfwGoXC6lKtSBgMvmGQ=
=Ukc3
-----END PGP SIGNATURE-----
-------------- next part --------------
A non-text attachment was scrubbed...
Name: allocateBuffer.patch
Type: text/x-patch
Size: 402 bytes
Desc: not available
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210318/95c0fb5f/attachment.bin>

From ken.dickey at whidbey.com  Thu Mar 18 15:28:31 2021
From: ken.dickey at whidbey.com (ken.dickey at whidbey.com)
Date: Thu, 18 Mar 2021 08:28:31 -0700
Subject: [Vm-dev] (Rescued) Re:  Copy Down to avoid Looking Up
In-Reply-To: <7c0465b0-dfb9-6300-ce81-f29b19d1aa57@klix.ch>
References: <ed4fd791dfcc2ac54cacdd69f0fe99f8@whidbey.com>
 <7c0465b0-dfb9-6300-ce81-f29b19d1aa57@klix.ch>
Message-ID: <05fbfbca72dd7a3c445a8a18a12a02af@whidbey.com>

HI Gerald,

Thanks for the feedback.

I am aware of Virtual Function Tables, Sparse Array index compression, 
lots of strategies for Dynamic Dispatch, many I have used to implement 
various object systems.

One can arrange the hash/ID of selectors, classes, and method tables.

I am looking for something which is space compact, simple to explain, 
runtime efficient, supports dynamic updates, and "fails softly" (an 
"imperfect hash" still works in a Dictionary).  Also, deals with 
polymorphism and super and does not require an expensive #isSubclassOf: 
test.

My thought with "Copy Down" is that while loading a bunch of code, one 
can build simple method dictionaries, then apply a #deepRehash after the 
load.  This could be done to affected method dictionaries each time a 
method is accepted during development as UI's (i.e. users) are slow.

The #deepRehash for each method dictionary could use simple trials to 
generate a compact table/array and set the hash for a method Dictionary 
so that (<selector> hash) <logical-op> (<mtable> hash>) finds the 
<selector>'s method without collisions.

The #deepRehash only requires local knowledge (all used selectors are 
known), not a global analysis to renumber classes or selectors.

Perfect hash would be something competitive, speed wise, with PICs but 
does not require cache management or inline code changes.  With lookup 
in only one table/dict one does not need to traverse the class hierarchy 
on a miss, because lookup never misses (DNU in every table which is 
answer on not found -- separate lookup if invoker wants nil instead).

A bit of work to craft hashfun creation, but seems worthwhile.  And an 
interesting problem to work on. :)

The idea seems simple enough that I suspect someone has looked at this 
strategy.

If this is a really bad idea, I don't have to spend any more thing 
thinking about it or worse, actually doing something.  ;^)

Thanks again,
-KenD


On 2021-03-18 07:25, nabble.42 at klix.ch wrote:
> Hi Ken,
> 
> I am by no means a VM expert, but this idea can
> be enhanced further:
> 
> Let's generate consecutive
> numbers for all the symbols used as message selectors then this
> "perfect hash" degenerates
> to an array with pointers to the compiled
> method. Let's also ignore
> -- for the sake of simplicity --
> symbols that are garbage collected;
> the holes occurring can simply be filled
> by some sort "free-list-algorithm.
> 
> Now you have something the C++ programmer knows
> really well: A virtual function table.
> 
> I have no clue whether those VFTs will help.
> 
> 
> Just my 0.01€.
> 
> 
> Best Regards,
> 
> Gerald

From noreply at github.com  Thu Mar 18 19:09:01 2021
From: noreply at github.com (Eliot Miranda)
Date: Thu, 18 Mar 2021 12:09:01 -0700
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] a6fd91: Fix the camera
 size selection logic in the MacOS C...
Message-ID: <OpenSmalltalk/opensmalltalk-vm/push/refs/heads/Cog/ca9794-a6fd91@github.com>

  Branch: refs/heads/Cog
  Home:   https://github.com/OpenSmalltalk/opensmalltalk-vm
  Commit: a6fd91e98f0d3e51c45a1a66611483abdc4c7156
      https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/a6fd91e98f0d3e51c45a1a66611483abdc4c7156
  Author: Eliot Miranda <eliot.miranda at gmail.com>
  Date:   2021-03-18 (Thu, 18 Mar 2021)

  Changed paths:
    M platforms/iOS/plugins/CameraPlugin/AVFoundationVideoGrabber.m

  Log Message:
  -----------
  Fix the camera size selection logic in the MacOS CameraPlugin.



From ken.dickey at whidbey.com  Thu Mar 18 19:09:34 2021
From: ken.dickey at whidbey.com (ken.dickey at whidbey.com)
Date: Thu, 18 Mar 2021 12:09:34 -0700
Subject: [Vm-dev] Copy Down to avoid Looking Up
Message-ID: <3376723ee3fe09756548cd4edeb150f3@whidbey.com>

Jecel,

Thanks much for both your thoughtful reply and the paper reference.  
This latter was the gold I was looking for!  Too many Dispatch papers to 
search through!

Good on ya,
-KenD
========
> A nice overview of the different ways to compress the class vs selector
tables can be found in the ECOOP 95 paper "Message Dispatch on Pipelined
Processors" by Karel Driesen, Urs Hölzle and Jan Vitek:

> https://www.researchgate.net/publication/221496253_Message_Dispatch_on_Pipelined_Processors
========

From no-reply at appveyor.com  Thu Mar 18 19:27:42 2021
From: no-reply at appveyor.com (AppVeyor)
Date: Thu, 18 Mar 2021 19:27:42 +0000
Subject: [Vm-dev] Build failed: opensmalltalk-vm 1.0.2416
Message-ID: <20210318192742.1.9AA25BC72DED9E4D@appveyor.com>

An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210318/fb594c9f/attachment.html>

From eliot.miranda at gmail.com  Thu Mar 18 20:10:54 2021
From: eliot.miranda at gmail.com (Eliot Miranda)
Date: Thu, 18 Mar 2021 13:10:54 -0700
Subject: [Vm-dev] Copy Down to avoid Looking Up
In-Reply-To: <ed4fd791dfcc2ac54cacdd69f0fe99f8@whidbey.com>
References: <ed4fd791dfcc2ac54cacdd69f0fe99f8@whidbey.com>
Message-ID: <CAC20JE0edGY4DDg7-AmFKcXAca1sF783wH0R-Tqghn4TKUhfmQ@mail.gmail.com>

Hi Ken,

On Wed, Mar 17, 2021 at 8:13 AM <ken.dickey at whidbey.com> wrote:

>
> You can help me out.
>
> I keep getting this wacky idea which does not seem to go away.  You can
> tell me what is wrong with it.
>
> OK.  Here it is:
>
> To get rid of PICs and dynamic method lookups, each Class knows
> selectors of methods it defines.  Each successful method definition is
> added not only to the method dictionary of the class, but copied down
> transitively to all subclasses, and their sub-subclasses, who have not
> defined a method with the same selector.
>
> At #DeepRehash time, _each_ method dictionary knows all of its selectors
> and creates a "perfect hash" function, or something close to it.
>
> So lookup is just into the specific method dictionary of the instance's
> class which yields the proper method (backstop method is DNU).  Never a
> miss.  No class hierarchy search for inherited methods.
>
> Memory is cheap these days, but selectors and compiled methods are
> mostly shared and the memory cost of method dictionaries is probably
> balanced out by elimination of PICS at the call sites.
>
> The UI tools need to keep some simple rules (e,g, transitively deleting
> methods from subclass method dictionaries), but these should be simple
> and obvious.
>
> The above seems simple enough, but nobody seems to be doing it.
>

I don't think no one is doing this.  I expect for example that the SqueakJS
effort would explore this.


> Why not?
>

Speaking for myself when I did my threaded code version of BrouHaHa in the
late '80's/early '90's I added copy down in the bytecode-to-threaded-code
JIT so that self sends were statically bound.  t was an interesting
experiment, and placed some additional constraints on become: if become
changes any object's class then all activations must be visited and checked
for validity, changing the current method to match the receiver if its
class no longer matches the activation's copied down method.  This is easy
to do with a threaded code JIT because there is a direct mapping between
bytecode and threaded code, at least in my design, so that each byte of
bytecode produced 8 bytes of threaded code (function and parameter, 32 bit
arch).  But in general it can be difficult.  One wants to be able to take
advantage of the static receiver, but if one does, then mapping activations
between different versions of a copied down (& possibly optimized) method
at any suspension point can become problematic.

When I joined Parc Place in '95 I got to know the world's best Smalltalk
JIT at that time, Peter Deutsch's, HPS (Schiffman's context-to-stack mapper
& Jackson's and Ungar's GC).  This VM doesn't do copy-down and it
influenced my work on the VM (I improved method caching, co-designed a new
bytecode set and closure representation, designed a much improved
context-to-stack mapping scheme, improved old space free space management,
and did the 64-bit object representation which introduced class indices in
headers rather than class pointers.  This experience from '95 through '06
heavily influenced my work on Cog and Spur.  Indeed the architectural idea
for Sista/Scorch was inspired by performance analysis work Peter did,
written up in the internal documentation for HPS.  Peter had introduced an
in-line primitive bytecode so he could hand-generate static code to measure
where the time is spent executing Smalltalk.

With Sista/Scorch the entire policy for code management is moved up to the
image, and is no longer a VM decision.  Scorch is freely able to copy-down
and if it chooses to mediate become and class schema migration it has far
more power to manage more aggressive optimization and code duplication than
the VM.  So I'm looking forward to exploring copy-down in that context.  Of
course, now that Clément has gone to Google, this work won't get done
unless I can find collaborators.  And here I sound like a broken record.

>
> Thanks much for insights!
> -KenD
>


-- 
_,,,^..^,,,_
best, Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210318/f736f473/attachment.html>

From vanessa at codefrau.net  Thu Mar 18 21:01:38 2021
From: vanessa at codefrau.net (Vanessa Freudenberg)
Date: Thu, 18 Mar 2021 14:01:38 -0700
Subject: [Vm-dev] Copy Down to avoid Looking Up
In-Reply-To: <CAC20JE0edGY4DDg7-AmFKcXAca1sF783wH0R-Tqghn4TKUhfmQ@mail.gmail.com>
References: <ed4fd791dfcc2ac54cacdd69f0fe99f8@whidbey.com>
 <CAC20JE0edGY4DDg7-AmFKcXAca1sF783wH0R-Tqghn4TKUhfmQ@mail.gmail.com>
Message-ID: <CAOWGXNDAWd1ebFCRfk5D+URsZC+qPcoKDsFJ+adfPngFq3Q6-Q@mail.gmail.com>

On Thu, Mar 18, 2021 at 1:11 PM Eliot Miranda <eliot.miranda at gmail.com>
wrote:

>
> Hi Ken,
>
> On Wed, Mar 17, 2021 at 8:13 AM <ken.dickey at whidbey.com> wrote:
>
>>
>> You can help me out.
>>
>> I keep getting this wacky idea which does not seem to go away.  You can
>> tell me what is wrong with it.
>>
>> OK.  Here it is:
>>
>> To get rid of PICs and dynamic method lookups, each Class knows
>> selectors of methods it defines.  Each successful method definition is
>> added not only to the method dictionary of the class, but copied down
>> transitively to all subclasses, and their sub-subclasses, who have not
>> defined a method with the same selector.
>>
>> At #DeepRehash time, _each_ method dictionary knows all of its selectors
>> and creates a "perfect hash" function, or something close to it.
>>
>> So lookup is just into the specific method dictionary of the instance's
>> class which yields the proper method (backstop method is DNU).  Never a
>> miss.  No class hierarchy search for inherited methods.
>>
>> Memory is cheap these days, but selectors and compiled methods are
>> mostly shared and the memory cost of method dictionaries is probably
>> balanced out by elimination of PICS at the call sites.
>>
>> The UI tools need to keep some simple rules (e,g, transitively deleting
>> methods from subclass method dictionaries), but these should be simple
>> and obvious.
>>
>> The above seems simple enough, but nobody seems to be doing it.
>>
>
> I don't think no one is doing this.  I expect for example that the
> SqueakJS effort would explore this.
>

Nah. SqueakJS is supposed to be compatible with existing images, I'm not
interested in changing the image to suit the VM. So this copy-down would
have to happen at image load time or dynamically, *and* it would have to be
hidden from the image. I think inline caches are the safer bet, polymorphic
or not.

Plus, if you squint at them in the right way, PICs *are* equivalent to
copy-down. They "copy" the right method into the cache. That's a
late-bound version of copy-down, whereas actual copy-down would be
early-bound and require all tools to change to maintain the invariants.
Sure it can be done, but it also makes certain use cases a lot more
expensive: adding a method to Object is not a big deal today, but with
copy-down, you would have to copy it to every single class in the whole
system, making it extremely expensive. This sounds more like a deployment
mechanism to me, not like something I would use for live coding.

- Vanessa -
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210318/e00d9827/attachment-0001.html>

From nabble.42 at klix.ch  Fri Mar 19 11:08:49 2021
From: nabble.42 at klix.ch (nabble.42 at klix.ch)
Date: Fri, 19 Mar 2021 12:08:49 +0100
Subject: [Vm-dev] (Rescued) Re:  Copy Down to avoid Looking Up
In-Reply-To: <05fbfbca72dd7a3c445a8a18a12a02af@whidbey.com>
References: <ed4fd791dfcc2ac54cacdd69f0fe99f8@whidbey.com>
 <7c0465b0-dfb9-6300-ce81-f29b19d1aa57@klix.ch>
 <05fbfbca72dd7a3c445a8a18a12a02af@whidbey.com>
Message-ID: <86a40453-998b-5580-7be2-739dcaf64660@klix.ch>

Hi Ken,

... see below ...
On 2021-03-18 16:28, ken.dickey at whidbey.com wrote:
> HI Gerald,
> 
> Thanks for the feedback.
> 
> I am aware of Virtual Function Tables, Sparse Array index compression, 
> lots of strategies for Dynamic Dispatch, many I have used to implement 
> various object systems.
I suspected that you know better than me.
I just explained the train of thought I follow
when I muse about that problem. It always
ends with: "Now you reinvented the wheel
aka. VFTs"
> 
> One can arrange the hash/ID of selectors, classes, and method tables.
> 
> I am looking for something which is space compact, simple to explain, 
> runtime efficient, supports dynamic updates, and "fails softly" (an 
> "imperfect hash" still works in a Dictionary).  Also, deals with 
> polymorphism and super and does not require an expensive #isSubclassOf: 
> test.
I have a hunch that something like
(in pseudo-smalltalk):

receiver class
	caseOf: {
		[ Class1 ] -> [<send directly>]
		[ Class2 ] -> [ ... ] }
	otherwise: [
		<just do the normal thing > ]

will translate to byte code that executes quickly on an interpreter an 
can easily be
(translated|jitted) to fast machine code.
AFIR branch prediction for the resulting
chain of jumps was much better than for
indirect calls. I don't know if this
is still the case with modern processors.
Maybe Elliot can enlighten me here.

I also imagine that this can be done
easily with Sista.

> 
> My thought with "Copy Down" is that while loading a bunch of code, one 
> can build simple method dictionaries, then apply a #deepRehash after the 
> load.  This could be done to affected method dictionaries each time a 
> method is accepted during development as UI's (i.e. users) are slow.
> 
> The #deepRehash for each method dictionary could use simple trials to 
> generate a compact table/array and set the hash for a method Dictionary 
> so that (<selector> hash) <logical-op> (<mtable> hash>) finds the 
> <selector>'s method without collisions.
> 
> The #deepRehash only requires local knowledge (all used selectors are 
> known), not a global analysis to renumber classes or selectors.
> 
> Perfect hash would be something competitive, speed wise, with PICs but 
> does not require cache management or inline code changes.  With lookup 
> in only one table/dict one does not need to traverse the class hierarchy 
> on a miss, because lookup never misses (DNU in every table which is 
> answer on not found -- separate lookup if invoker wants nil instead).
> 
How big do these tables get?

> A bit of work to craft hashfun creation, but seems worthwhile.  And an 
> interesting problem to work on. :)
> 
> The idea seems simple enough that I suspect someone has looked at this 
> strategy.
> 
> If this is a really bad idea, I don't have to spend any more thing 
> thinking about it or worse, actually doing something.  ;^)
> 
> Thanks again,
> -KenD
> 
> 
> On 2021-03-18 07:25, nabble.42 at klix.ch wrote:
>> Hi Ken,
>>
>> I am by no means a VM expert, but this idea can
>> be enhanced further:
>>
>> Let's generate consecutive
>> numbers for all the symbols used as message selectors then this
>> "perfect hash" degenerates
>> to an array with pointers to the compiled
>> method. Let's also ignore
>> -- for the sake of simplicity --
>> symbols that are garbage collected;
>> the holes occurring can simply be filled
>> by some sort "free-list-algorithm.
>>
>> Now you have something the C++ programmer knows
>> really well: A virtual function table.
>>
>> I have no clue whether those VFTs will help.
>>
>>
>> Just my 0.01€.
>>
>>
>> Best Regards,
>>
>> Gerald
Again just my sundry thoughts.


Best Regards,

Gerald

From vanessa at codefrau.net  Fri Mar 19 16:11:52 2021
From: vanessa at codefrau.net (Vanessa Freudenberg)
Date: Fri, 19 Mar 2021 09:11:52 -0700
Subject: [Vm-dev] regression: TestObjectsAsMethods crash in 4.19.5 (not
	in 4.16.7)
In-Reply-To: <20210302000410.GA79682@shell.msen.com>
References: <949389016.25753821.1613919758428.JavaMail.zimbra@telenet.be>
 <605876499.30501015.1614071179388.JavaMail.zimbra@telenet.be>
 <766189717.50931693.1614619258680.JavaMail.zimbra@telenet.be>
 <20210302000410.GA79682@shell.msen.com>
Message-ID: <CAOWGXNDAH+6AER966TA4FMYorKRJWyupdQrtqKRs86togJ6dOA@mail.gmail.com>

On Mon, Mar 1, 2021 at 16:05 David T. Lewis <lewis at mail.msen.com> wrote:

>
> On Mon, Mar 01, 2021 at 06:20:58PM +0100, stes at PANDORA.BE wrote:
> >
> > > Install primitiveArrayBecomeOneWayNoCopyHash as primitive 248
> > > replacing obsolete primitiveInvokeObjectAsMethod.
> >
> > Any news on this Tests-ObjectsAsMethods issue ?
> >
> > The purpose of those tests are not clear to me;
> >
> > Perhaps the change for the "primitives" which was made, can be reverted ?
> >
> > Possibly that creates a new problem for
> primitiveArrayBecomeOneWayNoCopyHash.
> >
> > David Stes
> >
>
> You can consider this a bug in the most recent interpreter VMs. I had
> failed to notice that there is a hard coded reference to that primitive
> number within the VM itself, and this reference is now resulting in
> invocation of the wrong function.
>
> A trivial fix is to rearrange some primtive number assignments, but the
> correct thing to do is adopt the logic that Eliot uses in oscog so that
> the function is not invoked as a "primitive" at all.
>
> I'll try to follow up on this next week.
>
> Dave



This test crashes SqueakJS, too. Could be the same primitive reassignment
confusion.

Can you describe this “hard coded reference”? Under what circumstances is a
primitive being called without being an actual primitive?

Vanessa

>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210319/1b7f351b/attachment.html>

From lewis at mail.msen.com  Fri Mar 19 18:47:28 2021
From: lewis at mail.msen.com (David T. Lewis)
Date: Fri, 19 Mar 2021 14:47:28 -0400
Subject: [Vm-dev] regression: TestObjectsAsMethods crash in 4.19.5 (not
	in 4.16.7)
In-Reply-To: <CAOWGXNDAH+6AER966TA4FMYorKRJWyupdQrtqKRs86togJ6dOA@mail.gmail.com>
References: <949389016.25753821.1613919758428.JavaMail.zimbra@telenet.be>
 <605876499.30501015.1614071179388.JavaMail.zimbra@telenet.be>
 <766189717.50931693.1614619258680.JavaMail.zimbra@telenet.be>
 <20210302000410.GA79682@shell.msen.com>
 <CAOWGXNDAH+6AER966TA4FMYorKRJWyupdQrtqKRs86togJ6dOA@mail.gmail.com>
Message-ID: <20210319184728.GA24254@shell.msen.com>

On Fri, Mar 19, 2021 at 09:11:52AM -0700, Vanessa Freudenberg wrote:
>  
> On Mon, Mar 1, 2021 at 16:05 David T. Lewis <lewis at mail.msen.com> wrote:
> 
> >
> > On Mon, Mar 01, 2021 at 06:20:58PM +0100, stes at PANDORA.BE wrote:
> > >
> > > > Install primitiveArrayBecomeOneWayNoCopyHash as primitive 248
> > > > replacing obsolete primitiveInvokeObjectAsMethod.
> > >
> > > Any news on this Tests-ObjectsAsMethods issue ?
> > >
> > > The purpose of those tests are not clear to me;
> > >
> > > Perhaps the change for the "primitives" which was made, can be reverted ?
> > >
> > > Possibly that creates a new problem for
> > primitiveArrayBecomeOneWayNoCopyHash.
> > >
> > > David Stes
> > >
> >
> > You can consider this a bug in the most recent interpreter VMs. I had
> > failed to notice that there is a hard coded reference to that primitive
> > number within the VM itself, and this reference is now resulting in
> > invocation of the wrong function.
> >
> > A trivial fix is to rearrange some primtive number assignments, but the
> > correct thing to do is adopt the logic that Eliot uses in oscog so that
> > the function is not invoked as a "primitive" at all.
> >
> > I'll try to follow up on this next week.
> >
> > Dave
> 
> 
> 
> This test crashes SqueakJS, too. Could be the same primitive reassignment
> confusion.
> 
> Can you describe this ???hard coded reference???? Under what circumstances is a
> primitive being called without being an actual primitive?
> 

Hi Vanessa,

Sorry I did not think about SqueakJS and yes I am sure it is the same issue.

The hard coded reference to primitive 248 is in ContextInterpreter>>lookupMethodInDictionary:
which assumes that primitiveInvokeObjectAsMethod is installed as primitive 248.

I tried assigning primitiveInvokeObjectAsMethod to primitive number 575 in
initializePrimitiveTable (last available entry in the current table, which
appeared to be unused), and changed lookupMethodInDictionary: to call 575
rather than 248.

That worked fine until I tried running a trunk-level V3 image, which failed
because we are now calling primitive 575 for primitiveHighBit.

I'm sure there's some other primitive number we could use to hide
primitiveInvokeObjectAsMethod elsewhere in the primitive table, but this
approach seems obviously wrong to me, and the right thing would be to do
whatever Eliot did in oscog to avoid trying to pretend that this method
is a primitive (which it is not despite its name).

At that point I had no more time to follow up on this, sorry. But that's
where the problem lies, and I'm sure there is a way to do this correctly
based on Eliot's implementation.

Dave


From vanessa at codefrau.net  Fri Mar 19 20:14:32 2021
From: vanessa at codefrau.net (Vanessa Freudenberg)
Date: Fri, 19 Mar 2021 13:14:32 -0700
Subject: [Vm-dev] regression: TestObjectsAsMethods crash in 4.19.5 (not
	in 4.16.7)
In-Reply-To: <20210319184728.GA24254@shell.msen.com>
References: <949389016.25753821.1613919758428.JavaMail.zimbra@telenet.be>
 <605876499.30501015.1614071179388.JavaMail.zimbra@telenet.be>
 <766189717.50931693.1614619258680.JavaMail.zimbra@telenet.be>
 <20210302000410.GA79682@shell.msen.com>
 <CAOWGXNDAH+6AER966TA4FMYorKRJWyupdQrtqKRs86togJ6dOA@mail.gmail.com>
 <20210319184728.GA24254@shell.msen.com>
Message-ID: <CAOWGXNA6b=CT37h+AKTHRosAjOTdinGVN_dk8pJ1Acgr5ggBOA@mail.gmail.com>

On Fri, Mar 19, 2021 at 11:47 AM David T. Lewis <lewis at mail.msen.com> wrote:

>
> On Fri, Mar 19, 2021 at 09:11:52AM -0700, Vanessa Freudenberg wrote:
> >
> > On Mon, Mar 1, 2021 at 16:05 David T. Lewis <lewis at mail.msen.com> wrote:
> >
> > >
> > > On Mon, Mar 01, 2021 at 06:20:58PM +0100, stes at PANDORA.BE wrote:
> > > >
> > > > > Install primitiveArrayBecomeOneWayNoCopyHash as primitive 248
> > > > > replacing obsolete primitiveInvokeObjectAsMethod.
> > > >
> > > > Any news on this Tests-ObjectsAsMethods issue ?
> > > >
> > > > The purpose of those tests are not clear to me;
> > > >
> > > > Perhaps the change for the "primitives" which was made, can be
> reverted ?
> > > >
> > > > Possibly that creates a new problem for
> > > primitiveArrayBecomeOneWayNoCopyHash.
> > > >
> > > > David Stes
> > > >
> > >
> > > You can consider this a bug in the most recent interpreter VMs. I had
> > > failed to notice that there is a hard coded reference to that primitive
> > > number within the VM itself, and this reference is now resulting in
> > > invocation of the wrong function.
> > >
> > > A trivial fix is to rearrange some primtive number assignments, but the
> > > correct thing to do is adopt the logic that Eliot uses in oscog so that
> > > the function is not invoked as a "primitive" at all.
> > >
> > > I'll try to follow up on this next week.
> > >
> > > Dave
> >
> >
> >
> > This test crashes SqueakJS, too. Could be the same primitive reassignment
> > confusion.
> >
> > Can you describe this ???hard coded reference???? Under what
> circumstances is a
> > primitive being called without being an actual primitive?
> >
>
> Hi Vanessa,
>
> Sorry I did not think about SqueakJS and yes I am sure it is the same
> issue.
>
> The hard coded reference to primitive 248 is in
> ContextInterpreter>>lookupMethodInDictionary:
> which assumes that primitiveInvokeObjectAsMethod is installed as primitive
> 248.
>
> I tried assigning primitiveInvokeObjectAsMethod to primitive number 575 in
> initializePrimitiveTable (last available entry in the current table, which
> appeared to be unused), and changed lookupMethodInDictionary: to call 575
> rather than 248.
>
> That worked fine until I tried running a trunk-level V3 image, which failed
> because we are now calling primitive 575 for primitiveHighBit.
>
> I'm sure there's some other primitive number we could use to hide
> primitiveInvokeObjectAsMethod elsewhere in the primitive table, but this
> approach seems obviously wrong to me, and the right thing would be to do
> whatever Eliot did in oscog to avoid trying to pretend that this method
> is a primitive (which it is not despite its name).
>
> At that point I had no more time to follow up on this, sorry. But that's
> where the problem lies, and I'm sure there is a way to do this correctly
> based on Eliot's implementation.
>
> Dave
>

Ah that makes sense. Yes, in SqueakJS it is used directly too:

https://github.com/codefrau/SqueakJS/blob/02d88e6cd5592029071c1939d85181b79a8f0d7a/vm.interpreter.js#L934


... and the easiest fix would indeed be to assign an unused primitive
number, because that would require no changes to the rest of the
execution mechanism. Putting it in the lookup phase, and caching it, seems
quite elegant. Otherwise we would have to find another place to check that
the method dict entry is actually a method, and invoke the run:with:in:
mechanism if not.

Vanessa
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210319/638cf584/attachment-0001.html>

From stes at telenet.be  Sat Mar 20 14:33:34 2021
From: stes at telenet.be (stes@PANDORA.BE)
Date: Sat, 20 Mar 2021 15:33:34 +0100 (CET)
Subject: [Vm-dev] patch for allocateBuffer in AsyncFilePlugin
In-Reply-To: <669270513.57271174.1616076614691.JavaMail.zimbra@telenet.be>
References: <669270513.57271174.1616076614691.JavaMail.zimbra@telenet.be>
Message-ID: <1491228737.65919207.1616250814727.JavaMail.zimbra@telenet.be>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256


Hello,

Can the allocateBuffer patch be added/committed to subversion, please ... ?

Thanks!

Regards,
David Stes

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJgVgepAAoJEAwpOKXMq1Ma4mMH/0vW0KPYBMtrvbeHl+9ec47x
6cIGQ/EHadOH6Ae2+sj96I2ArR9uHGXRPWVKa3Wf2G3NQwPMCUvKlqNCPfzZo8g9
4wYzjpvjrMuSgrnP7A9jCvYShQ/TZY7QwBWl83kbnW+7w4oLZxkXUIcOG6RhD71s
gXmNlFpx6xLTqe+Z1ljeR58/vh7TfxBE3lJy3arxY+apyyCw05BVeErn8zqCDMzx
KDnfJ7XrINJt0MubpfdvzFf4KvoPNwFSuT9MTqMOE0zxAqKLz5sCsFQfevhnZ3Qq
l4SRQhs5Mo01a+x4hCNT3yrHxgnJ8aGpWjxddW9wJ+3F0QTav9htAAb3wPBUCgk=
=+txC
-----END PGP SIGNATURE-----

From ken.dickey at whidbey.com  Sat Mar 20 15:18:34 2021
From: ken.dickey at whidbey.com (ken.dickey at whidbey.com)
Date: Sat, 20 Mar 2021 08:18:34 -0700
Subject: [Vm-dev] [Re: Copy Down to avoid Looking Up]
Message-ID: <d63eddc62811aca63a1dd67ca6690bab@whidbey.com>

Thanks all for contributing to my education.  Much to think about.

Even with memory "on chip" these days for SOC, it seems IC (MonoMorphic) 
+ PIC (PolyMorphic) win over copy-down based on CPU cache locality.

For MegaMorphic case, one _might_ profitably specialize such selectors 
to contain/point-to caching dict of class->method as the lookups fault 
in.  Have to test to see if caching the larger number of lookup results 
to avoid chronic dynamic lookup pays off.  Simpler cache strategy to 
manage than copy-down, but equivalent based on Vanessa's squint test.  
Probably data dependent..

Thanks again to all!
-KenD

From lewis at mail.msen.com  Sat Mar 20 15:45:28 2021
From: lewis at mail.msen.com (David T. Lewis)
Date: Sat, 20 Mar 2021 11:45:28 -0400
Subject: [Vm-dev] patch for allocateBuffer in AsyncFilePlugin
In-Reply-To: <1491228737.65919207.1616250814727.JavaMail.zimbra@telenet.be>
References: <669270513.57271174.1616076614691.JavaMail.zimbra@telenet.be>
 <1491228737.65919207.1616250814727.JavaMail.zimbra@telenet.be>
Message-ID: <20210320154528.GA16758@shell.msen.com>

On Sat, Mar 20, 2021 at 03:33:34PM +0100, stes at PANDORA.BE wrote:
> 
> Hello,
> 
> Can the allocateBuffer patch be added/committed to subversion, please ... ?
> 
> Thanks!
> 
> Regards,
> David Stes
>

Done.

Thanks David 
 

From lewis at mail.msen.com  Sat Mar 20 16:01:22 2021
From: lewis at mail.msen.com (David T. Lewis)
Date: Sat, 20 Mar 2021 12:01:22 -0400
Subject: [Vm-dev] regression: TestObjectsAsMethods crash in 4.19.5 (not
	in 4.16.7)
In-Reply-To: <CAOWGXNA6b=CT37h+AKTHRosAjOTdinGVN_dk8pJ1Acgr5ggBOA@mail.gmail.com>
References: <949389016.25753821.1613919758428.JavaMail.zimbra@telenet.be>
 <605876499.30501015.1614071179388.JavaMail.zimbra@telenet.be>
 <766189717.50931693.1614619258680.JavaMail.zimbra@telenet.be>
 <20210302000410.GA79682@shell.msen.com>
 <CAOWGXNDAH+6AER966TA4FMYorKRJWyupdQrtqKRs86togJ6dOA@mail.gmail.com>
 <20210319184728.GA24254@shell.msen.com>
 <CAOWGXNA6b=CT37h+AKTHRosAjOTdinGVN_dk8pJ1Acgr5ggBOA@mail.gmail.com>
Message-ID: <20210320160122.GB16758@shell.msen.com>

On Fri, Mar 19, 2021 at 01:14:32PM -0700, Vanessa Freudenberg wrote:
>  
> 
> Ah that makes sense. Yes, in SqueakJS it is used directly too:
> 
> https://github.com/codefrau/SqueakJS/blob/02d88e6cd5592029071c1939d85181b79a8f0d7a/vm.interpreter.js#L934
> 
> 
> ... and the easiest fix would indeed be to assign an unused primitive
> number, because that would require no changes to the rest of the
> execution mechanism. Putting it in the lookup phase, and caching it, seems
> quite elegant. Otherwise we would have to find another place to check that
> the method dict entry is actually a method, and invoke the run:with:in:
> mechanism if not.
> 

Hmmm that gives me an idea. We can extend the actual size of the primitive
table beyond MaxPrimitiveIndex and hide primitiveInvokeObjectAsMethod there.
That protects it against accidental invocation from the image as more numbered
primitives get added (yuk). I'll give that a try and see how it looks.

Dave


From commits at source.squeak.org  Sat Mar 20 17:32:31 2021
From: commits at source.squeak.org (commits at source.squeak.org)
Date: Sat, 20 Mar 2021 17:32:31 0000
Subject: [Vm-dev] VM Maker: VMMaker-dtl.423.mcz
Message-ID: <E1lNfSn-0003Yt-70@andreas>

David T. Lewis uploaded a new version of VMMaker to project VM Maker:
http://source.squeak.org/VMMaker/VMMaker-dtl.423.mcz

==================== Summary ====================

Name: VMMaker-dtl.423
Author: dtl
Time: 20 March 2021, 12:27:52.291 pm
UUID: b744c4a9-0a90-4104-8680-6e815df0f064
Ancestors: VMMaker-dtl.422

VMMaker 4.19.6 restore support for objects as methods.
Extend the internal size of primitiveTable beyond MaxPrimitiveIndex and reinstall primitiveInvokeObjectAsMethod in the protected region. Allows invocation from lookupMethodInDictionary without exposing the method to the image as a numbered primitive. Also clean up a duplicate currentByteCode in class Interpreter declaration, not sure how that happened.

=============== Diff against VMMaker-dtl.422 ===============

Item was changed:
  ----- Method: ContextInterpreter class>>declareCVarsIn: (in category 'translation') -----
  declareCVarsIn: aCCodeGenerator 
  	aCCodeGenerator addHeaderFile: '<setjmp.h>'.
  	aCCodeGenerator var: #statGCTime type: #sqLong.
  	aCCodeGenerator var: #statFullGCMSecs type: #sqLong.
  	aCCodeGenerator var: #statIGCDeltaTime type: #sqLong.
  	aCCodeGenerator var: #statIncrGCMSecs type: #sqLong.
  	aCCodeGenerator var: #compilerHooks declareC: 'sqInt (*compilerHooks[' , (CompilerHooksSize + 1) printString , '])()'.
  	aCCodeGenerator var: #interpreterVersion declareC: 'const char *interpreterVersion = "' , SmalltalkImage current datedVersion , ' [' , SmalltalkImage current lastUpdateString , ']"'.
  	self declareCAsOop: {#newMethod. #activeContext. #theHomeContext} in: aCCodeGenerator.
  	"Reinitialized at interpreter entry by #initializeImageFormatVersion"
  	aCCodeGenerator var: #imageFormatVersionNumber declareC: 'sqInt imageFormatVersionNumber = 0'.
  	"Declared here to prevent inclusion in foo struct by
  	CCodeGeneratorGlobalStructure"
  	aCCodeGenerator var: #imageFormatInitialVersion declareC: 'sqInt imageFormatInitialVersion = 0'.
+ 	aCCodeGenerator var: #primitiveTable declareC: 'void (*primitiveTable[', (MaxPrimitiveIndex + ProtectedPrimitiveCount + 1) asString, '])(void)'.
- 	aCCodeGenerator var: #primitiveTable declareC: 'void (*primitiveTable[', MaxPrimitiveIndex asString, '])(void)'.
  	aCCodeGenerator var: #primitiveTableDefaults declareC: 'void (*primitiveTableDefaults[' , (MaxPrimitiveIndex + 2) printString , '] )(void)= ' , self primitiveTableString.
  !

Item was changed:
  ----- Method: ContextInterpreter>>installPrimitiveTable (in category 'primitive table') -----
  installPrimitiveTable
  	<inline: false>
+ 	0 to: MaxPrimitiveIndex + ProtectedPrimitiveCount do: [ :i |
- 	0 to: MaxPrimitiveIndex do: [ :i |
  		primitiveTable at: i put: (primitiveTableDefaults at: i) ].
  	self updatePrimitiveTable.
  !

Item was changed:
  ----- Method: ContextInterpreter>>lookupMethodInDictionary: (in category 'message sending') -----
  lookupMethodInDictionary: dictionary 
  	"This method lookup tolerates integers as Dictionary keys to 
  	support execution of images in which Symbols have been 
  	compacted out"
  	| length index mask wrapAround nextSelector methodArray |
  	<inline: true>
  	length := objectMemory fetchWordLengthOf: dictionary.
  	mask := length - SelectorStart - 1.
  	(objectMemory isIntegerObject: messageSelector)
  		ifTrue: [index := (mask bitAnd: (objectMemory integerValueOf: messageSelector)) + SelectorStart]
  		ifFalse: [index := (mask bitAnd: (objectMemory hashBitsOf: messageSelector)) + SelectorStart].
  
  	"It is assumed that there are some nils in this dictionary, and search will 
  	stop when one is encountered. However, if there are no nils, then wrapAround 
  	will be detected the second time the loop gets to the end of the table."
  	wrapAround := false.
  	[true]
  		whileTrue: [nextSelector := objectMemory fetchPointer: index ofObject: dictionary.
  			nextSelector = objectMemory getNilObj ifTrue: [^ false].
  			nextSelector = messageSelector
  				ifTrue: [methodArray := objectMemory fetchPointer: MethodArrayIndex ofObject: dictionary.
  					newMethod := objectMemory fetchPointer: index - SelectorStart ofObject: methodArray.
  					"Check if newMethod is a CompiledMethod."
  					(objectMemory isCompiledMethod: newMethod)
  						ifTrue: [primitiveIndex := self primitiveIndexOf: newMethod.
  							primitiveIndex > MaxPrimitiveIndex
  								ifTrue: ["If primitiveIndex is out of range, set to zero before putting in 
  									cache. This is equiv to primFail, and avoids the need to check on 
  									every send."
  									primitiveIndex := 0]]
  						ifFalse: ["indicate that this is no compiled method - use primitiveInvokeObjectAsMethod"
+ 							self flag: #primitiveInvokeObjectAsMethod.
+ 							primitiveIndex := MaxPrimitiveIndex + 1].
- 							primitiveIndex := 248].
  					^ true].
  			index := index + 1.
  			index = length
  				ifTrue: [wrapAround
  						ifTrue: [^ false].
  					wrapAround := true.
  					index := SelectorStart]]!

Item was changed:
  InterpreterPrimitives subclass: #Interpreter
+ 	instanceVariableNames: 'currentBytecode semaphoresUseBufferA semaphoresToSignalA semaphoresToSignalCountA semaphoresToSignalB semaphoresToSignalCountB extraVMMemory globalSessionID method instructionPointer stackPointer localIP localSP localReturnValue messageSelector primitiveFunctionPointer methodCache atCache lkupClass interruptPending savedWindowSize deferDisplayUpdates pendingFinalizationSignals interpreterProxy showSurfaceFn externalPrimitiveTable jmpBuf jmpDepth suspendedCallbacks suspendedMethods'
+ 	classVariableNames: 'AtCacheEntries AtCacheFixedFields AtCacheFmt AtCacheMask AtCacheOop AtCacheSize AtCacheTotalSize AtPutBase MaxExternalPrimitiveTableSize MaxJumpBuf MaxPrimitiveIndex ProtectedPrimitiveCount SemaphoresToSignalSize'
- 	instanceVariableNames: 'currentBytecode semaphoresUseBufferA semaphoresToSignalA semaphoresToSignalCountA semaphoresToSignalB semaphoresToSignalCountB extraVMMemory globalSessionID method instructionPointer stackPointer localIP localSP localReturnValue messageSelector currentByteCode primitiveFunctionPointer methodCache atCache lkupClass interruptPending savedWindowSize deferDisplayUpdates pendingFinalizationSignals interpreterProxy showSurfaceFn externalPrimitiveTable jmpBuf jmpDepth suspendedCallbacks suspendedMethods'
- 	classVariableNames: 'AtCacheEntries AtCacheFixedFields AtCacheFmt AtCacheMask AtCacheOop AtCacheSize AtCacheTotalSize AtPutBase MaxExternalPrimitiveTableSize MaxJumpBuf MaxPrimitiveIndex SemaphoresToSignalSize'
  	poolDictionaries: 'VMMethodCacheConstants'
  	category: 'VMMaker-Interpreter'!
  
  !Interpreter commentStamp: 'dtl 4/22/2016 22:14' prior: 0!
  This class is a complete implementation of the Smalltalk-80 virtual machine, derived originally from the Blue Book specification but quite different in some areas.
  
  ContextInterpreter is the Squeak interpreter VM as originally implemented by Dan Ingalls.
  
  StackInterpreter is the stack mapped interpreter by Eliot Miranda, which provides the basis for later Cog and Spur VMs.!

Item was changed:
  ----- Method: Interpreter class>>declareCVarsIn: (in category 'translation') -----
  declareCVarsIn: aCCodeGenerator
  
  	aCCodeGenerator
  		var: #interpreterProxy  type: #'struct VirtualMachine*'.
  	aCCodeGenerator var: 'semaphoresToSignalA' declareC: 'sqInt semaphoresToSignalA[' , (SemaphoresToSignalSize + 1) printString , ']'.
  	aCCodeGenerator var: 'semaphoresToSignalB' declareC: 'sqInt semaphoresToSignalB[' , (SemaphoresToSignalSize + 1) printString , ']'.
+ 	self declareC: #(stackPointer instructionPointer method)
- 	self declareC: #(instructionPointer method)
  		as: #usqInt
  		in: aCCodeGenerator.
+ 	self declareC: #(localIP localSP)
- 	self declareC: #(stackPointer localIP localSP)
  		as: #'char *'
  		in: aCCodeGenerator.
  	aCCodeGenerator
  		var: #primitiveFunctionPointer
  		declareC: 'void (*primitiveFunctionPointer)()'.
  	aCCodeGenerator
  		var: #methodCache
  		declareC: 'long methodCache[MethodCacheSize + 1 /* ', (MethodCacheSize + 1) printString, ' */]'.
  	aCCodeGenerator
  		var: #atCache
  		declareC: 'sqInt atCache[AtCacheTotalSize + 1 /* ', (AtCacheTotalSize + 1) printString, ' */]'.
  	aCCodeGenerator
  		var: #showSurfaceFn
  		type: #'void *'.
  	aCCodeGenerator
  		var: #externalPrimitiveTable
  		declareC: 'void (*externalPrimitiveTable[' , (MaxExternalPrimitiveTableSize + 1) printString , '])(void)'.
  	aCCodeGenerator
  		var: #jmpBuf
  		declareC: 'jmp_buf jmpBuf[MaxJumpBuf + 1 /* ', (MaxJumpBuf + 1) printString, ' */]'.
  	aCCodeGenerator
  		var: #suspendedCallbacks
  		declareC: 'usqInt suspendedCallbacks[MaxJumpBuf + 1 /* ', (MaxJumpBuf + 1) printString, ' */]'.
  	aCCodeGenerator
  		var: #suspendedMethods
  		declareC: 'usqInt suspendedMethods[MaxJumpBuf + 1 /* ', (MaxJumpBuf + 1) printString, ' */]'.
  
  
  
  !

Item was changed:
  ----- Method: Interpreter class>>initializePrimitiveTable (in category 'initialization') -----
(excessive size, no diff calculated)

Item was changed:
  ----- Method: VMMaker class>>versionString (in category 'version testing') -----
  versionString
  
  	"VMMaker versionString"
  
+ 	^'4.19.6'!
- 	^'4.19.5'!


From lewis at mail.msen.com  Sat Mar 20 17:37:03 2021
From: lewis at mail.msen.com (David T. Lewis)
Date: Sat, 20 Mar 2021 13:37:03 -0400
Subject: [Vm-dev] regression: TestObjectsAsMethods crash in 4.19.5 (not
	in 4.16.7)
In-Reply-To: <576861635.57225754.1616076329186.JavaMail.zimbra@telenet.be>
References: <949389016.25753821.1613919758428.JavaMail.zimbra@telenet.be>
 <605876499.30501015.1614071179388.JavaMail.zimbra@telenet.be>
 <857058996.42907317.1615746625509.JavaMail.zimbra@telenet.be>
 <1310783440.43126857.1615752926405.JavaMail.zimbra@telenet.be>
 <576861635.57225754.1616076329186.JavaMail.zimbra@telenet.be>
Message-ID: <20210320173703.GA33107@shell.msen.com>

On Thu, Mar 18, 2021 at 03:05:29PM +0100, stes at PANDORA.BE wrote:
> 
> 
> If a good solution exists, it's better of course than a quick 
> (and possibly) wrong short term solution ...
> 
> Thanks for having looked into this.
> 
> David Stes
>

Hi David,

I found a "good enough" solution. Sources are updated in SVN r3799.

Dave
 

From tim at rowledge.org  Sat Mar 20 18:17:23 2021
From: tim at rowledge.org (tim Rowledge)
Date: Sat, 20 Mar 2021 11:17:23 -0700
Subject: [Vm-dev] regression: TestObjectsAsMethods crash in 4.19.5 (not
 in 4.16.7)
In-Reply-To: <20210320160122.GB16758@shell.msen.com>
References: <949389016.25753821.1613919758428.JavaMail.zimbra@telenet.be>
 <605876499.30501015.1614071179388.JavaMail.zimbra@telenet.be>
 <766189717.50931693.1614619258680.JavaMail.zimbra@telenet.be>
 <20210302000410.GA79682@shell.msen.com>
 <CAOWGXNDAH+6AER966TA4FMYorKRJWyupdQrtqKRs86togJ6dOA@mail.gmail.com>
 <20210319184728.GA24254@shell.msen.com>
 <CAOWGXNA6b=CT37h+AKTHRosAjOTdinGVN_dk8pJ1Acgr5ggBOA@mail.gmail.com>
 <20210320160122.GB16758@shell.msen.com>
Message-ID: <BDAF94C3-C3FB-48CD-9101-A04C2D838F17@rowledge.org>



> On 2021-03-20, at 9:01 AM, David T. Lewis <lewis at mail.msen.com> wrote:
> 
> That protects it against accidental invocation from the image as more numbered
> primitives get added (yuk). I'll give that a try and see how it looks.

I really wish for no numbered primitives at all. I thought I had worked out how once upon a time but not quite.

tim
--
tim Rowledge; tim at rowledge.org; http://www.rowledge.org/tim
Diagnostics are the programs that run when nothing else will.



From stes at telenet.be  Sat Mar 20 19:14:30 2021
From: stes at telenet.be (stes@PANDORA.BE)
Date: Sat, 20 Mar 2021 20:14:30 +0100 (CET)
Subject: [Vm-dev] regression: TestObjectsAsMethods crash in 4.19.5 (not
 in 4.16.7)
In-Reply-To: <576861635.57225754.1616076329186.JavaMail.zimbra@telenet.be>
References: <949389016.25753821.1613919758428.JavaMail.zimbra@telenet.be>
 <605876499.30501015.1614071179388.JavaMail.zimbra@telenet.be>
 <857058996.42907317.1615746625509.JavaMail.zimbra@telenet.be>
 <1310783440.43126857.1615752926405.JavaMail.zimbra@telenet.be>
 <576861635.57225754.1616076329186.JavaMail.zimbra@telenet.be>
Message-ID: <1473812179.66521259.1616267670305.JavaMail.zimbra@telenet.be>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256


Thanks, I've tested the 4.19.6 version and it works.

By it works, I mean that the Tests-ObjectsAsMethods is no longer crashing,
tested on OpenIndiana.

Also I have ran all 3763 tests again (takes a while) to see if the
rest works or continues to work.

I'm having a few failures on Solaris and OpenIndiana in test-suite,
but I'm not worrying too much about them;  a failure is not as bad as a crash.

With 4.19.6 I have the following results on OpenIndiana:

3763 run, 3724 passes, 24 expected failures, 15 failures, 0 errors, 0 unexpected passes

The run:with:in: crash was worse but a crash aborted the entire test-suite.

However that is fixed now - so the abort/crash is no longer happening.

Perhaps run:with:in: is a little bit of an obscure feature of Squeak,
but it looked bad that it was crashing (SIGSEGV) which for a new user,
may be confusing and giving a bad impression (worse than it really is).

Also I'll try to test SqueakJS which is something I have no experience with,
at all (JS = Javascript).

As far as I understand, SqueakJS is a Javascript runtime for Squeak.

I'll try to test that as well ...

I'm learning about all the different runtimes ...

  - squeak-5 (OpenSmalltalk) and squeak-5 Cog Spur (COG VM)
  - squeak-4 classical VM
  - squeakJS javascript

The best results related to run:with:in were with OpenSmalltalk squeak-5 (COG),
but I think the various runtimes have their own strenghts and weaknesses.

Regards,
David Stes

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJgVkkRAAoJEAwpOKXMq1MaQbcH/ix2VkUQ2mln6TaayK9AV+ME
ocFg6s6Q1eWJx9Y7fkCoKSilYqOEJAadvmzI9a+2cuMFWPI3lzwAj2UCf0zSikkt
9b2hsXYmqDZv2sAGWqSOsokbul00JNHszJP01dVNMBGkw3KgEZ6FPFo3OvMGUyik
ObY/KQs+ltqBXC6WBrBB8n2MWPcsOVdSjpTZ+wwNYfa5QjE3h5LBtObiSOSO0V1n
q2vGNbQ2N6FdPEzDUjG3XQs53YqSrTKxidHFTvjrBmzfOJxCu/KLqZxSnS9OWQ2i
uHbMbbGZLAcL+hWz4RsLjJn/6VGtPDt/ScW5TAgbDDLcTBnQQK9xtQ5VmV/Yt2w=
=qtca
-----END PGP SIGNATURE-----

From lewis at mail.msen.com  Sat Mar 20 22:14:20 2021
From: lewis at mail.msen.com (David T. Lewis)
Date: Sat, 20 Mar 2021 18:14:20 -0400
Subject: [Vm-dev] regression: TestObjectsAsMethods crash in 4.19.5 (not
	in 4.16.7)
In-Reply-To: <1473812179.66521259.1616267670305.JavaMail.zimbra@telenet.be>
References: <949389016.25753821.1613919758428.JavaMail.zimbra@telenet.be>
 <605876499.30501015.1614071179388.JavaMail.zimbra@telenet.be>
 <857058996.42907317.1615746625509.JavaMail.zimbra@telenet.be>
 <1310783440.43126857.1615752926405.JavaMail.zimbra@telenet.be>
 <576861635.57225754.1616076329186.JavaMail.zimbra@telenet.be>
 <1473812179.66521259.1616267670305.JavaMail.zimbra@telenet.be>
Message-ID: <20210320221420.GA70542@shell.msen.com>

On Sat, Mar 20, 2021 at 08:14:30PM +0100, stes at PANDORA.BE wrote:
> 
> Also I'll try to test SqueakJS which is something I have no experience with,
> at all (JS = Javascript).
> 
> As far as I understand, SqueakJS is a Javascript runtime for Squeak.
> 
> I'll try to test that as well ...
> 
> I'm learning about all the different runtimes ...
> 
>   - squeak-5 (OpenSmalltalk) and squeak-5 Cog Spur (COG VM)
>   - squeak-4 classical VM
>   - squeakJS javascript
>

I certainly encourage you to get acquainted with SqueakJS. It is not the
fastest VM for Squeak, but is it certainly the most portable with respect
to supporting various Squeak images, and it demonstrates that the virtual
machine that runs the Squeak image does not necessarily need to be tied
to any specific operating system or implementation language.

Dave
 

From nicolas.cellier.aka.nice at gmail.com  Sat Mar 20 23:00:04 2021
From: nicolas.cellier.aka.nice at gmail.com (Nicolas Cellier)
Date: Sun, 21 Mar 2021 00:00:04 +0100
Subject: [Vm-dev] regression: TestObjectsAsMethods crash in 4.19.5 (not
	in 4.16.7)
In-Reply-To: <20210320221420.GA70542@shell.msen.com>
References: <949389016.25753821.1613919758428.JavaMail.zimbra@telenet.be>
 <605876499.30501015.1614071179388.JavaMail.zimbra@telenet.be>
 <857058996.42907317.1615746625509.JavaMail.zimbra@telenet.be>
 <1310783440.43126857.1615752926405.JavaMail.zimbra@telenet.be>
 <576861635.57225754.1616076329186.JavaMail.zimbra@telenet.be>
 <1473812179.66521259.1616267670305.JavaMail.zimbra@telenet.be>
 <20210320221420.GA70542@shell.msen.com>
Message-ID: <CAKnRiT4+Kin1VTPfenvJPqDPLVWddvtiKBxarTJZTPQnbgbycw@mail.gmail.com>

Hi all,
just FYI, I had to use a numbered primitive for highBit because to my
knowledge, named primitives cannot be jitted yet...
And the main interest for this primitive is to be jitted.
https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/418

Le sam. 20 mars 2021 à 23:14, David T. Lewis <lewis at mail.msen.com> a écrit :
>
>
> On Sat, Mar 20, 2021 at 08:14:30PM +0100, stes at PANDORA.BE wrote:
> >
> > Also I'll try to test SqueakJS which is something I have no experience with,
> > at all (JS = Javascript).
> >
> > As far as I understand, SqueakJS is a Javascript runtime for Squeak.
> >
> > I'll try to test that as well ...
> >
> > I'm learning about all the different runtimes ...
> >
> >   - squeak-5 (OpenSmalltalk) and squeak-5 Cog Spur (COG VM)
> >   - squeak-4 classical VM
> >   - squeakJS javascript
> >
>
> I certainly encourage you to get acquainted with SqueakJS. It is not the
> fastest VM for Squeak, but is it certainly the most portable with respect
> to supporting various Squeak images, and it demonstrates that the virtual
> machine that runs the Squeak image does not necessarily need to be tied
> to any specific operating system or implementation language.
>
> Dave
>

From lewis at mail.msen.com  Sun Mar 21 00:11:25 2021
From: lewis at mail.msen.com (David T. Lewis)
Date: Sat, 20 Mar 2021 20:11:25 -0400
Subject: [Vm-dev] Fast named primitives (was: regression:
	TestObjectsAsMethods crash in 4.19.5 (not in 4.16.7))
In-Reply-To: <BDAF94C3-C3FB-48CD-9101-A04C2D838F17@rowledge.org>
References: <949389016.25753821.1613919758428.JavaMail.zimbra@telenet.be>
 <605876499.30501015.1614071179388.JavaMail.zimbra@telenet.be>
 <766189717.50931693.1614619258680.JavaMail.zimbra@telenet.be>
 <20210302000410.GA79682@shell.msen.com>
 <CAOWGXNDAH+6AER966TA4FMYorKRJWyupdQrtqKRs86togJ6dOA@mail.gmail.com>
 <20210319184728.GA24254@shell.msen.com>
 <CAOWGXNA6b=CT37h+AKTHRosAjOTdinGVN_dk8pJ1Acgr5ggBOA@mail.gmail.com>
 <20210320160122.GB16758@shell.msen.com>
 <BDAF94C3-C3FB-48CD-9101-A04C2D838F17@rowledge.org>
Message-ID: <20210321001125.GA85947@shell.msen.com>

On Sun, Mar 21, 2021 at 12:00:04AM +0100, Nicolas Cellier wrote:
>
> On Sat, Mar 20, 2021 at 11:17:23AM -0700, tim Rowledge wrote:
> > 
> > > On 2021-03-20, at 9:01 AM, David T. Lewis <lewis at mail.msen.com> wrote:
> > > 
> > > That protects it against accidental invocation from the image as more numbered
> > > primitives get added (yuk). I'll give that a try and see how it looks.
> > 
> > I really wish for no numbered primitives at all. I thought I had worked out how once upon a time but not quite.
> > 
>
> just FYI, I had to use a numbered primitive for highBit because to my
> knowledge, named primitives cannot be jitted yet...
> And the main interest for this primitive is to be jitted.
> https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/418
>

I wonder if there might be a solution that meets both goals:

  1) From the image, I want to call named primitives

  2) In the VM, I need the primitives to be numbered for fast JIT

Is there some way that a table of primitive numbers could be created
at runtime to support the JIT without requiring a declarative table
of primitive numbers at VM compile time?

Dave
 

From vanessa at codefrau.net  Sun Mar 21 07:18:59 2021
From: vanessa at codefrau.net (Vanessa Freudenberg)
Date: Sun, 21 Mar 2021 00:18:59 -0700
Subject: [Vm-dev] regression: TestObjectsAsMethods crash in 4.19.5 (not
	in 4.16.7)
In-Reply-To: <20210320173703.GA33107@shell.msen.com>
References: <949389016.25753821.1613919758428.JavaMail.zimbra@telenet.be>
 <605876499.30501015.1614071179388.JavaMail.zimbra@telenet.be>
 <857058996.42907317.1615746625509.JavaMail.zimbra@telenet.be>
 <1310783440.43126857.1615752926405.JavaMail.zimbra@telenet.be>
 <576861635.57225754.1616076329186.JavaMail.zimbra@telenet.be>
 <20210320173703.GA33107@shell.msen.com>
Message-ID: <CAOWGXNBzRUzKmWRTKbNg_DM-GiojZ6a7aFEq7+iGcQ8TUBm29Q@mail.gmail.com>

On Sat, Mar 20, 2021 at 10:37 AM David T. Lewis <lewis at mail.msen.com> wrote:

>
> On Thu, Mar 18, 2021 at 03:05:29PM +0100, stes at PANDORA.BE wrote:
> >
> >
> > If a good solution exists, it's better of course than a quick
> > (and possibly) wrong short term solution ...
> >
> > Thanks for having looked into this.
> >
> > David Stes
> >
>
> Hi David,
>
> I found a "good enough" solution. Sources are updated in SVN r3799.
>
> Dave
>

Wow, I didn't realize until just now you're still maintaining the
interpreter sources on SVN. I mean I knew you did it somewhere, I just
never bothered to look at it 😳

Also found your VMMaker-dtl.423 with the changes. I just did the same hack
in SqueakJS. It was good enough to not crash the image anymore.

I also had to add a VMMaker version to vm attribute 1004, because of
ImageSegmentTest>>testSelectors 😬

Then 2 out of 3 objects-as-methods test were working, the 3rd one revealed
a bug, with the second line in testAddNumbers erroring.

Tracking that one down was slightly tricky because it looked like it had to
do with the perform. But after banging on it for a while, I found that if
the method/object was found in the cache, the current message selector
global would not be updated. The only place where that is used is in
primitiveInvokeObjectAsMethod. 😜

Now all 3 tests are green. Yay! Thanks for the original report, David S!

- Vanessa -
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210321/fab27144/attachment.html>

From noreply at github.com  Mon Mar 22 01:53:02 2021
From: noreply at github.com (David T Lewis)
Date: Sun, 21 Mar 2021 18:53:02 -0700
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] d49424: Restore
 sqUnixEvent.c which was mistakenly deleted...
Message-ID: <OpenSmalltalk/opensmalltalk-vm/push/refs/heads/Cog/a6fd91-d49424@github.com>

  Branch: refs/heads/Cog
  Home:   https://github.com/OpenSmalltalk/opensmalltalk-vm
  Commit: d494240736f7c0309e3e819784feb1d53ed0985a
      https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/d494240736f7c0309e3e819784feb1d53ed0985a
  Author: David T. Lewis <lewis at mail.msen.com>
  Date:   2021-03-21 (Sun, 21 Mar 2021)

  Changed paths:
    A platforms/unix/vm/sqUnixEvent.c

  Log Message:
  -----------
  Restore sqUnixEvent.c which was mistakenly deleted in an earlier commit



From lewis at mail.msen.com  Mon Mar 22 01:55:51 2021
From: lewis at mail.msen.com (David T. Lewis)
Date: Sun, 21 Mar 2021 21:55:51 -0400
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] sqUnixEvent.c missing
	from platforms/unix/vm (#555)
In-Reply-To: <OpenSmalltalk/opensmalltalk-vm/issues/555/799327721@github.com>
References: <OpenSmalltalk/opensmalltalk-vm/issues/555@github.com>
 <OpenSmalltalk/opensmalltalk-vm/issues/555/799327721@github.com>
Message-ID: <20210322015551.GA13754@shell.msen.com>

On Mon, Mar 15, 2021 at 04:02:43AM -0700, Tom Beckmann wrote:
>  
> The commit that removed it was this one: https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/924a24bb54870a621c5283f23631261d5a792000
> 
> I saw that you did some cleanup of old event primitives, as documented in the commit message, @eliotmiranda. Was sqUnixEvent.c deleted by accident as part of this?
>

I restored the missing sqUnixEvent.c file, which I assume was mistakenly
deleted in that earlier commit.

Dave
 

From no-reply at appveyor.com  Mon Mar 22 02:13:12 2021
From: no-reply at appveyor.com (AppVeyor)
Date: Mon, 22 Mar 2021 02:13:12 +0000
Subject: [Vm-dev] Build failed: opensmalltalk-vm 1.0.2417
Message-ID: <20210322021312.1.B5A4CC074018AE7F@appveyor.com>

An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210322/55ad292f/attachment.html>

From bruce.oneel at pckswarms.ch  Mon Mar 22 10:46:27 2021
From: bruce.oneel at pckswarms.ch (Bruce O'Neel)
Date: Mon, 22 Mar 2021 11:46:27 +0100
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] sqUnixEvent.c missing
 from platforms/unix/vm (#555)
In-Reply-To: <20210322015551.GA13754@shell.msen.com>
References: <20210322015551.GA13754@shell.msen.com>
 <OpenSmalltalk/opensmalltalk-vm/issues/555@github.com>
 <OpenSmalltalk/opensmalltalk-vm/issues/555/799327721@github.com>
Message-ID: <1616409987-38356102e760c938a2dcb06d23dad2a9@pckswarms.ch>


Hi,  
  
That fixed one problem I was tripping over on the ARMv8 build.  
  
There still seems to be a problem with the x86-64 build.    
  
Thanks.  
  
bruce  
  
clang -Wall  -g -O2 -DNDEBUG -DDEBUGVM=0 -msse2 -DCOGMTVM=0 -pthread -DLSB_FIRST=1 -m64 -Wno-missing-braces -Wno-unknown-pragmas -Wno-unused-value -Wno-unu   
sed-label -Wno-unused-function -Wno-unused-variable  -DHAVE_CONFIG_H  -DSQUEAK_BUILTIN_PLUGIN -I/home/edoneel/tmp/opensmalltalk-vm/build.linux64x64/squeak.  
[cog.spur/build](http://cog.spur/build) -I/home/edoneel/tmp/opensmalltalk-vm/build.linux64x64/[squeak.cog.spur/build](http://squeak.cog.spur/build) -I/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm -I/home/e  
doneel/tmp/opensmalltalk-vm/platforms/Cross/vm -I/home/edoneel/tmp/opensmalltalk-vm/spur64src/vm -I/usr/local/include -I/home/edoneel/tmp/opensmalltalk-vm/  
platforms/Cross/vm -I/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm -I/home/edoneel/tmp/opensmalltalk-vm/spur64src/vm -I/home/edoneel/tmp/opensmallta  
lk-vm/platforms/Cross/plugins/FilePlugin -I/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/plugins/B3DAcceleratorPlugin -m64 -Wno-missing-braces -Wno-unk  
nown-pragmas -Wno-unused-value -Wno-unused-label -Wno-unused-function -Wno-unused-variable   -c -o sqUnixMain.o /home/edoneel/tmp/opensmalltalk-vm/platform  
s/unix/vm/sqUnixMain.c  
/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:872:3: warning: implicit declaration of function 'pushOutputFile' is invalid in C99 [-Wim  
plicit-function-declaration]  
  pushOutputFile((char *)STDERR_FILENO);  
  ^  
/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:926:35: error: use of undeclared identifier 'REG_RBP'  
                        void *fp = (void *)(uap ? uap->_FP_IN_UCONTEXT : 0);  
                                                       ^  
/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/include_ucontext.h:78:44: note: expanded from macro '_FP_IN_UCONTEXT'  
# define _FP_IN_UCONTEXT uc_mcontext.gregs[REG_RBP]  
                                           ^  
/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:927:35: error: use of undeclared identifier 'REG_RSP'  
                        void *sp = (void *)(uap ? uap->_SP_IN_UCONTEXT : 0);  
                                                       ^  
/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/include_ucontext.h:79:44: note: expanded from macro '_SP_IN_UCONTEXT'  
# define _SP_IN_UCONTEXT uc_mcontext.gregs[REG_RSP]  
                                           ^  
/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:992:9: error: use of undeclared identifier 'REG_RAX'  
                        regs[REG_RAX], regs[REG_RBX], regs[REG_RCX], regs[REG_RDX],  
                             ^  
/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:992:24: error: use of undeclared identifier 'REG_RBX'  
                        regs[REG_RAX], regs[REG_RBX], regs[REG_RCX], regs[REG_RDX],  
                                            ^  
/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:992:39: error: use of undeclared identifier 'REG_RCX'  
                        regs[REG_RAX], regs[REG_RBX], regs[REG_RCX], regs[REG_RDX],  
                                                           ^  
/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:992:54: error: use of undeclared identifier 'REG_RDX'  
                        regs[REG_RAX], regs[REG_RBX], regs[REG_RCX], regs[REG_RDX],  
                                                                          ^  
/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:993:9: error: use of undeclared identifier 'REG_RDI'  
                        regs[REG_RDI], regs[REG_RSI], regs[REG_RBP], regs[REG_RSP],  
                             ^  
/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:993:24: error: use of undeclared identifier 'REG_RSI'  
                        regs[REG_RDI], regs[REG_RSI], regs[REG_RBP], regs[REG_RSP],  
                                            ^  
/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:993:39: error: use of undeclared identifier 'REG_RBP'  
                        regs[REG_RDI], regs[REG_RSI], regs[REG_RBP], regs[REG_RSP],  
                                                           ^  
/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:993:54: error: use of undeclared identifier 'REG_RSP'  
                        regs[REG_RDI], regs[REG_RSI], regs[REG_RBP], regs[REG_RSP],  
                                                                          ^  
/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:994:9: error: use of undeclared identifier 'REG_R8'  
                        regs[REG_R8 ], regs[REG_R9 ], regs[REG_R10], regs[REG_R11],  
                             ^  
  
  
  
  

> On Mon, Mar 15, 2021 at 04:02:43AM -0700, Tom Beckmann wrote:  
>  >  
>  > The commit that removed it was this one: https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/924a24bb54870a621c5283f23631261d5a792000  
>  >  
>  > I saw that you did some cleanup of old event primitives, as documented in the commit message, @eliotmiranda. Was sqUnixEvent.c deleted by accident as part of this?  
>  >  
>   
>  I restored the missing sqUnixEvent.c file, which I assume was mistakenly  
>  deleted in that earlier commit.  
>   
>  Dave
  

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210322/1df799b5/attachment.html>

From notifications at github.com  Mon Mar 22 12:24:46 2021
From: notifications at github.com (Christoph Thiede)
Date: Mon, 22 Mar 2021 05:24:46 -0700
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] primitiveSSLCreate
 fails sporadically on Windows (#554)
In-Reply-To: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
References: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
Message-ID: <OpenSmalltalk/opensmalltalk-vm/issues/554/804022442@github.com>

Sorry for the late reply. I'm not yet very familiar with building the VM and did not debug it before at all 😅

As far as I can tell, everything looks fine when `LoadLibrary()` is called (I debugged `build.win64x64/squeak.cog.spur/builddbg`):

![image](https://user-images.githubusercontent.com/38782922/111986111-d045c080-8b0d-11eb-83ab-6f7303b88a34.png)

![image](https://user-images.githubusercontent.com/38782922/111989144-aabab600-8b11-11eb-91a4-684807b78944.png)

Still, these calls raise an access violation. The other attempts with `32`/`.dll` postfixes and image path prefix fail, too. As far as I understand it, the screenshotted call should succeed because the `Win32OSProcessPlugin.dll` is in the same directory as `Squeak.exe`.

Does it matter that according to the Process Explorer, the current working directory of Squeak.exe points to the location of the image file rather than to the location of the VM? According to the [docs](https://docs.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order#standard-search-order-for-desktop-applications), this should not matter?

By the way, the SSL plugin can be successfully loaded when running the same image in WSL (Ubuntu). So it's indeed a VM problem.

---

BTW: When trying to run a 64-bit image in a 32-bit VM (which does not work, of course), an exception is generated from the following stack:

```
Squeak.exe!abortMessage(char * fmt) Line 63
Squeak.exe!printUsage(int level) Line 3116
Squeak.exe!sqMain(int argc, char * * argv) Line 1636
Squeak.exe!WinMain(HINSTANCE__ * hInst, HINSTANCE__ * hPrevInstance, char * lpCmdLine, int nCmdShow) Line 1779
Squeak.exe!main(int flags, char * * cmdline, char * * inst) Line 20
Squeak.exe!__tmainCRTStartup() Line 341
```

Shouldn't a meaningful error window be shown here instead? Do you have any idea why this is broken? I also noticed that the `args` variable seems to be corrupted. Or is it just pending to be initialized properly?

![image](https://user-images.githubusercontent.com/38782922/111917579-ff622080-8a80-11eb-9a79-fb17086f8c08.png)


-- 
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/554#issuecomment-804022442
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210322/284b7546/attachment-0001.html>

From lewis at mail.msen.com  Mon Mar 22 23:14:18 2021
From: lewis at mail.msen.com (David T. Lewis)
Date: Mon, 22 Mar 2021 19:14:18 -0400
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] sqUnixEvent.c missing
	from platforms/unix/vm (#555)
In-Reply-To: <1616409987-38356102e760c938a2dcb06d23dad2a9@pckswarms.ch>
References: <20210322015551.GA13754@shell.msen.com>
 <OpenSmalltalk/opensmalltalk-vm/issues/555@github.com>
 <OpenSmalltalk/opensmalltalk-vm/issues/555/799327721@github.com>
 <1616409987-38356102e760c938a2dcb06d23dad2a9@pckswarms.ch>
Message-ID: <20210322231418.GA2171@shell.msen.com>

On Mon, Mar 22, 2021 at 11:46:27AM +0100, Bruce O'Neel wrote:
>  
> 
> Hi,  
>   
> That fixed one problem I was tripping over on the ARMv8 build.  
>   
> There still seems to be a problem with the x86-64 build.????  
>   
> Thanks.  
>   
> bruce  

I see the same issue on my Linux x86-64 PC. I cannot spot the exact cause of
the problem, but it seems to have been introduced in this commit:

commit 6d74c4b652c7780110fe327f97e98aaef5242fbc
Author: Eliot Miranda <eliot.miranda at gmail.com>
Date:   Wed Feb 10 21:41:31 2021 -0800

    Revisions to core include files to get EXPORT defined correctly at the relevant points.
    Essentially move the default EXPORT definitions from sq.h to sqMemoryAccess.h.
    Include fbdev support in build.linux32ARMv6 builds and add -m32 to their CFLAGS
    to attempt to get the 32-bit VMs to be compilable on 64-bit ARM. [ci skip]
    cuz new cogit files will arrive soon.


The changes seem straightforward, but for some reason the header files must
not be getting included properly.

Dave




>   
> clang -Wall?? -g -O2 -DNDEBUG -DDEBUGVM=0 -msse2 -DCOGMTVM=0 -pthread -DLSB_FIRST=1 -m64 -Wno-missing-braces -Wno-unknown-pragmas -Wno-unused-value -Wno-unu   
> sed-label -Wno-unused-function -Wno-unused-variable?? -DHAVE_CONFIG_H?? -DSQUEAK_BUILTIN_PLUGIN -I/home/edoneel/tmp/opensmalltalk-vm/build.linux64x64/squeak.  
> [cog.spur/build](http://cog.spur/build) -I/home/edoneel/tmp/opensmalltalk-vm/build.linux64x64/[squeak.cog.spur/build](http://squeak.cog.spur/build) -I/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm -I/home/e  
> doneel/tmp/opensmalltalk-vm/platforms/Cross/vm -I/home/edoneel/tmp/opensmalltalk-vm/spur64src/vm -I/usr/local/include -I/home/edoneel/tmp/opensmalltalk-vm/  
> platforms/Cross/vm -I/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm -I/home/edoneel/tmp/opensmalltalk-vm/spur64src/vm -I/home/edoneel/tmp/opensmallta  
> lk-vm/platforms/Cross/plugins/FilePlugin -I/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/plugins/B3DAcceleratorPlugin -m64 -Wno-missing-braces -Wno-unk  
> nown-pragmas -Wno-unused-value -Wno-unused-label -Wno-unused-function -Wno-unused-variable???? -c -o sqUnixMain.o /home/edoneel/tmp/opensmalltalk-vm/platform  
> s/unix/vm/sqUnixMain.c  
> /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:872:3: warning: implicit declaration of function 'pushOutputFile' is invalid in C99 [-Wim  
> plicit-function-declaration]  
> ?? pushOutputFile((char *)STDERR_FILENO);  
> ?? ^  
> /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:926:35: error: use of undeclared identifier 'REG_RBP'  
> ?????????????????????????????????????????????? void *fp = (void *)(uap ? uap->_FP_IN_UCONTEXT : 0);  
> ???????????????????????????????????????????????????????????????????????????????????????????????????????????? ^  
> /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/include_ucontext.h:78:44: note: expanded from macro '_FP_IN_UCONTEXT'  
> # define _FP_IN_UCONTEXT uc_mcontext.gregs[REG_RBP]  
> ???????????????????????????????????????????????????????????????????????????????????? ^  
> /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:927:35: error: use of undeclared identifier 'REG_RSP'  
> ?????????????????????????????????????????????? void *sp = (void *)(uap ? uap->_SP_IN_UCONTEXT : 0);  
> ???????????????????????????????????????????????????????????????????????????????????????????????????????????? ^  
> /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/include_ucontext.h:79:44: note: expanded from macro '_SP_IN_UCONTEXT'  
> # define _SP_IN_UCONTEXT uc_mcontext.gregs[REG_RSP]  
> ???????????????????????????????????????????????????????????????????????????????????? ^  
> /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:992:9: error: use of undeclared identifier 'REG_RAX'  
> ?????????????????????????????????????????????? regs[REG_RAX], regs[REG_RBX], regs[REG_RCX], regs[REG_RDX],  
> ???????????????????????????????????????????????????????? ^  
> /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:992:24: error: use of undeclared identifier 'REG_RBX'  
> ?????????????????????????????????????????????? regs[REG_RAX], regs[REG_RBX], regs[REG_RCX], regs[REG_RDX],  
> ?????????????????????????????????????????????????????????????????????????????????????? ^  
> /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:992:39: error: use of undeclared identifier 'REG_RCX'  
> ?????????????????????????????????????????????? regs[REG_RAX], regs[REG_RBX], regs[REG_RCX], regs[REG_RDX],  
> ???????????????????????????????????????????????????????????????????????????????????????????????????????????????????? ^  
> /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:992:54: error: use of undeclared identifier 'REG_RDX'  
> ?????????????????????????????????????????????? regs[REG_RAX], regs[REG_RBX], regs[REG_RCX], regs[REG_RDX],  
> ?????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????? ^  
> /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:993:9: error: use of undeclared identifier 'REG_RDI'  
> ?????????????????????????????????????????????? regs[REG_RDI], regs[REG_RSI], regs[REG_RBP], regs[REG_RSP],  
> ???????????????????????????????????????????????????????? ^  
> /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:993:24: error: use of undeclared identifier 'REG_RSI'  
> ?????????????????????????????????????????????? regs[REG_RDI], regs[REG_RSI], regs[REG_RBP], regs[REG_RSP],  
> ?????????????????????????????????????????????????????????????????????????????????????? ^  
> /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:993:39: error: use of undeclared identifier 'REG_RBP'  
> ?????????????????????????????????????????????? regs[REG_RDI], regs[REG_RSI], regs[REG_RBP], regs[REG_RSP],  
> ???????????????????????????????????????????????????????????????????????????????????????????????????????????????????? ^  
> /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:993:54: error: use of undeclared identifier 'REG_RSP'  
> ?????????????????????????????????????????????? regs[REG_RDI], regs[REG_RSI], regs[REG_RBP], regs[REG_RSP],  
> ?????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????? ^  
> /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:994:9: error: use of undeclared identifier 'REG_R8'  
> ?????????????????????????????????????????????? regs[REG_R8 ], regs[REG_R9 ], regs[REG_R10], regs[REG_R11],  
> ???????????????????????????????????????????????????????? ^  
>   
>   
>   
>   
> 
> > On Mon, Mar 15, 2021 at 04:02:43AM -0700, Tom Beckmann wrote:  
> >  >  
> >  > The commit that removed it was this one: https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/924a24bb54870a621c5283f23631261d5a792000  
> >  >  
> >  > I saw that you did some cleanup of old event primitives, as documented in the commit message, @eliotmiranda. Was sqUnixEvent.c deleted by accident as part of this?  
> >  >  
> >   
> >  I restored the missing sqUnixEvent.c file, which I assume was mistakenly  
> >  deleted in that earlier commit.  
> >   
> >  Dave
>   
> 


From eliot.miranda at gmail.com  Wed Mar 24 18:54:55 2021
From: eliot.miranda at gmail.com (Eliot Miranda)
Date: Wed, 24 Mar 2021 11:54:55 -0700
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] sqUnixEvent.c missing
 from platforms/unix/vm (#555)
In-Reply-To: <20210322231418.GA2171@shell.msen.com>
References: <20210322015551.GA13754@shell.msen.com>
 <OpenSmalltalk/opensmalltalk-vm/issues/555@github.com>
 <OpenSmalltalk/opensmalltalk-vm/issues/555/799327721@github.com>
 <1616409987-38356102e760c938a2dcb06d23dad2a9@pckswarms.ch>
 <20210322231418.GA2171@shell.msen.com>
Message-ID: <CAC20JE0k53LZC0hRzyDh56Lkc2DH-EVaRoU+Ja9oEaCQPxyL6A@mail.gmail.com>

Hi David,

On Mon, Mar 22, 2021 at 4:14 PM David T. Lewis <lewis at mail.msen.com> wrote:

>
> On Mon, Mar 22, 2021 at 11:46:27AM +0100, Bruce O'Neel wrote:
> >
> >
> > Hi,
> >
> > That fixed one problem I was tripping over on the ARMv8 build.
> >
> > There still seems to be a problem with the x86-64 build.????
> >
> > Thanks.
> >
> > bruce
>
> I see the same issue on my Linux x86-64 PC. I cannot spot the exact cause
> of
> the problem, but it seems to have been introduced in this commit:
>
> commit 6d74c4b652c7780110fe327f97e98aaef5242fbc
> Author: Eliot Miranda <eliot.miranda at gmail.com>
> Date:   Wed Feb 10 21:41:31 2021 -0800
>
>     Revisions to core include files to get EXPORT defined correctly at the
> relevant points.
>     Essentially move the default EXPORT definitions from sq.h to
> sqMemoryAccess.h.
>     Include fbdev support in build.linux32ARMv6 builds and add -m32 to
> their CFLAGS
>     to attempt to get the 32-bit VMs to be compilable on 64-bit ARM. [ci
> skip]
>     cuz new cogit files will arrive soon.
>
>
> The changes seem straightforward, but for some reason the header files must
> not be getting included properly.
>
> Dave
>
>
>
>
> >
> > clang -Wall?? -g -O2 -DNDEBUG -DDEBUGVM=0 -msse2 -DCOGMTVM=0 -pthread
> -DLSB_FIRST=1 -m64 -Wno-missing-braces -Wno-unknown-pragmas
> -Wno-unused-value -Wno-unu
> > sed-label -Wno-unused-function -Wno-unused-variable?? -DHAVE_CONFIG_H??
> -DSQUEAK_BUILTIN_PLUGIN
> -I/home/edoneel/tmp/opensmalltalk-vm/build.linux64x64/squeak.
> > [cog.spur/build](http://cog.spur/build)
> -I/home/edoneel/tmp/opensmalltalk-vm/build.linux64x64/[squeak.cog.spur/build](
> http://squeak.cog.spur/build)
> -I/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm -I/home/e
> > doneel/tmp/opensmalltalk-vm/platforms/Cross/vm
> -I/home/edoneel/tmp/opensmalltalk-vm/spur64src/vm -I/usr/local/include
> -I/home/edoneel/tmp/opensmalltalk-vm/
> > platforms/Cross/vm
> -I/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm
> -I/home/edoneel/tmp/opensmalltalk-vm/spur64src/vm
> -I/home/edoneel/tmp/opensmallta
> > lk-vm/platforms/Cross/plugins/FilePlugin
> -I/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/plugins/B3DAcceleratorPlugin
> -m64 -Wno-missing-braces -Wno-unk
> > nown-pragmas -Wno-unused-value -Wno-unused-label -Wno-unused-function
> -Wno-unused-variable???? -c -o sqUnixMain.o
> /home/edoneel/tmp/opensmalltalk-vm/platform
> > s/unix/vm/sqUnixMain.c
> > /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:872:3:
> warning: implicit declaration of function 'pushOutputFile' is invalid in
> C99 [-Wim
> > plicit-function-declaration]
> > ?? pushOutputFile((char *)STDERR_FILENO);
> > ?? ^
> >
> /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:926:35:
> error: use of undeclared identifier 'REG_RBP'
>

If you have a look at the code here the file is trying to
- find out what platform it is on
- pull in the relevant version of ucontext.h containing the relevant
defines for extracting register values from an interrupt context
- print out the registers for a crash report
So please look to
- find out if platforms/unix/vm/include_ucontext.h is working correctly on
this platform
- find out what are the platform defines (-E -dM is your friend here)
- find out how ucontext.h defines the registers in a ucontext

> ?????????????????????????????????????????????? void *fp = (void *)(uap ?
> uap->_FP_IN_UCONTEXT : 0);
> >
> ????????????????????????????????????????????????????????????????????????????????????????????????????????????
> ^
> >
> /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/include_ucontext.h:78:44:
> note: expanded from macro '_FP_IN_UCONTEXT'
> > # define _FP_IN_UCONTEXT uc_mcontext.gregs[REG_RBP]
> >
> ????????????????????????????????????????????????????????????????????????????????????
> ^
> >
> /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:927:35:
> error: use of undeclared identifier 'REG_RSP'
> > ?????????????????????????????????????????????? void *sp = (void *)(uap ?
> uap->_SP_IN_UCONTEXT : 0);
> >
> ????????????????????????????????????????????????????????????????????????????????????????????????????????????
> ^
> >
> /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/include_ucontext.h:79:44:
> note: expanded from macro '_SP_IN_UCONTEXT'
> > # define _SP_IN_UCONTEXT uc_mcontext.gregs[REG_RSP]
> >
> ????????????????????????????????????????????????????????????????????????????????????
> ^
> > /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:992:9:
> error: use of undeclared identifier 'REG_RAX'
> > ?????????????????????????????????????????????? regs[REG_RAX],
> regs[REG_RBX], regs[REG_RCX], regs[REG_RDX],
> > ???????????????????????????????????????????????????????? ^
> >
> /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:992:24:
> error: use of undeclared identifier 'REG_RBX'
> > ?????????????????????????????????????????????? regs[REG_RAX],
> regs[REG_RBX], regs[REG_RCX], regs[REG_RDX],
> >
> ??????????????????????????????????????????????????????????????????????????????????????
> ^
> >
> /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:992:39:
> error: use of undeclared identifier 'REG_RCX'
> > ?????????????????????????????????????????????? regs[REG_RAX],
> regs[REG_RBX], regs[REG_RCX], regs[REG_RDX],
> >
> ????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
> ^
> >
> /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:992:54:
> error: use of undeclared identifier 'REG_RDX'
> > ?????????????????????????????????????????????? regs[REG_RAX],
> regs[REG_RBX], regs[REG_RCX], regs[REG_RDX],
> >
> ??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
> ^
> > /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:993:9:
> error: use of undeclared identifier 'REG_RDI'
> > ?????????????????????????????????????????????? regs[REG_RDI],
> regs[REG_RSI], regs[REG_RBP], regs[REG_RSP],
> > ???????????????????????????????????????????????????????? ^
> >
> /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:993:24:
> error: use of undeclared identifier 'REG_RSI'
> > ?????????????????????????????????????????????? regs[REG_RDI],
> regs[REG_RSI], regs[REG_RBP], regs[REG_RSP],
> >
> ??????????????????????????????????????????????????????????????????????????????????????
> ^
> >
> /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:993:39:
> error: use of undeclared identifier 'REG_RBP'
> > ?????????????????????????????????????????????? regs[REG_RDI],
> regs[REG_RSI], regs[REG_RBP], regs[REG_RSP],
> >
> ????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
> ^
> >
> /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:993:54:
> error: use of undeclared identifier 'REG_RSP'
> > ?????????????????????????????????????????????? regs[REG_RDI],
> regs[REG_RSI], regs[REG_RBP], regs[REG_RSP],
> >
> ??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
> ^
> > /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:994:9:
> error: use of undeclared identifier 'REG_R8'
> > ?????????????????????????????????????????????? regs[REG_R8 ],
> regs[REG_R9 ], regs[REG_R10], regs[REG_R11],
> > ???????????????????????????????????????????????????????? ^
> >
> >
> >
> >
> >
> > > On Mon, Mar 15, 2021 at 04:02:43AM -0700, Tom Beckmann wrote:
> > >  >
> > >  > The commit that removed it was this one:
> https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/924a24bb54870a621c5283f23631261d5a792000
>
> > >  >
> > >  > I saw that you did some cleanup of old event primitives, as
> documented in the commit message, @eliotmiranda. Was sqUnixEvent.c deleted
> by accident as part of this?
> > >  >
> > >
> > >  I restored the missing sqUnixEvent.c file, which I assume was
> mistakenly
> > >  deleted in that earlier commit.
> > >
> > >  Dave
> >
> >
>
>

-- 
_,,,^..^,,,_
best, Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210324/0229e571/attachment-0001.html>

From noreply at github.com  Wed Mar 24 18:57:47 2021
From: noreply at github.com (Eliot Miranda)
Date: Wed, 24 Mar 2021 11:57:47 -0700
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] 4b5f05: Improve the
 comment in platforms/unix/vm/include_u...
Message-ID: <OpenSmalltalk/opensmalltalk-vm/push/refs/heads/Cog/d49424-4f6b75@github.com>

  Branch: refs/heads/Cog
  Home:   https://github.com/OpenSmalltalk/opensmalltalk-vm
  Commit: 4b5f053169427eb4e17587d5fa2cbffdddbcf292
      https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/4b5f053169427eb4e17587d5fa2cbffdddbcf292
  Author: Eliot Miranda <eliot.miranda at gmail.com>
  Date:   2021-03-24 (Wed, 24 Mar 2021)

  Changed paths:
    M platforms/unix/vm/include_ucontext.h

  Log Message:
  -----------
  Improve the comment in platforms/unix/vm/include_ucontext.h [ci skip]


  Commit: 4f6b75b56b314c20b1842078872bd845d61344a4
      https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/4f6b75b56b314c20b1842078872bd845d61344a4
  Author: Eliot Miranda <eliot.miranda at gmail.com>
  Date:   2021-03-24 (Wed, 24 Mar 2021)

  Changed paths:
    A platforms/unix/vm/sqUnixEvent.c

  Log Message:
  -----------
  Merge branch 'Cog' of https://github.com/OpenSmalltalk/opensmalltalk-vm into Cog


Compare: https://github.com/OpenSmalltalk/opensmalltalk-vm/compare/d494240736f7...4f6b75b56b31

From no-reply at appveyor.com  Wed Mar 24 19:16:26 2021
From: no-reply at appveyor.com (AppVeyor)
Date: Wed, 24 Mar 2021 19:16:26 +0000
Subject: [Vm-dev] Build failed: opensmalltalk-vm 1.0.2418
Message-ID: <20210324191626.1.A674708C275BA5F6@appveyor.com>

An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210324/76d76f0b/attachment.html>

From lewis at mail.msen.com  Wed Mar 24 19:58:11 2021
From: lewis at mail.msen.com (David T. Lewis)
Date: Wed, 24 Mar 2021 15:58:11 -0400
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] sqUnixEvent.c missing
	from platforms/unix/vm (#555)
In-Reply-To: <CAC20JE0k53LZC0hRzyDh56Lkc2DH-EVaRoU+Ja9oEaCQPxyL6A@mail.gmail.com>
References: <20210322015551.GA13754@shell.msen.com>
 <OpenSmalltalk/opensmalltalk-vm/issues/555@github.com>
 <OpenSmalltalk/opensmalltalk-vm/issues/555/799327721@github.com>
 <1616409987-38356102e760c938a2dcb06d23dad2a9@pckswarms.ch>
 <20210322231418.GA2171@shell.msen.com>
 <CAC20JE0k53LZC0hRzyDh56Lkc2DH-EVaRoU+Ja9oEaCQPxyL6A@mail.gmail.com>
Message-ID: <20210324195811.GA81078@shell.msen.com>

On Wed, Mar 24, 2021 at 11:54:55AM -0700, Eliot Miranda wrote:
>  
> Hi David,
> 
> On Mon, Mar 22, 2021 at 4:14 PM David T. Lewis <lewis at mail.msen.com> wrote:
> 
> >
> > On Mon, Mar 22, 2021 at 11:46:27AM +0100, Bruce O'Neel wrote:
> > >
> > >
> > > Hi,
> > >
> > > That fixed one problem I was tripping over on the ARMv8 build.
> > >
> > > There still seems to be a problem with the x86-64 build.????
> > >
> > > Thanks.
> > >
> > > bruce
> >
> > I see the same issue on my Linux x86-64 PC. I cannot spot the exact cause
> > of
> > the problem, but it seems to have been introduced in this commit:
> >
> > commit 6d74c4b652c7780110fe327f97e98aaef5242fbc
> > Author: Eliot Miranda <eliot.miranda at gmail.com>
> > Date:   Wed Feb 10 21:41:31 2021 -0800
> >
> >     Revisions to core include files to get EXPORT defined correctly at the
> > relevant points.
> >     Essentially move the default EXPORT definitions from sq.h to
> > sqMemoryAccess.h.
> >     Include fbdev support in build.linux32ARMv6 builds and add -m32 to
> > their CFLAGS
> >     to attempt to get the 32-bit VMs to be compilable on 64-bit ARM. [ci
> > skip]
> >     cuz new cogit files will arrive soon.
> >
> >
> > The changes seem straightforward, but for some reason the header files must
> > not be getting included properly.
> >
> > Dave
> >
> > >
> > /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:926:35:
> > error: use of undeclared identifier 'REG_RBP'
> >
> 
> If you have a look at the code here the file is trying to
> - find out what platform it is on
> - pull in the relevant version of ucontext.h containing the relevant
> defines for extracting register values from an interrupt context
> - print out the registers for a crash report
> So please look to
> - find out if platforms/unix/vm/include_ucontext.h is working correctly on
> this platform
> - find out what are the platform defines (-E -dM is your friend here)
> - find out how ucontext.h defines the registers in a ucontext
> 

Hi Eliot,

I think that platforms/unix/vm/include_ucontext.h is working correctly.
Using -E -dM, I can confirm:

  #define __linux__ 1
  #define __x86_64 1

Also, include_ucontext.h was not modified in commit 924a24bb54870a621c5283f23631261d5a792000
which is where I think the issue originates(?).

So there is something else going wrong with the includes for Linux, but
I am blind and cannot spot it.

Dave


From bruce.oneel at pckswarms.ch  Thu Mar 25 08:43:40 2021
From: bruce.oneel at pckswarms.ch (Bruce O'Neel)
Date: Thu, 25 Mar 2021 09:43:40 +0100
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] sqUnixEvent.c missing
 from platforms/unix/vm (#555)
In-Reply-To: <20210324195811.GA81078@shell.msen.com>
References: <20210324195811.GA81078@shell.msen.com>
 <20210322015551.GA13754@shell.msen.com>
 <OpenSmalltalk/opensmalltalk-vm/issues/555@github.com>
 <OpenSmalltalk/opensmalltalk-vm/issues/555/799327721@github.com>
 <1616409987-38356102e760c938a2dcb06d23dad2a9@pckswarms.ch>
 <20210322231418.GA2171@shell.msen.com>
 <CAC20JE0k53LZC0hRzyDh56Lkc2DH-EVaRoU+Ja9oEaCQPxyL6A@mail.gmail.com>
Message-ID: <1616661820-4e55f9814ec607b58360daea15516cde@pckswarms.ch>


Hi,  
  
Yea that's right, but, I think that is not where the problem is.  
  
What seems to have happened is that someone got clever with  which is included in the beginning of   and which must have changed between different versions of Linux.  So I'm on what is basically Ubuntu 20.4 and they have played around what __USE_GNU means.  Now it seams that the magic #define is _GNU_SOURCE.  
  
The easiest patch I found is below and attached above.  I also attached features.h just so you can see the nightmare that has become.  
  
diff --git a/build.linux64x64/[squeak.cog.spur/build/mvm](http://squeak.cog.spur/build/mvm) b/build.linux64x64/[squeak.cog.spur/build/mvm](http://squeak.cog.spur/build/mvm)  
index 29b710460..eb67f677e 100755  
--- a/build.linux64x64/[squeak.cog.spur/build/mvm](http://squeak.cog.spur/build/mvm)  
+++ b/build.linux64x64/[squeak.cog.spur/build/mvm](http://squeak.cog.spur/build/mvm)  
@@ -34,7 +34,7 @@ test -f config.h || ../../../platforms/unix/config/configure --without-npsqueak  
--with-src=spur64src \  
TARGET_ARCH="-m64" \  
CC=clang \  
-	CFLAGS="$CFLAGS" \  
+	CFLAGS="$CFLAGS -D_GNU_SOURCE=1" \  
LIBS="$LIBS" \  
LDFLAGS="$LDFLAGS"  
rm -f vm/sqUnixMain.o # nuke version info  
  
  
I tried to play the same ifdef games currently with __USE_GNU in include_ucontext.h but I was not successful.  
  
This builds both with clang and with gcc on Ubuntu 20.4 on x86_64.  
  
cheers  
  
bruce  
  

> On Wed, Mar 24, 2021 at 11:54:55AM -0700, Eliot Miranda wrote:  
>  >  
>  > Hi David,  
>  >  
>  > On Mon, Mar 22, 2021 at 4:14 PM David T. Lewis wrote:  
>  >  
>  > >  
>  > > On Mon, Mar 22, 2021 at 11:46:27AM +0100, Bruce O'Neel wrote:  
>  > > >  
>  > > >  
>  > > > Hi,  
>  > > >  
>  > > > That fixed one problem I was tripping over on the ARMv8 build.  
>  > > >  
>  > > > There still seems to be a problem with the x86-64 build.????  
>  > > >  
>  > > > Thanks.  
>  > > >  
>  > > > bruce  
>  > >  
>  > > I see the same issue on my Linux x86-64 PC. I cannot spot the exact cause  
>  > > of  
>  > > the problem, but it seems to have been introduced in this commit:  
>  > >  
>  > > commit 6d74c4b652c7780110fe327f97e98aaef5242fbc  
>  > > Author: Eliot Miranda  
>  > > Date: Wed Feb 10 21:41:31 2021 -0800  
>  > >  
>  > > Revisions to core include files to get EXPORT defined correctly at the  
>  > > relevant points.  
>  > > Essentially move the default EXPORT definitions from sq.h to  
>  > > sqMemoryAccess.h.  
>  > > Include fbdev support in build.linux32ARMv6 builds and add -m32 to  
>  > > their CFLAGS  
>  > > to attempt to get the 32-bit VMs to be compilable on 64-bit ARM. [ci  
>  > > skip]  
>  > > cuz new cogit files will arrive soon.  
>  > >  
>  > >  
>  > > The changes seem straightforward, but for some reason the header files must  
>  > > not be getting included properly.  
>  > >  
>  > > Dave  
>  > >  
>  > > >  
>  > > /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:926:35:  
>  > > error: use of undeclared identifier 'REG_RBP'  
>  > >  
>  >  
>  > If you have a look at the code here the file is trying to  
>  > - find out what platform it is on  
>  > - pull in the relevant version of ucontext.h containing the relevant  
>  > defines for extracting register values from an interrupt context  
>  > - print out the registers for a crash report  
>  > So please look to  
>  > - find out if platforms/unix/vm/include_ucontext.h is working correctly on  
>  > this platform  
>  > - find out what are the platform defines (-E -dM is your friend here)  
>  > - find out how ucontext.h defines the registers in a ucontext  
>  >  
>   
>  Hi Eliot,  
>   
>  I think that platforms/unix/vm/include_ucontext.h is working correctly.  
>  Using -E -dM, I can confirm:  
>   
>  #define __linux__ 1  
>  #define __x86_64 1  
>   
>  Also, include_ucontext.h was not modified in commit 924a24bb54870a621c5283f23631261d5a792000  
>  which is where I think the issue originates(?).  
>   
>  So there is something else going wrong with the includes for Linux, but  
>  I am blind and cannot spot it.  
>   
>  Dave
  

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210325/2f188818/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: diff
Type: text/x-diff
Size: 532 bytes
Desc: not available
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210325/2f188818/attachment-0001.diff>
-------------- next part --------------
/* Copyright (C) 1991-2020 Free Software Foundation, Inc.
   This file is part of the GNU C Library.

   The GNU C Library is free software; you can redistribute it and/or
   modify it under the terms of the GNU Lesser General Public
   License as published by the Free Software Foundation; either
   version 2.1 of the License, or (at your option) any later version.

   The GNU C Library is distributed in the hope that it will be useful,
   but WITHOUT ANY WARRANTY; without even the implied warranty of
   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
   Lesser General Public License for more details.

   You should have received a copy of the GNU Lesser General Public
   License along with the GNU C Library; if not, see
   <https://www.gnu.org/licenses/>.  */

#ifndef	_FEATURES_H
#define	_FEATURES_H	1

/* These are defined by the user (or the compiler)
   to specify the desired environment:

   __STRICT_ANSI__	ISO Standard C.
   _ISOC99_SOURCE	Extensions to ISO C89 from ISO C99.
   _ISOC11_SOURCE	Extensions to ISO C99 from ISO C11.
   _ISOC2X_SOURCE	Extensions to ISO C99 from ISO C2X.
   __STDC_WANT_LIB_EXT2__
			Extensions to ISO C99 from TR 27431-2:2010.
   __STDC_WANT_IEC_60559_BFP_EXT__
			Extensions to ISO C11 from TS 18661-1:2014.
   __STDC_WANT_IEC_60559_FUNCS_EXT__
			Extensions to ISO C11 from TS 18661-4:2015.
   __STDC_WANT_IEC_60559_TYPES_EXT__
			Extensions to ISO C11 from TS 18661-3:2015.

   _POSIX_SOURCE	IEEE Std 1003.1.
   _POSIX_C_SOURCE	If ==1, like _POSIX_SOURCE; if >=2 add IEEE Std 1003.2;
			if >=199309L, add IEEE Std 1003.1b-1993;
			if >=199506L, add IEEE Std 1003.1c-1995;
			if >=200112L, all of IEEE 1003.1-2004
			if >=200809L, all of IEEE 1003.1-2008
   _XOPEN_SOURCE	Includes POSIX and XPG things.  Set to 500 if
			Single Unix conformance is wanted, to 600 for the
			sixth revision, to 700 for the seventh revision.
   _XOPEN_SOURCE_EXTENDED XPG things and X/Open Unix extensions.
   _LARGEFILE_SOURCE	Some more functions for correct standard I/O.
   _LARGEFILE64_SOURCE	Additional functionality from LFS for large files.
   _FILE_OFFSET_BITS=N	Select default filesystem interface.
   _ATFILE_SOURCE	Additional *at interfaces.
   _GNU_SOURCE		All of the above, plus GNU extensions.
   _DEFAULT_SOURCE	The default set of features (taking precedence over
			__STRICT_ANSI__).

   _FORTIFY_SOURCE	Add security hardening to many library functions.
			Set to 1 or 2; 2 performs stricter checks than 1.

   _REENTRANT, _THREAD_SAFE
			Obsolete; equivalent to _POSIX_C_SOURCE=199506L.

   The `-ansi' switch to the GNU C compiler, and standards conformance
   options such as `-std=c99', define __STRICT_ANSI__.  If none of
   these are defined, or if _DEFAULT_SOURCE is defined, the default is
   to have _POSIX_SOURCE set to one and _POSIX_C_SOURCE set to
   200809L, as well as enabling miscellaneous functions from BSD and
   SVID.  If more than one of these are defined, they accumulate.  For
   example __STRICT_ANSI__, _POSIX_SOURCE and _POSIX_C_SOURCE together
   give you ISO C, 1003.1, and 1003.2, but nothing else.

   These are defined by this file and are used by the
   header files to decide what to declare or define:

   __GLIBC_USE (F)	Define things from feature set F.  This is defined
			to 1 or 0; the subsequent macros are either defined
			or undefined, and those tests should be moved to
			__GLIBC_USE.
   __USE_ISOC11		Define ISO C11 things.
   __USE_ISOC99		Define ISO C99 things.
   __USE_ISOC95		Define ISO C90 AMD1 (C95) things.
   __USE_ISOCXX11	Define ISO C++11 things.
   __USE_POSIX		Define IEEE Std 1003.1 things.
   __USE_POSIX2		Define IEEE Std 1003.2 things.
   __USE_POSIX199309	Define IEEE Std 1003.1, and .1b things.
   __USE_POSIX199506	Define IEEE Std 1003.1, .1b, .1c and .1i things.
   __USE_XOPEN		Define XPG things.
   __USE_XOPEN_EXTENDED	Define X/Open Unix things.
   __USE_UNIX98		Define Single Unix V2 things.
   __USE_XOPEN2K        Define XPG6 things.
   __USE_XOPEN2KXSI     Define XPG6 XSI things.
   __USE_XOPEN2K8       Define XPG7 things.
   __USE_XOPEN2K8XSI    Define XPG7 XSI things.
   __USE_LARGEFILE	Define correct standard I/O things.
   __USE_LARGEFILE64	Define LFS things with separate names.
   __USE_FILE_OFFSET64	Define 64bit interface as default.
   __USE_MISC		Define things from 4.3BSD or System V Unix.
   __USE_ATFILE		Define *at interfaces and AT_* constants for them.
   __USE_GNU		Define GNU extensions.
   __USE_FORTIFY_LEVEL	Additional security measures used, according to level.

   The macros `__GNU_LIBRARY__', `__GLIBC__', and `__GLIBC_MINOR__' are
   defined by this file unconditionally.  `__GNU_LIBRARY__' is provided
   only for compatibility.  All new code should use the other symbols
   to test for features.

   All macros listed above as possibly being defined by this file are
   explicitly undefined if they are not explicitly defined.
   Feature-test macros that are not defined by the user or compiler
   but are implied by the other feature-test macros defined (or by the
   lack of any definitions) are defined by the file.

   ISO C feature test macros depend on the definition of the macro
   when an affected header is included, not when the first system
   header is included, and so they are handled in
   <bits/libc-header-start.h>, which does not have a multiple include
   guard.  Feature test macros that can be handled from the first
   system header included are handled here.  */


/* Undefine everything, so we get a clean slate.  */
#undef	__USE_ISOC11
#undef	__USE_ISOC99
#undef	__USE_ISOC95
#undef	__USE_ISOCXX11
#undef	__USE_POSIX
#undef	__USE_POSIX2
#undef	__USE_POSIX199309
#undef	__USE_POSIX199506
#undef	__USE_XOPEN
#undef	__USE_XOPEN_EXTENDED
#undef	__USE_UNIX98
#undef	__USE_XOPEN2K
#undef	__USE_XOPEN2KXSI
#undef	__USE_XOPEN2K8
#undef	__USE_XOPEN2K8XSI
#undef	__USE_LARGEFILE
#undef	__USE_LARGEFILE64
#undef	__USE_FILE_OFFSET64
#undef	__USE_MISC
#undef	__USE_ATFILE
#undef	__USE_GNU
#undef	__USE_FORTIFY_LEVEL
#undef	__KERNEL_STRICT_NAMES
#undef	__GLIBC_USE_ISOC2X
#undef	__GLIBC_USE_DEPRECATED_GETS
#undef	__GLIBC_USE_DEPRECATED_SCANF

/* Suppress kernel-name space pollution unless user expressedly asks
   for it.  */
#ifndef _LOOSE_KERNEL_NAMES
# define __KERNEL_STRICT_NAMES
#endif

/* Convenience macro to test the version of gcc.
   Use like this:
   #if __GNUC_PREREQ (2,8)
   ... code requiring gcc 2.8 or later ...
   #endif
   Note: only works for GCC 2.0 and later, because __GNUC_MINOR__ was
   added in 2.0.  */
#if defined __GNUC__ && defined __GNUC_MINOR__
# define __GNUC_PREREQ(maj, min) \
	((__GNUC__ << 16) + __GNUC_MINOR__ >= ((maj) << 16) + (min))
#else
# define __GNUC_PREREQ(maj, min) 0
#endif

/* Similarly for clang.  Features added to GCC after version 4.2 may
   or may not also be available in clang, and clang's definitions of
   __GNUC(_MINOR)__ are fixed at 4 and 2 respectively.  Not all such
   features can be queried via __has_extension/__has_feature.  */
#if defined __clang_major__ && defined __clang_minor__
# define __glibc_clang_prereq(maj, min) \
  ((__clang_major__ << 16) + __clang_minor__ >= ((maj) << 16) + (min))
#else
# define __glibc_clang_prereq(maj, min) 0
#endif

/* Whether to use feature set F.  */
#define __GLIBC_USE(F)	__GLIBC_USE_ ## F

/* _BSD_SOURCE and _SVID_SOURCE are deprecated aliases for
   _DEFAULT_SOURCE.  If _DEFAULT_SOURCE is present we do not
   issue a warning; the expectation is that the source is being
   transitioned to use the new macro.  */
#if (defined _BSD_SOURCE || defined _SVID_SOURCE) \
    && !defined _DEFAULT_SOURCE
# warning "_BSD_SOURCE and _SVID_SOURCE are deprecated, use _DEFAULT_SOURCE"
# undef  _DEFAULT_SOURCE
# define _DEFAULT_SOURCE	1
#endif

/* If _GNU_SOURCE was defined by the user, turn on all the other features.  */
#ifdef _GNU_SOURCE
# undef  _ISOC95_SOURCE
# define _ISOC95_SOURCE	1
# undef  _ISOC99_SOURCE
# define _ISOC99_SOURCE	1
# undef  _ISOC11_SOURCE
# define _ISOC11_SOURCE	1
# undef  _ISOC2X_SOURCE
# define _ISOC2X_SOURCE	1
# undef  _POSIX_SOURCE
# define _POSIX_SOURCE	1
# undef  _POSIX_C_SOURCE
# define _POSIX_C_SOURCE	200809L
# undef  _XOPEN_SOURCE
# define _XOPEN_SOURCE	700
# undef  _XOPEN_SOURCE_EXTENDED
# define _XOPEN_SOURCE_EXTENDED	1
# undef	 _LARGEFILE64_SOURCE
# define _LARGEFILE64_SOURCE	1
# undef  _DEFAULT_SOURCE
# define _DEFAULT_SOURCE	1
# undef  _ATFILE_SOURCE
# define _ATFILE_SOURCE	1
#endif

/* If nothing (other than _GNU_SOURCE and _DEFAULT_SOURCE) is defined,
   define _DEFAULT_SOURCE.  */
#if (defined _DEFAULT_SOURCE					\
     || (!defined __STRICT_ANSI__				\
	 && !defined _ISOC99_SOURCE && !defined _ISOC11_SOURCE	\
	 && !defined _ISOC2X_SOURCE				\
	 && !defined _POSIX_SOURCE && !defined _POSIX_C_SOURCE	\
	 && !defined _XOPEN_SOURCE))
# undef  _DEFAULT_SOURCE
# define _DEFAULT_SOURCE	1
#endif

/* This is to enable the ISO C2X extension.  */
#if (defined _ISOC2X_SOURCE \
     || (defined __STDC_VERSION__ && __STDC_VERSION__ > 201710L))
# define __GLIBC_USE_ISOC2X	1
#else
# define __GLIBC_USE_ISOC2X	0
#endif

/* This is to enable the ISO C11 extension.  */
#if (defined _ISOC11_SOURCE || defined _ISOC2X_SOURCE \
     || (defined __STDC_VERSION__ && __STDC_VERSION__ >= 201112L))
# define __USE_ISOC11	1
#endif

/* This is to enable the ISO C99 extension.  */
#if (defined _ISOC99_SOURCE || defined _ISOC11_SOURCE			\
     || defined _ISOC2X_SOURCE						\
     || (defined __STDC_VERSION__ && __STDC_VERSION__ >= 199901L))
# define __USE_ISOC99	1
#endif

/* This is to enable the ISO C90 Amendment 1:1995 extension.  */
#if (defined _ISOC99_SOURCE || defined _ISOC11_SOURCE			\
     || defined _ISOC2X_SOURCE						\
     || (defined __STDC_VERSION__ && __STDC_VERSION__ >= 199409L))
# define __USE_ISOC95	1
#endif

#ifdef __cplusplus
/* This is to enable compatibility for ISO C++17.  */
# if __cplusplus >= 201703L
#  define __USE_ISOC11	1
# endif
/* This is to enable compatibility for ISO C++11.
   Check the temporary macro for now, too.  */
# if __cplusplus >= 201103L || defined __GXX_EXPERIMENTAL_CXX0X__
#  define __USE_ISOCXX11	1
#  define __USE_ISOC99	1
# endif
#endif

/* If none of the ANSI/POSIX macros are defined, or if _DEFAULT_SOURCE
   is defined, use POSIX.1-2008 (or another version depending on
   _XOPEN_SOURCE).  */
#ifdef _DEFAULT_SOURCE
# if !defined _POSIX_SOURCE && !defined _POSIX_C_SOURCE
#  define __USE_POSIX_IMPLICITLY	1
# endif
# undef  _POSIX_SOURCE
# define _POSIX_SOURCE	1
# undef  _POSIX_C_SOURCE
# define _POSIX_C_SOURCE	200809L
#endif

#if ((!defined __STRICT_ANSI__					\
      || (defined _XOPEN_SOURCE && (_XOPEN_SOURCE - 0) >= 500))	\
     && !defined _POSIX_SOURCE && !defined _POSIX_C_SOURCE)
# define _POSIX_SOURCE	1
# if defined _XOPEN_SOURCE && (_XOPEN_SOURCE - 0) < 500
#  define _POSIX_C_SOURCE	2
# elif defined _XOPEN_SOURCE && (_XOPEN_SOURCE - 0) < 600
#  define _POSIX_C_SOURCE	199506L
# elif defined _XOPEN_SOURCE && (_XOPEN_SOURCE - 0) < 700
#  define _POSIX_C_SOURCE	200112L
# else
#  define _POSIX_C_SOURCE	200809L
# endif
# define __USE_POSIX_IMPLICITLY	1
#endif

/* Some C libraries once required _REENTRANT and/or _THREAD_SAFE to be
   defined in all multithreaded code.  GNU libc has not required this
   for many years.  We now treat them as compatibility synonyms for
   _POSIX_C_SOURCE=199506L, which is the earliest level of POSIX with
   comprehensive support for multithreaded code.  Using them never
   lowers the selected level of POSIX conformance, only raises it.  */
#if ((!defined _POSIX_C_SOURCE || (_POSIX_C_SOURCE - 0) < 199506L) \
     && (defined _REENTRANT || defined _THREAD_SAFE))
# define _POSIX_SOURCE   1
# undef  _POSIX_C_SOURCE
# define _POSIX_C_SOURCE 199506L
#endif

#if (defined _POSIX_SOURCE					\
     || (defined _POSIX_C_SOURCE && _POSIX_C_SOURCE >= 1)	\
     || defined _XOPEN_SOURCE)
# define __USE_POSIX	1
#endif

#if defined _POSIX_C_SOURCE && _POSIX_C_SOURCE >= 2 || defined _XOPEN_SOURCE
# define __USE_POSIX2	1
#endif

#if defined _POSIX_C_SOURCE && (_POSIX_C_SOURCE - 0) >= 199309L
# define __USE_POSIX199309	1
#endif

#if defined _POSIX_C_SOURCE && (_POSIX_C_SOURCE - 0) >= 199506L
# define __USE_POSIX199506	1
#endif

#if defined _POSIX_C_SOURCE && (_POSIX_C_SOURCE - 0) >= 200112L
# define __USE_XOPEN2K		1
# undef __USE_ISOC95
# define __USE_ISOC95		1
# undef __USE_ISOC99
# define __USE_ISOC99		1
#endif

#if defined _POSIX_C_SOURCE && (_POSIX_C_SOURCE - 0) >= 200809L
# define __USE_XOPEN2K8		1
# undef  _ATFILE_SOURCE
# define _ATFILE_SOURCE	1
#endif

#ifdef	_XOPEN_SOURCE
# define __USE_XOPEN	1
# if (_XOPEN_SOURCE - 0) >= 500
#  define __USE_XOPEN_EXTENDED	1
#  define __USE_UNIX98	1
#  undef _LARGEFILE_SOURCE
#  define _LARGEFILE_SOURCE	1
#  if (_XOPEN_SOURCE - 0) >= 600
#   if (_XOPEN_SOURCE - 0) >= 700
#    define __USE_XOPEN2K8	1
#    define __USE_XOPEN2K8XSI	1
#   endif
#   define __USE_XOPEN2K	1
#   define __USE_XOPEN2KXSI	1
#   undef __USE_ISOC95
#   define __USE_ISOC95		1
#   undef __USE_ISOC99
#   define __USE_ISOC99		1
#  endif
# else
#  ifdef _XOPEN_SOURCE_EXTENDED
#   define __USE_XOPEN_EXTENDED	1
#  endif
# endif
#endif

#ifdef _LARGEFILE_SOURCE
# define __USE_LARGEFILE	1
#endif

#ifdef _LARGEFILE64_SOURCE
# define __USE_LARGEFILE64	1
#endif

#if defined _FILE_OFFSET_BITS && _FILE_OFFSET_BITS == 64
# define __USE_FILE_OFFSET64	1
#endif

#if defined _DEFAULT_SOURCE
# define __USE_MISC	1
#endif

#ifdef	_ATFILE_SOURCE
# define __USE_ATFILE	1
#endif

#ifdef	_GNU_SOURCE
# define __USE_GNU	1
#endif

#if defined _FORTIFY_SOURCE && _FORTIFY_SOURCE > 0 \
    && __GNUC_PREREQ (4, 1) && defined __OPTIMIZE__ && __OPTIMIZE__ > 0
# if _FORTIFY_SOURCE > 1
#  define __USE_FORTIFY_LEVEL 2
# else
#  define __USE_FORTIFY_LEVEL 1
# endif
#else
# define __USE_FORTIFY_LEVEL 0
#endif

/* The function 'gets' existed in C89, but is impossible to use
   safely.  It has been removed from ISO C11 and ISO C++14.  Note: for
   compatibility with various implementations of <cstdio>, this test
   must consider only the value of __cplusplus when compiling C++.  */
#if defined __cplusplus ? __cplusplus >= 201402L : defined __USE_ISOC11
# define __GLIBC_USE_DEPRECATED_GETS 0
#else
# define __GLIBC_USE_DEPRECATED_GETS 1
#endif

/* GNU formerly extended the scanf functions with modified format
   specifiers %as, %aS, and %a[...] that allocate a buffer for the
   input using malloc.  This extension conflicts with ISO C99, which
   defines %a as a standalone format specifier that reads a floating-
   point number; moreover, POSIX.1-2008 provides the same feature
   using the modifier letter 'm' instead (%ms, %mS, %m[...]).

   We now follow C99 unless GNU extensions are active and the compiler
   is specifically in C89 or C++98 mode (strict or not).  For
   instance, with GCC, -std=gnu11 will have C99-compliant scanf with
   or without -D_GNU_SOURCE, but -std=c89 -D_GNU_SOURCE will have the
   old extension.  */
#if (defined __USE_GNU							\
     && (defined __cplusplus						\
	 ? (__cplusplus < 201103L && !defined __GXX_EXPERIMENTAL_CXX0X__) \
	 : (!defined __STDC_VERSION__ || __STDC_VERSION__ < 199901L)))
# define __GLIBC_USE_DEPRECATED_SCANF 1
#else
# define __GLIBC_USE_DEPRECATED_SCANF 0
#endif

/* Get definitions of __STDC_* predefined macros, if the compiler has
   not preincluded this header automatically.  */
#include <stdc-predef.h>

/* This macro indicates that the installed library is the GNU C Library.
   For historic reasons the value now is 6 and this will stay from now
   on.  The use of this variable is deprecated.  Use __GLIBC__ and
   __GLIBC_MINOR__ now (see below) when you want to test for a specific
   GNU C library version and use the values in <gnu/lib-names.h> to get
   the sonames of the shared libraries.  */
#undef  __GNU_LIBRARY__
#define __GNU_LIBRARY__ 6

/* Major and minor version number of the GNU C library package.  Use
   these macros to test for features in specific releases.  */
#define	__GLIBC__	2
#define	__GLIBC_MINOR__	31

#define __GLIBC_PREREQ(maj, min) \
	((__GLIBC__ << 16) + __GLIBC_MINOR__ >= ((maj) << 16) + (min))

/* This is here only because every header file already includes this one.  */
#ifndef __ASSEMBLER__
# ifndef _SYS_CDEFS_H
#  include <sys/cdefs.h>
# endif

/* If we don't have __REDIRECT, prototypes will be missing if
   __USE_FILE_OFFSET64 but not __USE_LARGEFILE[64]. */
# if defined __USE_FILE_OFFSET64 && !defined __REDIRECT
#  define __USE_LARGEFILE	1
#  define __USE_LARGEFILE64	1
# endif

#endif	/* !ASSEMBLER */

/* Decide whether we can define 'extern inline' functions in headers.  */
#if __GNUC_PREREQ (2, 7) && defined __OPTIMIZE__ \
    && !defined __OPTIMIZE_SIZE__ && !defined __NO_INLINE__ \
    && defined __extern_inline
# define __USE_EXTERN_INLINES	1
#endif


/* This is here only because every header file already includes this one.
   Get the definitions of all the appropriate `__stub_FUNCTION' symbols.
   <gnu/stubs.h> contains `#define __stub_FUNCTION' when FUNCTION is a stub
   that will always return failure (and set errno to ENOSYS).  */
#include <gnu/stubs.h>


#endif	/* features.h  */

From Das.Linux at gmx.de  Thu Mar 25 09:14:53 2021
From: Das.Linux at gmx.de (Tobias Pape)
Date: Thu, 25 Mar 2021 10:14:53 +0100
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] sqUnixEvent.c missing
 from platforms/unix/vm (#555)
In-Reply-To: <1616661820-4e55f9814ec607b58360daea15516cde@pckswarms.ch>
References: <20210324195811.GA81078@shell.msen.com>
 <20210322015551.GA13754@shell.msen.com>
 <OpenSmalltalk/opensmalltalk-vm/issues/555@github.com>
 <OpenSmalltalk/opensmalltalk-vm/issues/555/799327721@github.com>
 <1616409987-38356102e760c938a2dcb06d23dad2a9@pckswarms.ch>
 <20210322231418.GA2171@shell.msen.com>
 <CAC20JE0k53LZC0hRzyDh56Lkc2DH-EVaRoU+Ja9oEaCQPxyL6A@mail.gmail.com>
 <1616661820-4e55f9814ec607b58360daea15516cde@pckswarms.ch>
Message-ID: <20DE3234-D4D8-4BDC-B0E2-0F7BF35E432F@gmx.de>

Hi Bruce and all

> On 25. Mar 2021, at 09:43, Bruce O'Neel <bruce.oneel at pckswarms.ch> wrote:
> 
> Hi,
> 
> Yea that's right, but, I think that is not where the problem is.
> 
> What seems to have happened is that someone got clever with <features.h> which is included in the beginning of  <sys/ucontext.h> and which must have changed between different versions of Linux.  So I'm on what is basically Ubuntu 20.4 and they have played around what __USE_GNU means.


>  Now it seams that the magic #define is _GNU_SOURCE.

This is so quite a long time already.
I just checked a system frozen in time in 2012, it is the same there.
All __USE_* get undefined in features.h and only set with the respective _*_SOURCE options.

The __USE_GNU dance in
	 https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/4f6b75b56b314c20b1842078872bd845d61344a4/platforms/unix/vm/include_ucontext.h#L15

is hence sadly ineffective.


> 
> The easiest patch I found is below and attached above.  I also attached features.h just so you can see the nightmare that has become.
> 
> diff --git a/build.linux64x64/squeak.cog.spur/build/mvm b/build.linux64x64/squeak.cog.spur/build/mvm
> index 29b710460..eb67f677e 100755
> --- a/build.linux64x64/squeak.cog.spur/build/mvm
> +++ b/build.linux64x64/squeak.cog.spur/build/mvm
> @@ -34,7 +34,7 @@ test -f config.h || ../../../platforms/unix/config/configure --without-npsqueak
> --with-src=spur64src \
> TARGET_ARCH="-m64" \
> CC=clang \
> - CFLAGS="$CFLAGS" \
> + CFLAGS="$CFLAGS -D_GNU_SOURCE=1" \
> LIBS="$LIBS" \
> LDFLAGS="$LDFLAGS"
> rm -f vm/sqUnixMain.o # nuke version info
> 
Configure should define that!

AC_USE_SYSTEM_EXTENSIONS in 
	https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/4f6b75b56b314c20b1842078872bd845d61344a4/platforms/unix/config/configure.ac#L42
does exactly that: if GNU Extensions are avaialbe, define _GNU_SORUCE.

This define should be in config.h

This is the unconfigured file: https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/unix/config/config.h.in#L392

Once you run ./configure, it _will_ define _GNU_SOURCE.

However, it must be sur that config.h is included. 
This is done unconditionally in sqMemoryAccess.h and conditionally (HAVE_CONFIG_H) in sqConfig.h (which is pulled by sq.h)
(HAVE_CONFIG_H is set as I see in your email from 2 days ago)


Based on your command in your email, can yous please send the output of

clang -Wall  -g -O2 -DNDEBUG -DDEBUGVM=0 -msse2 -DCOGMTVM=0 -pthread -DLSB_FIRST=1 -m64 -Wno-missing-braces -Wno-unknown-pragmas -Wno-unused-value -Wno-unused-label -Wno-unused-function -Wno-unused-variable  -DHAVE_CONFIG_H  -DSQUEAK_BUILTIN_PLUGIN -I/home/edoneel/tmp/opensmalltalk-vm/build.linux64x64/squeak.cog.spur/build -I/home/edoneel/tmp/opensmalltalk-vm/build.linux64x64/squeak.cog.spur/build -I/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm -I/home/edoneel/tmp/opensmalltalk-vm/platforms/Cross/vm -I/home/edoneel/tmp/opensmalltalk-vm/spur64src/vm -I/usr/local/include -I/home/edoneel/tmp/opensmalltalk-vm/platforms/Cross/vm -I/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm -I/home/edoneel/tmp/opensmalltalk-vm/spur64src/vm -I/home/edoneel/tmp/opensmalltalk-vm/platforms/Cross/plugins/FilePlugin -I/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/plugins/B3DAcceleratorPlugin -m64 -Wno-missing-braces -Wno-unknown-pragmas -Wno-unused-value -Wno-unused-label -Wno-unused-function -Wno-unused-variable   -E -o sqUnixMain.i /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c

and the resulting sqUnixMain.i?


Best regards
	-Tobias


> I tried to play the same ifdef games currently with __USE_GNU in include_ucontext.h but I was not successful.
> 
> This builds both with clang and with gcc on Ubuntu 20.4 on x86_64.
> 
> cheers
> 
> bruce
> 
> 24 March 2021 20:58 "David T. Lewis" <lewis at mail.msen.com> wrote:
> 
> On Wed, Mar 24, 2021 at 11:54:55AM -0700, Eliot Miranda wrote:
> >
> > Hi David,
> >
> > On Mon, Mar 22, 2021 at 4:14 PM David T. Lewis wrote:
> >
> > >
> > > On Mon, Mar 22, 2021 at 11:46:27AM +0100, Bruce O'Neel wrote:
> > > >
> > > >
> > > > Hi,
> > > >
> > > > That fixed one problem I was tripping over on the ARMv8 build.
> > > >
> > > > There still seems to be a problem with the x86-64 build.????
> > > >
> > > > Thanks.
> > > >
> > > > bruce
> > >
> > > I see the same issue on my Linux x86-64 PC. I cannot spot the exact cause
> > > of
> > > the problem, but it seems to have been introduced in this commit:
> > >
> > > commit 6d74c4b652c7780110fe327f97e98aaef5242fbc
> > > Author: Eliot Miranda
> > > Date: Wed Feb 10 21:41:31 2021 -0800
> > >
> > > Revisions to core include files to get EXPORT defined correctly at the
> > > relevant points.
> > > Essentially move the default EXPORT definitions from sq.h to
> > > sqMemoryAccess.h.
> > > Include fbdev support in build.linux32ARMv6 builds and add -m32 to
> > > their CFLAGS
> > > to attempt to get the 32-bit VMs to be compilable on 64-bit ARM. [ci
> > > skip]
> > > cuz new cogit files will arrive soon.
> > >
> > >
> > > The changes seem straightforward, but for some reason the header files must
> > > not be getting included properly.
> > >
> > > Dave
> > >
> > > >
> > > /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:926:35:
> > > error: use of undeclared identifier 'REG_RBP'
> > >
> >
> > If you have a look at the code here the file is trying to
> > - find out what platform it is on
> > - pull in the relevant version of ucontext.h containing the relevant
> > defines for extracting register values from an interrupt context
> > - print out the registers for a crash report
> > So please look to
> > - find out if platforms/unix/vm/include_ucontext.h is working correctly on
> > this platform
> > - find out what are the platform defines (-E -dM is your friend here)
> > - find out how ucontext.h defines the registers in a ucontext
> >
> 
> Hi Eliot,
> 
> I think that platforms/unix/vm/include_ucontext.h is working correctly.
> Using -E -dM, I can confirm:
> 
> #define __linux__ 1
> #define __x86_64 1
> 
> Also, include_ucontext.h was not modified in commit 924a24bb54870a621c5283f23631261d5a792000
> which is where I think the issue originates(?).
> 
> So there is something else going wrong with the includes for Linux, but
> I am blind and cannot spot it.
> 
> Dave
> 
> 
> <diff><features.h>



From noreply at github.com  Thu Mar 25 09:29:45 2021
From: noreply at github.com (Tobias Pape)
Date: Thu, 25 Mar 2021 02:29:45 -0700
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] b4afea: Play the
	features.h dance
Message-ID: <OpenSmalltalk/opensmalltalk-vm/push/refs/heads/Cog/4f6b75-b4afea@github.com>

  Branch: refs/heads/Cog
  Home:   https://github.com/OpenSmalltalk/opensmalltalk-vm
  Commit: b4afeac5f51acafd24035e14dc370eb3954c57f1
      https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/b4afeac5f51acafd24035e14dc370eb3954c57f1
  Author: Tobias Pape <tobias at netshed.de>
  Date:   2021-03-25 (Thu, 25 Mar 2021)

  Changed paths:
    M platforms/unix/vm/include_ucontext.h

  Log Message:
  -----------
  Play the features.h dance

Since configure takes care that `_GNU_SOURCE` is there
whenever supported, _not_ having that means we cannot reasonably
ensure that we can access reg names, as defined via `__USE_GNU`.

The `features.h` makes sure that `__USE_GNU` is set if and only if
`_GNU_SOURCE` is set. So Bailing when not having it is the right choice.



From bruce.oneel at pckswarms.ch  Thu Mar 25 09:32:46 2021
From: bruce.oneel at pckswarms.ch (Bruce O'Neel)
Date: Thu, 25 Mar 2021 10:32:46 +0100
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] sqUnixEvent.c missing
 from platforms/unix/vm (#555)
In-Reply-To: <20DE3234-D4D8-4BDC-B0E2-0F7BF35E432F@gmx.de>
References: <20DE3234-D4D8-4BDC-B0E2-0F7BF35E432F@gmx.de>
 <20210324195811.GA81078@shell.msen.com>
 <20210322015551.GA13754@shell.msen.com>
 <OpenSmalltalk/opensmalltalk-vm/issues/555@github.com>
 <OpenSmalltalk/opensmalltalk-vm/issues/555/799327721@github.com>
 <1616409987-38356102e760c938a2dcb06d23dad2a9@pckswarms.ch>
 <20210322231418.GA2171@shell.msen.com>
 <CAC20JE0k53LZC0hRzyDh56Lkc2DH-EVaRoU+Ja9oEaCQPxyL6A@mail.gmail.com>
 <1616661820-4e55f9814ec607b58360daea15516cde@pckswarms.ch>
Message-ID: <1616664766-7a9a05fb15c4caf6a693dd4d0fe96497@pckswarms.ch>


Hi,  
  
I've attached sqUnixMain.i from the command below.  
  
And yes it is in config.h  
  
  
#ifndef _GNU_SOURCE  
# define _GNU_SOURCE 1  
#endif  
  
Thanks.  
  
bruce  
  
  
  
  

> Hi Bruce and all  
>   
>  > On 25. Mar 2021, at 09:43, Bruce O'Neel wrote:  
>  >  
>  > Hi,  
>  >  
>  > Yea that's right, but, I think that is not where the problem is.  
>  >  
>  > What seems to have happened is that someone got clever with which is included in the beginning of and which must have changed between different versions of Linux. So I'm on what is basically Ubuntu 20.4 and they have played around what __USE_GNU means.  
>   
>   
>  > Now it seams that the magic #define is _GNU_SOURCE.  
>   
>  This is so quite a long time already.  
>  I just checked a system frozen in time in 2012, it is the same there.  
>  All __USE_* get undefined in features.h and only set with the respective _*_SOURCE options.  
>   
>  The __USE_GNU dance in  
> [https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/4f6b75b56b314c20b1842078872bd845d61344a4/platforms/unix/vm/include_ucontext.h#L15](https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/4f6b75b56b314c20b1842078872bd845d61344a4/platforms/unix/vm/include_ucontext.h#L15)  
>   
>  is hence sadly ineffective.  
>   
>   
>  >  
>  > The easiest patch I found is below and attached above. I also attached features.h just so you can see the nightmare that has become.  
>  >  
>  > diff --git a/build.linux64x64/[squeak.cog.spur/build/mvm](http://squeak.cog.spur/build/mvm) b/build.linux64x64/[squeak.cog.spur/build/mvm](http://squeak.cog.spur/build/mvm)  
>  > index 29b710460..eb67f677e 100755  
>  > --- a/build.linux64x64/[squeak.cog.spur/build/mvm](http://squeak.cog.spur/build/mvm)  
>  > +++ b/build.linux64x64/[squeak.cog.spur/build/mvm](http://squeak.cog.spur/build/mvm)  
>  > @@ -34,7 +34,7 @@ test -f config.h || ../../../platforms/unix/config/configure --without-npsqueak  
>  > --with-src=spur64src \  
>  > TARGET_ARCH="-m64" \  
>  > CC=clang \  
>  > - CFLAGS="$CFLAGS" \  
>  > + CFLAGS="$CFLAGS -D_GNU_SOURCE=1" \  
>  > LIBS="$LIBS" \  
>  > LDFLAGS="$LDFLAGS"  
>  > rm -f vm/sqUnixMain.o # nuke version info  
>  >  
>  Configure should define that!  
>   
>  AC_USE_SYSTEM_EXTENSIONS in  
> [https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/4f6b75b56b314c20b1842078872bd845d61344a4/platforms/unix/config/configure.ac#L42](https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/4f6b75b56b314c20b1842078872bd845d61344a4/platforms/unix/config/configure.ac#L42)  
>  does exactly that: if GNU Extensions are avaialbe, define _GNU_SORUCE.  
>   
>  This define should be in config.h  
>   
>  This is the unconfigured file: [https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/unix/config/config.h.in#L392](https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/unix/config/config.h.in#L392)  
>   
>  Once you run ./configure, it _will_ define _GNU_SOURCE.  
>   
>  However, it must be sur that config.h is included.  
>  This is done unconditionally in sqMemoryAccess.h and conditionally (HAVE_CONFIG_H) in sqConfig.h (which is pulled by sq.h)  
>  (HAVE_CONFIG_H is set as I see in your email from 2 days ago)  
>   
>   
>  Based on your command in your email, can yous please send the output of  
>   
>  clang -Wall -g -O2 -DNDEBUG -DDEBUGVM=0 -msse2 -DCOGMTVM=0 -pthread -DLSB_FIRST=1 -m64 -Wno-missing-braces -Wno-unknown-pragmas -Wno-unused-value -Wno-unused-label -Wno-unused-function -Wno-unused-variable -DHAVE_CONFIG_H -DSQUEAK_BUILTIN_PLUGIN -I/home/edoneel/tmp/opensmalltalk-vm/build.linux64x64/[squeak.cog.spur/build](http://squeak.cog.spur/build) -I/home/edoneel/tmp/opensmalltalk-vm/build.linux64x64/[squeak.cog.spur/build](http://squeak.cog.spur/build) -I/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm -I/home/edoneel/tmp/opensmalltalk-vm/platforms/Cross/vm -I/home/edoneel/tmp/opensmalltalk-vm/spur64src/vm -I/usr/local/include -I/home/edoneel/tmp/opensmalltalk-vm/platforms/Cross/vm -I/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm -I/home/edoneel/tmp/opensmalltalk-vm/spur64src/vm -I/home/edoneel/tmp/opensmalltalk-vm/platforms/Cross/plugins/FilePlugin -I/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/plugins/B3DAcceleratorPlugin -m64 -Wno-missing-braces -Wno-unknown-pragmas -Wno-unused-value -Wno-unused-label -Wno-unused-function -Wno-unused-variable -E -o sqUnixMain.i /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c  
>   
>  and the resulting sqUnixMain.i?  
>   
>   
>  Best regards  
>  -Tobias  
>   
>   
>  > I tried to play the same ifdef games currently with __USE_GNU in include_ucontext.h but I was not successful.  
>  >  
>  > This builds both with clang and with gcc on Ubuntu 20.4 on x86_64.  
>  >  
>  > cheers  
>  >  
>  > bruce  
>  >  
>  > 24 March 2021 20:58 "David T. Lewis" wrote:  
>  >  
>  > On Wed, Mar 24, 2021 at 11:54:55AM -0700, Eliot Miranda wrote:  
>  > >  
>  > > Hi David,  
>  > >  
>  > > On Mon, Mar 22, 2021 at 4:14 PM David T. Lewis wrote:  
>  > >  
>  > > >  
>  > > > On Mon, Mar 22, 2021 at 11:46:27AM +0100, Bruce O'Neel wrote:  
>  > > > >  
>  > > > >  
>  > > > > Hi,  
>  > > > >  
>  > > > > That fixed one problem I was tripping over on the ARMv8 build.  
>  > > > >  
>  > > > > There still seems to be a problem with the x86-64 build.????  
>  > > > >  
>  > > > > Thanks.  
>  > > > >  
>  > > > > bruce  
>  > > >  
>  > > > I see the same issue on my Linux x86-64 PC. I cannot spot the exact cause  
>  > > > of  
>  > > > the problem, but it seems to have been introduced in this commit:  
>  > > >  
>  > > > commit 6d74c4b652c7780110fe327f97e98aaef5242fbc  
>  > > > Author: Eliot Miranda  
>  > > > Date: Wed Feb 10 21:41:31 2021 -0800  
>  > > >  
>  > > > Revisions to core include files to get EXPORT defined correctly at the  
>  > > > relevant points.  
>  > > > Essentially move the default EXPORT definitions from sq.h to  
>  > > > sqMemoryAccess.h.  
>  > > > Include fbdev support in build.linux32ARMv6 builds and add -m32 to  
>  > > > their CFLAGS  
>  > > > to attempt to get the 32-bit VMs to be compilable on 64-bit ARM. [ci  
>  > > > skip]  
>  > > > cuz new cogit files will arrive soon.  
>  > > >  
>  > > >  
>  > > > The changes seem straightforward, but for some reason the header files must  
>  > > > not be getting included properly.  
>  > > >  
>  > > > Dave  
>  > > >  
>  > > > >  
>  > > > /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:926:35:  
>  > > > error: use of undeclared identifier 'REG_RBP'  
>  > > >  
>  > >  
>  > > If you have a look at the code here the file is trying to  
>  > > - find out what platform it is on  
>  > > - pull in the relevant version of ucontext.h containing the relevant  
>  > > defines for extracting register values from an interrupt context  
>  > > - print out the registers for a crash report  
>  > > So please look to  
>  > > - find out if platforms/unix/vm/include_ucontext.h is working correctly on  
>  > > this platform  
>  > > - find out what are the platform defines (-E -dM is your friend here)  
>  > > - find out how ucontext.h defines the registers in a ucontext  
>  > >  
>  >  
>  > Hi Eliot,  
>  >  
>  > I think that platforms/unix/vm/include_ucontext.h is working correctly.  
>  > Using -E -dM, I can confirm:  
>  >  
>  > #define __linux__ 1  
>  > #define __x86_64 1  
>  >  
>  > Also, include_ucontext.h was not modified in commit 924a24bb54870a621c5283f23631261d5a792000  
>  > which is where I think the issue originates(?).  
>  >  
>  > So there is something else going wrong with the includes for Linux, but  
>  > I am blind and cannot spot it.  
>  >  
>  > Dave  
>  >  
>  >  
>  >
  

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210325/7422d2ad/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sqUnixMain.i
Type: text/x-c
Size: 259298 bytes
Desc: not available
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210325/7422d2ad/attachment-0001.bin>

From no-reply at appveyor.com  Thu Mar 25 09:48:08 2021
From: no-reply at appveyor.com (AppVeyor)
Date: Thu, 25 Mar 2021 09:48:08 +0000
Subject: [Vm-dev] Build failed: opensmalltalk-vm 1.0.2419
Message-ID: <20210325094808.1.59254FB59551559C@appveyor.com>

An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210325/f45ea397/attachment.html>

From Das.Linux at gmx.de  Thu Mar 25 10:04:08 2021
From: Das.Linux at gmx.de (Tobias Pape)
Date: Thu, 25 Mar 2021 11:04:08 +0100
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] sqUnixEvent.c missing
 from platforms/unix/vm (#555)
In-Reply-To: <1616664766-7a9a05fb15c4caf6a693dd4d0fe96497@pckswarms.ch>
References: <20DE3234-D4D8-4BDC-B0E2-0F7BF35E432F@gmx.de>
 <20210324195811.GA81078@shell.msen.com>
 <20210322015551.GA13754@shell.msen.com>
 <OpenSmalltalk/opensmalltalk-vm/issues/555@github.com>
 <OpenSmalltalk/opensmalltalk-vm/issues/555/799327721@github.com>
 <1616409987-38356102e760c938a2dcb06d23dad2a9@pckswarms.ch>
 <20210322231418.GA2171@shell.msen.com>
 <CAC20JE0k53LZC0hRzyDh56Lkc2DH-EVaRoU+Ja9oEaCQPxyL6A@mail.gmail.com>
 <1616661820-4e55f9814ec607b58360daea15516cde@pckswarms.ch>
 <1616664766-7a9a05fb15c4caf6a693dd4d0fe96497@pckswarms.ch>
Message-ID: <54C1D004-EFE8-4239-BBB5-B04688A1A786@gmx.de>

Hi Bruce, all


> On 25. Mar 2021, at 10:32, Bruce O'Neel <bruce.oneel at pckswarms.ch> wrote:
> 
> Hi,
> 
> I've attached sqUnixMain.i from the command below.

Thanks, that helps.

The problem is here:

# 1 "/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c" 2
# 36 "/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c"
# 1 "/home/edoneel/tmp/opensmalltalk-vm/platforms/Cross/vm/sq.h" 1
# 12 "/home/edoneel/tmp/opensmalltalk-vm/platforms/Cross/vm/sq.h"
# 1 "/usr/include/stdio.h" 1 3 4
# 27 "/usr/include/stdio.h" 3 4
# 1 "/usr/include/x86_64-linux-gnu/bits/libc-header-start.h" 1 3 4
# 33 "/usr/include/x86_64-linux-gnu/bits/libc-header-start.h" 3 4
# 1 "/usr/include/features.h" 1 3 4
# 439 "/usr/include/features.h" 3 4
# 1 "/usr/include/stdc-predef.h" 1 3 4


This is due to aafcb78371c7e576073a8dbf2f1f32667568e05e, Where the include of "sqConfig.h" was demoted to _after_ stdio.h and such.

This seems to deal with an interdependency between EXPORT and sqPlatformSpecific.h 
but I don't understand what that has to do with sqConfig.h

I see that _only_ win32 is doing a lot of stuff in sqConfig.h (via sqWin32.h) which _most probably_ would belong in sqPlatformSpecific.h in the first place.

Since Eliot strongly-worded to not change that, I wont commit a fix.

best
	-tobias



> 
> And yes it is in config.h
> 
> 
> #ifndef _GNU_SOURCE
> # define _GNU_SOURCE 1
> #endif
> 
> Thanks.
> 
> bruce
> 
> 
> 
> 25 March 2021 10:14 Tobias Pape <Das.Linux at gmx.de> wrote:
> Hi Bruce and all
> 
> > On 25. Mar 2021, at 09:43, Bruce O'Neel wrote:
> >
> > Hi,
> >
> > Yea that's right, but, I think that is not where the problem is.
> >
> > What seems to have happened is that someone got clever with which is included in the beginning of and which must have changed between different versions of Linux. So I'm on what is basically Ubuntu 20.4 and they have played around what __USE_GNU means.
> 
> 
> > Now it seams that the magic #define is _GNU_SOURCE.
> 
> This is so quite a long time already.
> I just checked a system frozen in time in 2012, it is the same there.
> All __USE_* get undefined in features.h and only set with the respective _*_SOURCE options.
> 
> The __USE_GNU dance in
> https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/4f6b75b56b314c20b1842078872bd845d61344a4/platforms/unix/vm/include_ucontext.h#L15
> 
> is hence sadly ineffective.
> 
> 
> >
> > The easiest patch I found is below and attached above. I also attached features.h just so you can see the nightmare that has become.
> >
> > diff --git a/build.linux64x64/squeak.cog.spur/build/mvm b/build.linux64x64/squeak.cog.spur/build/mvm
> > index 29b710460..eb67f677e 100755
> > --- a/build.linux64x64/squeak.cog.spur/build/mvm
> > +++ b/build.linux64x64/squeak.cog.spur/build/mvm
> > @@ -34,7 +34,7 @@ test -f config.h || ../../../platforms/unix/config/configure --without-npsqueak
> > --with-src=spur64src \
> > TARGET_ARCH="-m64" \
> > CC=clang \
> > - CFLAGS="$CFLAGS" \
> > + CFLAGS="$CFLAGS -D_GNU_SOURCE=1" \
> > LIBS="$LIBS" \
> > LDFLAGS="$LDFLAGS"
> > rm -f vm/sqUnixMain.o # nuke version info
> >
> Configure should define that!
> 
> AC_USE_SYSTEM_EXTENSIONS in
> https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/4f6b75b56b314c20b1842078872bd845d61344a4/platforms/unix/config/configure.ac#L42
> does exactly that: if GNU Extensions are avaialbe, define _GNU_SORUCE.
> 
> This define should be in config.h
> 
> This is the unconfigured file: https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/unix/config/config.h.in#L392
> 
> Once you run ./configure, it _will_ define _GNU_SOURCE.
> 
> However, it must be sur that config.h is included.
> This is done unconditionally in sqMemoryAccess.h and conditionally (HAVE_CONFIG_H) in sqConfig.h (which is pulled by sq.h)
> (HAVE_CONFIG_H is set as I see in your email from 2 days ago)
> 
> 
> Based on your command in your email, can yous please send the output of
> 
> clang -Wall -g -O2 -DNDEBUG -DDEBUGVM=0 -msse2 -DCOGMTVM=0 -pthread -DLSB_FIRST=1 -m64 -Wno-missing-braces -Wno-unknown-pragmas -Wno-unused-value -Wno-unused-label -Wno-unused-function -Wno-unused-variable -DHAVE_CONFIG_H -DSQUEAK_BUILTIN_PLUGIN -I/home/edoneel/tmp/opensmalltalk-vm/build.linux64x64/squeak.cog.spur/build -I/home/edoneel/tmp/opensmalltalk-vm/build.linux64x64/squeak.cog.spur/build -I/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm -I/home/edoneel/tmp/opensmalltalk-vm/platforms/Cross/vm -I/home/edoneel/tmp/opensmalltalk-vm/spur64src/vm -I/usr/local/include -I/home/edoneel/tmp/opensmalltalk-vm/platforms/Cross/vm -I/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm -I/home/edoneel/tmp/opensmalltalk-vm/spur64src/vm -I/home/edoneel/tmp/opensmalltalk-vm/platforms/Cross/plugins/FilePlugin -I/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/plugins/B3DAcceleratorPlugin -m64 -Wno-missing-braces -Wno-unknown-pragmas -Wno-unused-value -Wno-unused-label -Wno-unused-function -Wno-unused-variable -E -o sqUnixMain.i /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c
> 
> and the resulting sqUnixMain.i?
> 
> 
> Best regards
> -Tobias
> 
> 
> > I tried to play the same ifdef games currently with __USE_GNU in include_ucontext.h but I was not successful.
> >
> > This builds both with clang and with gcc on Ubuntu 20.4 on x86_64.
> >
> > cheers
> >
> > bruce
> >
> > 24 March 2021 20:58 "David T. Lewis" wrote:
> >
> > On Wed, Mar 24, 2021 at 11:54:55AM -0700, Eliot Miranda wrote:
> > >
> > > Hi David,
> > >
> > > On Mon, Mar 22, 2021 at 4:14 PM David T. Lewis wrote:
> > >
> > > >
> > > > On Mon, Mar 22, 2021 at 11:46:27AM +0100, Bruce O'Neel wrote:
> > > > >
> > > > >
> > > > > Hi,
> > > > >
> > > > > That fixed one problem I was tripping over on the ARMv8 build.
> > > > >
> > > > > There still seems to be a problem with the x86-64 build.????
> > > > >
> > > > > Thanks.
> > > > >
> > > > > bruce
> > > >
> > > > I see the same issue on my Linux x86-64 PC. I cannot spot the exact cause
> > > > of
> > > > the problem, but it seems to have been introduced in this commit:
> > > >
> > > > commit 6d74c4b652c7780110fe327f97e98aaef5242fbc
> > > > Author: Eliot Miranda
> > > > Date: Wed Feb 10 21:41:31 2021 -0800
> > > >
> > > > Revisions to core include files to get EXPORT defined correctly at the
> > > > relevant points.
> > > > Essentially move the default EXPORT definitions from sq.h to
> > > > sqMemoryAccess.h.
> > > > Include fbdev support in build.linux32ARMv6 builds and add -m32 to
> > > > their CFLAGS
> > > > to attempt to get the 32-bit VMs to be compilable on 64-bit ARM. [ci
> > > > skip]
> > > > cuz new cogit files will arrive soon.
> > > >
> > > >
> > > > The changes seem straightforward, but for some reason the header files must
> > > > not be getting included properly.
> > > >
> > > > Dave
> > > >
> > > > >
> > > > /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:926:35:
> > > > error: use of undeclared identifier 'REG_RBP'
> > > >
> > >
> > > If you have a look at the code here the file is trying to
> > > - find out what platform it is on
> > > - pull in the relevant version of ucontext.h containing the relevant
> > > defines for extracting register values from an interrupt context
> > > - print out the registers for a crash report
> > > So please look to
> > > - find out if platforms/unix/vm/include_ucontext.h is working correctly on
> > > this platform
> > > - find out what are the platform defines (-E -dM is your friend here)
> > > - find out how ucontext.h defines the registers in a ucontext
> > >
> >
> > Hi Eliot,
> >
> > I think that platforms/unix/vm/include_ucontext.h is working correctly.
> > Using -E -dM, I can confirm:
> >
> > #define __linux__ 1
> > #define __x86_64 1
> >
> > Also, include_ucontext.h was not modified in commit 924a24bb54870a621c5283f23631261d5a792000
> > which is where I think the issue originates(?).
> >
> > So there is something else going wrong with the includes for Linux, but
> > I am blind and cannot spot it.
> >
> > Dave
> >
> >
> >
> 
> 
> 
> <sqUnixMain.i>



From noreply at github.com  Thu Mar 25 10:10:54 2021
From: noreply at github.com (Tobias Pape)
Date: Thu, 25 Mar 2021 03:10:54 -0700
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] 25e4f4: config.h cannot
 work after linux features.h
Message-ID: <OpenSmalltalk/opensmalltalk-vm/push/refs/heads/Cog/b4afea-25e4f4@github.com>

  Branch: refs/heads/Cog
  Home:   https://github.com/OpenSmalltalk/opensmalltalk-vm
  Commit: 25e4f4a834e6203d59f668d565e2b4d97d08bb65
      https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/25e4f4a834e6203d59f668d565e2b4d97d08bb65
  Author: Tobias Pape <tobias at netshed.de>
  Date:   2021-03-25 (Thu, 25 Mar 2021)

  Changed paths:
    M platforms/unix/config/config.h.in
    M platforms/unix/config/configure.ac

  Log Message:
  -----------
  config.h cannot work after linux features.h

Bail if that's the case



From no-reply at appveyor.com  Thu Mar 25 10:29:51 2021
From: no-reply at appveyor.com (AppVeyor)
Date: Thu, 25 Mar 2021 10:29:51 +0000
Subject: [Vm-dev] Build failed: opensmalltalk-vm 1.0.2420
Message-ID: <20210325102951.1.73483189D5980E79@appveyor.com>

An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210325/02f35811/attachment.html>

From notifications at github.com  Thu Mar 25 18:09:44 2021
From: notifications at github.com (Tobias Pape)
Date: Thu, 25 Mar 2021 11:09:44 -0700
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] primitiveSSLCreate
 fails sporadically on Windows (#554)
In-Reply-To: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
References: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
Message-ID: <OpenSmalltalk/opensmalltalk-vm/issues/554/807208940@github.com>

I tried to work with the windows stuff today.
Since I cannot get anything to work I cant help.
sorry

-- 
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/554#issuecomment-807208940
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210325/9d834e56/attachment.html>

From notifications at github.com  Thu Mar 25 18:59:50 2021
From: notifications at github.com (Christoph Thiede)
Date: Thu, 25 Mar 2021 11:59:50 -0700
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] primitiveSSLCreate
 fails sporadically on Windows (#554)
In-Reply-To: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
References: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
Message-ID: <OpenSmalltalk/opensmalltalk-vm/issues/554/807288897@github.com>

Hm ... for me, the cygwin approach has worked in the end ... anyone else?

-- 
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/554#issuecomment-807288897
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210325/1bd2f580/attachment-0001.html>

From notifications at github.com  Thu Mar 25 19:05:14 2021
From: notifications at github.com (Marcel Taeumel)
Date: Thu, 25 Mar 2021 12:05:14 -0700
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] primitiveSSLCreate
 fails sporadically on Windows (#554)
In-Reply-To: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
References: <OpenSmalltalk/opensmalltalk-vm/issues/554@github.com>
Message-ID: <OpenSmalltalk/opensmalltalk-vm/issues/554/807297230@github.com>

Since I also observed this issue with the FFI-Plugin and OSProcess-Plugin under Windows 10, I suspect an issue with module loading in general. On my machine, this sometimes happens after, for example, performing a Windows update. I suspect some DLL caching issue. If I had the time, I would take a closer look at how the VM loads modules on Windows and then cosult the MSDN documentation to then reason about the circumstances and edge cases.

-- 
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/554#issuecomment-807297230
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210325/2962dd09/attachment.html>

From eliot.miranda at gmail.com  Thu Mar 25 21:43:41 2021
From: eliot.miranda at gmail.com (Eliot Miranda)
Date: Thu, 25 Mar 2021 14:43:41 -0700
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] sqUnixEvent.c missing
 from platforms/unix/vm (#555)
In-Reply-To: <54C1D004-EFE8-4239-BBB5-B04688A1A786@gmx.de>
References: <20DE3234-D4D8-4BDC-B0E2-0F7BF35E432F@gmx.de>
 <20210324195811.GA81078@shell.msen.com>
 <20210322015551.GA13754@shell.msen.com>
 <OpenSmalltalk/opensmalltalk-vm/issues/555@github.com>
 <OpenSmalltalk/opensmalltalk-vm/issues/555/799327721@github.com>
 <1616409987-38356102e760c938a2dcb06d23dad2a9@pckswarms.ch>
 <20210322231418.GA2171@shell.msen.com>
 <CAC20JE0k53LZC0hRzyDh56Lkc2DH-EVaRoU+Ja9oEaCQPxyL6A@mail.gmail.com>
 <1616661820-4e55f9814ec607b58360daea15516cde@pckswarms.ch>
 <1616664766-7a9a05fb15c4caf6a693dd4d0fe96497@pckswarms.ch>
 <54C1D004-EFE8-4239-BBB5-B04688A1A786@gmx.de>
Message-ID: <CAC20JE03dHq8-zxTgPpTeRYfD9PRf4SWKL9MVbRekMEYoWkQ8A@mail.gmail.com>

Hi Tobias, Hi All,

On Thu, Mar 25, 2021 at 3:04 AM Tobias Pape <Das.Linux at gmx.de> wrote:

> Hi Bruce, all
>
>
> > On 25. Mar 2021, at 10:32, Bruce O'Neel <bruce.oneel at pckswarms.ch>
> wrote:
> >
> > Hi,
> >
> > I've attached sqUnixMain.i from the command below.
>
> Thanks, that helps.
>
> The problem is here:
>
> # 1 "/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c" 2
> # 36 "/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c"
> # 1 "/home/edoneel/tmp/opensmalltalk-vm/platforms/Cross/vm/sq.h" 1
> # 12 "/home/edoneel/tmp/opensmalltalk-vm/platforms/Cross/vm/sq.h"
> # 1 "/usr/include/stdio.h" 1 3 4
> # 27 "/usr/include/stdio.h" 3 4
> # 1 "/usr/include/x86_64-linux-gnu/bits/libc-header-start.h" 1 3 4
> # 33 "/usr/include/x86_64-linux-gnu/bits/libc-header-start.h" 3 4
> # 1 "/usr/include/features.h" 1 3 4
> # 439 "/usr/include/features.h" 3 4
> # 1 "/usr/include/stdc-predef.h" 1 3 4
>
>
> This is due to aafcb78371c7e576073a8dbf2f1f32667568e05e, Where the include
> of "sqConfig.h" was demoted to _after_ stdio.h and such.
>
> This seems to deal with an interdependency between EXPORT and
> sqPlatformSpecific.h
> but I don't understand what that has to do with sqConfig.h
>
> I see that _only_ win32 is doing a lot of stuff in sqConfig.h (via
> sqWin32.h) which _most probably_ would belong in sqPlatformSpecific.h in
> the first place.
>
> Since Eliot strongly-worded to not change that, I wont commit a fix.
>

I am trying to maintain a complex product across Windows and MacOS, and
soon enough linux ARMv8.  In doing this I hade to change the order of
includes for sqPlatformSpecific.h to get the EXPORT & IMPORT macros
correctly defined to be able to share variables and functions between the
VM and dlls on Windows.  This *must not* be broken.

I do not object to including a config.h file before any platform includes,
provided that that config.h is generated *and only defines values
meaningful to the platform include files*.  I find sqConfig.h noise at
best.  Its purpose is unclear and differs across platforms.

I said many years ago that I find the current automake code in
platforms/unix/config a disaster:
- it does not run reliably on many linuxes
- it does not produce the same configure script when run on different
distros and different versions of the same distro
- it is two phase, one must first compile, and then configure
- the makefiles it produces are awal: they don't include dependency
information so one must manually delete objects to get proper recompilation

I have said on numerous occasions that I believe that the right way to
tackle these problems is to
- discard the automake nonsense in favour of a small configure script that
generates a sqConfig.h in the root of the relevant build directory (e.g.
build.linux64x64/sqConfig.h, build.linux64ARMv8/sqConfig.h, etc). -- we
can't call it configh.h wit5hout affecting plugins that include code which
has its own config.h
- use proper static makefiles that are parameterised by plugins.int,
plugins.ext and the mvm script

But no one is stepping up to help.  So I'm just going to ensure that Terf
works (it includes everything Squeak does and then much more) on the
platforms we care about at the moment, and try and try and address the
structural problems with the current platform sources when I have time, or
when (as the IMPORT/EXPORT issue did) issues force addressing them.

I've written and maintained the static makefiles for MacOS.  I've written
the static MSVC Makefiles for WIndows.  I've maintained with Nicholas the
static Makefiles for MinGW/Cygwin that Andreas contributed.  I've spent far
too long struggling with the autoconf mess on unix.  Anybody prepared to
step up and move to statuic makefiles, a config.h in every build directory
and move all generated and platform source files to conform to the template

#include "sqConfig.h"
#include <platform headers>
#include "our readers

?


and test across at least 64-bit macos, windows, msvc & mingw/cygwin, & at
least one linux platform?

best
>         -tobias
>
>
>
> >
> > And yes it is in config.h
> >
> >
> > #ifndef _GNU_SOURCE
> > # define _GNU_SOURCE 1
> > #endif
> >
> > Thanks.
> >
> > bruce
> >
> >
> >
> > 25 March 2021 10:14 Tobias Pape <Das.Linux at gmx.de> wrote:
> > Hi Bruce and all
> >
> > > On 25. Mar 2021, at 09:43, Bruce O'Neel wrote:
> > >
> > > Hi,
> > >
> > > Yea that's right, but, I think that is not where the problem is.
> > >
> > > What seems to have happened is that someone got clever with which is
> included in the beginning of and which must have changed between different
> versions of Linux. So I'm on what is basically Ubuntu 20.4 and they have
> played around what __USE_GNU means.
> >
> >
> > > Now it seams that the magic #define is _GNU_SOURCE.
> >
> > This is so quite a long time already.
> > I just checked a system frozen in time in 2012, it is the same there.
> > All __USE_* get undefined in features.h and only set with the respective
> _*_SOURCE options.
> >
> > The __USE_GNU dance in
> >
> https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/4f6b75b56b314c20b1842078872bd845d61344a4/platforms/unix/vm/include_ucontext.h#L15
> >
> > is hence sadly ineffective.
> >
> >
> > >
> > > The easiest patch I found is below and attached above. I also attached
> features.h just so you can see the nightmare that has become.
> > >
> > > diff --git a/build.linux64x64/squeak.cog.spur/build/mvm
> b/build.linux64x64/squeak.cog.spur/build/mvm
> > > index 29b710460..eb67f677e 100755
> > > --- a/build.linux64x64/squeak.cog.spur/build/mvm
> > > +++ b/build.linux64x64/squeak.cog.spur/build/mvm
> > > @@ -34,7 +34,7 @@ test -f config.h ||
> ../../../platforms/unix/config/configure --without-npsqueak
> > > --with-src=spur64src \
> > > TARGET_ARCH="-m64" \
> > > CC=clang \
> > > - CFLAGS="$CFLAGS" \
> > > + CFLAGS="$CFLAGS -D_GNU_SOURCE=1" \
> > > LIBS="$LIBS" \
> > > LDFLAGS="$LDFLAGS"
> > > rm -f vm/sqUnixMain.o # nuke version info
> > >
> > Configure should define that!
> >
> > AC_USE_SYSTEM_EXTENSIONS in
> >
> https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/4f6b75b56b314c20b1842078872bd845d61344a4/platforms/unix/config/configure.ac#L42
> > does exactly that: if GNU Extensions are avaialbe, define _GNU_SORUCE.
> >
> > This define should be in config.h
> >
> > This is the unconfigured file:
> https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/unix/config/config.h.in#L392
> >
> > Once you run ./configure, it _will_ define _GNU_SOURCE.
> >
> > However, it must be sur that config.h is included.
> > This is done unconditionally in sqMemoryAccess.h and conditionally
> (HAVE_CONFIG_H) in sqConfig.h (which is pulled by sq.h)
> > (HAVE_CONFIG_H is set as I see in your email from 2 days ago)
> >
> >
> > Based on your command in your email, can yous please send the output of
> >
> > clang -Wall -g -O2 -DNDEBUG -DDEBUGVM=0 -msse2 -DCOGMTVM=0 -pthread
> -DLSB_FIRST=1 -m64 -Wno-missing-braces -Wno-unknown-pragmas
> -Wno-unused-value -Wno-unused-label -Wno-unused-function
> -Wno-unused-variable -DHAVE_CONFIG_H -DSQUEAK_BUILTIN_PLUGIN
> -I/home/edoneel/tmp/opensmalltalk-vm/build.linux64x64/squeak.cog.spur/build
> -I/home/edoneel/tmp/opensmalltalk-vm/build.linux64x64/squeak.cog.spur/build
> -I/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm
> -I/home/edoneel/tmp/opensmalltalk-vm/platforms/Cross/vm
> -I/home/edoneel/tmp/opensmalltalk-vm/spur64src/vm -I/usr/local/include
> -I/home/edoneel/tmp/opensmalltalk-vm/platforms/Cross/vm
> -I/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm
> -I/home/edoneel/tmp/opensmalltalk-vm/spur64src/vm
> -I/home/edoneel/tmp/opensmalltalk-vm/platforms/Cross/plugins/FilePlugin
> -I/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/plugins/B3DAcceleratorPlugin
> -m64 -Wno-missing-braces -Wno-unknown-pragmas -Wno-unused-value
> -Wno-unused-label -Wno-unused-function -Wno-unused-variable -E -o
> sqUnixMain.i
> /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c
> >
> > and the resulting sqUnixMain.i?
> >
> >
> > Best regards
> > -Tobias
> >
> >
> > > I tried to play the same ifdef games currently with __USE_GNU in
> include_ucontext.h but I was not successful.
> > >
> > > This builds both with clang and with gcc on Ubuntu 20.4 on x86_64.
> > >
> > > cheers
> > >
> > > bruce
> > >
> > > 24 March 2021 20:58 "David T. Lewis" wrote:
> > >
> > > On Wed, Mar 24, 2021 at 11:54:55AM -0700, Eliot Miranda wrote:
> > > >
> > > > Hi David,
> > > >
> > > > On Mon, Mar 22, 2021 at 4:14 PM David T. Lewis wrote:
> > > >
> > > > >
> > > > > On Mon, Mar 22, 2021 at 11:46:27AM +0100, Bruce O'Neel wrote:
> > > > > >
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > That fixed one problem I was tripping over on the ARMv8 build.
> > > > > >
> > > > > > There still seems to be a problem with the x86-64 build.????
> > > > > >
> > > > > > Thanks.
> > > > > >
> > > > > > bruce
> > > > >
> > > > > I see the same issue on my Linux x86-64 PC. I cannot spot the
> exact cause
> > > > > of
> > > > > the problem, but it seems to have been introduced in this commit:
> > > > >
> > > > > commit 6d74c4b652c7780110fe327f97e98aaef5242fbc
> > > > > Author: Eliot Miranda
> > > > > Date: Wed Feb 10 21:41:31 2021 -0800
> > > > >
> > > > > Revisions to core include files to get EXPORT defined correctly at
> the
> > > > > relevant points.
> > > > > Essentially move the default EXPORT definitions from sq.h to
> > > > > sqMemoryAccess.h.
> > > > > Include fbdev support in build.linux32ARMv6 builds and add -m32 to
> > > > > their CFLAGS
> > > > > to attempt to get the 32-bit VMs to be compilable on 64-bit ARM.
> [ci
> > > > > skip]
> > > > > cuz new cogit files will arrive soon.
> > > > >
> > > > >
> > > > > The changes seem straightforward, but for some reason the header
> files must
> > > > > not be getting included properly.
> > > > >
> > > > > Dave
> > > > >
> > > > > >
> > > > >
> /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:926:35:
> > > > > error: use of undeclared identifier 'REG_RBP'
> > > > >
> > > >
> > > > If you have a look at the code here the file is trying to
> > > > - find out what platform it is on
> > > > - pull in the relevant version of ucontext.h containing the relevant
> > > > defines for extracting register values from an interrupt context
> > > > - print out the registers for a crash report
> > > > So please look to
> > > > - find out if platforms/unix/vm/include_ucontext.h is working
> correctly on
> > > > this platform
> > > > - find out what are the platform defines (-E -dM is your friend here)
> > > > - find out how ucontext.h defines the registers in a ucontext
> > > >
> > >
> > > Hi Eliot,
> > >
> > > I think that platforms/unix/vm/include_ucontext.h is working correctly.
> > > Using -E -dM, I can confirm:
> > >
> > > #define __linux__ 1
> > > #define __x86_64 1
> > >
> > > Also, include_ucontext.h was not modified in commit
> 924a24bb54870a621c5283f23631261d5a792000
> > > which is where I think the issue originates(?).
> > >
> > > So there is something else going wrong with the includes for Linux, but
> > > I am blind and cannot spot it.
> > >
> > > Dave
> > >
> > >
> > >
> >
> >
> >
> > <sqUnixMain.i>
>
>
>

-- 
_,,,^..^,,,_
best, Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20210325/1b55c8fb/attachment-0001.html>

From Das.Linux at gmx.de  Thu Mar 25 21:50:51 2021
From: Das.Linux at gmx.de (Tobias Pape)
Date: Thu, 25 Mar 2021 22:50:51 +0100
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] sqUnixEvent.c missing
 from platforms/unix/vm (#555)
In-Reply-To: <CAC20JE03dHq8-zxTgPpTeRYfD9PRf4SWKL9MVbRekMEYoWkQ8A@mail.gmail.com>
References: <20DE3234-D4D8-4BDC-B0E2-0F7BF35E432F@gmx.de>
 <20210324195811.GA81078@shell.msen.com>
 <20210322015551.GA13754@shell.msen.com>
 <OpenSmalltalk/opensmalltalk-vm/issues/555@github.com>
 <OpenSmalltalk/opensmalltalk-vm/issues/555/799327721@github.com>
 <1616409987-38356102e760c938a2dcb06d23dad2a9@pckswarms.ch>
 <20210322231418.GA2171@shell.msen.com>
 <CAC20JE0k53LZC0hRzyDh56Lkc2DH-EVaRoU+Ja9oEaCQPxyL6A@mail.gmail.com>
 <1616661820-4e55f9814ec607b58360daea15516cde@pckswarms.ch>
 <1616664766-7a9a05fb15c4caf6a693dd4d0fe96497@pckswarms.ch>
 <54C1D004-EFE8-4239-BBB5-B04688A1A786@gmx.de>
 <CAC20JE03dHq8-zxTgPpTeRYfD9PRf4SWKL9MVbRekMEYoWkQ8A@mail.gmail.com>
Message-ID: <98DE6AC8-836D-4F44-9B39-24AB72DEDC5D@gmx.de>

Hi Eliot

> On 25. Mar 2021, at 22:43, Eliot Miranda <eliot.miranda at gmail.com> wrote:
> 
> Hi Tobias, Hi All,
> 


I just wanted to respect your request _not_ rewriting what you committed in that one commit.


> On Thu, Mar 25, 2021 at 3:04 AM Tobias Pape <Das.Linux at gmx.de> wrote:
> Hi Bruce, all
> 
> 
> > On 25. Mar 2021, at 10:32, Bruce O'Neel <bruce.oneel at pckswarms.ch> wrote:
> > 
> > Hi,
> > 
> > I've attached sqUnixMain.i from the command below.
> 
> Thanks, that helps.
> 
> The problem is here:
> 
> # 1 "/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c" 2
> # 36 "/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c"
> # 1 "/home/edoneel/tmp/opensmalltalk-vm/platforms/Cross/vm/sq.h" 1
> # 12 "/home/edoneel/tmp/opensmalltalk-vm/platforms/Cross/vm/sq.h"
> # 1 "/usr/include/stdio.h" 1 3 4
> # 27 "/usr/include/stdio.h" 3 4
> # 1 "/usr/include/x86_64-linux-gnu/bits/libc-header-start.h" 1 3 4
> # 33 "/usr/include/x86_64-linux-gnu/bits/libc-header-start.h" 3 4
> # 1 "/usr/include/features.h" 1 3 4
> # 439 "/usr/include/features.h" 3 4
> # 1 "/usr/include/stdc-predef.h" 1 3 4
> 
> 
> This is due to aafcb78371c7e576073a8dbf2f1f32667568e05e, Where the include of "sqConfig.h" was demoted to _after_ stdio.h and such.
> 
> This seems to deal with an interdependency between EXPORT and sqPlatformSpecific.h 
> but I don't understand what that has to do with sqConfig.h
> 
> I see that _only_ win32 is doing a lot of stuff in sqConfig.h (via sqWin32.h) which _most probably_ would belong in sqPlatformSpecific.h in the first place.
> 
> Since Eliot strongly-worded to not change that, I wont commit a fix.
> 
> I am trying to maintain a complex product across Windows and MacOS, and soon enough linux ARMv8.  In doing this I hade to change the order of includes for sqPlatformSpecific.h to get the EXPORT & IMPORT macros correctly defined to be able to share variables and functions between the VM and dlls on Windows.  This *must not* be broken.
> 
> I do not object to including a config.h file before any platform includes, provided that that config.h is generated *and only defines values meaningful to the platform include files*.  I find sqConfig.h noise at best.  Its purpose is unclear and differs across platforms.
> 
> I said many years ago that I find the current automake code in platforms/unix/config a disaster:
> - it does not run reliably on many linuxes
> - it does not produce the same configure script when run on different distros and different versions of the same distro
> - it is two phase, one must first compile, and then configure
> - the makefiles it produces are awal: they don't include dependency information so one must manually delete objects to get proper recompilation
> 
> I have said on numerous occasions that I believe that the right way to tackle these problems is to
> - discard the automake nonsense in favour of a small configure script that generates a sqConfig.h in the root of the relevant build directory (e.g. build.linux64x64/sqConfig.h, build.linux64ARMv8/sqConfig.h, etc). -- we can't call it configh.h wit5hout affecting plugins that include code which has its own config.h
> - use proper static makefiles that are parameterised by plugins.int, plugins.ext and the mvm script
> 
> But no one is stepping up to help.  So I'm just going to ensure that Terf works (it includes everything Squeak does and then much more) on the platforms we care about at the moment, and try and try and address the structural problems with the current platform sources when I have time, or when (as the IMPORT/EXPORT issue did) issues force addressing them.
> 
> I've written and maintained the static makefiles for MacOS.  I've written the static MSVC Makefiles for WIndows.  I've maintained with Nicholas the static Makefiles for MinGW/Cygwin that Andreas contributed.  I've spent far too long struggling with the autoconf mess on unix.  Anybody prepared to step up and move to statuic makefiles, a config.h in every build directory and move all generated and platform source files to conform to the template
> 
> #include "sqConfig.h"
> #include <platform headers>
> #include "our readers
> 
> ?
> 
> 
> and test across at least 64-bit macos, windows, msvc & mingw/cygwin, & at least one linux platform?


I know your stance regarding the configure stuff and hence did not want to interfere.
I still disagree with the hand-crafted solution. I deem it not viable.
But as I have not nearly as much contributed here, I won't interfere.
I'm just tired of hand-written static makefiles. Sorry.

Best regards
	-Tobi



> 
> best
>         -tobias
> 
> 
> 
> > 
> > And yes it is in config.h
> > 
> > 
> > #ifndef _GNU_SOURCE
> > # define _GNU_SOURCE 1
> > #endif
> > 
> > Thanks.
> > 
> > bruce
> > 
> > 
> > 
> > 25 March 2021 10:14 Tobias Pape <Das.Linux at gmx.de> wrote:
> > Hi Bruce and all
> > 
> > > On 25. Mar 2021, at 09:43, Bruce O'Neel wrote:
> > >
> > > Hi,
> > >
> > > Yea that's right, but, I think that is not where the problem is.
> > >
> > > What seems to have happened is that someone got clever with which is included in the beginning of and which must have changed between different versions of Linux. So I'm on what is basically Ubuntu 20.4 and they have played around what __USE_GNU means.
> > 
> > 
> > > Now it seams that the magic #define is _GNU_SOURCE.
> > 
> > This is so quite a long time already.
> > I just checked a system frozen in time in 2012, it is the same there.
> > All __USE_* get undefined in features.h and only set with the respective _*_SOURCE options.
> > 
> > The __USE_GNU dance in
> > https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/4f6b75b56b314c20b1842078872bd845d61344a4/platforms/unix/vm/include_ucontext.h#L15
> > 
> > is hence sadly ineffective.
> > 
> > 
> > >
> > > The easiest patch I found is below and attached above. I also attached features.h just so you can see the nightmare that has become.
> > >
> > > diff --git a/build.linux64x64/squeak.cog.spur/build/mvm b/build.linux64x64/squeak.cog.spur/build/mvm
> > > index 29b710460..eb67f677e 100755
> > > --- a/build.linux64x64/squeak.cog.spur/build/mvm
> > > +++ b/build.linux64x64/squeak.cog.spur/build/mvm
> > > @@ -34,7 +34,7 @@ test -f config.h || ../../../platforms/unix/config/configure --without-npsqueak
> > > --with-src=spur64src \
> > > TARGET_ARCH="-m64" \
> > > CC=clang \
> > > - CFLAGS="$CFLAGS" \
> > > + CFLAGS="$CFLAGS -D_GNU_SOURCE=1" \
> > > LIBS="$LIBS" \
> > > LDFLAGS="$LDFLAGS"
> > > rm -f vm/sqUnixMain.o # nuke version info
> > >
> > Configure should define that!
> > 
> > AC_USE_SYSTEM_EXTENSIONS in
> > https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/4f6b75b56b314c20b1842078872bd845d61344a4/platforms/unix/config/configure.ac#L42
> > does exactly that: if GNU Extensions are avaialbe, define _GNU_SORUCE.
> > 
> > This define should be in config.h
> > 
> > This is the unconfigured file: https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/unix/config/config.h.in#L392
> > 
> > Once you run ./configure, it _will_ define _GNU_SOURCE.
> > 
> > However, it must be sur that config.h is included.
> > This is done unconditionally in sqMemoryAccess.h and conditionally (HAVE_CONFIG_H) in sqConfig.h (which is pulled by sq.h)
> > (HAVE_CONFIG_H is set as I see in your email from 2 days ago)
> > 
> > 
> > Based on your command in your email, can yous please send the output of
> > 
> > clang -Wall -g -O2 -DNDEBUG -DDEBUGVM=0 -msse2 -DCOGMTVM=0 -pthread -DLSB_FIRST=1 -m64 -Wno-missing-braces -Wno-unknown-pragmas -Wno-unused-value -Wno-unused-label -Wno-unused-function -Wno-unused-variable -DHAVE_CONFIG_H -DSQUEAK_BUILTIN_PLUGIN -I/home/edoneel/tmp/opensmalltalk-vm/build.linux64x64/squeak.cog.spur/build -I/home/edoneel/tmp/opensmalltalk-vm/build.linux64x64/squeak.cog.spur/build -I/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm -I/home/edoneel/tmp/opensmalltalk-vm/platforms/Cross/vm -I/home/edoneel/tmp/opensmalltalk-vm/spur64src/vm -I/usr/local/include -I/home/edoneel/tmp/opensmalltalk-vm/platforms/Cross/vm -I/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm -I/home/edoneel/tmp/opensmalltalk-vm/spur64src/vm -I/home/edoneel/tmp/opensmalltalk-vm/platforms/Cross/plugins/FilePlugin -I/home/edoneel/tmp/opensmalltalk-vm/platforms/unix/plugins/B3DAcceleratorPlugin -m64 -Wno-missing-braces -Wno-unknown-pragmas -Wno-unused-value -Wno-unused-label -Wno-unused-function -Wno-unused-variable -E -o sqUnixMain.i /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c
> > 
> > and the resulting sqUnixMain.i?
> > 
> > 
> > Best regards
> > -Tobias
> > 
> > 
> > > I tried to play the same ifdef games currently with __USE_GNU in include_ucontext.h but I was not successful.
> > >
> > > This builds both with clang and with gcc on Ubuntu 20.4 on x86_64.
> > >
> > > cheers
> > >
> > > bruce
> > >
> > > 24 March 2021 20:58 "David T. Lewis" wrote:
> > >
> > > On Wed, Mar 24, 2021 at 11:54:55AM -0700, Eliot Miranda wrote:
> > > >
> > > > Hi David,
> > > >
> > > > On Mon, Mar 22, 2021 at 4:14 PM David T. Lewis wrote:
> > > >
> > > > >
> > > > > On Mon, Mar 22, 2021 at 11:46:27AM +0100, Bruce O'Neel wrote:
> > > > > >
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > That fixed one problem I was tripping over on the ARMv8 build.
> > > > > >
> > > > > > There still seems to be a problem with the x86-64 build.????
> > > > > >
> > > > > > Thanks.
> > > > > >
> > > > > > bruce
> > > > >
> > > > > I see the same issue on my Linux x86-64 PC. I cannot spot the exact cause
> > > > > of
> > > > > the problem, but it seems to have been introduced in this commit:
> > > > >
> > > > > commit 6d74c4b652c7780110fe327f97e98aaef5242fbc
> > > > > Author: Eliot Miranda
> > > > > Date: Wed Feb 10 21:41:31 2021 -0800
> > > > >
> > > > > Revisions to core include files to get EXPORT defined correctly at the
> > > > > relevant points.
> > > > > Essentially move the default EXPORT definitions from sq.h to
> > > > > sqMemoryAccess.h.
> > > > > Include fbdev support in build.linux32ARMv6 builds and add -m32 to
> > > > > their CFLAGS
> > > > > to attempt to get the 32-bit VMs to be compilable on 64-bit ARM. [ci
> > > > > skip]
> > > > > cuz new cogit files will arrive soon.
> > > > >
> > > > >
> > > > > The changes seem straightforward, but for some reason the header files must
> > > > > not be getting included properly.
> > > > >
> > > > > Dave
> > > > >
> > > > > >
> > > > > /home/edoneel/tmp/opensmalltalk-vm/platforms/unix/vm/sqUnixMain.c:926:35:
> > > > > error: use of undeclared identifier 'REG_RBP'
> > > > >
> > > >
> > > > If you have a look at the code here the file is trying to
> > > > - find out what platform it is on
> > > > - pull in the relevant version of ucontext.h containing the relevant
> > > > defines for extracting register values from an interrupt context
> > > > - print out the registers for a crash report
> > > > So please look to
> > > > - find out if platforms/unix/vm/include_ucontext.h is working correctly on
> > > > this platform
> > > > - find out what are the platform defines (-E -dM is your friend here)
> > > > - find out how ucontext.h defines the registers in a ucontext
> > > >
> > >
> > > Hi Eliot,
> > >
> > > I think that platforms/unix/vm/include_ucontext.h is working correctly.
> > > Using -E -dM, I can confirm:
> > >
> > > #define __linux__ 1
> > > #define __x86_64 1
> > >
> > > Also, include_ucontext.h was not modified in commit 924a24bb54870a621c5283f23631261d5a792000
> > > which is where I think the issue originates(?).
> > >
> > > So there is something else going wrong with the includes for Linux, but
> > > I am blind and cannot spot it.
> > >
> > > Dave
> > >
> > >
> > >
> > 
> > 
> > 
> > <sqUnixMain.i>




From tim at rowledge.org  Thu Mar 25 23:13:45 2021
From: tim at rowledge.org (tim Rowledge)
Date: Thu, 25 Mar 2021 16:13:45 -0700
Subject: [Vm-dev] [squeak-dev] BitBlt performance work for ARM32 & 64
In-Reply-To: <20210325011538.GA24870@shell.msen.com>
References: <8D641010-1D65-4E11-A4C7-EB95A3F30CE0@rowledge.org>
 <20210325011538.GA24870@shell.msen.com>
Message-ID: <7DFA8808-8821-4ED0-B96C-C4A499363E21@rowledge.org>



> On 2021-03-24, at 6:15 PM, David T. Lewis <lewis at mail.msen.com> wrote:
> 
> I am not certain of the status in the VM, but I think these links provide the background:
> 
>  https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/505
> 
>  http://source.squeak.org/VMMaker/VMMaker.oscog-nice.2909.diff

OK, passed them on.

Ben is clearly having a fun time with this. His test suite has 524288 tests! Right now he's down to just 6 failing, all relating to addWord/subWord stuff, specifically when the line length takes up less than a word. 

Hell, rather than trying to edit I'll just include his comments - 

> The next problem I hit appears to be a genuine bug in the existing code - specifically the function rgbComponentAlphawith() - one which has simply become more visible due to a buffer overrun fix in copyLoop().
> 
> What's happening is that source and destination images are 8 bpp, but the source and destination positions differ in 32-bit word alignment. Yet 32-bit blocks of pixels are passed in and out of rgbComponentAlphawith(), aligned to words at the destination. Therefore in a case like the following:
> 
> Word boundaries at source   |       |       |
> Pixels                         X X X X X X X
> Word boundaries at dest   |       |       |       |
> 
> previously, copyLoop() would read from the word following the last source pixel when constructing the final source word to pass to rgbComponentAlphawith() - but this is unsafe if the last source pixel is at the end of the image and the image ends at memory page alignment, because the following page might not be mapped in. The new version of copyLoop() correctly skips this load; however, it arbitrarily chooses to zero the relevant bits instead, and this triggers a shortcut in rgbComponentAlphawith() to be taken when it wasn't before.
> 
> This shortcut leaves the destination word unchanged if the source word was all 0. At first glance this seems OK, since even at < 32bpp, the source pixels are made up of bitfields, and for a component alpha operation, each field is used as an alpha weight to blend a planar colour into the destination. The problem is in the way the blend calculation is done: it's (dest * (0xFF - alpha) + planar_source * alpha) >> 8. Thus when alpha == 0, it will subtract 1 from each colour component value. It was particularly noticeable in my test because it also utilised random gamma tables, so the off-by-one value was very stark.
> 
> Ideally, the blending algorithm would be fixed so that an alpha of 0 leaves the destination unchanged. However, in all but a straight-through 1:1 gamma LUT, there will still be at least some incidences of a many-to-one mapping, so a round trip through gamma and ungamma could cause colour component changes even with an alpha of 0 and a better blending algorithm. So the simple solution is to remove the invalid shortcut in rgbComponentAlphawith(), and once you do so, the old and new builds produce the same result for this test. The change does impact the results of some other tests too, so I needed to recalculate the correct CRCs with this change in place.
> 
> By now I was down to 190 failing tests out of 524288. Surveying the first 10, their combinationRules are all either addWord or subWord. Bearing the above in mind, I immediately suspected what the problem might be. Since pixels are stored left-to-right in most-significant-to-least-significant order, and carry/borrow propagate from less-significant to more-significant bits, that means that memory locations after the nominal end of the source row can impact the result. That means that the change that zeroes the remaining bits rather than loading them has the potential to change the results whenever word-relative alignment differs between source and destination.
> 
> This time I tried backporting the buffer overflow fix to the old build (it's a good thing not to read off the end of the input buffer in any case) and recalculating the CRCs again.
> 
> Now we were down to 6 failures out of 524288 tests. They were all addWord or subWord again, but the extra distinguishing feature this time was that the line lengths were very short - no more than 1 pixel for 16bpp, 3 pixels for 8bpp and (though there weren't enough examples to prove it) presumably no more than 7 pixels for 4bpp and so on.
> 
> After a bit more digging, I think this change is responsible:
> https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/426
> 
> in the sense that it also causes bits that are outside the positions that are masked into the destination to be zeroed when they weren't before, but in such a way that they affect whether or not a carry or borrow into bit positions that *do* matter happens.
> 
> Unfortunately the patch from the resolution of that issue doesn't cleanly apply back onto the old source tree - it seems there were other intervening changes in how the "preload" flag was calculated. However, I'm reasonably confident that since there are now only 6 failures, they're probably all caused by this one remaining issue. It's very reassuring that I now get the same results from the new source tree irrespective of whether the 32-bit ARM assembly fast paths are enabled or not. So the easiest thing is to simply recalculate the CRCs again from the new source tree and use them as a baseline for the AArch64 conversion.
> 
> One thing that occurs to me is that, given that there are some combinationRules that have this "leaking across pixel boundaries" behaviour (and maybe it might be deliberately added one day - say for a "blur" operation) it arguably should be the case that the source and destination words are masked to only include the bits relating to pixels included in the bounding box *before* calling the combinationRule as well as afterwards. For example, assuming an operation on a single 8bpp pixel:
> 
> Words at source        |AA AA AA AA|BB BB BB BB|
> Pixel                         **
> Words at destination      |FF FF FF FF|
> 
> In the old Squeak VM, addWordwith() would have been passed 0xAAAAAABB and 0xFFFFFFFF. New VMs skip the load of the second source word and pass 0xAAAAAA00 and 0xFFFFFFFF. What I'm saying is, perhaps it should be 0x00AA0000 and 0x00FF0000 instead. (Had this rule been in place already, none of the issues I hit today would have shown up.)

Anything anyone can think of that might help would be appreciated. Although the intent here is to make the ARM64 system as fast as possible for Pi benefit, it will have probably benefits for other machine because some of his improvements are just C code. Bug fixes should be passed up to all systems easily enough.

tim
--
tim Rowledge; tim at rowledge.org; http://www.rowledge.org/tim
At the end of the day, a cliché walks into a bar -- fresh as a daisy, cute as a button, and sharp as a tack.