Hi Nicolas --
Thank you for the advice. Found and fixed the regression via VMMaker.oscog-mt.3181.
It's just that we don't have (or don't run ?) enough tests to notice the regression.
I found this regression by running the FFI tests in a 32-bit build, which is not possible on a recent macOS. I will now regenerate and push the SqueakFFIPrims sources and update the "long long" tests in a way that they show this kind of problem more quickly in the future.
Maybe I should also test 32-bit builds here: https://github.com/marceltaeumel/squeak-ffi [https://github.com/marceltaeumel/squeak-ffi]
There still seems to be an issue with FFI on macOS.
Best, Marcel Am 12.04.2022 22:40:01 schrieb Nicolas Cellier nicolas.cellier.aka.nice@gmail.com: Hi Marcel, Because there will still be problems passing 64bits (Large) integers to FFI, such a partial patch has not enough value. If we pass a 64 bits value, but only use the 32 low bits, the patch rather tends to obscure things. I'm pretty sure that passing a LargeInteger to FFI with 64 bit (signed/unsigned) int type did work back in 2016, so it's not a new feature, just a regression. For example we had specific #ffiPushUnsignedLongLongOop: #ffiPushSignedLongLongOop: at that time. So we would not necessarily go through the generic coercion #ffiIntegerValueOf:. Eliot did refactor since, probably so as to support many other architectures - which is a good thing. It's just that we don't have (or don't run ?) enough tests to notice the regression.
The fix would be to resort to some kind of #ffiPushUnsignedLongLongOop: #ffiPushSignedLongLongOop etc... All the coercion methods could resort to generic #ffiIntegerValueOf:, but the 2 above on 32 bits arch. The C compiler will probably easily factor out the #ffiIntegerValueOf: common to each case branch on 64 bits arch, we don't have to bother for handcrafted optimization. I ain't got much time to push it further, but that's not that hard if someone want to try it out.
Le mar. 12 avr. 2022 à 15:27, Marcel Taeumel a écrit :
Hi Nicolas --
Would you mind looking at VMMaker.oscog-mt.3180 in VMMakerInbox?
Do you think this is a good thing to have for the OSVM release? If so, I will fix the long-long tests to document this.
Best, Marcel
Am 12.04.2022 10:55:33 schrieb Marcel Taeumel : Hi Nicolas --
Thanks for the tip. I will have a look.
Do you know whether Squeak FFI 32-bit does (or should) support "long long" (or "int64_t") for Large(Positive|Negative)Integer outside the range of int32_t?
On 32-bit Squeak, we need both SmallInteger and Large(Positive|Negative)Integer to cover int32_t. Not so on 64-bit Squeak. So, I am not sure whether there is actually support for int64_t on 32-bit Squeak in this regard...
So, am I looking for a bug or would this rather be a new feature? :-)
Best, Marcel
Am 11.04.2022 21:31:58 schrieb Nicolas Cellier < nicolas.cellier.aka.nice@gmail.com>: Hi Marcel, Sounds like 32bits -1 missing a sign extension when converted to 64bits... At least, the result is correctly 64bits for sure.
Le lun. 11 avr. 2022 à 18:42, Marcel Taeumel a écrit :
Hi all --
This is not working in 32-bit on Windows:
FFITestLibrary ffiTest8LongLongSum: -1 with: -1 with: -1 with: -1 with:
-1
with: -1 with: -1 with: -1.
It answers " 34359738360 " instead of " - 8 ". It's working fine in a 64-bit build. The debugging output in this test functions says:
8 long longs came in as i1 = 4294967295 (ffffffff) i2 = 4294967295 (ffffffff) i3 = 4294967295 (ffffffff) i4 = 4294967295 (ffffffff) i5 = 4294967295 (ffffffff) i6 = 4294967295 (ffffffff) i7 = 4294967295 (ffffffff) i8 = 4294967295 (ffffffff)
... this is clearly wrong. Or is it? On 64-bit, the same function says:
8 long longs came in as i1 = -1 (ffffffffffffffff) i2 = -1 (ffffffffffffffff) i3 = -1 (ffffffffffffffff) i4 = -1 (ffffffffffffffff) i5 = -1 (ffffffffffffffff) i6 = -1 (ffffffffffffffff) i7 = -1 (ffffffffffffffff) i8 = -1 (ffffffffffffffff)
Both are formatted via " %lld (%llx) " I am missing something here...
did
we change the representation of negative integers in Squeak and forgot
to
update something along ThreadedFFIPlugin >> #ffiIntegerValueOf:? :-/
Best, Marcel
EXPORT(long long) ffiTest8LongLongSum(long long c1, long long c2, long long c3, long long c4, long long c5, long long c6, long long c7, long
long
c8) { printf("8 long longs came in as\ni1 = %lld (%llx)\ni2 = %lld (%llx)\ni3
=
%lld (%llx)\ni4 = %lld (%llx)\ni5 = %lld (%llx)\ni6 = %lld (%llx)\ni7 = %lld (%llx)\ni8 = %lld (%llx)\n", c1, c1, c2, c2, c3, c3, c4, c4, c5,
c5,
c6, c6, c7, c7, c8, c8); return c1 + c2 + c3 + c4 + c5 + c6 + c7 + c8; }
Hi Marcel, Sounds like 32bits -1 missing a sign extension when converted to 64bits... At least, the result is correctly 64bits for sure.
Le lun. 11 avr. 2022 à 18:42, Marcel Taeumel a écrit :
Hi all --
This is not working in 32-bit on Windows:
FFITestLibrary ffiTest8LongLongSum: -1 with: -1 with: -1 with: -1 with: -1 with: -1 with: -1 with: -1.
It answers " 34359738360 " instead of " - 8 ". It's working fine in a 64-bit build. The debugging output in this test functions says:
8 long longs came in as i1 = 4294967295 (ffffffff) i2 = 4294967295 (ffffffff) i3 = 4294967295 (ffffffff) i4 = 4294967295 (ffffffff) i5 = 4294967295 (ffffffff) i6 = 4294967295 (ffffffff) i7 = 4294967295 (ffffffff) i8 = 4294967295 (ffffffff)
... this is clearly wrong. Or is it? On 64-bit, the same function says:
8 long longs came in as i1 = -1 (ffffffffffffffff) i2 = -1 (ffffffffffffffff) i3 = -1 (ffffffffffffffff) i4 = -1 (ffffffffffffffff) i5 = -1 (ffffffffffffffff) i6 = -1 (ffffffffffffffff) i7 = -1 (ffffffffffffffff) i8 = -1 (ffffffffffffffff)
Both are formatted via " %lld (%llx) " I am missing something here... did we change the representation of negative integers in Squeak and forgot to update something along ThreadedFFIPlugin >> #ffiIntegerValueOf:? :-/
Best, Marcel
EXPORT(long long) ffiTest8LongLongSum(long long c1, long long c2, long long c3, long long c4, long long c5, long long c6, long long c7, long long c8) { printf("8 long longs came in as\ni1 = %lld (%llx)\ni2 = %lld (%llx)\ni3 = %lld (%llx)\ni4 = %lld (%llx)\ni5 = %lld (%llx)\ni6 = %lld (%llx)\ni7 = %lld (%llx)\ni8 = %lld (%llx)\n", c1, c1, c2, c2, c3, c3, c4, c4, c5, c5, c6, c6, c7, c7, c8, c8); return c1 + c2 + c3 + c4 + c5 + c6 + c7 + c8; }
Hi Marcel, Because there will still be problems passing 64bits (Large) integers to FFI, such a partial patch has not enough value. If we pass a 64 bits value, but only use the 32 low bits, the patch rather tends to obscure things.
I'm pretty sure that passing a LargeInteger to FFI with 64 bit (signed/unsigned) int type did work back in 2016, so it's not a new feature, just a regression. For example we had specific #ffiPushUnsignedLongLongOop: #ffiPushSignedLongLongOop: at that time. So we would not necessarily go through the generic coercion #ffiIntegerValueOf:. Eliot did refactor since, probably so as to support many other architectures - which is a good thing. It's just that we don't have (or don't run ?) enough tests to notice the regression.
The fix would be to resort to some kind of #ffiPushUnsignedLongLongOop: #ffiPushSignedLongLongOop etc...
All the coercion methods could resort to generic #ffiIntegerValueOf:, but the 2 above on 32 bits arch. The C compiler will probably easily factor out the #ffiIntegerValueOf: common to each case branch on 64 bits arch, we don't have to bother for handcrafted optimization.
I ain't got much time to push it further, but that's not that hard if someone want to try it out.
Le mar. 12 avr. 2022 à 15:27, Marcel Taeumel <marcel.taeumel@hpi.de [mailto:marcel.taeumel@hpi.de]> a écrit :
Hi Nicolas --
Would you mind looking at VMMaker.oscog-mt.3180 in VMMakerInbox?
Do you think this is a good thing to have for the OSVM release? If so, I will fix the long-long tests to document this.
Best, Marcel
Am 12.04.2022 10:55:33 schrieb Marcel Taeumel <marcel.taeumel@hpi.de [mailto:marcel.taeumel@hpi.de]>:
Hi Nicolas --
Thanks for the tip. I will have a look.
Do you know whether Squeak FFI 32-bit does (or should) support "long long" (or "int64_t") for Large(Positive|Negative)Integer outside the range of int32_t?
On 32-bit Squeak, we need both SmallInteger and Large(Positive|Negative)Integer to cover int32_t. Not so on 64-bit Squeak. So, I am not sure whether there is actually support for int64_t on 32-bit Squeak in this regard...
So, am I looking for a bug or would this rather be a new feature? :-)
Best, Marcel
Am 11.04.2022 21:31:58 schrieb Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com [mailto:nicolas.cellier.aka.nice@gmail.com]>: Hi Marcel,
Sounds like 32bits -1 missing a sign extension when converted to 64bits...
At least, the result is correctly 64bits for sure.
Le lun. 11 avr. 2022 à 18:42, Marcel Taeumel a
écrit :
Hi all --
This is not working in 32-bit on Windows:
FFITestLibrary ffiTest8LongLongSum: -1 with: -1 with: -1 with: -1 with: -1
with: -1 with: -1 with: -1.
It answers " 34359738360 " instead of " - 8 ". It's working fine in a
64-bit build. The debugging output in this test functions says:
8 long longs came in as
i1 = 4294967295 (ffffffff)
i2 = 4294967295 (ffffffff)
i3 = 4294967295 (ffffffff)
i4 = 4294967295 (ffffffff)
i5 = 4294967295 (ffffffff)
i6 = 4294967295 (ffffffff)
i7 = 4294967295 (ffffffff)
i8 = 4294967295 (ffffffff)
... this is clearly wrong. Or is it? On 64-bit, the same function says:
8 long longs came in as
i1 = -1 (ffffffffffffffff)
i2 = -1 (ffffffffffffffff)
i3 = -1 (ffffffffffffffff)
i4 = -1 (ffffffffffffffff)
i5 = -1 (ffffffffffffffff)
i6 = -1 (ffffffffffffffff)
i7 = -1 (ffffffffffffffff)
i8 = -1 (ffffffffffffffff)
Both are formatted via " %lld (%llx) " I am missing something here... did
we change the representation of negative integers in Squeak and forgot to
update something along ThreadedFFIPlugin >> #ffiIntegerValueOf:? :-/
Best,
Marcel
EXPORT(long long) ffiTest8LongLongSum(long long c1, long long c2, long
long c3, long long c4, long long c5, long long c6, long long c7, long long
c8)
{
printf("8 long longs came in as\ni1 = %lld (%llx)\ni2 = %lld (%llx)\ni3 =
%lld (%llx)\ni4 = %lld (%llx)\ni5 = %lld (%llx)\ni6 = %lld (%llx)\ni7 =
%lld (%llx)\ni8 = %lld (%llx)\n", c1, c1, c2, c2, c3, c3, c4, c4, c5, c5,
c6, c6, c7, c7, c8, c8);
return c1 + c2 + c3 + c4 + c5 + c6 + c7 + c8;
}
Hi Marcel, Sounds like 32bits -1 missing a sign extension when converted to 64bits... At least, the result is correctly 64bits for sure.
Le lun. 11 avr. 2022 à 18:42, Marcel Taeumel <marcel.taeumel@hpi.de [mailto:marcel.taeumel@hpi.de]> a écrit :
Hi all --
This is not working in 32-bit on Windows:
FFITestLibrary ffiTest8LongLongSum: -1 with: -1 with: -1 with: -1 with: -1 with: -1 with: -1 with: -1.
It answers " 34359738360 " instead of " - 8 ". It's working fine in a 64-bit build. The debugging output in this test functions says:
8 long longs came in as i1 = 4294967295 (ffffffff) i2 = 4294967295 (ffffffff) i3 = 4294967295 (ffffffff) i4 = 4294967295 (ffffffff) i5 = 4294967295 (ffffffff) i6 = 4294967295 (ffffffff) i7 = 4294967295 (ffffffff) i8 = 4294967295 (ffffffff)
... this is clearly wrong. Or is it? On 64-bit, the same function says:
8 long longs came in as i1 = -1 (ffffffffffffffff) i2 = -1 (ffffffffffffffff) i3 = -1 (ffffffffffffffff) i4 = -1 (ffffffffffffffff) i5 = -1 (ffffffffffffffff) i6 = -1 (ffffffffffffffff) i7 = -1 (ffffffffffffffff) i8 = -1 (ffffffffffffffff)
Both are formatted via " %lld (%llx) " I am missing something here... did we change the representation of negative integers in Squeak and forgot to update something along ThreadedFFIPlugin >> #ffiIntegerValueOf:? :-/
Best, Marcel
***
EXPORT(long long) ffiTest8LongLongSum(long long c1, long long c2, long long c3, long long c4, long long c5, long long c6, long long c7, long long c8) { printf("8 long longs came in as\ni1 = %lld (%llx)\ni2 = %lld (%llx)\ni3 = %lld (%llx)\ni4 = %lld (%llx)\ni5 = %lld (%llx)\ni6 = %lld (%llx)\ni7 = %lld (%llx)\ni8 = %lld (%llx)\n", c1, c1, c2, c2, c3, c3, c4, c4, c5, c5, c6, c6, c7, c7, c8, c8); return c1 + c2 + c3 + c4 + c5 + c6 + c7 + c8; }