[Vm-dev] [Pharo-dev] FFI Struct Argument Pass By Value Fails on Mac 64 bit
Todd Blanchard
tblanchard at mac.com
Thu Mar 1 17:40:02 UTC 2018
I got pulled off to do some $ work and haven't got back to this but I would like to get back to it.
Has there been any progress on this in the publicly available 64bit VM?
I see there is a completely separate FFI plugin for 32 vs 64 and worry 64 is still not quite ready for real work.
> On Dec 3, 2017, at 7:18 AM, Ben Coman <btc at openinworld.com> wrote:
>
>
>
> On 27 November 2017 at 00:24, Todd Blanchard <tblanchard at mac.com <mailto:tblanchard at mac.com>> wrote:
>
> i'm getting the idea that we should probably write a test suite/library for FFI
>
> I noticed these...
> * https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/unix/plugins/SqueakFFIPrims/ffi-test-main.c <https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/unix/plugins/SqueakFFIPrims/ffi-test-main.c>
> * https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/Cross/plugins/SqueakFFIPrims/sqFFITestFuncs.c <https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/Cross/plugins/SqueakFFIPrims/sqFFITestFuncs.c>
> * https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/Cross/plugins/IA32ABI/AlienSUnitTestProcedures.c <https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/Cross/plugins/IA32ABI/AlienSUnitTestProcedures.c>
>
> These look like c-code test frames for FFI. Could these be built by the OpenSmalltalk CI to be normally shipped with the VM so that Image-side CI can test against them?
>
> And also some general info...
> * https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/unix/plugins/SqueakFFIPrims/00README <https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/unix/plugins/SqueakFFIPrims/00README>
>
>
>
>> On Nov 24, 2017, at 12:54 AM, Ben Coman <btc at openinworld.com <mailto:btc at openinworld.com>> wrote:
>>
>>
>> On 24 November 2017 at 13:16, Ben Coman <btc at openinworld.com <mailto:btc at openinworld.com>> wrote:
>>
>>
>> On 22 November 2017 at 21:59, Ben Coman <btc at openinworld.com <mailto:btc at openinworld.com>> wrote:
>>
>>
>> On 22 November 2017 at 13:38, Todd Blanchard <tblanchard at mac.com <mailto:tblanchard at mac.com>> wrote:
>>
>> I've been trying to track this down for a couple weeks now.
>
>>
>> I have concluded that structs passed by value to functions on the 64 bit VM are not properly populated. The struct's memory is all zero'd.
>>
>> I found this while trying to work with LibClang and found that functions that fetched code locations from code ranges always returned invalid zero'd locations. After spending some time with lldb I have traced the problem into the native code and found that the argument is not correct.
>>
>> I've carved out the wee bit of clang to reproduce this in a tiny library.
>>
>> The gist of it is below and the entire file is included. Basically the struct passed to the function clang_getRangeStart is zero'd memory regardless of the data I send from the image side.
>
>
> My last analysis discovered something interesting about strings defined inside shared libraries being handled differently, but only later realised I had chased the wrong rabbit down the hole.
>
> I've now investigated the premise you actually poses, and I agree, that structs being zero is some cases.
> The attached zipfile containing libstruct.c a few comparison cases - three "good" layouts that work fine and one "bad" that mostly gets zeros but sometimes other weird numbers. There is a Makefile with three main targets:
> 1. make layout - statically compiles libstruct.c and runs produced a.out to display structure layouts
>
> 2. make run - downloads Pharo, starts it loading LibStruct.st <http://libstruct.st/> , then you manually run LibStruct>>>LibStructTest>>#testStructs and observe structure values on console
>
> 3. make debug - starts LLDB to run Pharo with breakpoints pre-configured for when you run #testStructs.
> Note Pharo will freeze and you need to move to LLDB. Try these commands...
> frame variable
> call print_struct(&GoodStruct1_fmt, &aStruct)
> continue
>
> That works on Ubuntu 16.04 64 bit.
> You will need to tune it for OSX.
> clang and lldb are required.
>
> ================================
> The offsets configured in the class variables of all ExternalStructs
> correctly matches that reported by the C code test frame results here...
>
> $ make layout
> clang -g libstruct.c
> ./a.out
>
> GoodStruct1:
> uint32_t:int1: 01 4
> uint32_t:int2: 05 4
> GoodStruct2:
> uint32_t:int1: 01 4
> uint32_t:int2: 05 4
> void*:ptr_data: 09 8
> GoodStruct3:
> void*:ptr_data: 01 8
> uint32_t:int1: 09 4
> uint32_t:int2: 13 4
> BadStruct:
> void*:ptr_data1: 01 8
> void*:ptr_data2: 09 8
> uint32_t:int1: 17 4
> uint32_t:int2: 21 4
>
> ================================
>
> $ make run
> clang -g -o libstruct.so -shared -fPIC libstruct.c
> getpharo/pharo-vm/lib/pharo/5.0-201707201942/pharo getpharo/Pharo.image ../LibStruct.st <http://libstruct.st/>
>
> Now manually browse to and run LibStruct>>>LibStructTest>>#testStructs
> LibStructTest>>testStructs
> |good1Struct good2Struct good3Struct badStruct |
>
> good1Struct := GoodStruct1 new int1: 2; int2: 3.
> good2Struct := GoodStruct2 new int1: 2; int2: 3; ptr_force_int: 4.
> good3Struct := GoodStruct3 new int1: 2; int2: 3; ptr_force_int: 4.
> badStruct := BadStruct new int1: 2; int2: 3; ptr_force_int1: 4; ptr_force_int1: 5.
>
> self assert: (LibStruct tryGood1: good1Struct) equals: 6.
> self assert: (LibStruct tryGood2: good2Struct) equals: 6.
> self assert: (LibStruct tryGood3: good3Struct) equals: 6.
> "Problem exposed in next line"
> self assert: (LibStruct tryBad: badStruct) equals: 6.
>
> which on console produces...
>
> GoodStruct1:
> uint32_t:int1: 01 4 = 02 00 00 00
> uint32_t:int2: 05 4 = 03 00 00 00
> GoodStruct2:
> uint32_t:int1: 01 4 = 02 00 00 00
> uint32_t:int2: 05 4 = 03 00 00 00
> void*:ptr_data: 09 8 = 04 00 00 00 00 00 00 00
> GoodStruct3:
> void*:ptr_data: 01 8 = 04 00 00 00 00 00 00 00
> uint32_t:int1: 09 4 = 02 00 00 00
> uint32_t:int2: 13 4 = 03 00 00 00
> BadStruct:
> void*:ptr_data1: 01 8 = 00 00 00 00 00 00 00 00
> void*:ptr_data2: 09 8 = 00 00 00 00 00 00 00 00
> uint32_t:int1: 17 4 = 29 03 03 00
> uint32_t:int2: 21 4 = 00 00 00 00
>
> Comparing GoodStruct3 and BadStruct, it seems one pointer is handled fine, but not two.
> The first time this is run after Image boots seems like BadStruct gets some random data.
>
> ================================
> In same image, subsequent runs of LibStruct>>>LibStructTest>>testStructs
> give only all zeros for BadStruct, as Todd observed.
>
> GoodStruct1:
> uint32_t:int1: 01 4 = 02 00 00 00
> uint32_t:int2: 05 4 = 03 00 00 00
> GoodStruct2:
> uint32_t:int1: 01 4 = 02 00 00 00
> uint32_t:int2: 05 4 = 03 00 00 00
> void*:ptr_data: 09 8 = 00 00 00 00 00 00 00 00
> GoodStruct3:
> void*:ptr_data: 01 8 = 00 00 00 00 00 00 00 00
> uint32_t:int1: 09 4 = 02 00 00 00
> uint32_t:int2: 13 4 = 03 00 00 00
> BadStruct:
> void*:ptr_data1: 01 8 = 00 00 00 00 00 00 00 00
> void*:ptr_data2: 09 8 = 00 00 00 00 00 00 00 00
> uint32_t:int1: 17 4 = 00 00 00 00
> uint32_t:int2: 21 4 = 00 00 00 00
>
>
> cheers -ben
>
> <FFI64StructTest.zip>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20180301/28d6f2e7/attachment-0001.html>
More information about the Vm-dev
mailing list