[Vm-dev] [Pharo-dev] FFI Struct Argument Pass By Value Fails on Mac 64 bit

Eliot Miranda eliot.miranda at gmail.com
Thu Mar 1 18:11:02 UTC 2018


Todd,

On Thu, Mar 1, 2018 at 10:01 AM, Eliot Miranda <eliot.miranda at gmail.com>
wrote:

> Hi Todd,
>
> On Sun, Nov 26, 2017 at 8:24 AM, Todd Blanchard <tblanchard at mac.com>
> wrote:
>
>>
>> i'm getting the idea that we should probably write a test suite/library
>> for FFI
>>
>
> See the package FFI-Tests at http://source.squeak.org/FFI and the file
> sqFFITestFuncs.c
>

I suggest you extend these with an example that demonstrates your failing
case.  One thing to be very careful about is the platform.  Are you on Mac
OS X, linux or Win64?  Mac OS X & linux use the SysV ABI (see e.g.
https://software.intel.com/sites/default/files/article/402129/mpx-linux64-abi.pdf)
whereas Win64 uses Microsoft's own ABI (see
https://msdn.microsoft.com/en-us/library/ms235286.aspx?f=255&MSPPError=-2147217396).
In particular under the Sys V ABI certain structs (structs having two
64-bit fields) will be passed in registers, if two registers are available
at that point in the parameter list.  The ThreadedX64SysVFFIPlugin is
written to pass structs in this manner but there may be bugs in the code to
identify precisely those structs that fulfill the register passing
criterion.  Microsoft's own ABI passes structs of size 8, 16, 32, or 64
bits and __m64 in a single register.  The ThreadedX64Win64FFIPlugin is
written to pass structs in this manner but again there may be bugs in the
code to identify precisely those structs that fulfill the register passing
criterion.


HTH

On Nov 24, 2017, at 12:54 AM, Ben Coman <btc at openinworld.com> wrote:
>>
>>
>>
>> On 24 November 2017 at 13:16, Ben Coman <btc at openinworld.com> wrote:
>>
>>>
>>>
>>> On 22 November 2017 at 21:59, Ben Coman <btc at openinworld.com> wrote:
>>>
>>>>
>>>>
>>>> On 22 November 2017 at 13:38, Todd Blanchard <tblanchard at mac.com>
>>>> wrote:
>>>>
>>>>>
>>>>> I've been trying to track this down for a couple weeks now.
>>>>>
>>>>> I have concluded that structs passed by value to functions on the 64
>>>>> bit VM are not properly populated.  The struct's memory is all zero'd.
>>>>>
>>>>> I found this while trying to work with LibClang and found that
>>>>> functions that fetched code locations from code ranges always returned
>>>>> invalid zero'd locations.  After spending some time with lldb I have traced
>>>>> the problem into the native code and found that the argument is not correct.
>>>>>
>>>>> I've carved out the wee bit of clang to reproduce this in a tiny
>>>>> library.
>>>>>
>>>>> The gist of it is below and the entire file is included.  Basically
>>>>> the struct passed to the function clang_getRangeStart is zero'd memory
>>>>> regardless of the data I send from the image side.
>>>>>
>>>>> The build command I used on sierra is clang -shared -undefined
>>>>> dynamic_lookup -o microclang.dylib microclang.c
>>>>>
>>>>
>>>> On Ubuntu 16.04 I used...
>>>> $ clang -shared -fPIC -o libmicroclang.so microclang.c
>>>>
>>>> $ clang test.c -L. -l microclang
>>>>    test.c:6:53: error: no member named 'begin_int_data' in
>>>> 'CXSourceLocation'
>>>>    if(clang_getRangeStart(clang_getArbitraryRange()).begin_int_data ==
>>>> 0)
>>>>
>>>> I presume you meant...
>>>>     if(clang_getRangeStart(clang_getArbitraryRange()).int_data == 0)
>>>> so correcting and continuing...
>>>>
>>>> $ clang test.c -L. -l microclang
>>>> $ LD_LIBRARY_PATH=. ./a.out
>>>> That failed
>>>>
>>>> So I'm not sure how to proceed.
>>>> I was expecting that would work while Pharo failed.
>>>>
>>>> Now interestingly...
>>>> $ clang test.c microlang.c
>>>> $ ./a.out
>>>> That worked
>>>>
>>>>
>>>> So it seems a similar problem exists outside our FFI.
>>>>
>>>> cheers -ben
>>>>
>>>> P.S. I refactored you code to extract a header file (attached)
>>>>
>>>
>>> The issue is still beyond my ken, but I've made some progress towards
>>> isolating/understanding the issue.
>>> Attached zip exploded here for easy reference...
>>>
>>>
>>> ___microlang.h___
>>> typedef unsigned  uintptr_t;
>>>
>>> typedef struct {
>>>   const void *ptr_data[2];
>>> } CXSourceRange_;
>>>
>>> CXSourceRange_ clang_getArbitraryRange_();
>>> int clang_getRangeEnd_(CXSourceRange_ range);
>>>
>>>
>>>
>>> ___microclang.c___
>>> #include "microclang.h"
>>> const char* libraryString = "library_pointer";
>>>
>>> CXSourceRange_ clang_getArbitraryRange_()
>>> {       CXSourceRange_ range = {0};
>>>         range.ptr_data[0] = (void*)libraryString;
>>>         return range;
>>> }
>>>
>>> int clang_getRangeEnd_(CXSourceRange_ range)
>>> {       // Special decoding for CXSourceLocations for
>>> CXLoadedDiagnostics.
>>>         if ((uintptr_t)range.ptr_data[0] & 0x1)
>>>         {       return 0;       }
>>>         else
>>>         {       return 1;       }
>>> }
>>>
>>>
>>>
>>>
>>> ___test.c___
>>> #include <stdio.h>
>>> #include "microclang.h"
>>> const char* localString =  "local_pointer";
>>>
>>> void test( CXSourceRange_ range, char *note )
>>> {       int result = clang_getRangeEnd_(range);
>>>         if(result == 0)
>>>         {       printf("That failed (%s)\n", note); }
>>>         else
>>>         {       printf("That worked (%s)\n", note); }
>>> }
>>>
>>> int main()
>>> {       CXSourceRange_ range1 = clang_getArbitraryRange_();
>>>         test(range1, "library string");
>>>
>>>         CXSourceRange_ range2 = {0};
>>>         range2.ptr_data[0] = (void*)localString;
>>>         test(range2, "local string");
>>> }
>>>
>>>
>>>
>>> ___Makefile___
>>> default: clean static shared
>>>
>>> clean:
>>>         rm -f *so *App
>>>         @echo
>>>
>>> shared:
>>>         clang -g -o libmicroclang.so -shared -fPIC microclang.c
>>>         clang -g -o sharedApp test.c -L. -lmicroclang
>>>         LD_LIBRARY_PATH=. ./sharedApp
>>>         @echo
>>>
>>> static:
>>>         clang -g -o staticApp test.c microclang.c
>>>         ./staticApp
>>>         @echo
>>>
>>>
>>>
>>> Now running...
>>> $ make > report
>>>
>>> gives...
>>> ___report___
>>> rm -f *so *App
>>>
>>> clang -g -o staticApp test.c microclang.c
>>> ./staticApp
>>> That worked (library string)
>>> That worked (local string)
>>>
>>> clang -g -o libmicroclang.so -shared -fPIC microclang.c
>>> clang -g -o sharedApp test.c -L. -lmicroclang
>>> LD_LIBRARY_PATH=. ./sharedApp
>>> That failed (library string)
>>> That worked (local string)
>>>
>>
>>
>> Further simplification dealing *only* with strings (see attached
>> sharedLibString.zip)
>> (also attached is clang3.zip as a step along the way)
>>
>> ___microclang.c___
>> typedef unsigned  uintptr_t;
>> const char* myLibraryString = "library_pointer";
>>
>> const char * lib_getLibraryString()
>> {       return myLibraryString;
>> }
>>
>> int lib_testString( const char *aString )
>> {       unsigned test = (uintptr_t)aString & 0x1;
>>         printf("\n  test=%d, aString-->%d\n", test, (uintptr_t)aString);
>>         if (test)
>>         {       return 0;       }
>>         else
>>         {       return 1;       }
>> }
>>
>>
>> ___test.c___
>> #include <stdio.h>
>> int lib_testString( const char *aString );
>> const char *lib_getLibraryString();
>> const char *localString =  "local_pointer";
>>
>> void test( int result, char *note )
>> {       if(result == 0)
>>         {       printf("That failed (%s)\n", note); }
>>         else
>>         {       printf("That worked (%s)\n", note); }
>> }
>>
>> int main()
>> {       const char * libraryString = lib_getLibraryString();
>>         test(lib_testString(libraryString), "library string");
>>
>>         test(lib_testString(localString), "local string");
>> }
>>
>>
>> $ make > report
>>
>> ___report___
>> rm -f *so *App
>>
>> clang -g -o staticApp test.c microclang.c
>> ./staticApp
>>
>>   test=0, aString-->4196150
>>   That worked (library string)
>>
>>   test=0, aString-->4196068
>>   That worked (local string)
>>
>> clang -g -o libmicroclang.so -shared -fPIC microclang.c
>> clang -g -o sharedApp test.c -L. -lmicroclang
>> LD_LIBRARY_PATH=. ./sharedApp
>>
>>   test=1, aString-->-792512599
>>   That failed (library string)
>>
>>   test=0, aString-->4196484
>>   That worked (local string)
>>
>>
>> cheers -ben
>>
>> <sharedLibString.zip><clang3.zip>
>>
>>
>>
>>
>
>
> --
> _,,,^..^,,,_
> best, Eliot
>



-- 
_,,,^..^,,,_
best, Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20180301/aae3984e/attachment-0001.html>


More information about the Vm-dev mailing list