[Vm-dev] [Pharo-dev] FFI Struct Argument Pass By Value Fails on Mac 64 bit

Todd Blanchard tblanchard at mac.com
Tue Mar 6 19:55:21 UTC 2018


Silly me.

I have no idea how to update this library.  I thought it would be a file.  I've used Monticello years ago in Squeak - however trying to load this repository into Pharo 6.1-64 is impossible.  Invariably I end up with an undefined symbol and a couple of undismissable progress dialogs.

Also, its .c files that compile into a library, right?  Where would such files be in this repository?

I love Smalltalk.

I'm really starting to hate this though.

> On Mar 1, 2018, at 10:25 AM, todd blanchard <tblanchard at mac.com> wrote:
> 
> I'm mac osx 
> 
> I will update the test library and ping back
> 
>> On Mar 1, 2018, at 10:11 AM, Eliot Miranda <eliot.miranda at gmail.com <mailto:eliot.miranda at gmail.com>> wrote:
>> 
>> Todd,
>> 
>> On Thu, Mar 1, 2018 at 10:01 AM, Eliot Miranda <eliot.miranda at gmail.com <mailto:eliot.miranda at gmail.com>> wrote:
>> Hi Todd,
>> 
>> On Sun, Nov 26, 2017 at 8:24 AM, Todd Blanchard <tblanchard at mac.com <mailto:tblanchard at mac.com>> wrote:
>>  
>> i'm getting the idea that we should probably write a test suite/library for FFI
>> 
>> See the package FFI-Tests at http://source.squeak.org/FFI <http://source.squeak.org/FFI> and the file sqFFITestFuncs.c <http://sqffitestfuncs.c/>
>> 
>> I suggest you extend these with an example that demonstrates your failing case.  One thing to be very careful about is the platform.  Are you on Mac OS X, linux or Win64?  Mac OS X & linux use the SysV ABI (see e.g. https://software.intel.com/sites/default/files/article/402129/mpx-linux64-abi.pdf <https://software.intel.com/sites/default/files/article/402129/mpx-linux64-abi.pdf>) whereas Win64 uses Microsoft's own ABI (see https://msdn.microsoft.com/en-us/library/ms235286.aspx?f=255&MSPPError=-2147217396 <https://msdn.microsoft.com/en-us/library/ms235286.aspx?f=255&MSPPError=-2147217396>).  In particular under the Sys V ABI certain structs (structs having two 64-bit fields) will be passed in registers, if two registers are available at that point in the parameter list.  The ThreadedX64SysVFFIPlugin is written to pass structs in this manner but there may be bugs in the code to identify precisely those structs that fulfill the register passing criterion.  Microsoft's own ABI passes structs of size 8, 16, 32, or 64 bits and __m64 in a single register.  The ThreadedX64Win64FFIPlugin is written to pass structs in this manner but again there may be bugs in the code to identify precisely those structs that fulfill the register passing criterion.
>>  
>> 
>> HTH
>> 
>>> On Nov 24, 2017, at 12:54 AM, Ben Coman <btc at openinworld.com <mailto:btc at openinworld.com>> wrote:
>>> 
>>> 
>>> 
>>> On 24 November 2017 at 13:16, Ben Coman <btc at openinworld.com <mailto:btc at openinworld.com>> wrote:
>>> 
>>> 
>>> On 22 November 2017 at 21:59, Ben Coman <btc at openinworld.com <mailto:btc at openinworld.com>> wrote:
>>> 
>>> 
>>> On 22 November 2017 at 13:38, Todd Blanchard <tblanchard at mac.com <mailto:tblanchard at mac.com>> wrote:
>>>  
>>> I've been trying to track this down for a couple weeks now.
>>> 
>>> I have concluded that structs passed by value to functions on the 64 bit VM are not properly populated.  The struct's memory is all zero'd.
>>> 
>>> I found this while trying to work with LibClang and found that functions that fetched code locations from code ranges always returned invalid zero'd locations.  After spending some time with lldb I have traced the problem into the native code and found that the argument is not correct.
>>> 
>>> I've carved out the wee bit of clang to reproduce this in a tiny library.
>>> 
>>> The gist of it is below and the entire file is included.  Basically the struct passed to the function clang_getRangeStart is zero'd memory regardless of the data I send from the image side.
>>> 
>>> The build command I used on sierra is clang -shared -undefined dynamic_lookup -o microclang.dylib microclang.c
>>> 
>>> On Ubuntu 16.04 I used...
>>> $ clang -shared -fPIC -o libmicroclang.so microclang.c
>>> 
>>> $ clang test.c -L. -l microclang
>>>    test.c:6:53: error: no member named 'begin_int_data' in 'CXSourceLocation'
>>>    if(clang_getRangeStart(clang_getArbitraryRange()).begin_int_data == 0) 
>>> 
>>> I presume you meant...
>>>     if(clang_getRangeStart(clang_getArbitraryRange()).int_data == 0) 
>>> so correcting and continuing...
>>> 
>>> $ clang test.c -L. -l microclang
>>> $ LD_LIBRARY_PATH=. ./a.out
>>> That failed
>>> 
>>> So I'm not sure how to proceed.  
>>> I was expecting that would work while Pharo failed.
>>> 
>>> Now interestingly...
>>> $ clang test.c microlang.c
>>> $ ./a.out
>>> That worked
>>> 
>>> 
>>> So it seems a similar problem exists outside our FFI. 
>>> 
>>> cheers -ben
>>> 
>>> P.S. I refactored you code to extract a header file (attached)
>>> 
>>> The issue is still beyond my ken, but I've made some progress towards isolating/understanding the issue.
>>> Attached zip exploded here for easy reference...
>>> 
>>> 
>>> ___microlang.h___
>>> typedef unsigned  uintptr_t;
>>> 
>>> typedef struct {
>>>   const void *ptr_data[2];
>>> } CXSourceRange_;
>>> 
>>> CXSourceRange_ clang_getArbitraryRange_();
>>> int clang_getRangeEnd_(CXSourceRange_ range);
>>> 
>>> 
>>> 
>>> ___microclang.c___
>>> #include "microclang.h"
>>> const char* libraryString = "library_pointer";
>>> 
>>> CXSourceRange_ clang_getArbitraryRange_()
>>> {       CXSourceRange_ range = {0};
>>>         range.ptr_data[0] = (void*)libraryString;
>>>         return range;
>>> }
>>> 
>>> int clang_getRangeEnd_(CXSourceRange_ range)
>>> {       // Special decoding for CXSourceLocations for CXLoadedDiagnostics.
>>>         if ((uintptr_t)range.ptr_data[0] & 0x1)
>>>         {       return 0;       }
>>>         else
>>>         {       return 1;       }
>>> }
>>> 
>>> 
>>> 
>>> 
>>> ___test.c___
>>> #include <stdio.h>
>>> #include "microclang.h"
>>> const char* localString =  "local_pointer";
>>> 
>>> void test( CXSourceRange_ range, char *note )
>>> {       int result = clang_getRangeEnd_(range);
>>>         if(result == 0)
>>>         {       printf("That failed (%s)\n", note); }
>>>         else
>>>         {       printf("That worked (%s)\n", note); }
>>> }
>>> 
>>> int main()
>>> {       CXSourceRange_ range1 = clang_getArbitraryRange_();
>>>         test(range1, "library string");
>>> 
>>>         CXSourceRange_ range2 = {0};
>>>         range2.ptr_data[0] = (void*)localString;
>>>         test(range2, "local string");
>>> }
>>> 
>>> 
>>> 
>>> ___Makefile___
>>> default: clean static shared
>>> 
>>> clean:
>>>         rm -f *so *App
>>>         @echo
>>> 
>>> shared:
>>>         clang -g -o libmicroclang.so -shared -fPIC microclang.c
>>>         clang -g -o sharedApp test.c -L. -lmicroclang
>>>         LD_LIBRARY_PATH=. ./sharedApp
>>>         @echo
>>> 
>>> static:
>>>         clang -g -o staticApp test.c microclang.c
>>>         ./staticApp
>>>         @echo
>>> 
>>> 
>>> 
>>> Now running...
>>> $ make > report
>>> 
>>> gives...
>>> ___report___
>>> rm -f *so *App
>>> 
>>> clang -g -o staticApp test.c microclang.c
>>> ./staticApp
>>> That worked (library string)
>>> That worked (local string)
>>> 
>>> clang -g -o libmicroclang.so -shared -fPIC microclang.c
>>> clang -g -o sharedApp test.c -L. -lmicroclang
>>> LD_LIBRARY_PATH=. ./sharedApp
>>> That failed (library string)
>>> That worked (local string)
>>> 
>>> 
>>> Further simplification dealing *only* with strings (see attached sharedLibString.zip)
>>> (also attached is clang3.zip as a step along the way)
>>> 
>>> ___microclang.c___
>>> typedef unsigned  uintptr_t;
>>> const char* myLibraryString = "library_pointer";
>>> 
>>> const char * lib_getLibraryString()
>>> {       return myLibraryString;
>>> }
>>> 
>>> int lib_testString( const char *aString )
>>> {       unsigned test = (uintptr_t)aString & 0x1;
>>>         printf("\n  test=%d, aString-->%d\n", test, (uintptr_t)aString);
>>>         if (test)
>>>         {       return 0;       }
>>>         else
>>>         {       return 1;       }
>>> }
>>> 
>>> 
>>> ___test.c___
>>> #include <stdio.h>
>>> int lib_testString( const char *aString );
>>> const char *lib_getLibraryString();
>>> const char *localString =  "local_pointer";
>>> 
>>> void test( int result, char *note )
>>> {       if(result == 0)
>>>         {       printf("That failed (%s)\n", note); }
>>>         else 
>>>         {       printf("That worked (%s)\n", note); }
>>> }
>>> 
>>> int main()
>>> {       const char * libraryString = lib_getLibraryString();
>>>         test(lib_testString(libraryString), "library string");
>>> 
>>>         test(lib_testString(localString), "local string");
>>> }
>>> 
>>> 
>>> $ make > report
>>> 
>>> ___report___
>>> rm -f *so *App
>>> 
>>> clang -g -o staticApp test.c microclang.c
>>> ./staticApp
>>> 
>>>   test=0, aString-->4196150
>>>   That worked (library string)
>>> 
>>>   test=0, aString-->4196068
>>>   That worked (local string)
>>> 
>>> clang -g -o libmicroclang.so -shared -fPIC microclang.c
>>> clang -g -o sharedApp test.c -L. -lmicroclang
>>> LD_LIBRARY_PATH=. ./sharedApp
>>> 
>>>   test=1, aString-->-792512599
>>>   That failed (library string)
>>> 
>>>   test=0, aString-->4196484
>>>   That worked (local string)
>>> 
>>> 
>>> cheers -ben
>>> 
>>> <sharedLibString.zip><clang3.zip>
>> 
>> 
>> 
>> 
>> 
>> -- 
>> _,,,^..^,,,_
>> best, Eliot
>> 
>> 
>> 
>> -- 
>> _,,,^..^,,,_
>> best, Eliot
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20180306/1371e406/attachment-0001.html>


More information about the Vm-dev mailing list