[Vm-dev] [Pharo-dev] FFI Struct Argument Pass By Value Fails on Mac 64 bit

[Todd Blanchard]

i'm getting the idea that we should probably write a test suite/library for FFI

> On Nov 24, 2017, at 12:54 AM, Ben Coman <btc at openinworld.com> wrote:
> 
> 
> 
> On 24 November 2017 at 13:16, Ben Coman <btc at openinworld.com <mailto:btc at openinworld.com>> wrote:
> 
> 
> On 22 November 2017 at 21:59, Ben Coman <btc at openinworld.com <mailto:btc at openinworld.com>> wrote:
> 
> 
> On 22 November 2017 at 13:38, Todd Blanchard <tblanchard at mac.com <mailto:tblanchard at mac.com>> wrote:
>  
> I've been trying to track this down for a couple weeks now.
> 
> I have concluded that structs passed by value to functions on the 64 bit VM are not properly populated.  The struct's memory is all zero'd.
> 
> I found this while trying to work with LibClang and found that functions that fetched code locations from code ranges always returned invalid zero'd locations.  After spending some time with lldb I have traced the problem into the native code and found that the argument is not correct.
> 
> I've carved out the wee bit of clang to reproduce this in a tiny library.
> 
> The gist of it is below and the entire file is included.  Basically the struct passed to the function clang_getRangeStart is zero'd memory regardless of the data I send from the image side.
> 
> The build command I used on sierra is clang -shared -undefined dynamic_lookup -o microclang.dylib microclang.c
> 
> On Ubuntu 16.04 I used...
> $ clang -shared -fPIC -o libmicroclang.so microclang.c
> 
> $ clang test.c -L. -l microclang
>    test.c:6:53: error: no member named 'begin_int_data' in 'CXSourceLocation'
>    if(clang_getRangeStart(clang_getArbitraryRange()).begin_int_data == 0) 
> 
> I presume you meant...
>     if(clang_getRangeStart(clang_getArbitraryRange()).int_data == 0) 
> so correcting and continuing...
> 
> $ clang test.c -L. -l microclang
> $ LD_LIBRARY_PATH=. ./a.out
> That failed
> 
> So I'm not sure how to proceed.  
> I was expecting that would work while Pharo failed.
> 
> Now interestingly...
> $ clang test.c microlang.c
> $ ./a.out
> That worked
> 
> 
> So it seems a similar problem exists outside our FFI. 
> 
> cheers -ben
> 
> P.S. I refactored you code to extract a header file (attached)
> 
> The issue is still beyond my ken, but I've made some progress towards isolating/understanding the issue.
> Attached zip exploded here for easy reference...
> 
> 
> ___microlang.h___
> typedef unsigned  uintptr_t;
> 
> typedef struct {
>   const void *ptr_data[2];
> } CXSourceRange_;
> 
> CXSourceRange_ clang_getArbitraryRange_();
> int clang_getRangeEnd_(CXSourceRange_ range);
> 
> 
> 
> ___microclang.c___
> #include "microclang.h"
> const char* libraryString = "library_pointer";
> 
> CXSourceRange_ clang_getArbitraryRange_()
> {       CXSourceRange_ range = {0};
>         range.ptr_data[0] = (void*)libraryString;
>         return range;
> }
> 
> int clang_getRangeEnd_(CXSourceRange_ range)
> {       // Special decoding for CXSourceLocations for CXLoadedDiagnostics.
>         if ((uintptr_t)range.ptr_data[0] & 0x1)
>         {       return 0;       }
>         else
>         {       return 1;       }
> }
> 
> 
> 
> 
> ___test.c___
> #include <stdio.h>
> #include "microclang.h"
> const char* localString =  "local_pointer";
> 
> void test( CXSourceRange_ range, char *note )
> {       int result = clang_getRangeEnd_(range);
>         if(result == 0)
>         {       printf("That failed (%s)\n", note); }
>         else
>         {       printf("That worked (%s)\n", note); }
> }
> 
> int main()
> {       CXSourceRange_ range1 = clang_getArbitraryRange_();
>         test(range1, "library string");
> 
>         CXSourceRange_ range2 = {0};
>         range2.ptr_data[0] = (void*)localString;
>         test(range2, "local string");
> }
> 
> 
> 
> ___Makefile___
> default: clean static shared
> 
> clean:
>         rm -f *so *App
>         @echo
> 
> shared:
>         clang -g -o libmicroclang.so -shared -fPIC microclang.c
>         clang -g -o sharedApp test.c -L. -lmicroclang
>         LD_LIBRARY_PATH=. ./sharedApp
>         @echo
> 
> static:
>         clang -g -o staticApp test.c microclang.c
>         ./staticApp
>         @echo
> 
> 
> 
> Now running...
> $ make > report
> 
> gives...
> ___report___
> rm -f *so *App
> 
> clang -g -o staticApp test.c microclang.c
> ./staticApp
> That worked (library string)
> That worked (local string)
> 
> clang -g -o libmicroclang.so -shared -fPIC microclang.c
> clang -g -o sharedApp test.c -L. -lmicroclang
> LD_LIBRARY_PATH=. ./sharedApp
> That failed (library string)
> That worked (local string)
> 
> 
> Further simplification dealing *only* with strings (see attached sharedLibString.zip)
> (also attached is clang3.zip as a step along the way)
> 
> ___microclang.c___
> typedef unsigned  uintptr_t;
> const char* myLibraryString = "library_pointer";
> 
> const char * lib_getLibraryString()
> {       return myLibraryString;
> }
> 
> int lib_testString( const char *aString )
> {       unsigned test = (uintptr_t)aString & 0x1;
>         printf("\n  test=%d, aString-->%d\n", test, (uintptr_t)aString);
>         if (test)
>         {       return 0;       }
>         else
>         {       return 1;       }
> }
> 
> 
> ___test.c___
> #include <stdio.h>
> int lib_testString( const char *aString );
> const char *lib_getLibraryString();
> const char *localString =  "local_pointer";
> 
> void test( int result, char *note )
> {       if(result == 0)
>         {       printf("That failed (%s)\n", note); }
>         else 
>         {       printf("That worked (%s)\n", note); }
> }
> 
> int main()
> {       const char * libraryString = lib_getLibraryString();
>         test(lib_testString(libraryString), "library string");
> 
>         test(lib_testString(localString), "local string");
> }
> 
> 
> $ make > report
> 
> ___report___
> rm -f *so *App
> 
> clang -g -o staticApp test.c microclang.c
> ./staticApp
> 
>   test=0, aString-->4196150
>   That worked (library string)
> 
>   test=0, aString-->4196068
>   That worked (local string)
> 
> clang -g -o libmicroclang.so -shared -fPIC microclang.c
> clang -g -o sharedApp test.c -L. -lmicroclang
> LD_LIBRARY_PATH=. ./sharedApp
> 
>   test=1, aString-->-792512599
>   That failed (library string)
> 
>   test=0, aString-->4196484
>   That worked (local string)
> 
> 
> cheers -ben
> 
> <sharedLibString.zip><clang3.zip>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20171126/9253658c/attachment-0001.html>