<html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">i'm getting the idea that we should probably write a test suite/library for FFI<div class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Nov 24, 2017, at 12:54 AM, Ben Coman <<a href="mailto:btc@openinworld.com" class="">btc@openinworld.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div class="gmail_extra"><br class="Apple-interchange-newline"><br class=""><div class="gmail_quote">On 24 November 2017 at 13:16, Ben Coman<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:btc@openinworld.com" target="_blank" class="">btc@openinworld.com</a>></span><span class="Apple-converted-space"> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div dir="ltr" class=""><br class=""><div class="gmail_extra"><br class=""><div class="gmail_quote"><div class=""><div class="gmail-h5">On 22 November 2017 at 21:59, Ben Coman<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:btc@openinworld.com" target="_blank" class="">btc@openinworld.com</a>></span><span class="Apple-converted-space"> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div dir="ltr" class=""><br class=""><div class="gmail_extra"><br class=""><div class="gmail_quote"><span class="gmail-m_8524537758069216380gmail-m_-4929630517411459228gmail-">On 22 November 2017 at 13:38, Todd Blanchard<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:tblanchard@mac.com" target="_blank" class="">tblanchard@mac.com</a>></span><span class="Apple-converted-space"> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"> <br class=""><div style="word-wrap: break-word;" class="">I've been trying to track this down for a couple weeks now.<div class=""><br class=""></div><div class="">I have concluded that structs passed by value to functions on the 64 bit VM are not properly populated. The struct's memory is all zero'd.</div><div class=""><br class=""></div><div class="">I found this while trying to work with LibClang and found that functions that fetched code locations from code ranges always returned invalid zero'd locations. After spending some time with lldb I have traced the problem into the native code and found that the argument is not correct.</div><div class=""><br class=""></div><div class="">I've carved out the wee bit of clang to reproduce this in a tiny library.</div><div class=""><br class=""></div><div class="">The gist of it is below and the entire file is included. Basically the struct passed to the function clang_getRangeStart is zero'd memory regardless of the data I send from the image side.</div><div class=""><br class=""></div><div class="">The build command I used on sierra is <span style="font-family: Menlo; font-size: 11px; background-color: rgb(255, 255, 255);" class="">clang -shared -undefined dynamic_lookup -o microclang.dylib microclang.c</span></div></div></blockquote><div class=""><br class=""></div></span><div class="">On Ubuntu 16.04 I used...</div><div class=""><div class="">$ clang -shared -fPIC -o libmicroclang.so microclang.c</div></div><div class=""><div class=""><br class=""></div></div><div class="">$ clang test.c -L. -l microclang</div><div class=""><div class=""><div class=""> test.c:6:53: error: no member named 'begin_int_data' in 'CXSourceLocation'</div><div class=""> if(clang_getRangeStart(clang_<wbr class="">getArbitraryRange()).begin_int<wbr class="">_data == 0) </div></div></div><div class=""><br class=""></div><div class="">I presume you meant...</div><div class=""> if(clang_getRangeStart(clang<wbr class="">_getArbitraryRange()).int_data == 0) </div><div class="">so correcting and continuing...</div><div class=""><br class=""></div><div class="">$ clang test.c -L. -l microclang</div><div class=""><div class="">$ LD_LIBRARY_PATH=. ./a.out</div></div><div class=""><div class="">That failed</div></div><div class=""><br class=""></div><div class="">So I'm not sure how to proceed. </div><div class="">I was expecting that would work while Pharo failed.</div><div class=""><br class=""></div><div class="">Now interestingly...</div><div class=""><div class="">$ clang test.c microlang.c</div><div class="">$ ./a.out</div><div class=""></div></div><div class="">That worked</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">So it seems a similar problem exists outside our FFI. </div><div class=""><br class=""></div><div class="">cheers -ben</div><div class=""><br class=""></div><div class="">P.S. I refactored you code to extract a header file (attached)</div></div></div></div></blockquote><div class=""><br class=""></div></div></div><div class="">The issue is still beyond my ken, but I've made some progress towards isolating/understanding the issue.</div><div class="">Attached zip exploded here for easy reference...</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">___microlang.h___</div><div class=""><div class="">typedef unsigned uintptr_t;</div><span class="gmail-"><div class=""><br class=""></div><div class="">typedef struct {</div><div class=""> <span class="Apple-converted-space"> </span>const void *ptr_data[2];</div></span><div class="">} CXSourceRange_;</div><div class=""><br class=""></div><div class="">CXSourceRange_ clang_getArbitraryRange_();</div><div class="">int clang_getRangeEnd_(<wbr class="">CXSourceRange_ range);</div></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">___microclang.c___</div><div class=""><div class="">#include "microclang.h"</div><div class="">const char* libraryString = "library_pointer";</div><div class=""><br class=""></div><div class="">CXSourceRange_ clang_getArbitraryRange_()</div><div class="">{ CXSourceRange_ range = {0};</div><div class=""> <span class="Apple-converted-space"> </span>range.ptr_data[0] = (void*)libraryString;</div><div class=""> <span class="Apple-converted-space"> </span>return range;</div><div class="">}</div><div class=""><br class=""></div><div class="">int clang_getRangeEnd_(<wbr class="">CXSourceRange_ range)</div><span class="gmail-"><div class="">{ // Special decoding for CXSourceLocations for CXLoadedDiagnostics.</div><div class=""> <span class="Apple-converted-space"> </span>if ((uintptr_t)range.ptr_data[0] & 0x1)</div></span><div class=""> <span class="Apple-converted-space"> </span>{ return 0; }</div><div class=""> <span class="Apple-converted-space"> </span>else</div><div class=""> <span class="Apple-converted-space"> </span>{ return 1; }</div><div class="">}</div></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">___test.c___</div><div class=""><div class="">#include <stdio.h></div><div class="">#include "microclang.h"</div><div class="">const char* localString = "local_pointer";</div><div class=""><br class=""></div><div class="">void test( CXSourceRange_ range, char *note )</div><div class="">{ int result = clang_getRangeEnd_(range);</div><div class=""> <span class="Apple-converted-space"> </span>if(result == 0)</div><div class=""> <span class="Apple-converted-space"> </span>{ printf("That failed (%s)\n", note); }</div><div class=""> <span class="Apple-converted-space"> </span>else</div><div class=""> <span class="Apple-converted-space"> </span>{ printf("That worked (%s)\n", note); }</div><div class="">}</div><div class=""><br class=""></div><div class="">int main()</div><div class="">{ CXSourceRange_ range1 = clang_getArbitraryRange_();</div><div class=""> <span class="Apple-converted-space"> </span>test(range1, "library string");</div><div class=""><br class=""></div><div class=""> <span class="Apple-converted-space"> </span>CXSourceRange_ range2 = {0};</div><div class=""> <span class="Apple-converted-space"> </span>range2.ptr_data[0] = (void*)localString;</div><div class=""> <span class="Apple-converted-space"> </span>test(range2, "local string");</div><div class="">}</div><div class=""><br class=""></div></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">___Makefile___</div><div class=""><div class="">default: clean static shared</div><div class=""><br class=""></div><div class="">clean:</div><div class=""> <span class="Apple-converted-space"> </span>rm -f *so *App</div><div class=""> <span class="Apple-converted-space"> </span>@echo</div><div class=""><br class=""></div><div class="">shared:</div><div class=""> <span class="Apple-converted-space"> </span>clang -g -o libmicroclang.so -shared -fPIC microclang.c</div><div class=""> <span class="Apple-converted-space"> </span>clang -g -o sharedApp test.c -L. -lmicroclang</div><div class=""> <span class="Apple-converted-space"> </span>LD_LIBRARY_PATH=. ./sharedApp</div><div class=""> <span class="Apple-converted-space"> </span>@echo</div><div class=""><br class=""></div><div class="">static:</div><div class=""> <span class="Apple-converted-space"> </span>clang -g -o staticApp test.c microclang.c</div><div class=""> <span class="Apple-converted-space"> </span>./staticApp</div><div class=""> <span class="Apple-converted-space"> </span>@echo</div></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">Now running...</div><div class="">$ make > report</div><div class=""><br class=""></div><div class="">gives...</div><div class="">___report___<br class=""></div><div class=""><div class="">rm -f *so *App</div><div class=""><br class=""></div><div class="">clang -g -o staticApp test.c microclang.c</div><div class="">./staticApp</div><div class="">That worked (library string)</div><div class="">That worked (local string)</div><div class=""><br class=""></div><div class="">clang -g -o libmicroclang.so -shared -fPIC microclang.c</div><div class="">clang -g -o sharedApp test.c -L. -lmicroclang</div><div class="">LD_LIBRARY_PATH=. ./sharedApp</div><div class="">That failed (library string)</div><div class="">That worked (local string)</div><div class=""></div></div></div></div></div></blockquote><div class=""><br class=""></div><div class=""><br class=""></div><div class="">Further simplification dealing *only* with strings (see attached sharedLibString.zip)</div><div class="">(also attached is clang3.zip as a step along the way)</div><div class=""><br class=""></div><div class="">___microclang.c___<br class=""></div><div class=""><div class="">typedef unsigned uintptr_t;</div><div class="">const char* myLibraryString = "library_pointer";</div><div class=""><br class=""></div><div class="">const char * lib_getLibraryString()</div><div class="">{ return myLibraryString;</div><div class="">}</div><div class=""><br class=""></div><div class=""><div class="">int lib_testString( const char *aString )</div><div class="">{ unsigned test = (uintptr_t)aString & 0x1;</div><div class=""> printf("\n test=%d, aString-->%d\n", test, (uintptr_t)aString);</div><div class=""> if (test)</div><div class=""> { return 0; }</div><div class=""> else</div><div class=""> { return 1; }</div><div class="">}</div></div></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">___test.c___<br class=""></div><div class=""><div class="">#include <stdio.h></div><div class="">int lib_testString( const char *aString );</div><div class="">const char *lib_getLibraryString();</div><div class="">const char *localString = "local_pointer";</div><div class=""><br class=""></div><div class="">void test( int result, char *note )</div><div class="">{ if(result == 0)</div><div class=""> { printf("That failed (%s)\n", note); }</div><div class=""> else </div><div class=""> { printf("That worked (%s)\n", note); }</div><div class="">}</div><div class=""><br class=""></div><div class="">int main()</div><div class="">{ const char * libraryString = lib_getLibraryString();</div><div class=""> test(lib_testString(libraryString), "library string");</div><div class=""><br class=""></div><div class=""> test(lib_testString(localString), "local string");</div><div class="">}</div></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">$ make > report<br class=""></div><div class=""><br class=""></div><div class="">___report___<br class=""></div><div class=""><div class="">rm -f *so *App</div><div class=""><br class=""></div><div class="">clang -g -o staticApp test.c microclang.c</div><div class="">./staticApp</div><div class=""><br class=""></div><div class=""> test=0, aString-->4196150</div><div class=""> That worked (library string)</div><div class=""><br class=""></div><div class=""> test=0, aString-->4196068</div><div class=""> That worked (local string)</div><div class=""><br class=""></div><div class="">clang -g -o libmicroclang.so -shared -fPIC microclang.c</div><div class="">clang -g -o sharedApp test.c -L. -lmicroclang</div><div class="">LD_LIBRARY_PATH=. ./sharedApp</div><div class=""><br class=""></div><div class=""> test=1, aString-->-792512599</div><div class=""> That failed (library string)</div><div class=""><br class=""></div><div class=""> test=0, aString-->4196484</div><div class=""> That worked (local string)</div></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">cheers -ben</div><div class=""><br class=""></div></div></div></div><span id="cid:7C7686F0-08CC-4B97-A9E0-D60CECDF7A79"><sharedLibString.zip></span><span id="cid:13716AE6-42AA-48FB-BE5C-63E412916E9B"><clang3.zip></span></div></blockquote></div><br class=""></div></body></html>