<html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">I got pulled off to do some $ work and haven't got back to this but I would like to get back to it.<div class=""><br class=""></div><div class="">Has there been any progress on this in the publicly available 64bit VM?</div><div class=""><br class=""></div><div class="">I see there is a completely separate FFI plugin for 32 vs 64 and worry 64 is still not quite ready for real work.</div><div class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Dec 3, 2017, at 7:18 AM, Ben Coman <<a href="mailto:btc@openinworld.com" class="">btc@openinworld.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div class="gmail_extra"><br class="Apple-interchange-newline"><br class=""><div class="gmail_quote">On 27 November 2017 at 00:24, Todd Blanchard<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:tblanchard@mac.com" target="_blank" class="">tblanchard@mac.com</a>></span><span class="Apple-converted-space"> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"> <br class=""><div style="word-wrap: break-word;" class="">i'm getting the idea that we should probably write a test suite/library for FFI</div></blockquote><div class=""><br class=""></div><div class="">I noticed these...</div><div class="">*<span class="Apple-converted-space"> </span><a href="https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/unix/plugins/SqueakFFIPrims/ffi-test-main.c" target="_blank" class="">https://github.com/OpenSmallta<wbr class="">lk/opensmalltalk-vm/blob/Cog/<wbr class="">platforms/unix/plugins/<wbr class="">SqueakFFIPrims/ffi-test-main.c</a><br class=""></div><div class="">* <a href="https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/Cross/plugins/SqueakFFIPrims/sqFFITestFuncs.c" target="_blank" class="">https://github.com/OpenSmall<wbr class="">talk/opensmalltalk-vm/blob/<wbr class="">Cog/platforms/Cross/plugins/<wbr class="">SqueakFFIPrims/sqFFITestFuncs.<wbr class="">c</a></div><div class="">* <a href="https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/Cross/plugins/IA32ABI/AlienSUnitTestProcedures.c" target="_blank" class="">https://github.com/OpenSmall<wbr class="">talk/opensmalltalk-vm/blob/<wbr class="">Cog/platforms/Cross/plugins/<wbr class="">IA32ABI/AlienSUnitTestProcedur<wbr class="">es.c</a></div><div class=""><br class=""></div><div class="">These look like c-code test frames for FFI. Could these be built by the OpenSmalltalk CI to be normally shipped with the VM so that Image-side CI can test against them?</div><div class=""><br class=""></div><div class="">And also some general info...</div><div class="">* <a href="https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/unix/plugins/SqueakFFIPrims/00README" target="_blank" class="">https://github.com/OpenSmall<wbr class="">talk/opensmalltalk-vm/blob/<wbr class="">Cog/platforms/unix/plugins/<wbr class="">SqueakFFIPrims/00README</a></div><div class=""><br class=""></div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="word-wrap: break-word;" class=""><div class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="">On Nov 24, 2017, at 12:54 AM, Ben Coman <<a href="mailto:btc@openinworld.com" target="_blank" class="">btc@openinworld.com</a>> wrote:</div><div class=""><div dir="ltr" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class=""><div class="gmail_extra"><br class="gmail-m_-4607556448884927622m_-3452065927662814111gmail-m_7527885605307787862Apple-interchange-newline"><br class=""><div class="gmail_quote">On 24 November 2017 at 13:16, Ben Coman<span class="gmail-m_-4607556448884927622m_-3452065927662814111gmail-m_7527885605307787862Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:btc@openinworld.com" target="_blank" class="">btc@openinworld.com</a>></span><span class="gmail-m_-4607556448884927622m_-3452065927662814111gmail-m_7527885605307787862Apple-converted-space"> </span>wr<wbr class="">ote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div dir="ltr" class=""><br class=""><div class="gmail_extra"><br class=""><div class="gmail_quote"><div class=""><div class="gmail-m_-4607556448884927622m_-3452065927662814111gmail-m_7527885605307787862gmail-h5">On 22 November 2017 at 21:59, Ben Coman<span class="gmail-m_-4607556448884927622m_-3452065927662814111gmail-m_7527885605307787862Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:btc@openinworld.com" target="_blank" class="">btc@openinworld.com</a>></span><span class="gmail-m_-4607556448884927622m_-3452065927662814111gmail-m_7527885605307787862Apple-converted-space"> </span>wr<wbr class="">ote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div dir="ltr" class=""><br class=""><div class="gmail_extra"><br class=""><div class="gmail_quote"><span class="gmail-m_-4607556448884927622m_-3452065927662814111gmail-m_7527885605307787862gmail-m_8524537758069216380gmail-m_-4929630517411459228gmail-">On 22 November 2017 at 13:38, Todd Blanchard<span class="gmail-m_-4607556448884927622m_-3452065927662814111gmail-m_7527885605307787862Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:tblanchard@mac.com" target="_blank" class="">tblanchard@mac.com</a>></span><span class="gmail-m_-4607556448884927622m_-3452065927662814111gmail-m_7527885605307787862Apple-converted-space"><wbr class=""> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"> <br class=""><div style="word-wrap: break-word;" class="">I've been trying to track this down for a couple weeks now.</div></blockquote></span></div></div></div></blockquote></div></div></div></div></div></blockquote></div></div></div></div></blockquote></div></div></div></blockquote><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="word-wrap: break-word;" class=""><div class=""><div class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class=""><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class=""><div class="gmail-m_-4607556448884927622m_-3452065927662814111gmail-m_7527885605307787862gmail-h5"><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><span class="gmail-m_-4607556448884927622m_-3452065927662814111gmail-m_7527885605307787862gmail-m_8524537758069216380gmail-m_-4929630517411459228gmail-"><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="word-wrap: break-word;" class=""><div class=""><br class=""></div><div class="">I have concluded that structs passed by value to functions on the 64 bit VM are not properly populated. The struct's memory is all zero'd.</div><div class=""><br class=""></div><div class="">I found this while trying to work with LibClang and found that functions that fetched code locations from code ranges always returned invalid zero'd locations. After spending some time with lldb I have traced the problem into the native code and found that the argument is not correct.</div><div class=""><br class=""></div><div class="">I've carved out the wee bit of clang to reproduce this in a tiny library.</div><div class=""><br class=""></div><div class="">The gist of it is below and the entire file is included. Basically the struct passed to the function clang_getRangeStart is zero'd memory regardless of the data I send from the image side.</div></div></blockquote></span></div></div></div></blockquote></div></div></div></div></div></blockquote></div></div></div></div></blockquote></div></div></div></blockquote><div class=""><br class=""></div><div class="">My last analysis discovered something interesting about strings defined inside shared libraries being handled differently, but only later realised I had chased the wrong rabbit down the hole. </div><div class=""> <br class=""></div><div class="">I've now investigated the premise you actually poses, and I agree, that structs being zero is some cases. </div><div class="">The attached zipfile containing libstruct.c a few comparison cases - three "good" layouts that work fine and one "bad" that mostly gets zeros but sometimes other weird numbers. There is a Makefile with three main targets:</div><div class="">1. make layout - statically compiles libstruct.c and runs produced a.out to display structure layouts </div><div class=""><div class=""><br class=""></div><div class="">2. make run - downloads Pharo, starts it loading <span class="Apple-converted-space"> </span><a href="http://libstruct.st/" class="">LibStruct.st</a> , then you manually run LibStruct>>>LibStructTest>>#testStructs and observe structure values on console</div><div class=""><br class=""></div><div class="">3. make debug - starts LLDB to run Pharo with breakpoints pre-configured for when you run #testStructs. </div><div class=""> Note Pharo will freeze and you need to move to LLDB. Try these commands... </div><div class=""> frame variable</div><div class=""> call print_struct(&GoodStruct1_fmt, &aStruct)</div></div><div class=""> continue</div><div class=""><br class=""></div><div class="">That works on Ubuntu 16.04 64 bit. </div><div class="">You will need to tune it for OSX.</div><div class="">clang and lldb are required.</div><div class=""><br class=""></div><div class="">================================<br class=""></div><div class="">The offsets configured in the class variables of all ExternalStructs </div><div class="">correctly matches that reported by the C code test frame results here...</div><div class=""><br class=""></div><div class="">$ make layout </div><div class=""><div class="">clang -g libstruct.c</div><div class="">./a.out</div><div class=""><br class=""></div><div class="">GoodStruct1:</div><div class=""><span style="white-space: pre;" class=""> </span>uint32_t:int1: <span style="white-space: pre;" class=""> </span>01 4 </div><div class=""><span style="white-space: pre;" class=""> </span>uint32_t:int2: <span style="white-space: pre;" class=""> </span>05 4 </div><div class="">GoodStruct2:</div><div class=""><span style="white-space: pre;" class=""> </span>uint32_t:int1: <span style="white-space: pre;" class=""> </span>01 4 </div><div class=""><span style="white-space: pre;" class=""> </span>uint32_t:int2: <span style="white-space: pre;" class=""> </span>05 4 </div><div class=""><span style="white-space: pre;" class=""> </span>void*:ptr_data: <span style="white-space: pre;" class=""> </span>09 8 </div><div class="">GoodStruct3:</div><div class=""><span style="white-space: pre;" class=""> </span>void*:ptr_data: <span style="white-space: pre;" class=""> </span>01 8 </div><div class=""><span style="white-space: pre;" class=""> </span>uint32_t:int1: <span style="white-space: pre;" class=""> </span>09 4 </div><div class=""><span style="white-space: pre;" class=""> </span>uint32_t:int2: <span style="white-space: pre;" class=""> </span>13 4 </div><div class="">BadStruct:</div><div class=""><span style="white-space: pre;" class=""> </span>void*:ptr_data1: <span style="white-space: pre;" class=""> </span>01 8 </div><div class=""><span style="white-space: pre;" class=""> </span>void*:ptr_data2: <span style="white-space: pre;" class=""> </span>09 8 </div><div class=""><span style="white-space: pre;" class=""> </span>uint32_t:int1: <span style="white-space: pre;" class=""> </span>17 4 </div><div class=""><span style="white-space: pre;" class=""> </span>uint32_t:int2: <span style="white-space: pre;" class=""> </span>21 4 </div><div class=""><br class=""></div><div class="">================================</div><div class=""><br class=""></div><div class="">$ make run </div><div class=""><div class="">clang -g -o libstruct.so -shared -fPIC libstruct.c</div><div class="">getpharo/pharo-vm/lib/pharo/5.0-201707201942/pharo getpharo/Pharo.image ../<a href="http://libstruct.st/" class="">LibStruct.st</a></div></div><div class=""><br class=""></div><div class="">Now manually browse to and run LibStruct>>>LibStructTest>>#testStructs</div><div class=""><div class=""><div class="">LibStructTest>>testStructs</div><div class=""><span style="white-space: pre;" class=""> </span>|good1Struct good2Struct good3Struct badStruct |<br class=""></div><div class=""><span style="white-space: pre;" class=""> </span></div><div class=""><span style="white-space: pre;" class=""> </span>good1Struct := GoodStruct1 new int1: 2; int2: 3.</div><div class=""><span style="white-space: pre;" class=""> </span>good2Struct := GoodStruct2 new int1: 2; int2: 3; ptr_force_int: 4.</div><div class=""><span style="white-space: pre;" class=""> </span>good3Struct := GoodStruct3 new int1: 2; int2: 3; ptr_force_int: 4.</div><div class=""><span style="white-space: pre;" class=""> </span>badStruct := BadStruct new int1: 2; int2: 3; ptr_force_int1: 4; ptr_force_int1: 5.</div><div class=""><span style="white-space: pre;" class=""> </span></div><div class=""><span style="white-space: pre;" class=""> </span>self assert: (LibStruct tryGood1: good1Struct) equals: 6.<span style="white-space: pre;" class=""> </span></div><div class=""><span style="white-space: pre;" class=""> </span>self assert: (LibStruct tryGood2: good2Struct) equals: 6.</div><div class=""><span style="white-space: pre;" class=""> </span>self assert: (LibStruct tryGood3: good3Struct) equals: 6.</div><div class=""><span style="white-space: pre;" class=""> </span>"Problem exposed in next line"</div><div class=""><span style="white-space: pre;" class=""> </span>self assert: (LibStruct tryBad: badStruct) equals: 6.</div></div></div><div class=""><br class=""></div><div class="">which on console produces...</div><div class=""><br class=""></div><div class=""><div class="">GoodStruct1:</div><div class=""><span style="white-space: pre;" class=""> </span>uint32_t:int1: <span style="white-space: pre;" class=""> </span>01 4 = 02 00 00 00</div><div class=""><span style="white-space: pre;" class=""> </span>uint32_t:int2: <span style="white-space: pre;" class=""> </span>05 4 = 03 00 00 00</div><div class="">GoodStruct2:</div><div class=""><span style="white-space: pre;" class=""> </span>uint32_t:int1: <span style="white-space: pre;" class=""> </span>01 4 = 02 00 00 00</div><div class=""><span style="white-space: pre;" class=""> </span>uint32_t:int2: <span style="white-space: pre;" class=""> </span>05 4 = 03 00 00 00</div><div class=""><span style="white-space: pre;" class=""> </span>void*:ptr_data: <span style="white-space: pre;" class=""> </span>09 8 = 04 00 00 00 00 00 00 00</div><div class="">GoodStruct3:</div><div class=""><span style="white-space: pre;" class=""> </span>void*:ptr_data: <span style="white-space: pre;" class=""> </span>01 8 = 04 00 00 00 00 00 00 00</div><div class=""><span style="white-space: pre;" class=""> </span>uint32_t:int1: <span style="white-space: pre;" class=""> </span>09 4 = 02 00 00 00</div><div class=""><span style="white-space: pre;" class=""> </span>uint32_t:int2: <span style="white-space: pre;" class=""> </span>13 4 = 03 00 00 00</div><div class="">BadStruct:</div><div class=""><span style="white-space: pre;" class=""> </span>void*:ptr_data1: <span style="white-space: pre;" class=""> </span>01 8 = 00 00 00 00 00 00 00 00</div><div class=""><span style="white-space: pre;" class=""> </span>void*:ptr_data2: <span style="white-space: pre;" class=""> </span>09 8 = 00 00 00 00 00 00 00 00</div><div class=""><span style="white-space: pre;" class=""> </span>uint32_t:int1: <span style="white-space: pre;" class=""> </span>17 4 = 29 03 03 00</div><div class=""><span style="white-space: pre;" class=""> </span>uint32_t:int2: <span style="white-space: pre;" class=""> </span>21 4 = 00 00 00 00</div></div><div class=""><br class=""></div><div class=""><div class="">Comparing GoodStruct3 and BadStruct, it seems one pointer is handled fine, but not two.</div><div class="">The first time this is run after Image boots seems like BadStruct gets some random data.</div></div><div class=""><br class=""></div><div class="">================================</div><div class="">In same image, subsequent runs of LibStruct>>>LibStructTest>>testStructs </div><div class="">give only all zeros for BadStruct, as Todd observed.</div><div class=""><br class=""></div><div class="">GoodStruct1:</div><div class=""><span style="white-space: pre;" class=""> </span>uint32_t:int1: <span style="white-space: pre;" class=""> </span>01 4 = 02 00 00 00</div><div class=""><span style="white-space: pre;" class=""> </span>uint32_t:int2: <span style="white-space: pre;" class=""> </span>05 4 = 03 00 00 00</div><div class="">GoodStruct2:</div><div class=""><span style="white-space: pre;" class=""> </span>uint32_t:int1: <span style="white-space: pre;" class=""> </span>01 4 = 02 00 00 00</div><div class=""><span style="white-space: pre;" class=""> </span>uint32_t:int2: <span style="white-space: pre;" class=""> </span>05 4 = 03 00 00 00</div><div class=""><span style="white-space: pre;" class=""> </span>void*:ptr_data: <span style="white-space: pre;" class=""> </span>09 8 = 00 00 00 00 00 00 00 00</div><div class="">GoodStruct3:</div><div class=""><span style="white-space: pre;" class=""> </span>void*:ptr_data: <span style="white-space: pre;" class=""> </span>01 8 = 00 00 00 00 00 00 00 00</div><div class=""><span style="white-space: pre;" class=""> </span>uint32_t:int1: <span style="white-space: pre;" class=""> </span>09 4 = 02 00 00 00</div><div class=""><span style="white-space: pre;" class=""> </span>uint32_t:int2: <span style="white-space: pre;" class=""> </span>13 4 = 03 00 00 00</div><div class="">BadStruct:</div><div class=""><span style="white-space: pre;" class=""> </span>void*:ptr_data1: <span style="white-space: pre;" class=""> </span>01 8 = 00 00 00 00 00 00 00 00</div><div class=""><span style="white-space: pre;" class=""> </span>void*:ptr_data2: <span style="white-space: pre;" class=""> </span>09 8 = 00 00 00 00 00 00 00 00</div><div class=""><span style="white-space: pre;" class=""> </span>uint32_t:int1: <span style="white-space: pre;" class=""> </span>17 4 = 00 00 00 00</div><div class=""><span style="white-space: pre;" class=""> </span>uint32_t:int2: <span style="white-space: pre;" class=""> </span>21 4 = 00 00 00 00</div></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">cheers -ben</div><div class=""><br class=""></div></div></div></div><span id="cid:D7D707EA-C52A-495F-AE9F-35C99DA5D9D2"><FFI64StructTest.zip></span></div></blockquote></div><br class=""></div></body></html>