On Dec 3, 2017, at 7:18 AM, Ben Coman <btc@openinworld.com> wrote:<FFI64StructTest.zip>On 27 November 2017 at 00:24, Todd Blanchard <tblanchard@mac.com> wrote:i'm getting the idea that we should probably write a test suite/library for FFII noticed these...These look like c-code test frames for FFI. Could these be built by the OpenSmalltalk CI to be normally shipped with the VM so that Image-side CI can test against them?And also some general info...On Nov 24, 2017, at 12:54 AM, Ben Coman <btc@openinworld.com> wrote:On 24 November 2017 at 13:16, Ben Coman <btc@openinworld.com> wrote: On 22 November 2017 at 21:59, Ben Coman <btc@openinworld.com> wrote: On 22 November 2017 at 13:38, Todd Blanchard <tblanchard@mac.com>wrote: I've been trying to track this down for a couple weeks now.I have concluded that structs passed by value to functions on the 64 bit VM are not properly populated. The struct's memory is all zero'd.I found this while trying to work with LibClang and found that functions that fetched code locations from code ranges always returned invalid zero'd locations. After spending some time with lldb I have traced the problem into the native code and found that the argument is not correct.I've carved out the wee bit of clang to reproduce this in a tiny library.The gist of it is below and the entire file is included. Basically the struct passed to the function clang_getRangeStart is zero'd memory regardless of the data I send from the image side.My last analysis discovered something interesting about strings defined inside shared libraries being handled differently, but only later realised I had chased the wrong rabbit down the hole.I've now investigated the premise you actually poses, and I agree, that structs being zero is some cases.The attached zipfile containing libstruct.c a few comparison cases - three "good" layouts that work fine and one "bad" that mostly gets zeros but sometimes other weird numbers. There is a Makefile with three main targets:1. make layout - statically compiles libstruct.c and runs produced a.out to display structure layouts2. make run - downloads Pharo, starts it loading LibStruct.st , then you manually run LibStruct>>>LibStructTest>>#testStructs and observe structure values on console3. make debug - starts LLDB to run Pharo with breakpoints pre-configured for when you run #testStructs.Note Pharo will freeze and you need to move to LLDB. Try these commands...frame variablecall print_struct(&GoodStruct1_fmt, &aStruct)continueThat works on Ubuntu 16.04 64 bit.You will need to tune it for OSX.clang and lldb are required.================================The offsets configured in the class variables of all ExternalStructscorrectly matches that reported by the C code test frame results here...$ make layoutclang -g libstruct.c./a.outGoodStruct1:uint32_t:int1: 01 4uint32_t:int2: 05 4GoodStruct2:uint32_t:int1: 01 4uint32_t:int2: 05 4void*:ptr_data: 09 8GoodStruct3:void*:ptr_data: 01 8uint32_t:int1: 09 4uint32_t:int2: 13 4BadStruct:void*:ptr_data1: 01 8void*:ptr_data2: 09 8uint32_t:int1: 17 4uint32_t:int2: 21 4================================$ make runclang -g -o libstruct.so -shared -fPIC libstruct.cgetpharo/pharo-vm/lib/pharo/5.0-201707201942/pharo getpharo/Pharo.image ../LibStruct.stNow manually browse to and run LibStruct>>>LibStructTest>>#testStructsLibStructTest>>testStructs|good1Struct good2Struct good3Struct badStruct |good1Struct := GoodStruct1 new int1: 2; int2: 3.good2Struct := GoodStruct2 new int1: 2; int2: 3; ptr_force_int: 4.good3Struct := GoodStruct3 new int1: 2; int2: 3; ptr_force_int: 4.badStruct := BadStruct new int1: 2; int2: 3; ptr_force_int1: 4; ptr_force_int1: 5.self assert: (LibStruct tryGood1: good1Struct) equals: 6.self assert: (LibStruct tryGood2: good2Struct) equals: 6.self assert: (LibStruct tryGood3: good3Struct) equals: 6."Problem exposed in next line"self assert: (LibStruct tryBad: badStruct) equals: 6.which on console produces...GoodStruct1:uint32_t:int1: 01 4 = 02 00 00 00uint32_t:int2: 05 4 = 03 00 00 00GoodStruct2:uint32_t:int1: 01 4 = 02 00 00 00uint32_t:int2: 05 4 = 03 00 00 00void*:ptr_data: 09 8 = 04 00 00 00 00 00 00 00GoodStruct3:void*:ptr_data: 01 8 = 04 00 00 00 00 00 00 00uint32_t:int1: 09 4 = 02 00 00 00uint32_t:int2: 13 4 = 03 00 00 00BadStruct:void*:ptr_data1: 01 8 = 00 00 00 00 00 00 00 00void*:ptr_data2: 09 8 = 00 00 00 00 00 00 00 00uint32_t:int1: 17 4 = 29 03 03 00uint32_t:int2: 21 4 = 00 00 00 00Comparing GoodStruct3 and BadStruct, it seems one pointer is handled fine, but not two.The first time this is run after Image boots seems like BadStruct gets some random data.================================In same image, subsequent runs of LibStruct>>>LibStructTest>>testStructsgive only all zeros for BadStruct, as Todd observed.GoodStruct1:uint32_t:int1: 01 4 = 02 00 00 00uint32_t:int2: 05 4 = 03 00 00 00GoodStruct2:uint32_t:int1: 01 4 = 02 00 00 00uint32_t:int2: 05 4 = 03 00 00 00void*:ptr_data: 09 8 = 00 00 00 00 00 00 00 00GoodStruct3:void*:ptr_data: 01 8 = 00 00 00 00 00 00 00 00uint32_t:int1: 09 4 = 02 00 00 00uint32_t:int2: 13 4 = 03 00 00 00BadStruct:void*:ptr_data1: 01 8 = 00 00 00 00 00 00 00 00void*:ptr_data2: 09 8 = 00 00 00 00 00 00 00 00uint32_t:int1: 17 4 = 00 00 00 00uint32_t:int2: 21 4 = 00 00 00 00cheers -ben