[Vm-dev] [Pharo-users] FFI 64 bit libClang issue on Sierra.

Fri Nov 17 09:32:03 UTC 2017

On Nov 14, 2017, at 4:34 PM, Ben Coman <btc at openinworld.com> wrote:

On 15 November 2017 at 00:07, Todd Blanchard <tblanchard at mac.com> wrote:

> I've got it loaded, have fixed up the library path and fixed the test for
> version (built in lib returns very different version string).
>
> I have many tests green.  However, as before, tests involving
> CXSourceRange that return CXSourceLocations return all zero'd data.  The
> CXSourceRange struct looks sane.
>
> The calls to CXSourceLocation clang_getRangeStart(CXSourceRange range);
>
> and its twin RangeEnd result in zero'd structs.
>
> Any idea on where I can look to try to figure out where this is going
> wrong?
>
> I have compiled little C programs and verified that both struct sizes
> agree in FFI and native code (24 bytes each).
>
> Not sure what to try next.
>  c
> -Todd Blanchard
>

Compile your own debug-VM** and make a little C wrapper MyCXSourceRange to
forward to Clang's CXSourceRange, and set a gdb a breakpoint in
MyCXSourceRange and compare 64 bit and 32 bit versions.

That still leaves differences in the callout before your breakpoint in
MyCXSourceRange is reached, so just do two callouts in a row and trace from
the first to the second.

** https://github.com/OpenSmalltalk/opensmalltalk-
vm/tree/Cog/build.linux64x64/pharo.cog.spur/build.debug

cheers -ben

On 17 November 2017 at 14:57, Todd Blanchard <tblanchard at mac.com> wrote:

> Right, so I have my own debug VM (64 bit Pharo Cog) and a debug version of
> libclang.
>

> I can attach with lldb and set a breakpoint on clang_getRangeStart
>
> Then I debug the test that gets diagnostics and I see that the
> CXSourceRange has data.  Then I step into 'range start' and I land in my
> breakpoint for clang_getRangeStart and I can see that the argument to the
> function - supposedly a CXSourceRange passed by value - is all zero'd
> memory.
>
> So it appears that in the 64bit VM, structs passed by value doesn't work
> right.
>
> I've traced back up the stack and I confess the marshaling code is totally
> beyond my comprehension.  This is what I was able to get:
>
> Process 1126 stopped
> * thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
> 2.1
>     frame #0: 0x000000011e33136b libclang.dylib`::clang_
> getRangeStart(range=CXSourceRange @ 0x00007fff55f4e380) at
> CXSourceLocation.cpp:89
>    86
>    87  CXSourceLocation clang_getRangeStart(CXSourceRange range) {
>    88    // Special decoding for CXSourceLocations for
> CXLoadedDiagnostics.
> -> 89    if ((uintptr_t)range.ptr_data[0] & 0x1) {
>    90      CXSourceLocation Result = { { range.ptr_data[0], nullptr }, 0
> };
>    91      return Result;
>    92    }
> Target 0: (Pharo) stopped.
> (lldb) p range
> (CXSourceRange) $0 = {
>   ptr_data = ([0] = 0x0000000000000000, [1] = 0x0000000000000000)
>   begin_int_data = 0
>   end_int_data = 0
> }
> (lldb) bt
> * thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
> 2.1
>   * frame #0: 0x000000011e33136b libclang.dylib`::clang_
> getRangeStart(range=CXSourceRange @ 0x00007fff55f4e380) at
> CXSourceLocation.cpp:89
>     frame #1: 0x0000000109e20b5a Pharo`primitiveCalloutWithArgs at
> X64SysVFFIPlugin.c:6041
>     frame #2: 0x0000000109cb964c Pharo`primitiveExternalCall at
> gcc3x-cointerp.c:77188
>     frame #3: 0x0000000109cb1629 Pharo`interpretMethodFromMachineCode at
> gcc3x-cointerp.c:19719
>     frame #4: 0x0000000109cb77fc Pharo`ceSendsupertonumArgs(selector=4713336760,
> superNormalBar=0, rcvr=4692466872 <(469)%20246-6872>, numArgs=1) at
> gcc3x-cointerp.c:17767
>     frame #5: 0x000000011774a135
>     frame #6: 0x0000000109c76227 Pharo`interpret at gcc3x-cointerp.c:2703
>     frame #7: 0x0000000109dbbf31 Pharo`-[sqSqueakMainApplication
> runSqueak](self=0x00006080000d7370, _cmd="runSqueak") at
> sqSqueakMainApplication.m:201
>     frame #8: 0x00007fff81e5a8dd Foundation`__NSFirePerformWithOrder + 368
>     frame #9: 0x00007fff803b1d37 CoreFoundation`__CFRUNLOOP_IS_
> CALLING_OUT_TO_AN_OBSERVER_CALLBACK_FUNCTION__ + 23
>     frame #10: 0x00007fff803b1ca7 CoreFoundation`__CFRunLoopDoObservers +
> 391
>     frame #11: 0x00007fff803926d9 CoreFoundation`__CFRunLoopRun + 873
>     frame #12: 0x00007fff80392114 CoreFoundation`CFRunLoopRunSpecific +
> 420
>     frame #13: 0x00007fff7f8f2ebc HIToolbox`RunCurrentEventLoopInMode +
> 240
>     frame #14: 0x00007fff7f8f2bf9 HIToolbox`ReceiveNextEventCommon + 184
>     frame #15: 0x00007fff7f8f2b26 HIToolbox`_
> BlockUntilNextEventMatchingListInModeWithFilter + 71
>     frame #16: 0x00007fff7de89a54 AppKit`_DPSNextEvent + 1120
>     frame #17: 0x00007fff7e6057ee AppKit`-[NSApplication(NSEvent)
> _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 2796
>     frame #18: 0x00007fff7de7e3db AppKit`-[NSApplication run] + 926
>     frame #19: 0x00007fff7de48e0e AppKit`NSApplicationMain + 1237
>     frame #20: 0x0000000109db5baf Pharo`main(argc=1,
> argv=0x00007fff55f8bca8, envp=0x00007fff55f8bcb8) at main.m:52
>     frame #21: 0x00007fff95b18235 libdyld.dylib`start + 1
>     frame #22: 0x00007fff95b18235 libdyld.dylib`start + 1
> (lldb) up
> frame #1: 0x0000000109e20b5a Pharo`primitiveCalloutWithArgs at
> X64SysVFFIPlugin.c:6041
>    6038 result2 = floatObjectOf(floatRet1);
>    6039 goto l14;
>    6040 }
> -> 6041 intRet1 = dispatchFunctionPointerwithwithwithwithwithwith(((SixteenByteReturn
> (*)(sqIntptr_t, sqIntptr_t, sqIntptr_t, sqIntptr_t, sqIntptr_t,
> sqIntptr_t)) (((void *) address2))), ((calloutState1->integerRegisters))[0],
> ((calloutState1->integerRegisters))[1], ((calloutState1->integerRegisters))[2],
> ((calloutState1->integerRegisters))[3], ((calloutState1->integerRegisters))[4],
> ((calloutState1->integerRegisters))[5]);
>    6042 /* begin maybeOwnVM:threadIndex: */
>    6043
>    6044 #  if COGMTVM
>
> The struct that should have shown up appears in Pharo as
>
> "CXSourceRangeStruct (
> ptr_data1: (void*)@ 16r7FDF356C77B0
> ptr_data2: (void*)@ 16r7FDF356D0C70
> begin_int_data: 21
> end_int_data: 28
> )"
>
> Open to ideas at this point but I think to progress I would require deep
> knowledge of Intel calling conventions and I don't really know where to go
> next.
>
> Try to isolate the functions into a little dynamic lib and file a bug?
>

As you say, to minimize confounding factors the next step would be to try
to reproduce the "structs passed by value" problem in a minimal library,
and also someone trying to reproduce it doesn't need to compile the whole
libclang

Also now is a good time to move discussion to vm-dev.

cheers -ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20171117/39085cf1/attachment-0001.html>