[Vm-dev] [OpenSmalltalk/opensmalltalk-vm] Reproduceable Segmentation fault while saving images (#444)
John M McIntosh
johnmci at smalltalkconsulting.com
Thu Nov 14 06:37:38 UTC 2019
An interesting question to ask here is can you tag the image memory as read only during a FFI call out for debugging purposes? If writes to image memory are required can they sandboxed? If writes to a display area are required can that be protected by no read/write pages before/after the screen buffer to trap overwrites or reads?
....
John M. McIntosh. Corporate Smalltalk Consulting Ltd https://www.linkedin.com/in/smalltalk
On Wed, Nov 13, 2019 at 10:12 PM, Eliot Miranda <notifications at github.com> wrote:
> If one starts up the latest Pharo 8 64-bit image using an assert VM with leak checking turned on one ends that the image is already corrupted. It is not the script, nor the VM that is at fault, but the initial image, which is already corrupted.
>
> Here's what I get when I launch the latest 64-bit 8.0 image using an assert VM with leak checking turned on (commentary after the run):
>
> $ pharo64cavm --lldb --leakcheck 1 gccrash8-64.image gccrash.st
>
> run --leakcheck 1 /Users/eliot/Documents/Pharo/images/Pharo 8.0 - 64bit/gccrash8-64.image gccrash.st
>
> (lldb) target create "/Users/eliot/oscogvm/build.macos64x64/pharo.cog.spur/PharoAssert.app/Contents/MacOS/Pharo"
> Current executable set to '/Users/eliot/oscogvm/build.macos64x64/pharo.cog.spur/PharoAssert.app/Contents/MacOS/Pharo' (x86_64).
> (lldb) settings set -- target.run-args "/Users/eliot/Documents/Pharo/images/Pharo 8.0 - 64bit/gccrash8-64.image" "gccrash.st"
> (lldb) b Pharo`warning
> Breakpoint 1: where = Pharo`warning + 9 at gcc3x-cointerp.c:44, address = 0x0000000100002079
> (lldb) run --leakcheck 1 gccrash8-64.image gccrash.st
> Process 13469 launched: '/Users/eliot/oscogvm/build.macos64x64/pharo.cog.spur/PharoAssert.app/Contents/MacOS/Pharo' (x86_64)
> 2019-11-13 21:53:49.806855-0800 Pharo[13469:7168447] MessageTracer: load_domain_whitelist_search_tree:73: Search tree file's format version number (0) is not supported
> 2019-11-13 21:53:49.806878-0800 Pharo[13469:7168447] MessageTracer: Falling back to default whitelist
> object leak in 0x110cc6720 @ 0 = 0x11a542c50
> object leak in 0x110d402e0 @ 1 = 0x11a542c50
> object leak in 0x110d40418 @ 1 = 0x11a702018
> object leak in 0x110d40790 @ 1 = 0x11a53bec8
> object leak in 0x110d557f8 @ 4 = 0x11a542c88
> object leak in 0x110e1a940 @ 0 = 0x11a8ffb48
> object leak in 0x110e1a9a0 @ 0 = 0xfffffffffb3d0000
> object leak in 0x110e1a9f8 @ 0 = 0x11a70bdc0
> object leak in 0x110e1a9f8 @ 5 = 0x11a702018
> object leak in 0x110e1aa30 @ 4 = 0x11a73a320
> object leak in 0x110e1aa30 @ 5 = 0x11a702050
> object leak in 0x110e1aa68 @ 0 = 0xfffffffffb3d0000
> object leak in 0x110e1ab08 @ 1 = 0x11a70bdc0
> object leak in 0x110e1ab20 @ 1 = 0x11a70bdf8
> object leak in 0x110e1ab38 @ 1 = 0x11a70be68
> object leak in 0x110e1ab50 @ 1 = 0x11a73a7a8
> object leak in 0x110e1ab68 @ 1 = 0x11a70bea0
> object leak in 0x110e1ab80 @ 1 = 0x11a70bed8
> object leak in 0x110e1ab98 @ 1 = 0x11a70bf10
> object leak in 0x110e1abb0 @ 1 = 0x11a70be30
> object leak in 0x110e1ae00 @ 0 = 0x11a73af40
> object leak in 0x110e1ae20 @ 1 = 0x11a73af68
> object leak in 0x110e1e258 @ 0 = 0x11a73af98
> object leak in 0x110e1e290 @ 5 = 0x11a7058b0
> object leak in 0x110e1e448 @ 1 = 0x11a73bbe8
> object leak in 0x110e1e6a0 @ 1 = 0x11a73d3a0
> object leak in 0x110e1e6b8 @ 1 = 0x11a73d478
> object leak in 0x110e1ea48 @ 1 = 0x11a73f950
> object leak in 0x110e1ebe0 @ 1 = 0x11a740a48
> object leak in 0x110e1f348 @ 1 = 0x11a745698
> object leak in 0x110e1f648 @ 1 = 0x11a7475a0
> object leak in 0x110e1fbb8 @ 1 = 0x11a74ad98
> object leak in 0x110e20008 @ 1 = 0x11a74db10
> object leak in 0x110e20068 @ 1 = 0x11a74dee8
> object leak in 0x110e20128 @ 1 = 0x11a74e650
> object leak in 0x110e20188 @ 1 = 0x11a74ea28
> object leak in 0x110e20398 @ 1 = 0x11a74feb0
> object leak in 0x110e207e8 @ 1 = 0x11a752b08
> object leak in 0x110e217d8 @ 1 = 0x11a75cbb8
> object leak in 0x110e21958 @ 1 = 0x11a75dad0
> object leak in 0x110e21fa0 @ 1 = 0x11a761ac0
> object leak in 0x110e22180 @ 1 = 0x11a762dc8
> object leak in 0x110e222d0 @ 1 = 0x11a763b00
> object leak in 0x110e22420 @ 1 = 0x11a764820
> object leak in 0x110e225b8 @ 1 = 0x11a765828
> object leak in 0x110e22798 @ 1 = 0x11a766ad0
> object leak in 0x110e22948 @ 1 = 0x11a767bf8
> object leak in 0x110e22bd0 @ 1 = 0x11a7695f0
> object leak in 0x110e22cf0 @ 1 = 0x11a76a1a8
> object leak in 0x110e22d80 @ 1 = 0x11a76a760
> object leak in 0x110e234a0 @ 1 = 0x11a76f050
> object leak in 0x110e23ce0 @ 1 = 0x11a774438
> Pharo was compiled with optimization - stepping may behave oddly; variables may not be available.
> Process 13469 stopped
> * thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
> frame #0: 0x0000000100002079 Pharo`warning(s="checkHeapIntegrityclassIndicesShouldBeValid(0, 1) 56761") at gcc3x-cointerp.c:44 [opt]
> 41 sqInt warnpid, erroronwarn;
> 42 void
> 43 warning(char *s) { /* Print an error message but don't necessarily exit. */
> -> 44 if (erroronwarn) error(s);
> 45 if (warnpid)
> 46 printf("\n%s pid %ld\n", s, (long)warnpid);
> 47 else
> Target 0: (Pharo) stopped.
> (lldb) thr b
> * thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
> * frame #0: 0x0000000100002079 Pharo`warning(s="checkHeapIntegrityclassIndicesShouldBeValid(0, 1) 56761") at gcc3x-cointerp.c:44 [opt]
> frame #1: 0x000000010003872f Pharo`runLeakCheckerFor(gcModes=1) at gcc3x-cointerp.c:56761 [opt]
> frame #2: 0x0000000100012d1e Pharo`loadInitialContext at gcc3x-cointerp.c:65476 [opt]
> frame #3: 0x000000010000231d Pharo`interpret at gcc3x-cointerp.c:2770 [opt]
> frame #4: 0x00000001000bd0c9 Pharo`-[sqSqueakMainApplication runSqueak](self=0x000000010045e840, _cmd=<unavailable>) at sqSqueakMainApplication.m:201 [opt]
> frame #5: 0x00007fff319e809c Foundation`__NSFirePerformWithOrder + 360
> frame #6: 0x00007fff2f85c257 CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_AN_OBSERVER_CALLBACK_FUNCTION__ + 23
> frame #7: 0x00007fff2f85c17f CoreFoundation`__CFRunLoopDoObservers + 527
> frame #8: 0x00007fff2f83e6f8 CoreFoundation`__CFRunLoopRun + 1240
> frame #9: 0x00007fff2f83df93 CoreFoundation`CFRunLoopRunSpecific + 483
> frame #10: 0x00007fff2eb28d96 HIToolbox`RunCurrentEventLoopInMode + 286
> frame #11: 0x00007fff2eb28a0f HIToolbox`ReceiveNextEventCommon + 366
> frame #12: 0x00007fff2eb28884 HIToolbox`_BlockUntilNextEventMatchingListInModeWithFilter + 64
> frame #13: 0x00007fff2cdd7a3b AppKit`_DPSNextEvent + 2085
> frame #14: 0x00007fff2d56de34 AppKit`-[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 3044
> frame #15: 0x00007fff2cdcc84d AppKit`-[NSApplication run] + 764
> frame #16: 0x00007fff2cd9ba3a AppKit`NSApplicationMain + 804
> frame #17: 0x00007fff5777c015 libdyld.dylib`start + 1
>
> My script pharo64cavm runs an assert VM, specifically /Users/eliot/oscogvm/build.macos64x64/pharo.cog.spur/PharoAssert.app/Contents/MacOS/Pharo. Supplying the --lldb argument has the script launch lldb (a low-level debugger for native executables) on the VM.
>
> This command places a breakpoint in the error/warning output routine called when the VM wants to report that asserts have failed, leaks in the heap have been found, etc.
>
> (lldb) b Pharo`warning
>
> This command then launches the VM under the control of the debugger.
>
> run --leakcheck 1 gccrash8-64.image gccrash.st
>
> The arguments to leak check are a combination of the following flags:
>
> SpurMemoryManager<<setCheckForLeaks: integerFlags
> " 0 = do nothing.
> 1 = check for leaks on fullGC (GCModeFull).
> 2 = check for leaks on scavenger (GCModeNewSpace).
> 4 = check for leaks on incremental (GCModeIncremental)
> 8 = check for leaks on become (GCModeBecome)
> 16 = check for leaks on image segments (GCModeImageSegment)"
> checkForLeaks := integerFlags
>
> If GCModeFull is set then the VM performs a leak check on loading the initial image. From the back trace you can see that the Vm has not yet started running, loadInitialCOntext being the routine that sets up the VM to run from the context that performed the snapshot:
>
> Pharo`warning(s="checkHeapIntegrityclassIndicesShouldBeValid(0, 1) 56761") at gcc3x-cointerp.c:44 [opt]
> frame #1: 0x000000010003872f Pharo`runLeakCheckerFor(gcModes=1) at gcc3x-cointerp.c:56761 [opt]
> frame #2: 0x0000000100012d1e Pharo`loadInitialContext at gcc3x-cointerp.c:65476 [opt]
> frame #3: 0x000000010000231d Pharo`interpret at gcc3x-cointerp.c:2770 [opt]
> frame #4: 0x00000001000bd0c9 Pharo`-[sqSqueakMainApplication runSqueak](self=0x000000010045e840, _cmd=<unavailable>) at sqSqueakMainApplication.m:201 [opt]
> frame #5: 0x00007fff319e809c Foundation`__NSFirePerformWithOrder + 360
>
> The following leak report shows that there are many leaks in this image:
>
> object leak in 0x110cc6720 @ 0 = 0x11a542c50
> object leak in 0x110d402e0 @ 1 = 0x11a542c50
> object leak in 0x110d40418 @ 1 = 0x11a702018
> object leak in 0x110d40790 @ 1 = 0x11a53bec8
> object leak in 0x110d557f8 @ 4 = 0x11a542c88
> object leak in 0x110e1a940 @ 0 = 0x11a8ffb48
> object leak in 0x110e1a9a0 @ 0 = 0xfffffffffb3d0000
> object leak in 0x110e1a9f8 @ 0 = 0x11a70bdc0
> object leak in 0x110e1a9f8 @ 5 = 0x11a702018
> object leak in 0x110e1aa30 @ 4 = 0x11a73a320
> object leak in 0x110e1aa30 @ 5 = 0x11a702050
> object leak in 0x110e1aa68 @ 0 = 0xfffffffffb3d0000
> ...
>
> Let's take a look at some of these objects. In lldb we can call the VM's debug printing routines, just as we can in the simulator:
>
> (lldb) call printOop(0x110cc6720)
> 0x110cc6720: a(n) FreeTypeCacheEntry
> 0x11a542c50 0x110ca0fb8 0x221 0x21 0x81ffae4000000004
> 0x110d25ea8
> (lldb) call printOop(0x110d402e0)
> 0x110d402e0: a(n) Association
> 0x21 0x11a542c50
> (lldb) call printOop(0x110d402e0)
> 0x110d402e0: a(n) Association
> 0x21 0x11a542c50
> (sqInt) $1 = 0
> (lldb) print whereIs(0x11a542c50)
> (char *) $2 = 0x0000000100110864 " is no where obvious"
> (lldb) call printOop(0x110d40418)
> 0x110d40418: a(n) Association
> 0x16000000001 0x11a702018
> (sqInt) $3 = 0
> (lldb) call printOop(0x110d40790)
> 0x110d40790: a(n) Association
> 0x21 0x11a53bec8
> (sqInt) $4 = 0
> (lldb) call printOop(0x110d557f8)
> 0x110d557f8: a(n) FreeTypeCacheEntry
> 0x110d55ad0 0x110ca0fb8 0x181 0x7c1 0x11a542c88
> 0x110d55718
>
> So the first suspect (to me) looks like external C memory management in FreeType font management. Let me suggest you add a step in the release process which involves checking the validity of images before they're released. Let me also suggest that you appoint a team to look at FreeType font management using the leak checker, et al, to find and fix these issues which I think have been around for quite a while.
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, [view it on GitHub](https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/444?email_source=notifications&email_token=AIJPEWY5B5FKVETVH5JI75LQTTT6PA5CNFSM4JNBNJH2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEAWYSA#issuecomment-553741384), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AIJPEW554N2J2HMLDO53QJTQTTT6PANCNFSM4JNBNJHQ).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20191114/7500d73a/attachment-0001.html>
More information about the Vm-dev
mailing list