[Vm-dev] [OpenSmalltalk/opensmalltalk-vm] BitBlt Segmentation Fault with Magic (2^31) value (#447)

Nicolas Cellier notifications at github.com
Thu Nov 21 21:41:45 UTC 2019


2^31 is most certainly a signed int overflow. it is not representable on 32 bits and will wrap to -2^31. Then, it will most probably result in buffer overflow (or underflow). 

If I compile debug VM with `-fsanitize=undefined` then run the snippet, I get this:

```
../../spur64src/vm/gcc3x-cointerp.c:52645:33: runtime error: signed integer overflow: -1752183416206057520 * 16807 cannot be represented in type 'long'

(((usqInt)destIndex)) < endOfDestination 2226

Segmentation fault Thu Nov 21 22:37:33 2019


VM: 201911090934-compile_legacy_Mac_OS nicolas at MBP-de-Nicolas:Smalltalk/OpenSmalltalk/opensmalltalk-vm
Date: Sat Nov 9 10:34:10 2019 CommitHash: 30315747a
Plugins: 201911090934-compile_legacy_Mac_OS nicolas at MBP-de-Nicolas:Smalltalk/OpenSmalltalk/opensmalltalk-vm

C stack backtrace & registers:
	rax 0xffffffff0fcd2df8 rbx 0x00000001064c9900 rcx 0xffffffff0fcd2df8 rdx 0x0000000000000001
	rdi 0x00007fff9da40f01 rsi 0x0000000000000000 rbp 0x00007ffeea108440 rsp 0x00007ffeea1082f0
	r8  0x00000000000130a8 r9  0x00007fff9da40f78 r10 0x0000000000000000 r11 0x00007fff9da40f70
	r12 0x00007fff9642c818 r13 0x00007fff9642c838 r14 0x000000010bf6e900 r15 0x000000010b9033a0
	rip 0x0000000105f39c51
0   Squeak                              0x0000000105f39c51 copyLoopNoSource + 993
1   Squeak                              0x0000000105e5517d reportStackState + 589
2   Squeak                              0x0000000105e55d19 sigsegv + 681
3   libsystem_platform.dylib            0x00007fff673b7b5d _sigtramp + 29
4   ???                                 0x0000000000000001 0x0 + 1
5   Squeak                              0x0000000105f2fb71 copyBitsLockedAndClipped + 4129
6   Squeak                              0x0000000105f296e6 copyBits + 134
7   Squeak                              0x0000000105f29a23 primitiveCopyBits + 67
8   ???                                 0x000000010b1e0670 0x0 + 4481484400
9   Squeak                              0x0000000105aafa65 interpret + 517
10  Squeak                              0x0000000105e59d87 -[sqSqueakMainApplication runSqueak] + 615
11  Foundation                          0x00007fff3d5ae756 __NSFirePerformWithOrder + 362
12  CoreFoundation                      0x00007fff3b29e6d8 __CFRUNLOOP_IS_CALLING_OUT_TO_AN_OBSERVER_CALLBACK_FUNCTION__ + 23
13  CoreFoundation                      0x00007fff3b29e60d __CFRunLoopDoObservers + 451
14  CoreFoundation                      0x00007fff3b240d30 __CFRunLoopRun + 1136
15  CoreFoundation                      0x00007fff3b24066e CFRunLoopRunSpecific + 455
16  HIToolbox                           0x00007fff3a49f1ab RunCurrentEventLoopInMode + 292
17  HIToolbox                           0x00007fff3a49eded ReceiveNextEventCommon + 355
18  HIToolbox                           0x00007fff3a49ec76 _BlockUntilNextEventMatchingListInModeWithFilter + 64
19  AppKit                              0x00007fff3883777d _DPSNextEvent + 1135
20  AppKit                              0x00007fff3883646b -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 1361
21  AppKit                              0x00007fff38830588 -[NSApplication run] + 699
22  AppKit                              0x00007fff3881fac8 NSApplicationMain + 777
23  Squeak                              0x0000000105e504ce main + 782
24  libdyld.dylib                       0x00007fff671cc3d5 start + 1


Smalltalk stack dump:
    0x7ffeea112aa8 M GrafPort>copyBits 0x10b9033a0: a(n) GrafPort
```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/447#issuecomment-557284790
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20191121/f059f7cb/attachment-0001.html>


More information about the Vm-dev mailing list