<p>The signed integer overflow reported by -fsanitize is not related, what is related is the assert warning:</p>
<pre><code>(((usqInt)destIndex)) < endOfDestination 2226
Process 77080 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0xffffffff0d50d768)
    frame #0: 0x000000010048bc51 Squeak`copyLoopNoSource at BitBltPlugin.c:2227:14
   2224                 }
   2225                 destMask = mask1;
   2226                 assert((((usqInt)destIndex)) < endOfDestination);
-> 2227                      destWord = long32At(destIndex);
   2228                 mergeWord = mergeFnwith(halftoneWord, destWord);
   2229                 destWord = (destMask & mergeWord) | (destWord & ((unsigned int)~destMask));
   2230                 long32Atput(destIndex, destWord);
Target 0: (Squeak) stopped.
</code></pre>
<pre><code>(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0xffffffff0d50d768)
  * frame #0: 0x000000010048bc51 Squeak`copyLoopNoSource at BitBltPlugin.c:2227:14
    frame #1: 0x0000000100481b71 Squeak`copyBitsLockedAndClipped at BitBltPlugin.c:1505:3
    frame #2: 0x000000010047b6e6 Squeak`copyBits at BitBltPlugin.c:1257:2
    frame #3: 0x000000010047ba23 Squeak`primitiveCopyBits at BitBltPlugin.c:5128:2
    frame #4: 0x0000000108a01670
    frame #5: 0x0000000100001a65 Squeak`interpret at gcc3x-cointerp.c:2772:3
    frame #6: 0x00000001003abd87 Squeak`-[sqSqueakMainApplication runSqueak](self=0x0000000101867010, _cmd="runSqueak") at sqSqueakMainApplication.m:201:2

(lldb) p/x destIndex
(usqInt) $8 = 0xffffffff0d50d768
(lldb) p/x endOfDestination
(usqInt) $9 = 0x000000010d741768
(lldb) p/x destX
(sqInt) $32 = 0x0000000080000000
(lldb) p/x dx
(int) $36 = 0x80000000
(lldb) print dx
(int) $34 = -2147483648
</code></pre>
<p><code>destX</code> is 2^31, since it is on 64bits (signed), there is no overflow at this stage.<br><br>
The overflow is on <code>dx</code> which is an <code>int</code>.<br><br>
It seems that <code>-fsanitize=undefined</code> does not signal case of overflow unsigned->signed, because it is not UB...</p>
<p>One solution is to change the type of dx and dy to sqInt.<br>
Then, with 64 bits, it might still be possible to craft a value that will overflow, this will have to be reviewed...</p>

<p style="font-size:small;-webkit-text-size-adjust:none;color:#666;">—<br />You are receiving this because you are subscribed to this thread.<br />Reply to this email directly, <a href="https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/447?email_source=notifications&email_token=AIJPEWYYEOOXJ3J3B7CSANLQU4BERA5CNFSM4JQC6HX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEE32IZY#issuecomment-557294695">view it on GitHub</a>, or <a href="https://github.com/notifications/unsubscribe-auth/AIJPEWY5RMFGSWDIWI34QBTQU4BERANCNFSM4JQC6HXQ">unsubscribe</a>.<img src="https://github.com/notifications/beacon/AIJPEW5NNXMQUVFVBSHW3LLQU4BERA5CNFSM4JQC6HX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEE32IZY.gif" height="1" width="1" alt="" /></p>
<script type="application/ld+json">[
{
"@context": "http://schema.org",
"@type": "EmailMessage",
"potentialAction": {
"@type": "ViewAction",
"target": "https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/447?email_source=notifications\u0026email_token=AIJPEWYYEOOXJ3J3B7CSANLQU4BERA5CNFSM4JQC6HX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEE32IZY#issuecomment-557294695",
"url": "https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/447?email_source=notifications\u0026email_token=AIJPEWYYEOOXJ3J3B7CSANLQU4BERA5CNFSM4JQC6HX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEE32IZY#issuecomment-557294695",
"name": "View Issue"
},
"description": "View this Issue on GitHub",
"publisher": {
"@type": "Organization",
"name": "GitHub",
"url": "https://github.com"
}
}
]</script>