[Vm-dev] VM crash in the Squeak3D plugin

Nicolas Cellier nicolas.cellier.aka.nice at gmail.com
Sat Dec 28 22:23:59 UTC 2019


I forgot to say that I had an assert failure report:

leftEdge && rightEdge 1398

Looking around line 1398:

                                        /*-- search for the next top edge
which will be the right edge --*/
                                        assert(aetStart < aetSize);
                                        if(!fillList->firstFace)
                                                rightEdge =
aetData[aetStart++]; /* If no current top fill just use the next edge */
                                        else while(aetStart < aetSize) { /*
Search for the next top edge in the AET */
                                                rightEdge =
aetData[aetStart];
                                                /* If we have an
intersection use the intersection edge */
                                                if(nextIntersection->xValue
<= rightEdge->xValue) {
                                                        rightEdge =
nextIntersection;
                                                        break;
                                                }
                                                aetStart++;
                                                /* Check if this edge is on
top */
                                                assert(fillList->firstFace);
                                                {
                                                        double xValue =
rightEdge->xValue * B3D_FixedToFloat;
                                                        B3DPrimitiveFace
*topFace = fillList->firstFace;
                                                        if(
rightEdge->leftFace == topFace ||

rightEdge->rightFace == topFace ||

rightEdge->zValue < zValueAt(topFace, xValue, yValue))
                                                                break; /*
rightEdge is on top */
                                                }
                                                /* If the edge is not on
top toggle its (back) fills */

b3dToggleBackFills(fillList, rightEdge, yValue, nextIntersection);
                                                rightEdge = NULL;
                                        }
                                        /*-- end of search for next top
edge --*/

                                        /*-- Now do the drawing from
leftEdge to rightEdge --*/
#if 1 /* This assert fails in rare cases; the fix is not understood. eem */
                                        assert(leftEdge && rightEdge);
#else
                                        if(!leftEdge || !rightEdge)

FAIL_UPDATING(B3D_NO_MORE_EDGES); // another segfault

//FAIL_PAINTING(B3D_NO_MORE_EDGES); // blow up in allocating edges
#endif
                                        if(fillList->firstFace) {
                                                /* Note: We fill
*including* leftX and rightX */
                                                int leftX =
(leftEdge->xValue >> B3D_FixedToIntShift) + 1;
                                                int rightX =
(rightEdge->xValue >> B3D_FixedToIntShift);

So I'm pretty sure that we went thru:

                                                /* If the edge is not on
top toggle its (back) fills */

b3dToggleBackFills(fillList, rightEdge, yValue, nextIntersection);
                                                rightEdge = NULL;

if intention of #if 1... assert(...) was probably to debug the case of
failure...
But it was left in the release. Maybe it would be safer to replace with
#ifdef DEBUG or something like that?
Eliot?

Le sam. 28 déc. 2019 à 23:04, Nicolas Cellier <
nicolas.cellier.aka.nice at gmail.com> a écrit :

> Running SqueakDebug.app in lldb, I get:
>
> Process 61332 stopped
> * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS
> (code=1, address=0x18)
>     frame #0: 0x09f17c07 Squeak3D`b3dMainLoop(state=0x09f1928c,
> stopReason=0) at b3dMain.c:1407:32
>    1404 if(fillList->firstFace) {
>    1405 /* Note: We fill *including* leftX and rightX */
>    1406 int leftX = (leftEdge->xValue >> B3D_FixedToIntShift) + 1;
> -> 1407 int rightX = (rightEdge->xValue >> B3D_FixedToIntShift);
>    1408 B3DPrimitiveFace *topFace = fillList->firstFace;
>    1409
>    1410 if(leftX < 0) leftX = 0;
> Target 0: (Squeak) stopped.
> (lldb) print rightEdge
> (B3DPrimitiveEdge *) $0 = 0x00000000
> (lldb) print leftEdge
> (B3DPrimitiveEdge *) $1 = 0x0617a49c
> (lldb) bt
> * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS
> (code=1, address=0x18)
>   * frame #0: 0x09f17c07 Squeak3D`b3dMainLoop(state=0x09f1928c,
> stopReason=0) at b3dMain.c:1407:32
>     frame #1: 0x09f0cb23 Squeak3D`b3dStartRasterizer at Squeak3D.c:1704:12
>     frame #2: 0x05d01398
>     frame #3: 0x00002f52 Squeak`interpret at gcc3x-cointerp.c:2749:3
>
> So why rightEdge is a null pointer?
> If you already examined the code, you may have an idea...
>
> Le sam. 28 déc. 2019 à 22:55, Nicolas Cellier <
> nicolas.cellier.aka.nice at gmail.com> a écrit :
>
>> Hi Stephane,
>> I confirm that I can reproduce the crash with an up-to-date VM (on OSX).
>>
>> Le sam. 28 déc. 2019 à 19:20, Stéphane Rollandin <lecteur at zogotounga.net>
>> a écrit :
>>
>>>
>>> Hello all,
>>>
>>> For my first post on this list, I would like to submit the bug that
>>> prevents one of my game (this one:
>>> http://www.zogotounga.net/comp/guardians.htm) to work reliably.
>>>
>>> I spent quite some time on this, and I ended up with a nice and concise
>>> way to crash the Squeak3D plugin. The image is available at:
>>>
>>> http://www.zogotounga.net/swap/crashlab3.zip
>>>
>>> Instructions to crash the VM are detailed therein.
>>>
>>> The 3D scene is very simple, there are only three objects. The crash
>>> seems related to a race condition, because inserting a simple delay in
>>> the code prevents it - this is detailed in the image itself.
>>>
>>> I have spent days studying the Smalltalk code, and could not see
>>> anything wrong with what is sent to the plugin. It looks to my
>>> uninformed eyes as a problem of shared memory, where one rasterizing
>>> operation messes up with another happening in parallel (although no
>>> processed is being forked on the Smalltalk side).
>>>
>>>
>>> Stef
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20191228/c1d2a240/attachment-0001.html>


More information about the Vm-dev mailing list