[squeak-dev] Fwd: doesn't Primitive 33 primitiveQuoLargeIntegers relies on C Undefined Behaviour ? uncover a BUG in Primitive 32 primitiveDivLargeIntegers and maybe some others

Nicolas Cellier nicolas.cellier.aka.nice at gmail.com
Tue Aug 28 21:36:43 UTC 2012


Sorry to post here, this bounced at vm-dev
But I think it's important

Nicolas


---------- Forwarded message ----------
From: Nicolas Cellier <nicolas.cellier.aka.nice at gmail.com>
Date: 2012/8/28
Subject: doesn't Primitive 33 primitiveQuoLargeIntegers relies on C
Undefined Behaviour ? uncover a BUG in Primitive 32
primitiveDivLargeIntegers and maybe some others
To: vm-dev-request at lists.squeakfoundation.org


(1 << 63) negated is converted to signed long long
0x8000000000000000LL by signed64BitValueOf:

So far so good...
Then in primitiveQuoLargeIntegers, we will try to take absolute value
of this beast...

integerRcvr > 0 ifTrue: [
                integerArg > 0
                        ifTrue: [result := integerRcvr // integerArg]
                        ifFalse: [result := 0 - (integerRcvr // (0 -
integerArg))].
        ] ifFalse: [
                integerArg > 0
                        ifTrue: [result := 0 - ((0 - integerRcvr) //
integerArg)]
                        ifFalse: [result := (0 - integerRcvr) // (0 -
integerArg)].
        ].

That is translated straightfully in C...

    sqLong integerArg;
    sqLong integerRcvr;
    sqLong result;

snip...

        if (integerRcvr > 0) {
                if (integerArg > 0) {
                        result = integerRcvr / integerArg;
                }
                else {
                        result = 0 - (integerRcvr / (0 - integerArg));
                }
        }
        else {
                if (integerArg > 0) {
                        result = 0 - ((0 - integerRcvr) / integerArg);
                }
                else {
                        result = (0 - integerRcvr) / (0 - integerArg);
                }
        }

Isn't this relying on UB? like
http://stackoverflow.com/questions/2539178/why-is-abs0x80000000-0x80000000
At worse, I would expect 0 -  0x8000000000000000LL to be converted to
itself  0x8000000000000000LL...
So can someone explain why this works?

    ((1 << 63) negated quo: 2) = (1 << 62) negated

I guess it must be some compiler optimization, like:

    0 - (0-a)/b -> 0 - (0/b-a/b) -> a/b

But then, why doing the sign checks at all?
Isn't it just result = integerRcvr / integerArg; whatever the signs?

Maybe the compiler is able to just do that, but it sounds like obfuscation...

Anyway, that won't work in primitiveDivLargeIntegers, i would expect:
    ((1 << 63) negated // -2) =  (1 << 62)
But I get
    ((1 << 63) negated // -2) =  (1 << 62) negated

Ah Ah, it looks like a bug this time, no Compiler optimization did save us

We have several solutions...
- let signed64BitValueOf: (1 << 63) negated fail
- protect each and every negation of those
- perform most arithmetic with unsigned, handle sign separately

Nicolas


More information about the Squeak-dev mailing list