# [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] Mixed SmallInteger/Float comparison is inexact in Spur64 when jitted (#417)

Nicolas Cellier notifications at github.com
Wed Aug 21 00:19:47 UTC 2019

```Comparison of SmallInteger and Float is simple in Spur32, we just have to convert the SmallInteger to a (double), which is an exact operation, and perform the comparison of (double).

But it's more tricky in Spur64 because SmallInteger has more precision (61 bits) than Float (53 bits) and thus conversion to (double) might be inexact...

Example:

si := 1<<(Float precision + 2)+1.
sf := si asFloat.
self deny: sf = si.
self deny: si = sf.

It's not obvious to find where the trick happens in the VM, because all code at upper level is equivalent to trivial asFloat conversion followed by comparison, and there are many possible path!

Let's concentrate on <=

- #primitiveLessOrEqual
- #genPrimitiveLessOrEqual (JIT)
- #primitiveFloatLessOrEqual
- #genPrimitiveFloatLessOrEqual (JIT)
- #primitiveSmallFloatLessOrEqual
- #genPrimitiveSmallFloatLessOrEqual (JIT)
All receivers (non jitted sender or stack VM only?):
- #bytecodePrimLessOrEqual

For generated C code, the protection is programmed in lower level #loadFloatOrIntFrom:

To check that it does not exceed 53 bits, we shift the integer left << and forth >> and verify integrity. If not, we fail the primitive.
With shift length as programmed currently, the primitive will fail for 52 bits and above. It should fail for 54 bits and above, but that's a detail, the primitive will fail more often than necessary...

The problem is that there is no such exactness check for jitted primitives!

I have sketched a solution in Squeak inbox (See http://source.squeak.org/inbox/Kernel-nice.1260.diff), demonstrated here for SmallInteger <=

^(asFloat := self asFloat) = aNumber
ifTrue: [self <= aNumber truncated]
ifFalse: [asFloat <= aNumber]

In C code that is:

if ( (double) si == sf ) return si <= (int64) sf;
else return (double) si <= sf;

It works for all the comparisons (though = and ~= could be a bit simpler, no need to perform the comparison twice).

And it's simple enough to be jitted (I have prototyped it this evening).

--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/417
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20190820/121d1c79/attachment.html>
```