[Vm-dev] [OpenSmalltalk/opensmalltalk-vm] Mixed SmallInteger/Float comparison is inexact in Spur64 when jitted (#417)

[David T. Lewis]

To check my understanding, the strategy can be summarized as:

When comparing an integer to a float, first check if the integer can be cast
to a float without loss of precision.  If so, then treat both values as
floating point, and compare the two float values. Otherwise fall back to
existing comparison logic.

This sounds like a good optimization to me. As you have said, it can
be done in the VM because the VM already understands the word sizes
and object storage formats.

Dave


On Tue, Aug 20, 2019 at 05:19:47PM -0700, Nicolas Cellier wrote:
>  
> Comparison of SmallInteger and Float is simple in Spur32, we just have to convert the SmallInteger to a (double), which is an exact operation, and perform the comparison of (double).
> 
> But it's more tricky in Spur64 because SmallInteger has more precision (61 bits) than Float (53 bits) and thus conversion to (double) might be inexact...
> 
> Example:
> 
> 	si := 1<<(Float precision + 2)+1.
> 	sf := si asFloat.
> 	self deny: sf = si.
> 	self deny: si = sf.
> 
> It's not obvious to find where the trick happens in the VM, because all code at upper level is equivalent to trivial asFloat conversion followed by comparison, and there are many possible path!
> 
> Let's concentrate on <=
> 
> SmallInteger receiver:
>    - #primitiveLessOrEqual
>    - #genPrimitiveLessOrEqual (JIT)
> BoxedFloat64 receiver:
>    - #primitiveFloatLessOrEqual
>    - #genPrimitiveFloatLessOrEqual (JIT)
> SmallFloat64 receiver:
>    - #primitiveSmallFloatLessOrEqual
>    - #genPrimitiveSmallFloatLessOrEqual (JIT)
> All receivers (non jitted sender or stack VM only?):
>    - #bytecodePrimLessOrEqual
> 
> For generated C code, the protection is programmed in lower level #loadFloatOrIntFrom:
> 
> To check that it does not exceed 53 bits, we shift the integer left << and forth >> and verify integrity. If not, we fail the primitive.
> With shift length as programmed currently, the primitive will fail for 52 bits and above. It should fail for 54 bits and above, but that's a detail, the primitive will fail more often than necessary...
> 
> The problem is that there is no such exactness check for jitted primitives!
> 
> I have sketched a solution in Squeak inbox (See http://source.squeak.org/inbox/Kernel-nice.1260.diff), demonstrated here for SmallInteger <=
> 
>     ^(asFloat := self asFloat) = aNumber
>          ifTrue: [self <= aNumber truncated]
>          ifFalse: [asFloat <= aNumber]
> 
> In C code that is:
> 
>     if ( (double) si == sf ) return si <= (int64) sf;
>     else return (double) si <= sf;
> 
> It works for all the comparisons (though = and ~= could be a bit simpler, no need to perform the comparison twice).
> 
> And it's simple enough to be jitted (I have prototyped it this evening).
> 
> 
> 
> 
> 
> 
> -- 
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly or view it on GitHub:
> https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/417