## [BUG] Fraction>>asFloat returns infinity if numerator and denominator both very large

David N. Smith (IBM) dnsmith at watson.ibm.com
Mon Oct 9 20:47:51 UTC 2000

```At 7:39 -0700 10/9/00, Martin McClure wrote:
>I ran into this surprising result last night:
>
>  aFraction rounded
>
>returned a reasonable value (318, IIRC) but
>
>  aFraction asFloat
>
>returned Infinity.
>
>I dug a bit and found that the cause is that both the denominator and numerator were very large.
>
>A test case:
>
>  ((10 raisedToInteger: 309) + 1 / (10 raisedToInteger: 309)) rounded
>
>returns 1, which seems correct, but
>
>  ((10 raisedToInteger: 309) + 1 / (10 raisedToInteger: 309)) asFloat
>
>returns Infinity, which does not seem correct. I expect to get exactly 1.0 in this case, since that is the Float value that most closely approximates the Fraction's value.

I think I disagree. The first example is done entirely in integers and answers correctly.

The second becomes two Infinites divided by each other as they are being converted to float, which cannot answer a number no matter what. The code in #asFloat (below) checks only for the special case where the fraction, as written, might have a denormalized floating-point result, a special case allowed by IEEE.

I'm not quite sure what the '+1' in the numerator is for. If it was to force a valid float over the limit and make it Infinite, it fails since 10^309 is already infinite. The highest IEEE double positive value is:

16r1FFFFFFFFFFFFF * (2 raisedToInteger: 16r3FF - 52)

which converts to the floating point number:

1.797693134862315e308.

Let's take this number with or without the added 1. The one makes no difference at all since it will change only the right most digit of 308 digits (ignoring overflow) and a float represents only about 15 digits. See the third line of each:

| f |
f := 16r1FFFFFFFFFFFFF * (2 raisedToInteger: 16r3FF - 52).
(f + 1 / f) asFloat
1.0

and:

| f |
f := 16r1FFFFFFFFFFFFF * (2 raisedToInteger: 16r3FF - 52).
(f / f) asFloat
1.0

To force the value to be too big we can do this. First, the case where we don't add anything (see second line):

| g |
g := 16r1FFFFFFFFFFFFF + 0 * (2 raisedToInteger: 16r3FF - 52).
g asFloat
1.797693134862315e308

then the case where we add 1:

| h |
h := 16r1FFFFFFFFFFFFF + 1 * (2 raisedToInteger: 16r3FF - 52).
h asFloat
Infinity

>
>
>asFloat
>	"Answer a Float that represents the same value as does the receiver."
>
>	| df scaleFactor scaledDenominator |
>	df _ denominator asFloat.
>	^ df isInfinite
>		ifTrue:
>			["might be representable as a denormalized Float"
>			scaleFactor _ 2 raisedToInteger: 53.
>			scaledDenominator _ (denominator / scaleFactor) rounded.
>			numerator asFloat / scaleFactor asFloat / scaledDenominator asFloat]
>		ifFalse:
>			[numerator asFloat / df]
>
>This code assumes that if the denominator isInfinite, that the numerator is *not* Infinite.  Scaling down both numerator and denominator before converting would avoid this problem, but it's not clear to me exactly how much to scale to avoid losing precision unnecessarily.
>
>If the more numerically-minded folks on the list have an opinion on this, please send it, otherwise I'll give it more thought this evening.

I may be misreading this but I think it will answer Infinite if the numerator is Infinite, regardless of whether or not the denominator is infinite.

Note that if a and b are fractions each with more than 308 digits, then (a/b) might have a result with much smaller numerator and denominator due to reduction. Then an expected Infinity might not happen. Try (g/h) asFloat (as defined in the last two examples). The numerator and denominator are much smaller than 308 digits.

If you want to see inside a Fraction try #asTrueFraction which answers a fraction exactly representing the floating-point number. The numbers in such examples often make more sense in hex, so sometimes try:

float asTrueFraction hex

It is also useful to have a description of the IEEE double floating point number handy.

Have fun!

Dave
_______________________________
David N. Smith
IBM T J Watson Research Center
Hawthorne, NY
_______________________________
Any opinions or recommendations
herein are those of the author
and not of his employer.

```