[BUG] Fraction>>asFloat returns infinity if numerator and
denominator both very large
Martin McClure
martin at hand2mouse.com
Mon Oct 9 14:39:14 UTC 2000
I ran into this surprising result last night:
aFraction rounded
returned a reasonable value (318, IIRC) but
aFraction asFloat
returned Infinity.
I dug a bit and found that the cause is that both the denominator and
numerator were very large.
A test case:
((10 raisedToInteger: 309) + 1 / (10 raisedToInteger: 309)) rounded
returns 1, which seems correct, but
((10 raisedToInteger: 309) + 1 / (10 raisedToInteger: 309)) asFloat
returns Infinity, which does not seem correct. I expect to get
exactly 1.0 in this case, since that is the Float value that most
closely approximates the Fraction's value.
Fraction>>asFloat reads:
asFloat
"Answer a Float that represents the same value as does the receiver."
| df scaleFactor scaledDenominator |
df _ denominator asFloat.
^ df isInfinite
ifTrue:
["might be representable as a denormalized Float"
scaleFactor _ 2 raisedToInteger: 53.
scaledDenominator _ (denominator /
scaleFactor) rounded.
numerator asFloat / scaleFactor asFloat /
scaledDenominator asFloat]
ifFalse:
[numerator asFloat / df]
This code assumes that if the denominator isInfinite, that the
numerator is *not* Infinite. Scaling down both numerator and
denominator before converting would avoid this problem, but it's not
clear to me exactly how much to scale to avoid losing precision
unnecessarily.
If the more numerically-minded folks on the list have an opinion on
this, please send it, otherwise I'll give it more thought this
evening.
And a historical note: Andres found a problem in Fraction>>asFloat a
couple of years ago, copied below, which may or may not be the same
problem.
-Martin
At 10:08 PM -0300 11/30/98, sqrmax at cvtci.com.ar wrote:
>Hi.
>
>I found a problem with Fraction>>asFloat. And I've found a part of the
>problem. Integer>>asFloat for largeIntegers is broken. It can
>consider as few as
>49 bits of data when fraction parts of floats are 52 + implicit bits long.
>This can happen when the largeInteger in question is longer than 7 bytes. This
>is fixed, and a faster asFloat was developed. It can be more than 20% faster
>than its original counterpart, plus it answers correctly.
>
>But Fraction>>asFloat still needs work.
>
>I include my profiler so you can measure the difference with precision.
>
>Andres.
More information about the Squeak-dev
mailing list
|