Numerics question: reading floating point constants

Andreas Raab andreas.raab at gmx.de
Sun Apr 9 01:12:22 UTC 2006


Hi Folks,

This post is one for the numerics fraction. I was just in the process of 
transcribing some interesting numerical code from C to Squeak while I 
was running into an issue with floating point constants. The code that I 
have has constants like here:

   pio2_2  =  6.07710050630396597660e-11, /* 0x3DD0B461, 0x1A600000 */

but naively transcribing this into, e.g.,

   pio22 := 6.07710050630396597660e-11.

turned out to get some of the expected results wrong. Why? Because the 
bit pattern is different, e.g., what we get is this:

   6.07710050630396597660e-11 hex => '3DD0B4611A5FFFFF'

Notice how the last hex digits are different.

My question here is the following: While I can understand why *printing* 
might give different results based on the algorithm chosen (e.g., what 
amount of error do we allow when printing floating point numbers etc) is 
it to be expected that *reading* has similar properties? In particular 
considering that, e.g.,

   (607710050630396597660 * 1.0e-31) hex => '3DD0B4611A600000'

gives the "correct" bit pattern makes me think that there might be 
something broken in our way of reading floating point numbers.

Obviously, in a case like mine the best way to work around this problem 
is to construct the floats from the bit patterns directly but I can't 
stop but wonder what the impact of a subtle bug like this might be on 
code which does *not* include the hex denotations for the relevant bit 
patterns.

So. Is this a bug?

Cheers,
   - Andreas



More information about the Squeak-dev mailing list