Numerics question: reading floating point constants

stéphane ducasse ducasse at iam.unibe.ch
Sun Apr 9 07:34:02 UTC 2006

```Hi andreas

I knew you much stronger on your statements :) so why would not it be
a bug?
Do you imply that we could have an implementation justification of
float? I'm puzzled because reading seems to me the only easy things
with floats :)

Stef

> This post is one for the numerics fraction. I was just in the
> process of transcribing some interesting numerical code from C to
> Squeak while I was running into an issue with floating point
> constants. The code that I have has constants like here:
>
>   pio2_2  =  6.07710050630396597660e-11, /* 0x3DD0B461, 0x1A600000 */
>
> but naively transcribing this into, e.g.,
>
>   pio22 := 6.07710050630396597660e-11.
>
> turned out to get some of the expected results wrong. Why? Because
> the bit pattern is different, e.g., what we get is this:
>
>   6.07710050630396597660e-11 hex => '3DD0B4611A5FFFFF'
>
> Notice how the last hex digits are different.
>
> My question here is the following: While I can understand why
> *printing* might give different results based on the algorithm
> chosen (e.g., what amount of error do we allow when printing
> floating point numbers etc) is it to be expected that *reading* has
> similar properties? In particular considering that, e.g.,
>
>   (607710050630396597660 * 1.0e-31) hex => '3DD0B4611A600000'
>
> gives the "correct" bit pattern makes me think that there might be
> something broken in our way of reading floating point numbers.
>
> Obviously, in a case like mine the best way to work around this
> problem is to construct the floats from the bit patterns directly
> but I can't stop but wonder what the impact of a subtle bug like
> this might be on code which does *not* include the hex denotations
> for the relevant bit patterns.
>
> So. Is this a bug?

```