[Newbies] Floating point arithmetic

Richard Karpinski dick at cfcl.com
Sun Feb 17 22:18:06 UTC 2008


Would rounding help?

The problem is pretty well explained in that mailing, but surely,  
when we are doing arithmetic to two or three decimal places, there  
will be some way to ask that the result be rounded to that many  
places. As long as the numbers remain close to 1, it may be easier to  
deal with that than forcing the calculations into rational arithmetic.

I have observed rooms full of professors and hardware designers  
arguing over these details for months on end, eventually producing  
fifty to one hundred page documents to say how it should work. See  
IEEE-754 for a serious definition of a good computer arithmetic. It  
is indeed very hard to cover all the cases and make it useful as well.

Dick

On 2008, Feb 17, in beginners-request at lists.squeakfoundation.org:

>>> 1 - 0.2 -0.05 -0.3 -0.1= 0.35  "true"
>>> 1 - 0.2 -0.05 -0.3 -0.1 - 0.10= 0.25  "*****false*****"
>>> 1 - 0.2 -0.05 -0.3 -0.1 - 0.10= 0.24999999999999995   true
...
>>> Can somebody explain that or is it a bug ? I don't know yet if it's
>>> specific to my image, so does other have the same behavior?
>>> I can force the rounding of the result but find it strange.


More information about the Beginners mailing list