[squeak-dev] Re: The joys (or not) of floating point numbers

Nicolas Cellier nicolas.cellier.aka.nice at gmail.com
Tue Mar 26 13:05:31 UTC 2013


2013/3/26 Stéphane Rollandin <lecteur at zogotounga.net>:
> In my image I have:
>
> precision: aNumber
>
>         | pten |
>
>         self isInfinite ifTrue: [^ self].
>
>         pten := 10 raisedTo: aNumber.
>         ^ (pten * self) rounded / pten asFloat
>
>
> 54.25125260474855 precision: 1 ==> 54.3
>
> What do you think of it Nicolas ? Does it make sense ?
>
> Stef
>

Yes, it's better than roundTo: 0.1.
Pharo has implemented a similar method named #round:
http://code.google.com/p/pharo/issues/detail?id=5590
It's still not perfect because it cumulates two inexact operations:
- one inexact operation in the multiplication pten * self
- one inexact operation in the division / pten

Example (0.995 round: 2) -> 1.00 though 0.995 < (995/1000) so it
should round to 0.99.

Nicolas


More information about the Squeak-dev mailing list