[squeak-dev] Rounding floats

Nicolas Cellier nicolas.cellier.aka.nice at gmail.com
Tue Jul 19 00:26:39 UTC 2016


Hi Tim,
you can't round float that easily, they are not multiple of 1/5.
But you can round their printed decimal representation.
That's fair because this is what you want: print the damn float

292.37 printShowingMaxDecimalPlaces: 1


Technically, 2924*0.1 and 2924/10.0 differ.
The second one is guaranteed to be the closest float to (2924/10) because
there's only one inexact operation involved (the division).
The first one, not so, it's 2924 * (1/10.0), two inexact operations
Remember, 1/10 = 0.1 is false...
That's why roundTo: is failing: it uses the second formula.

Otherwise, of course, you would use ScaledDecimals or Fractions...

239.37 roundTo: 0.1s.
(239.37 roundTo: 1/10) asFloat.

But you want to print rounded, so just print rounded, don't bother with
round.

2016-07-19 1:20 GMT+02:00 tim Rowledge <tim at rowledge.org>:

> I know that it’s a hard thing to deal with floating point numbers around
> the fiddly edges, but are we really unable to do a decent job of rounding
> floats to reasonable numbers of decimal places? This can’t be a rare thing
> to need to do.
>
> For example, I’m reading temperature values for various places from the
> openweathermap.org site and many of them are specified to 2dp, which
> seems a bit optimistic for a weather site. But if I try, for example
> 292.37 roundTo: 0.1
> I get the very unhelpful 292.40000…00003 (with too many 0s to count by
> eye). I understand that one option is to use formatted printing of the
> number but that doesn’t seem practical for this usage right now.
>
>
> So, numerics aficionados, can we do any better than the sensible seeming
> but unsuccessful code in Number>>roundTo: ?
>
> tim
> --
> tim Rowledge; tim at rowledge.org; http://www.rowledge.org/tim
> Ubi dubium ibi libertas
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20160719/0acdcab3/attachment.htm


More information about the Squeak-dev mailing list