[squeak-dev] Rounding floats

tim Rowledge tim at rowledge.org
Mon Jul 18 23:20:08 UTC 2016


I know that it’s a hard thing to deal with floating point numbers around the fiddly edges, but are we really unable to do a decent job of rounding floats to reasonable numbers of decimal places? This can’t be a rare thing to need to do.

For example, I’m reading temperature values for various places from the openweathermap.org site and many of them are specified to 2dp, which seems a bit optimistic for a weather site. But if I try, for example
292.37 roundTo: 0.1
I get the very unhelpful 292.40000…00003 (with too many 0s to count by eye). I understand that one option is to use formatted printing of the number but that doesn’t seem practical for this usage right now. 


So, numerics aficionados, can we do any better than the sensible seeming but unsuccessful code in Number>>roundTo: ?

tim
--
tim Rowledge; tim at rowledge.org; http://www.rowledge.org/tim
Ubi dubium ibi libertas



More information about the Squeak-dev mailing list