[squeak-dev] Float precision

Michael Haupt mhaupt at gmail.com
Wed Dec 22 07:04:11 UTC 2010


Hi,

Am 22.12.2010 um 06:19 schrieb Markus Lampert <markuslampert at yahoo.com>:
> Evaluating the following:
> 
> 0.0 to: 10 by: 0.1 do: [ :v |
>    Transcript show: v printString; crlf]
> 
> gives some interesting numbers for 3.6, 3.7, 3.8, 3.9 and everything above 5.5.
> Is this expected given the rounding effects of Float?

inaccuracies are always expected with floating-point numbers. If you want accurate results, you should use BigDecimal (IIRC) or SmallInteger and manual decimal point management. 

Best,

Michael


More information about the Squeak-dev mailing list