[squeak-dev] Float precision

David T. Lewis lewis at mail.msen.com
Wed Dec 22 13:38:01 UTC 2010


On Wed, Dec 22, 2010 at 08:04:11AM +0100, Michael Haupt wrote:
> Hi,
> 
> Am 22.12.2010 um 06:19 schrieb Markus Lampert <markuslampert at yahoo.com>:
> > Evaluating the following:
> > 
> > 0.0 to: 10 by: 0.1 do: [ :v |
> >    Transcript show: v printString; crlf]
> > 
> > gives some interesting numbers for 3.6, 3.7, 3.8, 3.9 and everything above 5.5.
> > Is this expected given the rounding effects of Float?
> 
> inaccuracies are always expected with floating-point numbers. If you want accurate results, you should use BigDecimal (IIRC) or SmallInteger and manual decimal point management. 
> 
> Best,
> 
> Michael

Yes. And that would be ScaledDecimal (not BigDecimal).

This is a common source of confusion. Wikipedia gives a good overview
of the topic:

http://en.wikipedia.org/wiki/Floating_point

Dave



More information about the Squeak-dev mailing list