[squeak-dev] Float precision

Markus Lampert markuslampert at yahoo.com
Wed Dec 22 18:05:19 UTC 2010


Thanks, I was hoping I did something wrong ... ;)

Markus



----- Original Message ----
> From: David T. Lewis <lewis at mail.msen.com>
> To: The general-purpose Squeak developers list 
><squeak-dev at lists.squeakfoundation.org>
> Sent: Wed, December 22, 2010 5:38:01 AM
> Subject: Re: [squeak-dev] Float precision
> 
> On Wed, Dec 22, 2010 at 08:04:11AM +0100, Michael Haupt wrote:
> >  Hi,
> > 
> > Am 22.12.2010 um 06:19 schrieb Markus Lampert <markuslampert at yahoo.com>:
> >  > Evaluating the following:
> > > 
> > > 0.0 to: 10 by: 0.1 do:  [ :v |
> > >    Transcript show: v printString; crlf]
> >  > 
> > > gives some interesting numbers for 3.6, 3.7, 3.8, 3.9 and  everything above 
>5.5.
> > > Is this expected given the rounding effects  of Float?
> > 
> > inaccuracies are always expected with floating-point  numbers. If you want 
>accurate results, you should use BigDecimal (IIRC) or  SmallInteger and manual 
>decimal point management. 
>
> > 
> >  Best,
> > 
> > Michael
> 
> Yes. And that would be ScaledDecimal (not  BigDecimal).
> 
> This is a common source of confusion. Wikipedia gives a good  overview
> of the topic:
> 
> http://en.wikipedia.org/wiki/Floating_point
> 
> Dave
> 
> 





More information about the Squeak-dev mailing list