[Newbies] problem with numbers

[David T. Lewis]

On Mon, Jul 13, 2009 at 05:26:58PM -0300, JuanjoE wrote:
> Hello.
> I'm having a problem with numbers in squeak, I'm using a 3.10.2 image update
> #7179 on a vm 3.8.18b1 running on a OSX 10.5
> 
> If I open a workspace and evaluate 1203.22 - 1200 I get  3.220000000000027
> 
> It's seems that is something working wrong on this.
> 
> I tried with ScaledDecimal but when I print the numbers it appends a 's2'
> after it. I'm doing something like ((((1203.22 - 1200)*100) asInteger)/100)
> asFloat as a workarround, but I think that there may be a better way to work
> with this problem

Hi,

What you are seeing here seems confusing, but it is perfectly normal. The
number 1203.22 in your workspace is treated as a floating point number.
When you subtract an integer from it, the result is another floating
point number. The ...027 is just an artifact of the precision of floats.

In Squeak, floating point numbers are represented by class Float, and
are internally implemented as double precision float values.

If you want unlimited precision, your intuition was right in using a
ScaledDecimal instead of a Float. Interestingly, there as been a lot
of discussion recently on the squeak-dev and pharo lists about the
"right" way to handle this. Treating 1203.22 as a Float by default
meets the expectations of most people who do engineering sorts of
arithmetic, but is confusing for people who are not familiar with
the limits of floating point arithmetic precision.

Dave