[Newbies] problem with numbers

[johnps11 at bigpond.com]

Try searching on "subtractive cancellation" on google for information on
strategies to avoid this kind of thing.  One is to treat (a-b) as
(a^2 - b^2)/(a+b), which can reduce the relative error in some cases.
Unfortunately the exact strategy often depends on the numbers used and the
hardware it runs on.


> On Mon, Jul 13, 2009 at 05:26:58PM -0300, JuanjoE wrote:
>> Hello.
>> I'm having a problem with numbers in squeak, I'm using a 3.10.2 image
>> update
>> #7179 on a vm 3.8.18b1 running on a OSX 10.5
>>
>> If I open a workspace and evaluate 1203.22 - 1200 I get
>> 3.220000000000027
>>
>> It's seems that is something working wrong on this.
>>
>> I tried with ScaledDecimal but when I print the numbers it appends a
>> 's2'
>> after it. I'm doing something like ((((1203.22 - 1200)*100)
>> asInteger)/100)
>> asFloat as a workarround, but I think that there may be a better way to
>> work
>> with this problem
>
> Hi,
>
> What you are seeing here seems confusing, but it is perfectly normal. The
> number 1203.22 in your workspace is treated as a floating point number.
> When you subtract an integer from it, the result is another floating
> point number. The ...027 is just an artifact of the precision of floats.
>
> In Squeak, floating point numbers are represented by class Float, and
> are internally implemented as double precision float values.
>
> If you want unlimited precision, your intuition was right in using a
> ScaledDecimal instead of a Float. Interestingly, there as been a lot
> of discussion recently on the squeak-dev and pharo lists about the
> "right" way to handle this. Treating 1203.22 as a Float by default
> meets the expectations of most people who do engineering sorts of
> arithmetic, but is confusing for people who are not familiar with
> the limits of floating point arithmetic precision.
>
> Dave
>
> _______________________________________________
> Beginners mailing list
> Beginners at lists.squeakfoundation.org
> http://lists.squeakfoundation.org/mailman/listinfo/beginners
>