[Newbies] Re: [BUG] Inconsistent float soustraction
cdrick65 at gmail.com
Sun Feb 17 17:49:59 UTC 2008
> > Hi,
> > I noticed in my image (damien last beta so 3.10 - windows XP and
> > Vista) that I have a strange bug when soustracting floats
> > successively. Here is how I reproduce it in a workspace:
> > 1 - 0.2 -0.05 -0.3 = 0.45 "true"
> > 1 - 0.2 -0.05 -0.3 -0.1= 0.35 "true"
> > 1 - 0.2 -0.05 -0.3 -0.1 - 0.10= 0.25 "*****false*****"
> > 1 - 0.2 -0.05 -0.3 -0.1 - 0.10= 0.24999999999999995 true
> > or again:
> > 1 - 0.1 = 0.9 true
> > 1 - 0.1 - 0.1 = 0.8 true
> > 1 - 0.1 - 0.1 - 0.1 = 0.7 false
> > Can somebody explain that or is it a bug ? I don't know yet if it's
> > specific to my image, so does other have the same behavior?
> > I can force the rounding of the result but find it strange.
> > Cédrick
2008/2/17, danil osipchuk <danil.osipchuk at gmail.com>:
> This is not even squeak specific. Decimal floats not always can be
> represented precisely in binary form:
> Hi Cédrick,
> if you need to operate with exact precision just don't trust floats. Use ScaledDecimals instead. In Squeak ScaledDecimals are implemented with the fraction so it's matematically stronger than floats. Use floats at presentation or user interface time and evade them at domain model.
> Sebastian Sastre
uhm ok, good to know.
Actually, it was just a test that failed so I was surprised.
I still find the result odd.
Seeing ScaledDecimal>>testConvertFromFloat, I'm wandering if it would
be better to have Float>>= considering egality if the difference is
less than 1.0e-9.
aNumber isNumber ifFalse: [^ false].
^ (aNumber asFloat - self) abs < 1.0e-9
I would prefer having = working as I expect, even if it would be wrong
for smaller values than 1.0e-9. For strict egality, == can be used.
Is it possible to disable a primitive call like here (it's optimized no ?) ?
Anyway, thanks for your answers.
More information about the Beginners