[squeak-dev] The joys (or not) of floating point numbers

Louis LaBrunda Lou at Keystone-Software.com
Tue Mar 26 18:50:45 UTC 2013


On Tue, 26 Mar 2013 18:59:12 +0100, Nicolas Cellier
<nicolas.cellier.aka.nice at gmail.com> wrote:

>2013/3/26 Louis LaBrunda <Lou at keystone-software.com>:
>> On Tue, 26 Mar 2013 18:32:31 +0100, Stéphane Rollandin
>> <lecteur at zogotounga.net> wrote:
>>
>>>> Example (0.995 round: 2) -> 1.00 though 0.995 < (995/1000) so it
>>>> should round to 0.99.
>>>
>>>.. and I just noticed that
>>>
>>>       0.995 printShowingDecimalPlaces: 2   ==>  '1.00'
>>>
>>>Do we have a method returning the correct value ?
>>
>> That looks right to me, because 0.995 was rounded up.
>>
>> Lou
>
>No it's not, because 0.995 < 0.995s3 so it should be rounded down.
>In a recent trunk image, print is OK:
>(0.995 printShowingDecimalPlaces: 2) ->  '0.99'

In VA Smalltalk:

0.995 = 0.995s3 => true
0.995 < 0.995s3 => false

I think in VA Smalltalk there is some VM magic going on that makes the
above work the way it does.  The #= and #< of Float are implemented in a
primitive.  I guess when converting the '0.995' string to a float a little
is lost and that would make it less than 0.995s3 but there is a lot of code
floating around (sorry about the puns) to make floats look better.  In that
case I would think people would want (0.995 printShowingDecimalPlaces: 2)
to show '1.00' and not '0.99'.

Anyway, we should try not to use floats without a very, very good reason.
Like we have to send them outside of Smalltalk or we really need the speed
or decimals and fractions take up too much memory.  But then we must live
with their inaccuracies and display mess.

I have an untested theory that fractions can be close in speed to floats
because divisions (that are expensive) can be pushed to the end of a
computation because with fractions they are multiplies.

Lou
-----------------------------------------------------------
Louis LaBrunda
Keystone Software Corp.
SkypeMe callto://PhotonDemon
mailto:Lou at Keystone-Software.com http://www.Keystone-Software.com



More information about the Squeak-dev mailing list