[squeak-dev] truncatedTo: / roundTo: oddities with fractions

Nicolas Cellier nicolas.cellier.aka.nice at gmail.com
Mon Aug 12 14:16:43 UTC 2019


truncateTo: 0.01s or 1/100 is effectively less surprising... but is it good
enough?

66.66 truncatesTo: 0.01s as 66.65s.
It might look surprising, bt it's not because 66.66 < (6666/100).
Does it always work?

99.99 < 99.99s.
-> true

so we might expect that it truncates to 99.98.s
99.99 truncateTo: 0.01s
-> 99.99s

Nope...
(0 to: 10000) count: [:i | (i/100) asFloat < (i/100) and: [((i/100) asFloat
truncateTo: 0.01s) = (i/100)]].
-> 3720

For the other way (truncate to lower when it should not), I has to go to
5th digit, try this:

(1 to: 100000) select: [:i | (i/100000) asFloat >= (i/100000) and:
[((i/100000) asFloat truncateTo: 0.00001s) < (i/100000)]].

The last one is very interesting:

1.0 truncateTo: 0.00001s
-> 0.99999s5

We have 1.0 = 1, so we're dealing with exact decimal fraction operands, and
we could expect an exact result...
...but the intermediate 0.00001s asFloat used in (truncateTo:) does not.

One must use (aFloat asFraction truncateTo: 0.00001s) for exactness.




Le jeu. 1 août 2019 à 18:20, Chris Cunningham <cunningham.cb at gmail.com> a
écrit :

>
>
> On Wed, Jul 31, 2019 at 6:06 AM Levente Uzonyi <leves at caesar.elte.hu>
> wrote:
>
>> On Wed, 31 Jul 2019, David T. Lewis wrote:
>>
>> > On Wed, Jul 31, 2019 at 11:53:38AM +0000, Rein, Patrick wrote:
>> >> Hi everyone,
>> >>
>> >> a student recently encountered the following behavior:
>> >>
>> >> fraction := 5 / 9.
>> >> (fraction* 100) truncateTo: 0.01. ???55.550000000000004"
>> >> fraction:= 1/18.
>> >> (fraction * 100) roundTo: 0.01. ???5.5600000000000005"
>> >>
>> >> This is not what I would expect from #truncateTo: or #roundTo: even
>> when using Floats (especially roundTo:).
>> >>
>> >> Is this what we want or an open issue? I have not found any test
>> covering the protocol.
>> >>
>> >
>> > The results are unexpected, but not wrong.
>> >
>> > Why unexpected? When I look at the expression, I intuitively expect
>> > it to perform decimal arithmetic, and it does not do that.
>> >
>> > Why not wrong? 0.01 is a float, even though it may have been intended
>> > as an exact decimal number by the writer. Mixing float values in any
>> > arithmetic expression produces an inexact float result, as it should.
>> >
>> > To get a result that is both correct and intuitively right, the
>> expression
>> > might better be written like this:
>> >
>> >  5 / 9 * 100 truncateTo: (1/100) ==> (1111/20)
>>
>> I think ScaledDecimal is a better fit, as it gives a decimal result:
>>
>> 5 / 9 * 100 truncateTo: 0.01s "==> 55.55s2"
>>
>
> To be precise, it still gives you the same result as the fraction result,
> but it look like you would expect (i.e., it prints as a decimal, properly
> truncated).
>
> But, yes, probably a better fit.
>
> -cbc
>
>>
>>
>> Levente
>>
>> >
>> > Dave
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20190812/f254988a/attachment.html>


More information about the Squeak-dev mailing list