[squeak-dev] truncatedTo: / roundTo: oddities with fractions

Chris Cunningham cunningham.cb at gmail.com
Thu Aug 1 16:20:22 UTC 2019


On Wed, Jul 31, 2019 at 6:06 AM Levente Uzonyi <leves at caesar.elte.hu> wrote:

> On Wed, 31 Jul 2019, David T. Lewis wrote:
>
> > On Wed, Jul 31, 2019 at 11:53:38AM +0000, Rein, Patrick wrote:
> >> Hi everyone,
> >>
> >> a student recently encountered the following behavior:
> >>
> >> fraction := 5 / 9.
> >> (fraction* 100) truncateTo: 0.01. ???55.550000000000004"
> >> fraction:= 1/18.
> >> (fraction * 100) roundTo: 0.01. ???5.5600000000000005"
> >>
> >> This is not what I would expect from #truncateTo: or #roundTo: even
> when using Floats (especially roundTo:).
> >>
> >> Is this what we want or an open issue? I have not found any test
> covering the protocol.
> >>
> >
> > The results are unexpected, but not wrong.
> >
> > Why unexpected? When I look at the expression, I intuitively expect
> > it to perform decimal arithmetic, and it does not do that.
> >
> > Why not wrong? 0.01 is a float, even though it may have been intended
> > as an exact decimal number by the writer. Mixing float values in any
> > arithmetic expression produces an inexact float result, as it should.
> >
> > To get a result that is both correct and intuitively right, the
> expression
> > might better be written like this:
> >
> >  5 / 9 * 100 truncateTo: (1/100) ==> (1111/20)
>
> I think ScaledDecimal is a better fit, as it gives a decimal result:
>
> 5 / 9 * 100 truncateTo: 0.01s "==> 55.55s2"
>

To be precise, it still gives you the same result as the fraction result,
but it look like you would expect (i.e., it prints as a decimal, properly
truncated).

But, yes, probably a better fit.

-cbc

>
>
> Levente
>
> >
> > Dave
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20190801/779d15f9/attachment.html>


More information about the Squeak-dev mailing list