[squeak-dev] truncatedTo: / roundTo: oddities with fractions

Marcel Taeumel marcel.taeumel at hpi.de
Wed Jul 31 12:18:09 UTC 2019


Hi, all.

I think the issue can be reduced to:

555 * 0.01 "5.55"
5555 * 0.01 "55.550000000000004"

... because #quo: on a Fraction seems to work as expected.

((5/9) * 100 "(500/9)" / 0.01 "5555.555555555556") truncated "5555" * 0.01 "55.550000000000004"


Best,
Marcel
Am 31.07.2019 13:53:49 schrieb Rein, Patrick <patrick.rein at hpi.de>:
Hi everyone,

a student recently encountered the following behavior:

fraction := 5 / 9.
(fraction* 100) truncateTo: 0.01. “55.550000000000004"
fraction:= 1/18.
(fraction * 100) roundTo: 0.01. “5.5600000000000005"

This is not what I would expect from #truncateTo: or #roundTo: even when using Floats (especially roundTo:).

Is this what we want or an open issue? I have not found any test covering the protocol.

Bests
Patrick

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20190731/163c604b/attachment.html>


More information about the Squeak-dev mailing list