[squeak-dev] truncatedTo: / roundTo: oddities with fractions
Marcel Taeumel
marcel.taeumel at hpi.de
Wed Jul 31 12:18:09 UTC 2019
Hi, all.
I think the issue can be reduced to:
555 * 0.01 "5.55"
5555 * 0.01 "55.550000000000004"
... because #quo: on a Fraction seems to work as expected.
((5/9) * 100 "(500/9)" / 0.01 "5555.555555555556") truncated "5555" * 0.01 "55.550000000000004"
Best,
Marcel
Am 31.07.2019 13:53:49 schrieb Rein, Patrick <patrick.rein at hpi.de>:
Hi everyone,
a student recently encountered the following behavior:
fraction := 5 / 9.
(fraction* 100) truncateTo: 0.01. “55.550000000000004"
fraction:= 1/18.
(fraction * 100) roundTo: 0.01. “5.5600000000000005"
This is not what I would expect from #truncateTo: or #roundTo: even when using Floats (especially roundTo:).
Is this what we want or an open issue? I have not found any test covering the protocol.
Bests
Patrick
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20190731/163c604b/attachment.html>
More information about the Squeak-dev
mailing list
|