[squeak-dev] Float rounding issue

Chris Muller ma.chris.m at gmail.com
Mon May 7 15:16:20 UTC 2012


Ok, thanks Darius (and Levente).  At first I thought we had a
regression but it looks like we don't..   :)


On Mon, May 7, 2012 at 12:51 AM, Darius Clarke <socinian at gmail.com> wrote:
> I don't believe roundTo: is the method you're looking for.
> I've been confused by this before.
>
> roundTo: quantum
> "Answer the nearest number that is a multiple of quantum."
>
> ^(self / quantum) rounded * quantum
>
> Maybe try this:
>
> roundedToFloat: quantum
> "Round to decimal places."
> | aReciprocal |
> aReciprocal := quantum reciprocal rounded.
>
> ^(self * aReciprocal) rounded / aReciprocal asFloat
>
>
>
> On Sun, May 6, 2012 at 9:20 PM, Levente Uzonyi <leves at elte.hu> wrote:
>>
>> On Sun, 6 May 2012, Chris Muller wrote:
>>
>>> ( 0.9580000000000001 roundTo: 0.0001) = 0.9580   "false"
>>
>>
>> Still didn't look at the code, but 0.0001 is also a Float and it doesn't
>> have an exact representation in base 2. The following works as expected:
>> (0.9580000000000001 roundTo: 0.0001s4) "0.9580s4"
>> (0.9580000000000001 roundTo: 1/10000) "(479/500)"
>>
>>
>> Levente
>>
>>>
>>>
>>>
>>> On Sun, May 6, 2012 at 8:55 PM, Levente Uzonyi <leves at elte.hu> wrote:
>>>>
>>>> On Sun, 6 May 2012, Chris Muller wrote:
>>>>
>>>>> With the latest trunk updates, I'm having trouble rounding to my
>>>>> desired precision:
>>>>>
>>>>>    0.9580000000000001 roundTo: 0.0001    " 0.9580000000000001"
>>>>>
>>>>> In Squeak 4.3 I get what I expect:
>>>>>
>>>>>    0.9580000000000001 roundTo: 0.0001   " 0.958"
>>>>>
>>>>>
>>>>
>>>> I'm pretty sure it's just a difference in printing. 0.958 is unlikely to
>>>> have an exact base 2 representation.
>>>>
>>>>
>>>> Levente
>>>>
>>>
>>
>>
>>
>


More information about the Squeak-dev mailing list