[squeak-dev] Float storage as Double?
Levente Uzonyi
leves at elte.hu
Mon Apr 26 10:24:56 UTC 2010
On Mon, 26 Apr 2010, Lawson English wrote:
> Levente Uzonyi wrote:
>> On Mon, 26 Apr 2010, Lawson English wrote:
>>
>>> I've been playing around with the Mandelbrot set and I noticed that the
>>> implementation using Float looked somewhat vague. After researching a
>>> while, it turns out (corrections welcome) that while Float performs 64-bit
>>> FP calculations, only a limited number of decimal places are stored for
>>> subsequent calculations. Is there a way around this?
>>
>> That's not true, every Float stores 64 bits. Just because some digits are
>> not shown in the printString, it doesn't mean they aren't there:
>>
>> 1.01 printString. "==> '1.01'"
>> 1.01 printShowingDecimalPlaces: 60. "==>
>> '1.010000000000000008881784197001252323389053344726562500000000'"
>>
>
> I'm not sure what this means:
>
> 1.4142135623730950488016887242096980785696718753769480
> printShowingDecimalPlaces: 60
> '1.414213562373095145474621858738828450441360473632812500000000'
It means that your number cannot be represented accurately as a Float.
Details here:
http://en.wikipedia.org/wiki/Floating_point#Representable_numbers.2C_conversion_and_rounding
>
>
>>
>>>
>>> is there a faster way to truncate a ScaledDecimal than the above?
>>
>> Sure, don't convert it to strings. :) I think you shouldn't use
>> ScaledDecimal at all, since it's just a combination of a fraction and
>> scale. You only need the fraction. Use #truncateTo: to truncate a number.
> Still confused, sorry.
>
> Even with 15 decimal places, a Float-based M set generator is going to be
> inaccurate after a few levels of zoom. Fractions don't truncate the way I
> need them to.
How would you like them to truncate?
Levente
>
> Lawson
>
>
>
>
More information about the Squeak-dev
mailing list
|