[squeak-dev] Float storage as Double?

Levente Uzonyi leves at elte.hu
Mon Apr 26 10:24:56 UTC 2010


On Mon, 26 Apr 2010, Lawson English wrote:

> Levente Uzonyi wrote:
>> On Mon, 26 Apr 2010, Lawson English wrote:
>> 
>>> I've been playing around with the Mandelbrot set and I noticed that the 
>>> implementation using Float looked somewhat vague. After researching a 
>>> while, it turns out (corrections welcome) that while Float performs 64-bit 
>>> FP calculations, only a limited number of decimal places are stored for 
>>> subsequent calculations. Is there a way around this?
>> 
>> That's not true, every Float stores 64 bits. Just because some digits are 
>> not shown in the printString, it doesn't mean they aren't there:
>> 
>> 1.01 printString. "==> '1.01'"
>> 1.01 printShowingDecimalPlaces: 60. "==> 
>> '1.010000000000000008881784197001252323389053344726562500000000'"
>> 
>
> I'm not sure what this means:
>
> 1.4142135623730950488016887242096980785696718753769480 
> printShowingDecimalPlaces: 60
> '1.414213562373095145474621858738828450441360473632812500000000'

It means that your number cannot be represented accurately as a Float. 
Details here:
http://en.wikipedia.org/wiki/Floating_point#Representable_numbers.2C_conversion_and_rounding

>
>
>> 
>>> 
>>> is there a faster way to truncate a ScaledDecimal than the above?
>> 
>> Sure, don't convert it to strings. :) I think you shouldn't use 
>> ScaledDecimal at all, since it's just a combination of a fraction and 
>> scale. You only need the fraction. Use #truncateTo: to truncate a number.
> Still confused, sorry.
>
> Even with 15 decimal places, a Float-based M set generator is going to be 
> inaccurate after a few levels of zoom.  Fractions don't truncate the way I 
> need them to.

How would you like them to truncate?


Levente

>
> Lawson
>
>
>
>



More information about the Squeak-dev mailing list