[Seaside-dev] Troubles with WANumberPrinter>>printFloat:on:

[Lukas Renggli]

>> I raised this a few months ago, but I'm still having trouble with it. Squeak
>> represents Floats as 64bit, while VW represents them as 32bit. This causes
>> the printFloat:on: method to lose precision.
>
> I am pretty sure you can find a failing testcase for Squeak too.  It
> took me a lot of time to get float printing right on gst.

Don't tell me. The goal of WANumberPrinter is to have a platform
independent way to print numbers in a wide variety of ways. At the
time I wrote the code I did not realize that different Smalltalk
dialects use different sizes for Float. That feels like being thrown
back to C, where you never knew the size of an int ;-)

The code under discussion is in fact based on a discussion I had with
Nicolas Cellier. He is the expert in number conversion and number
crunching. He tole me me:

> A naive and exact algorithm could be to use:
>
>  aFloat asTrueFraction roundTo: (base raisedTo: precision) reciprocal.

The problem here is that #asTrueFraction (in some Smalltalk it is
called #asExactFraction) can be costly and is not part of ANSI. Thus I
took the other solution he was proposing:

> Another solution used by Number>>#printShowingDecimalPlaces:
> is to perform above operations in Floating point arithmetic.
>
>  aFloat roundTo: (base asFloat raisedTo: precision) reciprocal.
>
> It will be correct in most cases, but not always! See my last note
> at http://bugs.squeak.org/view.php?id=5640.

> Here are my concerns:
> a) We can't really expect floats to behave this nicely across platforms and smalltalks

I don't know, I guess I was wrongly assuming that rounding errors are
no issue when printing to a few digits only.

> b) I don't entirely know if my proposed solution is right

It makes some of the tests fail in Squeak. So that is not a solution either.

Maybe Nicolas has an idea? ;-)

Lukas

-- 
Lukas Renggli
http://www.lukas-renggli.ch