[squeak-dev] Float plugin

Randal L. Schwartz merlyn at stonehenge.com
Tue Sep 9 16:35:01 UTC 2008


>>>>> "Ang" == Ang Beepeng <beepeng86 at yahoo.com> writes:

Ang> The primitive returns correct answer, but only until fourth or fifth decimal
Ang> points. For example, if the answer is "4.88888", primitive returns
Ang> "4.888880014419556". I added several steps, line 8 , 10 & 12, trying to
Ang> return or declare the correct data type or object both in C and also in
Ang> Smalltalk. But I still do not get the correct answer.

Ignoring the "float" vs "double" issue, you *do* realize that
4.88888 has no precise IEEE floating point representation, and thus
will never be *precisely* that value, right?

Even 0.1 has no precise representation... it has to be truncated to fit within
the 53(?) bits of precision.  So 0.1 added 10 times might not be precisely
1.0.  Most "flost to string" converters stop when they start getting into the
"imaginary noise" bits, and if you ask for a longer value, you'll get garbage.

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn at stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion



More information about the Squeak-dev mailing list