Floating point
Ned Konz
ned at bike-nomad.com
Tue Dec 17 01:36:51 UTC 2002
On Monday 16 December 2002 05:19 pm, Ned Konz wrote:
> Notably, since there are a limited number of bits of precision and
> numbers are represented by a binary fraction and an exponent, you
> can only exactly represent a small subset of the possible real
> numbers.
ruby -e 'printf("%0.40f", 1.1)'
1.1000000000000000888178419700125232338905
--
Ned Konz
http://bike-nomad.com
GPG key ID: BEEA7EFE
More information about the Squeak-dev
mailing list
|