Shouldn't 7r6.5 be the same as 7r65 * 0.1
Wolfgang Helbig
helbig at Lehre.BA-Stuttgart.DE
Tue Jul 4 18:11:04 UTC 2006
Klaus, you asked:
>Would some good soul please explain what
>
> Number readFrom: '7r6.5'
And here is the good soul's answer:
7r6.5 = 6*7^0 + 5 * 7^(-1) = 6 + 5/7 = 6.714285714285... with the last
6 digits repeating at infinitum.
Whereas 7r65 * 0.1 should evaluate to
7r65 * 0.1 = (6*7^1 + 5*7^0) * 10^-1 = (6*7 + 5) / 10 = 4.7
(Notation: "^" is the exponentation operator, like 3^2 = 3 * 3 = 9).
Greetings,
Wolfgang
--
"Dijkstra is right, but you don't say such things!"
(A less courageous programmer)
More information about the Squeak-dev
mailing list
|