Shouldn't 7r6.5 be the same as 7r65 * 0.1

Wolfgang Helbig helbig at Lehre.BA-Stuttgart.DE
Tue Jul 4 18:11:04 UTC 2006


Klaus, you asked:
>Would some good soul please explain what
>
>	Number readFrom: '7r6.5'

And here is the good soul's answer:

7r6.5 = 6*7^0 + 5 * 7^(-1) = 6 + 5/7 = 6.714285714285... with the last
6 digits repeating at infinitum.

Whereas 7r65 * 0.1 should evaluate to
7r65 * 0.1 = (6*7^1 + 5*7^0) * 10^-1 = (6*7 + 5) / 10 = 4.7

(Notation: "^" is the exponentation operator, like 3^2 = 3 * 3 = 9).

Greetings,
Wolfgang

--
"Dijkstra is right, but you don't say such things!"
(A less courageous programmer)




More information about the Squeak-dev mailing list