On 16.07.2009, at 23:48, USeITWes wrote:
I was in hurry with the post above, so I kept it brief. I apologize for being unclear, but now I can elaborate. The member in question is using the read out from a pH probe that gives a measurement which is alway between 0 (acid) and 14 (base) which gives 7 digits of precision. He then tried to create a script to calculate the concentration of hydrogen ions [H+]. The formula to calculate that is 10^-pH = [H+]. For a substance that is extremely basic, say a 12.6 on the pH scale, that would result in a concentration of 0.000000000000251. He attempted to use that result in another script and another equation and received a divide-by-zero exception.
I hope that clears up any confusion. Thanks for the prompt response Mr. Cherlin. I will try your solution immediately.
The precision setting only affects the display. However, for some properties there is some rounding going on. Can you provide the script that failed?
- Bert -