Float equality? (was: [BUG] Float NaN's)

Richard A. O'Keefe ok at cs.otago.ac.nz
Wed Sep 22 00:27:48 UTC 2004


I wrote:
	> C'mon, "what floating-point is" is NOT one of those things you are entitled
	> to be ignorant of and still call yourself a programmer.  Everyone who leaves
	> this department with a BSc or BAppSci has met floating-point and its dangers
	> (but not numerical analysis) in 2nd year.
	
lex at cc.gatech.edu wrote:
	I fear we thave to agree to disagree at that point. ...
	So, I think you can very well be a programmer, and even a computer
	scientist, without being good with floating point.  You should simply
	know that you suck at it.
	
Clearly, we agree.  What I was writing about was precisely knowing enough
about FP to know that you are not an expert.
	
I wrote, making this clear:
	> "The time required to really learn it" is a straw man.
	> What's at issue is "the time required to learn about it well enough to know
	> that it isn't decimal and it isn't real arithmetic, that it's finite and
	> approximate."  Two lectures to do a good job, one if pressed for time.
	
lex at cc.gatech.edu wrote:
	It's important, because if you can convince me that two days is all you
	need, then I will retract my rule of thumb.
	
Remember, I am *not* saying that two days is enough for someone to be	
really 'good with floating point', but that it is enough for someone	
to 'know that you suck at it' and have some idea of where to look for help.		
	
	> If you don't care what the answer comes out to, there is no point in
	> running a program, you might as well throw yarrow sticks.
	> That's very difference from tolerating a *limited* amount of approximation.
	
	Actually, there are a lot of programs where you don't really care what
	the answers are.  Textbook examples come to mind, where you are trying
	to make a point and not necessarily make a useful program.

You are talking about a case where there is some point in *writing* the
program, some point in *reading* the program, but little or no point in
*running* it.

	Graphics is my favorite example:  In graphics you often want it
	to look right, not to be "accurate" in most senses of the word.

That is *not* a case where you "don't really care what the answers answer".
It's a case where you care VERY VERY MUCH what the answers are, but they
do not have to be infinitely precise.  (In fact Lex and I are really on
the wrong sides for this example.  Getting graphics output to look right
often requires extremely careful attention to the detail of the FP
calculations.)

	Yet another example is the
	enemy strategy in a computer game.  Do you really care if a zombie
	occasionally moves right when it is "supposed" to move left?  In such
	cases there often isn't a "correct" answer at all.
	
We have a computer games paper here, COSC360.  If a zombie occasionally
moves right when it is supposed to move left, you look into the problem,
because you don't know what else it might do wrong.  Once you understand
why it's happening, you might accept this; several games written here have
included features that were originally discovered by accident.  But the
things were *recognised* as accidents precisely because there *was* an
expected behaviour.

	Throwing yarrow sticks, whatever those are, would be fine for these,
	too, but sounds more tiring than using a computer.
	
Yarrow sticks are basically dried stalks of the yarrow plant,
and are the randomising device traditionally used for consulting
the I Ching.  I regard that as irrational.

	Yes, that one calculation is fine, but it doesn't generalize.  Put some
	+ and -'s in there and the story changes.  Put in a loop and things get
	really hairy.  Calculating a simple integral is already enough to screw
	you up, if your calculation of (f(x+d) - f(x)) can involve a
	catastrophic cancelation.
	
It is still true that if your inputs are accurate to only 1 part in 10**4,
it is absurd to insist on precise output.

	And that's the problem.  Some simple calculations are safe and
	convenient with floating point, but how do you know which ones, and what
	are the odds that a complex calculation will only involve safe
	operations?
	
The answer, of course, is that if you know that FP has problems, you
*don't* write that code yourself.  You write the simple code that says
what the function *is* yourself, but you call an integration routine
you get from a library.  (Like CALGO, Netlib, &c.)
	
	>  If you represent that as a whole number of
	> cents, or a whole number of mills, or even a whole number of hundredths
	> of a cent, then guess what?  You just approximated the answer.  (In fact
	> you just got a *worse* approximation than Float would give you...)
	
	Yes, but now the approximation mechanism is very easy to understand. 

I turn your own argument back on you.  When *one* operation is involved,
this is true.  When you combine *many* operations with *many* roundings,
the overall effect is extremely difficult to understand, and requires the
*same* kinds of reasoning used to get FP calculations right.

	It seems to me that if you are rounding things off yourself, then any
	programmer will have a good handle on what is going on.

In the context of calculations involving many operations,
experience suggests that this is not so.

	If they use floating point, then they will likely get version 1
	of the program completed sooner, but they won't have any idea how
	accurate it is -- and it may well not be accurate at all.

This is why Fortran has INTERVAL arithmetic and why Pascal-SC (and the
corresponding package for Ada) has what are effectively interval and
triplex data types together with single-rounding dot product.  It is
possible, with the right tools, to have calculations which at the end
*tell* you how precise the answer is, and the right tools use floating-
point arithmetic as the basic building block.  I have played with
Pascal-SC and would really like to see that kind of stuff become mainstream.

The basic problem is that computers have finite precision arithmetic
and that operations like division, roots, compound interest, and so on
mean you have to cope somehow with approximation, and while you can do
that easily for *one* operation, calculations involving many operations
require at least a year of training before you can be trusted to get them
right.  And that applies just as much to calculations where intermediate
results are stored as integers as to ones where they are stored as floats.
If programming wasn't hard, who would pay us to do it?
	
It's just like complexity.  If you don't know about complexity, you
may in all seriousness write a bet-your-business program whose cost is
O(2**(2**n)).  I once met such a case, and was able to tell them what
they needed to know.  It was amazing and disheartening how many people
said 'if the program is short it should take a short time'.  But again,
if programming wasn't hard, who would pay us to do it?

The thing we should *really* be worrying about is spreadsheets...




More information about the Squeak-dev mailing list