[Vm-dev] Integer overflow with BitBlt rule 20 and depth 32

[David T. Lewis]

On Fri, Oct 23, 2009 at 11:24:49AM -0300, Juan Vuletich wrote:
> 
> Ok. This is my first try at this. I went back to my old 6809 assembly 
> language book to remember by 2's complement aritmethic. The bit pattern 
> of the result of addition and substraction is not altered by considering 
> a number signed or unsigned. The only operations that are affected are 
> multiplication and comparisons. rgbMul works ok because it will never 
> use the most significant bit (the sign bit). So I added the correct 
> types only on those operations that needed to do correct comparisons. I 
> also added the check for overflow in rgbAdd (the only place where it is 
> needed).
> 
> I'm not sure if we should add the types everywhere, or it is ok to add 
> them just to a few functions as I did. I'm running out of time today, 
> anybody who can try to build a VM with this and test it, please do. (I 
> didn't!)

It should be OK to add the type declarations only where they are actually
needed for comparison and multiply.

Just FYI it is worth noting that the default sqInt is an 64 bit long
when generating code for a 64 bit image VM, so it is generally good
practice to explicitly declare the int and unsigned types when doing
32 bit arithmetic. Unfortunately this looks like it would be a huge
amount of tedious work for the bitblt plugin, so I would not worry
about it for now (but it would be good to make some unit tests for
these problems so it will be possible to validate the fixes on a 64
bit image later).

I'll try building a VM this weekend if nobody else has gotten to it by
then. Note, I have zero expertise with bitblt (but I may still be able
to help with the debugging).

Dave