[Vm-dev] [OpenSmalltalk/opensmalltalk-vm] Uninitialized scale in B3DShaderPlugin>>computeDirection (#434)

Nicolas Cellier notifications at github.com
Sun Oct 20 22:35:20 UTC 2019


As the compiler tells:

```
OpenSmalltalk/opensmalltalk-vm/src/plugins/Squeak3D/Squeak3D.c:2475:7: warning: variable 'scale' is used uninitialized whenever 'if' condition is false [-Wsometimes-uninitialized]
                if (!((l2vDistance == 0.0)
                    ^~~~~~~~~~~~~~~~~~~~~~
/media/psf/Home/Smalltalk/OpenSmalltalk/opensmalltalk-vm/src/plugins/Squeak3D/Squeak3D.c:2480:42: note: uninitialized use occurs here
                l2vDirection[0] = ((l2vDirection[0]) * scale);
                                                       ^~~~~

```

That's effectively a bug in Smalltalk source, scale might be used uninitialized:

```
computeDirection
	"Compute the direction for the current light and vertex"
	| scale |
	<inline: true>
	<var: #scale type: #double>
	(lightFlags anyMask: FlagPositional) ifTrue:[
		"Must compute the direction for this vertex"
		l2vDirection at: 0 put: (litVertex at: PrimVtxPositionX) - (primLight at: PrimLightPositionX).
		l2vDirection at: 1 put: (litVertex at: PrimVtxPositionY) - (primLight at: PrimLightPositionY).
		l2vDirection at: 2 put: (litVertex at: PrimVtxPositionZ) - (primLight at: PrimLightPositionZ).
		"l2vDistance := self dotProductOf: l2vDirection with: l2vDirection."
		l2vDistance := ((l2vDirection at: 0) * (l2vDirection at: 0)) +
						((l2vDirection at: 1) * (l2vDirection at: 1)) +
							((l2vDirection at: 2) * (l2vDirection at: 2)).
		(l2vDistance = 0.0 or:[l2vDistance = 1.0]) 
			ifFalse:[	l2vDistance := l2vDistance sqrt.
					scale := -1.0/l2vDistance].
		l2vDirection at: 0 put: (l2vDirection at: 0) * scale.
		l2vDirection at: 1 put: (l2vDirection at: 1) * scale.
		l2vDirection at: 2 put: (l2vDirection at: 2) * scale.
	] ifFalse:[
		(lightFlags anyMask: FlagDirectional) ifTrue:[
			l2vDirection at: 0 put: (primLight at: PrimLightDirectionX).
			l2vDirection at: 1 put: (primLight at: PrimLightDirectionY).
			l2vDirection at: 2 put: (primLight at: PrimLightDirectionZ).
		].
	]
```

IMO:
- the scaling is not needed when distance is 0 or 1
- the scaling should be inside the block when distance is neither 0 nor 1
- the scaling should be positive

If I decipher correctly `l2vDirection` means light to vertex direction, that is:

    LV vector = OV vector - OL vector

whatever origin O, and that is what we already do in code (PrimVtxPositionX - PrimLightPositionX). So I don't understand the -1.0 here...
If there is a -1.0, then we must also initialize scale to -1.0 when distance is 1.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/434
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20191020/601be31d/attachment.html>


More information about the Vm-dev mailing list