[Vm-dev] [OpenSmalltalk/opensmalltalk-vm] Uninitialized scale in B3DShaderPlugin>>computeDirection (#434)
Nicolas Cellier
notifications at github.com
Sun Oct 20 22:35:20 UTC 2019
As the compiler tells:
```
OpenSmalltalk/opensmalltalk-vm/src/plugins/Squeak3D/Squeak3D.c:2475:7: warning: variable 'scale' is used uninitialized whenever 'if' condition is false [-Wsometimes-uninitialized]
if (!((l2vDistance == 0.0)
^~~~~~~~~~~~~~~~~~~~~~
/media/psf/Home/Smalltalk/OpenSmalltalk/opensmalltalk-vm/src/plugins/Squeak3D/Squeak3D.c:2480:42: note: uninitialized use occurs here
l2vDirection[0] = ((l2vDirection[0]) * scale);
^~~~~
```
That's effectively a bug in Smalltalk source, scale might be used uninitialized:
```
computeDirection
"Compute the direction for the current light and vertex"
| scale |
<inline: true>
<var: #scale type: #double>
(lightFlags anyMask: FlagPositional) ifTrue:[
"Must compute the direction for this vertex"
l2vDirection at: 0 put: (litVertex at: PrimVtxPositionX) - (primLight at: PrimLightPositionX).
l2vDirection at: 1 put: (litVertex at: PrimVtxPositionY) - (primLight at: PrimLightPositionY).
l2vDirection at: 2 put: (litVertex at: PrimVtxPositionZ) - (primLight at: PrimLightPositionZ).
"l2vDistance := self dotProductOf: l2vDirection with: l2vDirection."
l2vDistance := ((l2vDirection at: 0) * (l2vDirection at: 0)) +
((l2vDirection at: 1) * (l2vDirection at: 1)) +
((l2vDirection at: 2) * (l2vDirection at: 2)).
(l2vDistance = 0.0 or:[l2vDistance = 1.0])
ifFalse:[ l2vDistance := l2vDistance sqrt.
scale := -1.0/l2vDistance].
l2vDirection at: 0 put: (l2vDirection at: 0) * scale.
l2vDirection at: 1 put: (l2vDirection at: 1) * scale.
l2vDirection at: 2 put: (l2vDirection at: 2) * scale.
] ifFalse:[
(lightFlags anyMask: FlagDirectional) ifTrue:[
l2vDirection at: 0 put: (primLight at: PrimLightDirectionX).
l2vDirection at: 1 put: (primLight at: PrimLightDirectionY).
l2vDirection at: 2 put: (primLight at: PrimLightDirectionZ).
].
]
```
IMO:
- the scaling is not needed when distance is 0 or 1
- the scaling should be inside the block when distance is neither 0 nor 1
- the scaling should be positive
If I decipher correctly `l2vDirection` means light to vertex direction, that is:
LV vector = OV vector - OL vector
whatever origin O, and that is what we already do in code (PrimVtxPositionX - PrimLightPositionX). So I don't understand the -1.0 here...
If there is a -1.0, then we must also initialize scale to -1.0 when distance is 1.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/issues/434
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20191020/601be31d/attachment.html>
More information about the Vm-dev
mailing list