[Vm-dev] #FastCPrimitiveFlag in SHA2Plugin (was: Next Release Candidate?)

Marcel Taeumel marcel.taeumel at hpi.de
Fri Dec 10 17:35:13 UTC 2021


Hi all --

Well, this is weird (plugins/SHA2Plugin/SHA2Plugin.c, generated):

174: (((unsigned long *) (((unsigned long long*) bytes))))[i] = (SQ_SWAP_8_BYTES(((((unsigned long *) doubleWords))[i])));

Why does this work on 32-bit platforms (ILP32)?
Why does this work on 64-bit Windows (LLP64)?
Why does this work on 64-bit Linux (LP64)?

Anyway. I tried to debug the strange behavior of "primitiveCopyDoubleWordsIntoBytesBigEndian" in the debug build on Windows.

End of July 2021, the #FastCPrimitiveFlag was set in that method. (CryptographyPlugins-eem.23) Maybe that was not a good idea. Or maybe we just found a bug in the "Fast C primitives" mechanism.

Eliot? :-)

I just removed the flag but the generated code (see above) still cuts of stuff like some type casting went wrong. (debug build)

!!! I removed all #FastCPrimitiveFlag from the SHA2Plugin. And the VM does not crash anymore! (fast build) But those tests bother me. See above that line of generated code. It's broken.

***

I suspect an interference of #FastCPrimitiveFlag and a Clang compiler optimization flag.

Best,
Marcel
Am 10.12.2021 15:47:24 schrieb Marcel Taeumel <marcel.taeumel at hpi.de>:
Hi all --

So, I am trying to debug this on Windows. I want to figure out why the SHA-384 and beyond are not working in the debug build.



Here is the code:

#(newMD5 newSHA1 newSHA224 newSHA256 newSHA384 newSHA512 newSHA512p224 newSHA512p256) collect: [:sym |
sym -> ((HashFunction perform: sym) hmac
key: 'key';
hashMessage: '') hex] as: OrderedDictionary

Best,
Marcel

Am 10.12.2021 15:30:49 schrieb Marcel Taeumel <marcel.taeumel at hpi.de>:
Interestingly, the debug build for Windows produces test failures. Debug builds on both Windows 10 and Ubuntu 18.04 have the same errors.


ERRORS
CryptoHashFunctionTest>>#testHMAC
CryptoHashFunctionTest>>#testHMACMD5Spec
CryptoHashFunctionTest>>#testHMACSHA1Spec
CryptoHashFunctionTest>>#testHMACSHA256Spec
CryptoHashFunctionTest>>#testHMACSHA512Spec
CryptoHashFunctionTest>>#testLargeSHA1
CryptoHashFunctionTest>>#testMD5
CryptoHashFunctionTest>>#testSHA1
CryptoHashFunctionTest>>#testSHA256
CryptoHashFunctionTest>>#testSHA512

Example #testMAC
MessageNotUnderstood: ByteArray >> asInteger

Is there some code missing here?

FAILURES (Windows 10 only)
HMACTest>>#testEmptyStringWithKeyKey
SHA384WithSHA2PluginTest>>#testInputStream
SHA384WithSHA2PluginTest>>#testInputs
SHA512WithSHA2PluginTest>>#testInputStream
SHA512WithSHA2PluginTest>>#testInputs
SHA512p224WithSHA2PluginTest>>#testInputStream
SHA512p224WithSHA2PluginTest>>#testInputs
SHA512p256WithSHA2PluginTest>>#testInputStream
SHA512p256WithSHA2PluginTest>>#testInputs

Example #testInputs
Expected: cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e
Actual: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Why is this working in the debug build on Ubuntu?
Am 10.12.2021 15:06:41 schrieb Marcel Taeumel <marcel.taeumel at hpi.de>:
Hmm... Clang seems to be part of the problem. A gcc build on Ubuntu 18.04 did not crash.

squeak.cog.spur_win64x64 (Windows 10 21H2)
Clang 13.0.0 - CRASH
Clang 8.0.1 - CRASH

squeak.cog.spur_linux64x64 (Ubuntu 18.04)
gcc 7.5.0 - OK (with some failing tests)
Clang 9.0.0 - CRASH
Am 10.12.2021 14:40:26 schrieb Marcel Taeumel <marcel.taeumel at hpi.de>:
Hi Levente --

Windows VM crashes, too. But at a different test: #testHMACSH512Spec.



And the filename for the crash-dmp is broken.

Best,
Marcel

Am 10.12.2021 13:43:45 schrieb Levente Uzonyi <leves at caesar.elte.hu>:

Hi Marcel,

The SHA2 plugin (primitiveSHA256ProcessBufferUpdatingHash) still
crashes with that VM on 64-bit linux.
The plugin code works with earlier versions, so it's either a VM
change of the past 6-9 months, a code generator bug or a compiler bug IMO.

To reproduce the crash, evaluate the following:

Installer ss
project: 'Registers';
install: 'Registers-Core'.
Installer ss
project: 'Cryptography';
addPackage: 'CryptographyHashing';
addPackage: 'CryptographyHashingTests';
install.
(Smalltalk classNamed: #SHA256WithSHA2PluginTest) run: #testInputs

Interestingly another test (SHA512WithSHA2PluginTest) using a very
similar primitive but with DoubleWords works fine. So perhaps it's an
alignment issue.

Assert and debug VMs do not have that issue, so it's not that easy to
debug it. What I found was that buffer's value at
https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/99f1116c0f7a4ba9a0bf88cf7deb276325aa2d79/src/plugins/SHA2Plugin/SHA2Plugin.c#L277
was not the expected pointer, so the subsequent copying into it resulted
in segmentation fault.


Levente


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20211210/c01fa01a/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 309840 bytes
Desc: not available
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20211210/c01fa01a/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 148630 bytes
Desc: not available
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20211210/c01fa01a/attachment-0003.png>


More information about the Vm-dev mailing list