[squeak-dev] [Vm-dev] SHA512 squeak implementation?

Robert robert.withers at pm.me
Mon Mar 9 20:38:44 UTC 2020


Hi Levente,

Have you seen my other posts on ProCrypto and the work I did to the 
Hashing package? After I ported your code, the size went up to 730 kb, 
while the rest of Cryptography was about 270 kb. So there was a lot of 
bloat. As we already have solutions for SHA1 and SHA256, I really only 
needed SHA512. So I removed all the other classes and tests and ported 
SHA512 onto the HashFunction defined in CryptoCore. I removed all the 
RGThirtyTwoBitRegisters and I plan to move this ported SHA512 off of 
using a RGSixtyFourBitRegister, to eliminate that code. The size is 
currently 204 kb for Hashing. I do not know how this plays with your 
plugin, but there should just be a SHA512Plugin.

Thanks for digging in and writing your plugin! Can it be changed to just 
doing SHA512, easily?

Kindly,
Robert

On 3/9/20 3:11 PM, Levente Uzonyi wrote:
> Hi Robert,
>
> During the weekend I wrote a new plugin, SHA2Plugin, which covers all
> cases of SHA2, therefore obsoletes SHA256Plugin. The code is available on
> SqueakSource:
> http://www.squeaksource.com/Cryptography/CryptographyPlugins-ul.18.mcz
>
> I've updated the HAHasher package with hash variants using the new plugin
> (and the old MD5 and SHA256 plugins), and split it up to Core and Tests,
> so that the tests with the huge strings can be loaded optionally.
> After building a VM with the new plugin, load:
>
> Installer ss
>       project: 'Registers';
>       install: 'Registers';
>       project: 'Hasher';
>       install: 'HAHasher-Core';
>       install: 'HAHasher-Tests'.
>
>
> However, SHA2Plugin is not complete yet. There's an issue with the VM's
> code generator affecting 32-bit platforms, and I want to change the
> methods swapping endianness before the plugin is added to the VMs.
>
> On Sat, 7 Mar 2020, Robert wrote:
>
>> Hi Levente,
>>
>> Between this and your HashFunctions and my email in response to Jakob, I
>> have merged your work into Cryptography, published as:
>>
>> Name: Cryptography-v5.3-rww.121
>> Author: rww
>> Time: 7 March 2020, 12:13:00.184627 pm
>> UUID: 2b69a713-eac3-4680-9d00-8104eea5a7da
>> Ancestors: Cryptography-v5.3-rww.120
>>
>> Ported in the Registers & HAHasher packages and merged into
>> Cryptography. CryptoGreen!
> I suppose my point didn't get through. In my opinion, the code in
> Hasher/HAHasher package should replace Cryptography's HashFunction and
> subclasses. Why?
> - it's a single package already (except for its dependency on Registers,
> but I see no problem with Cryptography having dependencies)
> - slimmer: fewer, simpler methods per class
> - faster: uses the highly optimized Registers package when plugins are not
> available; has plugin support for all implemented hashes (SHA2Plugin);
> highly optimized in general
> - way more hashes: supports all SHA-2 variants except for SHA-512/t for
> arbitrary t other than 224, 256, 384, but that can be implemented as well
> if there's need for it.
> - has more comments and tests (e.g. check out SHA1 hashMessage: '' in
> Cryptography)
> - no bad tricks (e.g. check out SHA256 new class, or MD5 new class)
>
>> I am now looking to recategorize to fragment Cryptography. Future work
>> should be released to the whole enchilada Cryptography package and
>> copied to the correct fragment. For example reorging, rebasing and
>> renaming HASHA512 to SHA512. As I fragment I will be adding required
>> packages. It seems like we are unable to specify a version to the
>> required package. Is this the case?
>>
>> QUESTION TO CRYPTOGRAPHY TEAM: How do y'all feel about fragmenting
>> Cryptography? Pros? Cons?
>>
>> ---
>>
>> Here is a profile of Cryptography and I see many leaves for the various
>> Registers, so there may be some optimization that could occur.
> Registers is a highly optimized package, which should perform better
> than ThirtyTwoBitRegister, especially on 64-bit platforms. If you think
> you can do even better in pure Smalltalk, I'm all ears.
>
>
> Levente
>
>> - 6250 tallies, 6325 msec.
>>
>> **Leaves**
>> 7.5% {476ms} RGSixtyFourBitRegister64>>loadFrom:
>> 7.5% {472ms} Random>>nextBytes:into:startingAt:
>> 6.8% {427ms} LargePositiveInteger(Integer)>>bitShift:
>> 5.2% {328ms} [] SystemProgressMorph(Morph)>>updateDropShadowCache
>> 4.4% {280ms} RGSixtyFourBitRegister64>>bitXor:
>> 3.5% {220ms} Random>>generateStates
>> 3.0% {192ms} RGThirtyTwoBitRegisterTest64(TestCase)>>timeout:after:
>> 2.9% {181ms} RGSixtyFourBitRegister64>>+=
>> 2.8% {178ms} HASHA256Inlined64>>processBuffer
>> 2.7% {170ms} RGThirtyTwoBitRegister64>>loadFrom:
>> 2.2% {140ms} RGThirtyTwoBitRegister64>>+=
>> 2.1% {134ms} RGSixtyFourBitRegisterTest64(TestCase)>>assert:description:
>> 2.1% {133ms} Point>>=
>> 1.7% {109ms} RGThirtyTwoBitRegister64>>bitXor:
>> 1.7% {107ms} SHA1>>hashStream:
>> 1.5% {96ms} RGSixtyFourBitRegister64>>leftRotateBy:
>> 1.4% {89ms} RGSixtyFourBitRegister64>>load:
>> 1.4% {89ms} RGSixtyFourBitRegister32>>load:
>> 1.4% {89ms} SmallInteger(Number)>>negative
>> 1.4% {88ms} []
>> RGThirtyTwoBitRegisterTest64(RGRegisterTest)>>testLeftRotateBy
>> 1.2% {77ms} LargePositiveInteger>>*
>> 1.2% {76ms} GrafPort>>fillRoundRect:radius:
>> 1.1% {70ms} GrafPort>>copyBits
>> 1.1% {67ms} DisplayScreen(Form)>>depth
>>
>> **Memory**
>>      old            +0 bytes
>>      young        +2,071,872 bytes
>>      used        +2,071,872 bytes
>>      free        -2,071,872 bytes
>>
>> **GCs**
>>      full            1 totalling 74 ms (1.17% uptime), avg 74 ms
>>      incr            223 totalling 43 ms (0.7% uptime), avg 0.2 ms
>>      tenures        5,503 (avg 0 GCs/tenure)
>>      root table    0 overflows
>>
>> K, r
>>
>> On 3/7/20 9:39 AM, Robert wrote:
>>> Hi Levente,
>>>
>>> Thanks for your comments, I totally understand your views. I have some
>>> response, but the bottom line is that I or others in Cryptography
>>> (looking for new college age recruits...) will harvest some from your
>>> packages. It may only be SHA512 that we port over to round out the hash
>>> function offerings. I appreciate your willingness to allow us to do that.
>>>
>>> Your main argument is that Cryptography is too bloated, both with
>>> obsolete algorithms as well as combining the various facets into one
>>> package. The prime function of the Cryptography package is to have one
>>> package with all the Crypto we have, for single click loading. This also
>>> supports the Crypto toolbox/sandbox approach providing an educational
>>> environment to learning Crypto. Crypto is a learning package, as well as
>>> being fully functional.
>>>
>>> Cryptography, while larger than most packages, comes in at 300 kb, with
>>> the recent addition of Blowfish. That is loaded into an image file that
>>> is 47 MB! This means that Cryptography is just 0.6% the size of the
>>> image. This does not register to me as too bloated. But I see your
>>> perspective with outdated Crypto being present as well as the
>>> unnecessary add-ons, like X09 and ASN1; ciphers and hashes.
>>>
>>> When looking to do end to end encryption, we would load all of
>>> Cryptography (304 kb), ParrotTalk (93 kb), SSL (350 kb), Telnet (101 kb)
>>> & SSH (49 kb), for a total of 900 kb. That is only 1.9 % of the total
>>> image. Note that as I migrate SSL & SSH over onto the ParrotTalk
>>> framework, I expect to see the sizes of those two packages drastically
>>> reduce, through adoption of reuse.
>>>
>>> 1) having other hashes completes the scope of Crypto even if not utilized.
>>>
>>> 2) I look forward to diving deeper and understand your point here as
>>> well as see how your instances are created.
>>>
>>> 3) I would port Registers, as well.
>>>
>>> The absence of an ability to easily link dependencies provides a
>>> challenge to breaking Cryptography up. If only Monticello allowed for
>>> dependencies,  It is doable to break up Cryptography.
>>>
>>> So let us imagine a future where Cryptography stabilizes again. Before
>>> declaring Crypto stable, I would include adding the Signal encryption
>>> needs into the Crypto package. This would include the double ratchet
>>> block cipher mode. As that point of stability is reached, we could
>>> preserve the total package for one click loading and also reduce and
>>> break it up into pieces. How would it sound to you if old obsolete
>>> functions & ciphers are removed, then the ASN1 and X509 is split off. At
>>> this point to your point let us consider splitting off the ciphers and
>>> the hash functions and the randomizers, leaving a Crypto base. Then a
>>> pro user would load Crypto-base, Randomizers, Hash Functions (No MD2...)
>>> and Ciphers (No DES...).
>>>
>>> I tried to use your example of use of the Installer to load Cryptography
>>> and it did not work. It could not find the versions.
>>>
>>> Installer ss
>>>       	project: 'Cryptography';
>>>       	install: 'Cryptography-v5.3'.
>>>
>>> If this could be made to work, then small scripts could load this split up Crypto pro solution. Optionally loading the different pieces.
>>>
>>> Installer ss
>>>       	project: 'Cryptography';
>>>       	install: 'Cryptography-base-v5.3';
>>>       	install: 'Cryptography-core-v5.3';
>>>       	install: 'Cryptography-hash-v5.3';
>>>       	install: 'Cryptography-cipher-v5.3';
>>>       	install: 'Cryptography-ASN1-v5.3';
>>>       	install: 'Cryptography-X509-v5.3';
>>>       	install: 'ParrotTalk';
>>>       	install: 'SSL';
>>>       	install: 'Telnet';
>>>       	install: 'SSH';
>>>       	install: 'Signal'.
>>>
>>> Kindly,
>>> Robert
>>>
>>>
>>>
>>> On 3/7/20 5:20 AM, Levente Uzonyi wrote:
>>>> Hi Robert,
>>>>
>>>> On Thu, 5 Mar 2020, Robert wrote:
>>>>
>>>>> Oh yes, Levente, I recall speaking with you about it. I would like to
>>>>> make a proposal. Do you think you could fold all those hash functions,
>>>>> without the HA, into the Cryptography library? We have a HashFunction
>>>>> class in there, I do not know how different they may be in their public
>>>>> interface. I think it would be valuable to combine them. To support TLS
>>>>> 1.3, we would also need elliptical Diffie-Hellmans, I think.
>>>>>
>>>>> Levente, would you be willing to fold your work into Cryptography?
>>>> The reason why I created a separate package was that I found the
>>>> Cryptography package too bloated. Cryptographic hash functions seem to be
>>>> more commonly needed than ciphers, CSPRNGS, ASN1, etc.
>>>>
>>>> It is possible to replace HashFunction and subclasses from Cryptography
>>>> with those in Hasher, but there would be some consequences:
>>>> 1) Hasher doesn't have MD2 or MD4, but those are obsolete and broken. I
>>>> see little to no value rewriting them to satisfy Hasher's HashFunction
>>>> requirements, but it shouldn't be too hard to do that.
>>>> 2) the way instances are created differ. I didn't want to do it the way
>>>> it's done in Cryptography's MD5, SHA1, SHA256, where class side #new may
>>>> return an object that is not of that class but a subclass. So, I added
>>>> instance creation methods to Hasher's HashFunction which return an
>>>> instance optimized for the current platform. So, a few methods need to
>>>> be changed in Cryptography to use the optimized hash functions.
>>>> 3) Cryptography would depend on the Registers package.
>>>>
>>>>
>>>> Levente
>>>>
>>>>> Kindly,
>>>>> Robert
>>>>>
>>>>> On 3/5/20 12:43 PM, Levente Uzonyi wrote:
>>>>>> Hi Robert,
>>>>>>
>>>>>> The mail you are looking for is here:
>>>>>> http://lists.squeakfoundation.org/pipermail/vm-dev/2020-March/032986.html
>>>>>>
>>>>>> Since that email, to make life easier to those who have the Cryptography
>>>>>> package loaded in their images, I've uploaded another variant of
>>>>>> Hasher: HAHasher. It's the same as the Hasher package but all class
>>>>>> names are prefixed with HA.
>>>>>> To load that, evaluate:
>>>>>>
>>>>>> Installer ss
>>>>>>       	project: 'Registers';
>>>>>>       	install: 'Registers';
>>>>>>       	project: 'Hasher';
>>>>>>       	install: 'HAHasher'.
>>>>>>
>>>>>> And then you can write
>>>>>>
>>>>>> HAHashFunction newSHA512 hashMessage: 'test'.
>>>>>>
>>>>>>
>>>>>> Levente
>>



More information about the Squeak-dev mailing list