Hi Robert,

On Mon, 9 Mar 2020, Robert wrote:

> Hi Levente,
>
> Have you seen my other posts on ProCrypto and the work I did to the
> Hashing package? After I ported your code, the size went up to 730 kb,
> while the rest of Cryptography was about 270 kb. So there was a lot of

Yes, I've seen your messages.

As I mentioned in my previous email, the size of the package is large 
because of the test strings. I split the package up into Core and Tests.
If you have a look at the Hasher repository, you'll find that 
HAHasher-Core.1 is 24.6 kB, while HAHasher-Tests.1 is 650kB.

By bloat I didn't mean that the size of the package is too large. I meant 
that there are too many methods and classes are added to the environment 
in a non-isolated way (no class prefixes, extension methods). And when you 
only need a small chunk of those, you've got no way to load just that.

> bloat. As we already have solutions for SHA1 and SHA256, I really only
> needed SHA512. So I removed all the other classes and tests and ported

I think community needs come first, so having all SHA2 hash functions is 
better than just having SHA512.

> SHA512 onto the HashFunction defined in CryptoCore. I removed all the
> RGThirtyTwoBitRegisters and I plan to move this ported SHA512 off of
> using a RGSixtyFourBitRegister, to eliminate that code. The size is

I see no value in doing that. The code is there, complete, ready for 
integration.

> currently 204 kb for Hashing. I do not know how this plays with your
> plugin, but there should just be a SHA512Plugin.
>
> Thanks for digging in and writing your plugin! Can it be changed to just
> doing SHA512, easily?

It is designed to support all SHA2 functions. You may be able to remove 
the SHA224/SHA256 stuff, but I see no value in that. This plugin is better 
than SHA256Plugin.


Levente

>
> Kindly,
> Robert
>
> On 3/9/20 3:11 PM, Levente Uzonyi wrote:
>> Hi Robert,
>>
>> During the weekend I wrote a new plugin, SHA2Plugin, which covers all
>> cases of SHA2, therefore obsoletes SHA256Plugin. The code is available on
>> SqueakSource:
>> http://www.squeaksource.com/Cryptography/CryptographyPlugins-ul.18.mcz
>>
>> I've updated the HAHasher package with hash variants using the new plugin
>> (and the old MD5 and SHA256 plugins), and split it up to Core and Tests,
>> so that the tests with the huge strings can be loaded optionally.
>> After building a VM with the new plugin, load:
>>
>> Installer ss
>>       project: 'Registers';
>>       install: 'Registers';
>>       project: 'Hasher';
>>       install: 'HAHasher-Core';
>>       install: 'HAHasher-Tests'.
>>
>>
>> However, SHA2Plugin is not complete yet. There's an issue with the VM's
>> code generator affecting 32-bit platforms, and I want to change the
>> methods swapping endianness before the plugin is added to the VMs.
>>
>> On Sat, 7 Mar 2020, Robert wrote:
>>
>>> Hi Levente,
>>>
>>> Between this and your HashFunctions and my email in response to Jakob, I
>>> have merged your work into Cryptography, published as:
>>>
>>> Name: Cryptography-v5.3-rww.121
>>> Author: rww
>>> Time: 7 March 2020, 12:13:00.184627 pm
>>> UUID: 2b69a713-eac3-4680-9d00-8104eea5a7da
>>> Ancestors: Cryptography-v5.3-rww.120
>>>
>>> Ported in the Registers & HAHasher packages and merged into
>>> Cryptography. CryptoGreen!
>> I suppose my point didn't get through. In my opinion, the code in
>> Hasher/HAHasher package should replace Cryptography's HashFunction and
>> subclasses. Why?
>> - it's a single package already (except for its dependency on Registers,
>> but I see no problem with Cryptography having dependencies)
>> - slimmer: fewer, simpler methods per class
>> - faster: uses the highly optimized Registers package when plugins are not
>> available; has plugin support for all implemented hashes (SHA2Plugin);
>> highly optimized in general
>> - way more hashes: supports all SHA-2 variants except for SHA-512/t for
>> arbitrary t other than 224, 256, 384, but that can be implemented as well
>> if there's need for it.
>> - has more comments and tests (e.g. check out SHA1 hashMessage: '' in
>> Cryptography)
>> - no bad tricks (e.g. check out SHA256 new class, or MD5 new class)
>>
>>> I am now looking to recategorize to fragment Cryptography. Future work
>>> should be released to the whole enchilada Cryptography package and
>>> copied to the correct fragment. For example reorging, rebasing and
>>> renaming HASHA512 to SHA512. As I fragment I will be adding required
>>> packages. It seems like we are unable to specify a version to the
>>> required package. Is this the case?
>>>
>>> QUESTION TO CRYPTOGRAPHY TEAM: How do y'all feel about fragmenting
>>> Cryptography? Pros? Cons?
>>>
>>> ---
>>>
>>> Here is a profile of Cryptography and I see many leaves for the various
>>> Registers, so there may be some optimization that could occur.
>> Registers is a highly optimized package, which should perform better
>> than ThirtyTwoBitRegister, especially on 64-bit platforms. If you think
>> you can do even better in pure Smalltalk, I'm all ears.
>>
>>
>> Levente
>>
>>> - 6250 tallies, 6325 msec.
>>>
>>> **Leaves**
>>> 7.5% {476ms} RGSixtyFourBitRegister64>>loadFrom:
>>> 7.5% {472ms} Random>>nextBytes:into:startingAt:
>>> 6.8% {427ms} LargePositiveInteger(Integer)>>bitShift:
>>> 5.2% {328ms} [] SystemProgressMorph(Morph)>>updateDropShadowCache
>>> 4.4% {280ms} RGSixtyFourBitRegister64>>bitXor:
>>> 3.5% {220ms} Random>>generateStates
>>> 3.0% {192ms} RGThirtyTwoBitRegisterTest64(TestCase)>>timeout:after:
>>> 2.9% {181ms} RGSixtyFourBitRegister64>>+=
>>> 2.8% {178ms} HASHA256Inlined64>>processBuffer
>>> 2.7% {170ms} RGThirtyTwoBitRegister64>>loadFrom:
>>> 2.2% {140ms} RGThirtyTwoBitRegister64>>+=
>>> 2.1% {134ms} RGSixtyFourBitRegisterTest64(TestCase)>>assert:description:
>>> 2.1% {133ms} Point>>=
>>> 1.7% {109ms} RGThirtyTwoBitRegister64>>bitXor:
>>> 1.7% {107ms} SHA1>>hashStream:
>>> 1.5% {96ms} RGSixtyFourBitRegister64>>leftRotateBy:
>>> 1.4% {89ms} RGSixtyFourBitRegister64>>load:
>>> 1.4% {89ms} RGSixtyFourBitRegister32>>load:
>>> 1.4% {89ms} SmallInteger(Number)>>negative
>>> 1.4% {88ms} []
>>> RGThirtyTwoBitRegisterTest64(RGRegisterTest)>>testLeftRotateBy
>>> 1.2% {77ms} LargePositiveInteger>>*
>>> 1.2% {76ms} GrafPort>>fillRoundRect:radius:
>>> 1.1% {70ms} GrafPort>>copyBits
>>> 1.1% {67ms} DisplayScreen(Form)>>depth
>>>
>>> **Memory**
>>>      old            +0 bytes
>>>      young        +2,071,872 bytes
>>>      used        +2,071,872 bytes
>>>      free        -2,071,872 bytes
>>>
>>> **GCs**
>>>      full            1 totalling 74 ms (1.17% uptime), avg 74 ms
>>>      incr            223 totalling 43 ms (0.7% uptime), avg 0.2 ms
>>>      tenures        5,503 (avg 0 GCs/tenure)
>>>      root table    0 overflows
>>>
>>> K, r
>>>
>>> On 3/7/20 9:39 AM, Robert wrote:
>>>> Hi Levente,
>>>>
>>>> Thanks for your comments, I totally understand your views. I have some
>>>> response, but the bottom line is that I or others in Cryptography
>>>> (looking for new college age recruits...) will harvest some from your
>>>> packages. It may only be SHA512 that we port over to round out the hash
>>>> function offerings. I appreciate your willingness to allow us to do that.
>>>>
>>>> Your main argument is that Cryptography is too bloated, both with
>>>> obsolete algorithms as well as combining the various facets into one
>>>> package. The prime function of the Cryptography package is to have one
>>>> package with all the Crypto we have, for single click loading. This also
>>>> supports the Crypto toolbox/sandbox approach providing an educational
>>>> environment to learning Crypto. Crypto is a learning package, as well as
>>>> being fully functional.
>>>>
>>>> Cryptography, while larger than most packages, comes in at 300 kb, with
>>>> the recent addition of Blowfish. That is loaded into an image file that
>>>> is 47 MB! This means that Cryptography is just 0.6% the size of the
>>>> image. This does not register to me as too bloated. But I see your
>>>> perspective with outdated Crypto being present as well as the
>>>> unnecessary add-ons, like X09 and ASN1; ciphers and hashes.
>>>>
>>>> When looking to do end to end encryption, we would load all of
>>>> Cryptography (304 kb), ParrotTalk (93 kb), SSL (350 kb), Telnet (101 kb)
>>>> & SSH (49 kb), for a total of 900 kb. That is only 1.9 % of the total
>>>> image. Note that as I migrate SSL & SSH over onto the ParrotTalk
>>>> framework, I expect to see the sizes of those two packages drastically
>>>> reduce, through adoption of reuse.
>>>>
>>>> 1) having other hashes completes the scope of Crypto even if not utilized.
>>>>
>>>> 2) I look forward to diving deeper and understand your point here as
>>>> well as see how your instances are created.
>>>>
>>>> 3) I would port Registers, as well.
>>>>
>>>> The absence of an ability to easily link dependencies provides a
>>>> challenge to breaking Cryptography up. If only Monticello allowed for
>>>> dependencies,  It is doable to break up Cryptography.
>>>>
>>>> So let us imagine a future where Cryptography stabilizes again. Before
>>>> declaring Crypto stable, I would include adding the Signal encryption
>>>> needs into the Crypto package. This would include the double ratchet
>>>> block cipher mode. As that point of stability is reached, we could
>>>> preserve the total package for one click loading and also reduce and
>>>> break it up into pieces. How would it sound to you if old obsolete
>>>> functions & ciphers are removed, then the ASN1 and X509 is split off. At
>>>> this point to your point let us consider splitting off the ciphers and
>>>> the hash functions and the randomizers, leaving a Crypto base. Then a
>>>> pro user would load Crypto-base, Randomizers, Hash Functions (No MD2...)
>>>> and Ciphers (No DES...).
>>>>
>>>> I tried to use your example of use of the Installer to load Cryptography
>>>> and it did not work. It could not find the versions.
>>>>
>>>> Installer ss
>>>>       	project: 'Cryptography';
>>>>       	install: 'Cryptography-v5.3'.
>>>>
>>>> If this could be made to work, then small scripts could load this split up Crypto pro solution. Optionally loading the different pieces.
>>>>
>>>> Installer ss
>>>>       	project: 'Cryptography';
>>>>       	install: 'Cryptography-base-v5.3';
>>>>       	install: 'Cryptography-core-v5.3';
>>>>       	install: 'Cryptography-hash-v5.3';
>>>>       	install: 'Cryptography-cipher-v5.3';
>>>>       	install: 'Cryptography-ASN1-v5.3';
>>>>       	install: 'Cryptography-X509-v5.3';
>>>>       	install: 'ParrotTalk';
>>>>       	install: 'SSL';
>>>>       	install: 'Telnet';
>>>>       	install: 'SSH';
>>>>       	install: 'Signal'.
>>>>
>>>> Kindly,
>>>> Robert
>>>>
>>>>
>>>>
>>>> On 3/7/20 5:20 AM, Levente Uzonyi wrote:
>>>>> Hi Robert,
>>>>>
>>>>> On Thu, 5 Mar 2020, Robert wrote:
>>>>>
>>>>>> Oh yes, Levente, I recall speaking with you about it. I would like to
>>>>>> make a proposal. Do you think you could fold all those hash functions,
>>>>>> without the HA, into the Cryptography library? We have a HashFunction
>>>>>> class in there, I do not know how different they may be in their public
>>>>>> interface. I think it would be valuable to combine them. To support TLS
>>>>>> 1.3, we would also need elliptical Diffie-Hellmans, I think.
>>>>>>
>>>>>> Levente, would you be willing to fold your work into Cryptography?
>>>>> The reason why I created a separate package was that I found the
>>>>> Cryptography package too bloated. Cryptographic hash functions seem to be
>>>>> more commonly needed than ciphers, CSPRNGS, ASN1, etc.
>>>>>
>>>>> It is possible to replace HashFunction and subclasses from Cryptography
>>>>> with those in Hasher, but there would be some consequences:
>>>>> 1) Hasher doesn't have MD2 or MD4, but those are obsolete and broken. I
>>>>> see little to no value rewriting them to satisfy Hasher's HashFunction
>>>>> requirements, but it shouldn't be too hard to do that.
>>>>> 2) the way instances are created differ. I didn't want to do it the way
>>>>> it's done in Cryptography's MD5, SHA1, SHA256, where class side #new may
>>>>> return an object that is not of that class but a subclass. So, I added
>>>>> instance creation methods to Hasher's HashFunction which return an
>>>>> instance optimized for the current platform. So, a few methods need to
>>>>> be changed in Cryptography to use the optimized hash functions.
>>>>> 3) Cryptography would depend on the Registers package.
>>>>>
>>>>>
>>>>> Levente
>>>>>
>>>>>> Kindly,
>>>>>> Robert
>>>>>>
>>>>>> On 3/5/20 12:43 PM, Levente Uzonyi wrote:
>>>>>>> Hi Robert,
>>>>>>>
>>>>>>> The mail you are looking for is here:
>>>>>>> http://lists.squeakfoundation.org/pipermail/vm-dev/2020-March/032986.html
>>>>>>>
>>>>>>> Since that email, to make life easier to those who have the Cryptography
>>>>>>> package loaded in their images, I've uploaded another variant of
>>>>>>> Hasher: HAHasher. It's the same as the Hasher package but all class
>>>>>>> names are prefixed with HA.
>>>>>>> To load that, evaluate:
>>>>>>>
>>>>>>> Installer ss
>>>>>>>       	project: 'Registers';
>>>>>>>       	install: 'Registers';
>>>>>>>       	project: 'Hasher';
>>>>>>>       	install: 'HAHasher'.
>>>>>>>
>>>>>>> And then you can write
>>>>>>>
>>>>>>> HAHashFunction newSHA512 hashMessage: 'test'.
>>>>>>>
>>>>>>>
>>>>>>> Levente
>>>
>
>