[Vm-dev] [squeak-dev] SHA512 squeak implementation?
Levente Uzonyi
leves at caesar.elte.hu
Tue Mar 10 20:48:07 UTC 2020
Hi Robert,
On Mon, 9 Mar 2020, Robert wrote:
> Hi Levente,
>
> On 3/9/20 5:14 PM, Levente Uzonyi wrote:
>> Hi Robert,
>>
>> On Mon, 9 Mar 2020, Robert wrote:
>>
>>> Hi Levente,
>>>
>>> Have you seen my other posts on ProCrypto and the work I did to the
>>> Hashing package? After I ported your code, the size went up to 730 kb,
>>> while the rest of Cryptography was about 270 kb. So there was a lot of
>> Yes, I've seen your messages.
>
> Too many...I ride like a storm. Always have done so. I do hope that you
> are not upset about my ripping SHA512 into a Crypto package. I feel very
> strongly that a commitment by provider implementers doing Crypto to
> subclass the Root classes: BlockCipher, HashFunction, BlockCipherMode,
> and importantly adopt the common protocol inherited from those classes.
> Then all Crypto implemeters will be able to easily interoperate.
> Producing a lookup interface, much like Java's provider lookup, will be
> a step we could take forwards with the shared and interoperable Crypto
> library packages.
Are there any other "Crypo implementers" having hash functions? I doubt
it. So, I don't think we need anything like this. We need simple code
that works.
>
>> As I mentioned in my previous email, the size of the package is large
>> because of the test strings. I split the package up into Core and Tests.
>> If you have a look at the Hasher repository, you'll find that
>> HAHasher-Core.1 is 24.6 kB, while HAHasher-Tests.1 is 650kB.
> I see, alright, that's fair dinkum. I also feel strongly that a minimum
> of tests be carried within the primary package for some code. If you
> look at the Crypto packages, you'll see Hashing tests in hashing, Random
> tests in Random, Cipher tests in Cipher. This way it is easy to run
> them, off a CI server or what have you. The tests come with the code, is
As others wrote, this is not a good idea. You don't need those tests in
your production images.
> a principle I try to follow. I suspect that maybe we could reduce the
> tests, for SHA512, to a minimum. I will look into this. I also feel like
> having a separate package with more robust testing is fair dinkum. I
> think that would make a good solution here, where you could have a
> couple of tests with the package and a separate Tests package for the
> rest of them.
>> By bloat I didn't mean that the size of the package is too large. I meant
>> that there are too many methods and classes are added to the environment
>> in a non-isolated way (no class prefixes, extension methods). And when you
>> only need a small chunk of those, you've got no way to load just that.
>>
>>> bloat. As we already have solutions for SHA1 and SHA256, I really only
>>> needed SHA512. So I removed all the other classes and tests and ported
>> I think community needs come first, so having all SHA2 hash functions is
>> better than just having SHA512.
> Alright, how does this proposed approach grab you? I have a Hashing
> package as a part of the ProCrypto offering. Dependencies are setup in
> Monticello for Random to pull in that Hashing package, which has all our
> traditional functions and now has SHA512, thanks to your offering. Here
> is my further thought, If you interface with the common Crypto
> superclasses (reclass HAHashFunction under HashFunction) and adopt the
> Crypto protocol, your package as a provider package that could be loaded
> side by side with Crypto's Hashing package. You would naturally extend
> our offerings. This is my approach with SSL and SSH, as the port
> progresses so slooowly, running as a specialization of the ParrotTalk
> code, so it just fits in and works as an option. I really see a
> potential explosion of such solutions, if colleges and universities get
> onboard. *dreaming about a better future*
>
> Would this solution work well for you, Levente?
I did't clean up and restructure HashFunction in Hasher just to
reintroduce the stuff I removed/replaced.
Hasher is a replacement for the whole HashFunction hierarchy.
>
>>> SHA512 onto the HashFunction defined in CryptoCore. I removed all the
>>> RGThirtyTwoBitRegisters and I plan to move this ported SHA512 off of
>>> using a RGSixtyFourBitRegister, to eliminate that code. The size is
>> I see no value in doing that. The code is there, complete, ready for
>> integration.
> As an alternate Crypto package, folks could choose your solution. This
> way we can align our efforts together and mix and match solution
> packages. That seems like the exact approach to managing Cryptography
> library and how we conducts our affairs.
Our community is not large enough for that.
>
>>
>>> currently 204 kb for Hashing. I do not know how this plays with your
>>> plugin, but there should just be a SHA512Plugin.
>>>
>>> Thanks for digging in and writing your plugin! Can it be changed to just
>>> doing SHA512, easily?
>> It is designed to support all SHA2 functions. You may be able to remove
>> the SHA224/SHA256 stuff, but I see no value in that. This plugin is better
>> than SHA256Plugin.
>
> I must tell you, that I am totally unfamiliar with the SHA256Plugin,
> someone else wrote it over a decade ago! As such, I cannot evaluate the
> quality of one over the other, so I will trust your views.
I wrote SHA256Plugin and MD5Plugin. I wrote the image side code for
SHA256, and I added plugin support to MD5 on the image side too.
Other than these hashes, the Cryptography package has SHA1, which is
buggy (try SHA1 hashMessage: ''), MD2 and MD4, which are obsolete, and
nothing else.
So, again, I suggest replacing the HashFunction hierarchy in Cryptography
with Hasher.
Levente
>
> K, r
>
>>
>> Levente
>>
>>> Kindly,
>>> Robert
>>>
>>> On 3/9/20 3:11 PM, Levente Uzonyi wrote:
>>>> Hi Robert,
>>>>
>>>> During the weekend I wrote a new plugin, SHA2Plugin, which covers all
>>>> cases of SHA2, therefore obsoletes SHA256Plugin. The code is available on
>>>> SqueakSource:
>>>> http://www.squeaksource.com/Cryptography/CryptographyPlugins-ul.18.mcz
>>>>
>>>> I've updated the HAHasher package with hash variants using the new plugin
>>>> (and the old MD5 and SHA256 plugins), and split it up to Core and Tests,
>>>> so that the tests with the huge strings can be loaded optionally.
>>>> After building a VM with the new plugin, load:
>>>>
>>>> Installer ss
>>>> project: 'Registers';
>>>> install: 'Registers';
>>>> project: 'Hasher';
>>>> install: 'HAHasher-Core';
>>>> install: 'HAHasher-Tests'.
>>>>
>>>>
>>>> However, SHA2Plugin is not complete yet. There's an issue with the VM's
>>>> code generator affecting 32-bit platforms, and I want to change the
>>>> methods swapping endianness before the plugin is added to the VMs.
>>>>
>>>> On Sat, 7 Mar 2020, Robert wrote:
>>>>
>>>>> Hi Levente,
>>>>>
>>>>> Between this and your HashFunctions and my email in response to Jakob, I
>>>>> have merged your work into Cryptography, published as:
>>>>>
>>>>> Name: Cryptography-v5.3-rww.121
>>>>> Author: rww
>>>>> Time: 7 March 2020, 12:13:00.184627 pm
>>>>> UUID: 2b69a713-eac3-4680-9d00-8104eea5a7da
>>>>> Ancestors: Cryptography-v5.3-rww.120
>>>>>
>>>>> Ported in the Registers & HAHasher packages and merged into
>>>>> Cryptography. CryptoGreen!
>>>> I suppose my point didn't get through. In my opinion, the code in
>>>> Hasher/HAHasher package should replace Cryptography's HashFunction and
>>>> subclasses. Why?
>>>> - it's a single package already (except for its dependency on Registers,
>>>> but I see no problem with Cryptography having dependencies)
>>>> - slimmer: fewer, simpler methods per class
>>>> - faster: uses the highly optimized Registers package when plugins are not
>>>> available; has plugin support for all implemented hashes (SHA2Plugin);
>>>> highly optimized in general
>>>> - way more hashes: supports all SHA-2 variants except for SHA-512/t for
>>>> arbitrary t other than 224, 256, 384, but that can be implemented as well
>>>> if there's need for it.
>>>> - has more comments and tests (e.g. check out SHA1 hashMessage: '' in
>>>> Cryptography)
>>>> - no bad tricks (e.g. check out SHA256 new class, or MD5 new class)
>>>>
>>>>> I am now looking to recategorize to fragment Cryptography. Future work
>>>>> should be released to the whole enchilada Cryptography package and
>>>>> copied to the correct fragment. For example reorging, rebasing and
>>>>> renaming HASHA512 to SHA512. As I fragment I will be adding required
>>>>> packages. It seems like we are unable to specify a version to the
>>>>> required package. Is this the case?
>>>>>
>>>>> QUESTION TO CRYPTOGRAPHY TEAM: How do y'all feel about fragmenting
>>>>> Cryptography? Pros? Cons?
>>>>>
>>>>> ---
>>>>>
>>>>> Here is a profile of Cryptography and I see many leaves for the various
>>>>> Registers, so there may be some optimization that could occur.
>>>> Registers is a highly optimized package, which should perform better
>>>> than ThirtyTwoBitRegister, especially on 64-bit platforms. If you think
>>>> you can do even better in pure Smalltalk, I'm all ears.
>>>>
>>>>
>>>> Levente
>>>>
>>>>> - 6250 tallies, 6325 msec.
>>>>>
>>>>> **Leaves**
>>>>> 7.5% {476ms} RGSixtyFourBitRegister64>>loadFrom:
>>>>> 7.5% {472ms} Random>>nextBytes:into:startingAt:
>>>>> 6.8% {427ms} LargePositiveInteger(Integer)>>bitShift:
>>>>> 5.2% {328ms} [] SystemProgressMorph(Morph)>>updateDropShadowCache
>>>>> 4.4% {280ms} RGSixtyFourBitRegister64>>bitXor:
>>>>> 3.5% {220ms} Random>>generateStates
>>>>> 3.0% {192ms} RGThirtyTwoBitRegisterTest64(TestCase)>>timeout:after:
>>>>> 2.9% {181ms} RGSixtyFourBitRegister64>>+=
>>>>> 2.8% {178ms} HASHA256Inlined64>>processBuffer
>>>>> 2.7% {170ms} RGThirtyTwoBitRegister64>>loadFrom:
>>>>> 2.2% {140ms} RGThirtyTwoBitRegister64>>+=
>>>>> 2.1% {134ms} RGSixtyFourBitRegisterTest64(TestCase)>>assert:description:
>>>>> 2.1% {133ms} Point>>=
>>>>> 1.7% {109ms} RGThirtyTwoBitRegister64>>bitXor:
>>>>> 1.7% {107ms} SHA1>>hashStream:
>>>>> 1.5% {96ms} RGSixtyFourBitRegister64>>leftRotateBy:
>>>>> 1.4% {89ms} RGSixtyFourBitRegister64>>load:
>>>>> 1.4% {89ms} RGSixtyFourBitRegister32>>load:
>>>>> 1.4% {89ms} SmallInteger(Number)>>negative
>>>>> 1.4% {88ms} []
>>>>> RGThirtyTwoBitRegisterTest64(RGRegisterTest)>>testLeftRotateBy
>>>>> 1.2% {77ms} LargePositiveInteger>>*
>>>>> 1.2% {76ms} GrafPort>>fillRoundRect:radius:
>>>>> 1.1% {70ms} GrafPort>>copyBits
>>>>> 1.1% {67ms} DisplayScreen(Form)>>depth
>>>>>
>>>>> **Memory**
>>>>> old +0 bytes
>>>>> young +2,071,872 bytes
>>>>> used +2,071,872 bytes
>>>>> free -2,071,872 bytes
>>>>>
>>>>> **GCs**
>>>>> full 1 totalling 74 ms (1.17% uptime), avg 74 ms
>>>>> incr 223 totalling 43 ms (0.7% uptime), avg 0.2 ms
>>>>> tenures 5,503 (avg 0 GCs/tenure)
>>>>> root table 0 overflows
>>>>>
>>>>> K, r
>>>>>
>>>>> On 3/7/20 9:39 AM, Robert wrote:
>>>>>> Hi Levente,
>>>>>>
>>>>>> Thanks for your comments, I totally understand your views. I have some
>>>>>> response, but the bottom line is that I or others in Cryptography
>>>>>> (looking for new college age recruits...) will harvest some from your
>>>>>> packages. It may only be SHA512 that we port over to round out the hash
>>>>>> function offerings. I appreciate your willingness to allow us to do that.
>>>>>>
>>>>>> Your main argument is that Cryptography is too bloated, both with
>>>>>> obsolete algorithms as well as combining the various facets into one
>>>>>> package. The prime function of the Cryptography package is to have one
>>>>>> package with all the Crypto we have, for single click loading. This also
>>>>>> supports the Crypto toolbox/sandbox approach providing an educational
>>>>>> environment to learning Crypto. Crypto is a learning package, as well as
>>>>>> being fully functional.
>>>>>>
>>>>>> Cryptography, while larger than most packages, comes in at 300 kb, with
>>>>>> the recent addition of Blowfish. That is loaded into an image file that
>>>>>> is 47 MB! This means that Cryptography is just 0.6% the size of the
>>>>>> image. This does not register to me as too bloated. But I see your
>>>>>> perspective with outdated Crypto being present as well as the
>>>>>> unnecessary add-ons, like X09 and ASN1; ciphers and hashes.
>>>>>>
>>>>>> When looking to do end to end encryption, we would load all of
>>>>>> Cryptography (304 kb), ParrotTalk (93 kb), SSL (350 kb), Telnet (101 kb)
>>>>>> & SSH (49 kb), for a total of 900 kb. That is only 1.9 % of the total
>>>>>> image. Note that as I migrate SSL & SSH over onto the ParrotTalk
>>>>>> framework, I expect to see the sizes of those two packages drastically
>>>>>> reduce, through adoption of reuse.
>>>>>>
>>>>>> 1) having other hashes completes the scope of Crypto even if not utilized.
>>>>>>
>>>>>> 2) I look forward to diving deeper and understand your point here as
>>>>>> well as see how your instances are created.
>>>>>>
>>>>>> 3) I would port Registers, as well.
>>>>>>
>>>>>> The absence of an ability to easily link dependencies provides a
>>>>>> challenge to breaking Cryptography up. If only Monticello allowed for
>>>>>> dependencies, It is doable to break up Cryptography.
>>>>>>
>>>>>> So let us imagine a future where Cryptography stabilizes again. Before
>>>>>> declaring Crypto stable, I would include adding the Signal encryption
>>>>>> needs into the Crypto package. This would include the double ratchet
>>>>>> block cipher mode. As that point of stability is reached, we could
>>>>>> preserve the total package for one click loading and also reduce and
>>>>>> break it up into pieces. How would it sound to you if old obsolete
>>>>>> functions & ciphers are removed, then the ASN1 and X509 is split off. At
>>>>>> this point to your point let us consider splitting off the ciphers and
>>>>>> the hash functions and the randomizers, leaving a Crypto base. Then a
>>>>>> pro user would load Crypto-base, Randomizers, Hash Functions (No MD2...)
>>>>>> and Ciphers (No DES...).
>>>>>>
>>>>>> I tried to use your example of use of the Installer to load Cryptography
>>>>>> and it did not work. It could not find the versions.
>>>>>>
>>>>>> Installer ss
>>>>>> project: 'Cryptography';
>>>>>> install: 'Cryptography-v5.3'.
>>>>>>
>>>>>> If this could be made to work, then small scripts could load this split up Crypto pro solution. Optionally loading the different pieces.
>>>>>>
>>>>>> Installer ss
>>>>>> project: 'Cryptography';
>>>>>> install: 'Cryptography-base-v5.3';
>>>>>> install: 'Cryptography-core-v5.3';
>>>>>> install: 'Cryptography-hash-v5.3';
>>>>>> install: 'Cryptography-cipher-v5.3';
>>>>>> install: 'Cryptography-ASN1-v5.3';
>>>>>> install: 'Cryptography-X509-v5.3';
>>>>>> install: 'ParrotTalk';
>>>>>> install: 'SSL';
>>>>>> install: 'Telnet';
>>>>>> install: 'SSH';
>>>>>> install: 'Signal'.
>>>>>>
>>>>>> Kindly,
>>>>>> Robert
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 3/7/20 5:20 AM, Levente Uzonyi wrote:
>>>>>>> Hi Robert,
>>>>>>>
>>>>>>> On Thu, 5 Mar 2020, Robert wrote:
>>>>>>>
>>>>>>>> Oh yes, Levente, I recall speaking with you about it. I would like to
>>>>>>>> make a proposal. Do you think you could fold all those hash functions,
>>>>>>>> without the HA, into the Cryptography library? We have a HashFunction
>>>>>>>> class in there, I do not know how different they may be in their public
>>>>>>>> interface. I think it would be valuable to combine them. To support TLS
>>>>>>>> 1.3, we would also need elliptical Diffie-Hellmans, I think.
>>>>>>>>
>>>>>>>> Levente, would you be willing to fold your work into Cryptography?
>>>>>>> The reason why I created a separate package was that I found the
>>>>>>> Cryptography package too bloated. Cryptographic hash functions seem to be
>>>>>>> more commonly needed than ciphers, CSPRNGS, ASN1, etc.
>>>>>>>
>>>>>>> It is possible to replace HashFunction and subclasses from Cryptography
>>>>>>> with those in Hasher, but there would be some consequences:
>>>>>>> 1) Hasher doesn't have MD2 or MD4, but those are obsolete and broken. I
>>>>>>> see little to no value rewriting them to satisfy Hasher's HashFunction
>>>>>>> requirements, but it shouldn't be too hard to do that.
>>>>>>> 2) the way instances are created differ. I didn't want to do it the way
>>>>>>> it's done in Cryptography's MD5, SHA1, SHA256, where class side #new may
>>>>>>> return an object that is not of that class but a subclass. So, I added
>>>>>>> instance creation methods to Hasher's HashFunction which return an
>>>>>>> instance optimized for the current platform. So, a few methods need to
>>>>>>> be changed in Cryptography to use the optimized hash functions.
>>>>>>> 3) Cryptography would depend on the Registers package.
>>>>>>>
>>>>>>>
>>>>>>> Levente
>>>>>>>
>>>>>>>> Kindly,
>>>>>>>> Robert
>>>>>>>>
>>>>>>>> On 3/5/20 12:43 PM, Levente Uzonyi wrote:
>>>>>>>>> Hi Robert,
>>>>>>>>>
>>>>>>>>> The mail you are looking for is here:
>>>>>>>>> http://lists.squeakfoundation.org/pipermail/vm-dev/2020-March/032986.html
>>>>>>>>>
>>>>>>>>> Since that email, to make life easier to those who have the Cryptography
>>>>>>>>> package loaded in their images, I've uploaded another variant of
>>>>>>>>> Hasher: HAHasher. It's the same as the Hasher package but all class
>>>>>>>>> names are prefixed with HA.
>>>>>>>>> To load that, evaluate:
>>>>>>>>>
>>>>>>>>> Installer ss
>>>>>>>>> project: 'Registers';
>>>>>>>>> install: 'Registers';
>>>>>>>>> project: 'Hasher';
>>>>>>>>> install: 'HAHasher'.
>>>>>>>>>
>>>>>>>>> And then you can write
>>>>>>>>>
>>>>>>>>> HAHashFunction newSHA512 hashMessage: 'test'.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Levente
>>>
>
>
More information about the Vm-dev
mailing list