Tobias Pape uploaded a new version of System to project The Inbox:
http://source.squeak.org/inbox/System-topa.775.mcz
==================== Summary ====================
Name: System-topa.775
Author: topa
Time: 29 October 2015, 1:06:23.749 am
UUID: a2ad05e2-8cf3-4974-84c4-7a5d93e304e3
Ancestors: System-ul.774
Cache NaturalLanguageTranslator when looking up yet inexisting ones.
Induced by tim Rowledge
(see http://forum.world.st/image-startup-complications-td4858390.html)
=============== Diff against System-ul.774 ===============
Item was changed:
----- Method: NaturalLanguageTranslator class>>availableForLocaleID: (in category 'accessing') -----
availableForLocaleID: localeID
"Answer available locale ID.
If translator is not found for correct locale ID, then isoLanguage is
attempted for the key."
^ self translators
at: localeID
+ ifAbsentPut: [localeID hasParent
- ifAbsent: [localeID hasParent
ifTrue: [self translators
at: localeID parent
ifAbsent: [self default]]
ifFalse: [self default]]!
One of the things that running Squeak in the vm simulator will teach you is that we have a lot of stuff going on during system startup. A slow startup might not seem like a big deal when you’re running on a fast machine (because it will be less-slow, duh) or planning to keep running for a while. However, on a slow machine (Pi, for example) or when you are trying to quickly spin up a wotsit for Docker (or other fashionable SaaS thingummy) the slowness can be a real annoyance.
As an example, I’ve just fired up an image under the vm sim to test a new CPIC design. After a lot of whirring you tend to get worried that something is recursively recursing and otherwise going around and around chasing its own tail recursion. So you interrupt to see where things are and … oh, yeah, going around and around in something to do with X. Proceed, wait a while, look again.. same stuff…
In this case it prompted me to look at MenuIcon initializeTranslations. What happens here is that the class MenuIcons is running down all the registered menu item icons to translate the attached strings. This involves a great deal of work to find the correct domain(s) to do the translating, potentially reading many files to build dictionaries etc. I have to do much the same in Scratch - and indeed probably ought to rework that to use the latest stuff.
Part of the job here is to find the translator for the current locale. See NaturalLanguageTranslator class>availableForLocaleID: where we check the list of known translators and if nothing is found, check for the parent of the requested locale, and if still nothing return a default. The problem is that finding the parent involves a long search which seems to mainly return a locale the same as the not-found one which of course is itself not found. So then we find the default - which involves making a new translator and going via that same long search.
Because this newly made default translator is not saved we go through this for every single string that wants a translation. I tallied the work and on a Pi with Cog VM it takes 373mS - 1/3rd of a second each time we start an image. On my iMac it’s only 37mS, which may seem too trivial to worry about but remember this is only one of many startup operations.
So, what to do? Well simply saving the new default translator back into the list of known translators - which I really suspect is what the original authors intended to do since it’s such an obvious thing - the time is cut to less than 1/5th, or 62mS. Since I’m not at all expert in the workings of the translation code I don’t want to just commit this.
It really is a trivial change-
NaturalLanguageTranslator class>>#availableForLocaleID:
availableForLocaleID: localeID
"Answer available locale ID.
If translator is not found for correct locale ID, then isoLanguage is
attempted for the key."
^ self translators
at: localeID
ifAbsent: [localeID hasParent
ifTrue: [self translators
at: localeID parent
ifAbsent: [self translators at: localeID put: self default]]
ifFalse: [self translators at: localeID put: self default]]
There’s a *lot* of senders of #translated that this will give a little boost to.
tim
--
tim Rowledge; tim(a)rowledge.org; http://www.rowledge.org/tim
All computers run at the same speed...with the power off.
I wanted to add a little cmd key hack to the debugger to print numbers in hex - useful when debugging the VM in sim etc. Eventually I found a way to solve my immediate needs but along the path I found a dazzlingly complex intertwingled mess of code for menus and cmd key handling.
Part of it is likely because of the confusion inherent to having two separate UI systems that aren’t quite separate. For example StringHolder class>>#yellowButtonMenuItems includes
{'make project link (P)' translated. #makeProjectLink}.
which
a) annoyed me because I want to use cmd-shift-p for my printItHex
b) seems to be a bit odd to have in a text editor menu, and very much so in a code editor menu
c) doesn’t even work in Morphic so far as I can tell (PluggableTextMorph does not understand #makeProjectLink)
Also when building menus - which is done for every menu button press! - we end up doing a scan of all the methods (for menu related pragmas) of every class that could possibly be involved. This is even more insane than using #isKindOf: within UI code. Perhaps one could argue that it isn’t quite completely insane on multi-core/multi-GHz/megaRam machines but on anything slower ( like the Pi, a rather important platform for public awareness and outreach) it can ruin the UI experience.
The current system reminds me unpleasantly of the PenPoint/Go UI dating back to ’89 or so; their hardware was nominally several times faster than our Active Book, and they proudly proclaimed how it was all carefully optimised code and yet it was grindingly slow to do anything in the UI. The Active Book was a mere 8MHz ARM2 (no caches, not even an instruction prefect queue, 1Mb RAM for everything including the screen and filing system) and running that terrible slow nonsense called Smalltalk that everyone knew was slow as slow. Many years later I met one of the developers and it turned out that their framework carefully built menus by checking here, looking there, seeing if a string needed translating, having a tea-break and finally asking a complex graphics subsystem to render something. The Active Book code bitblt’d a menu form to the screen.
Ideally menus should be pre-built and cached, with some cache-flushing algorithm connected to anything that changes what should be in them. Such as adding a new menu related pragma.
I also spotted some code where menu getting selectors are examined to see how many arguments they take and then assorted #perform incantations are used. In general abetter option is to make use of the execution machinery and just send the most flexible message. Implement it at some sensible root to re-try with a simpler version if needed.
I wish I had time to clean this up but I have to make VM changes to support a largish number of Scratch users and my brain is about ready to explode.
tim
--
tim Rowledge; tim(a)rowledge.org; http://www.rowledge.org/tim
29A, the hexadecimal of the Beast.
> On 28.10.2015, at 02:35, Jecel Assumpcao Jr. <jecel(a)merlintec.com> wrote:
>
> Bert Freudenberg wrote on Mon, 26 Oct 2015 00:51:55 +0100:
>>> On 24.10.2015, at 19:15, tim Rowledge wrote:
>>>> On 23-10-2015, at 11:28 AM, marcel.taeumel wrote:
>>>>
>>>> The image looks like the 1.1 on our FTP (http://ftp.squeak.org/1.1/) but it
>>>> will not open with the Windows VM from then. Was the image format somewhat
>>>> different between Windows/Mac back then?
>>>
>>> No, the image *format* was the same but the image was originally pretty solidly
>>> Mac specific until Andreas added support for Windows, Ian added it for *nix and
>>> I added it for RISC OS. It?s possible that image you have there simply predates
>>> any of that.
>
> Wasn't one of your RISC OS additions the ability to load images saved in
> the opposite "endian"? I would guess that any VMs (Mac, Windows and
> Unix) before that would be more picky about what images they could load,
> though a newer VM could easily be used to create the needed image.
I thought Ian created that … we should write up an article about the early community history of Squeak.
>> The image on the website is 1.1w meaning it has the WindowsFileDirectory class
>> which is needed to make it work on the Windows VM (it supports Mac and Windows).
>> IIRC it lacks the UnixFileDir though (those are labeled ?...u?).
>>
>> John?s image only has MacFileDirectory, so I think this is actually the oldest released
>> image found so far, and we should put it on the website.
>>
>> Would be fun if someone looked at the changes file time stamps to reconstruct the
>> evolution of early Squeak images ...
>
> The first image I ever saw was 1.13 until the relicensed 1.1 was
> released about 10 years later. I think that individual methods didn't
> have timestaps before 2.2 or so, but all the times the image was saved
> would be indicated in the .changes files and that should be good enough.
> It might even be possible to continue tracing back the evolution to the
> Apple Smalltalk and then Xerox Smalltalk this way.
I have a pre-release Squeak image from May 20 1996 (image format version 6501) but I’m not entirely sure of its licensing status …
- Bert -
I'm trying to track down the very oldest oldest squeak releases (mostly for historic purposes), the original 1.x (and earlier) releases for Mac. Of course, research.apple.com is long gone, and even ftp.squeak.org doesn't seem to have the .sit.hqx files
Anyone have any idea where the old distributions can be found?
Levente Uzonyi uploaded a new version of System to project The Trunk:
http://source.squeak.org/trunk/System-ul.773.mcz
==================== Summary ====================
Name: System-ul.773
Author: ul
Time: 26 October 2015, 3:58:12.896 am
UUID: 6bc40bec-54a9-43e1-ae6d-f4fc730cdc16
Ancestors: System-mt.772
SecureHashAlgorithm speedups:
- avoid allocations
- simplified hash function
=============== Diff against System-mt.772 ===============
Item was changed:
Object subclass: #SecureHashAlgorithm
+ instanceVariableNames: 'totalA totalB totalC totalD totalE totals expandBuffer t1 t2 a b c d e t'
+ classVariableNames: 'K1 K2 K3 K4 TA TB TC TD TE TI'
- instanceVariableNames: 'totalA totalB totalC totalD totalE totals'
- classVariableNames: 'K1 K2 K3 K4'
poolDictionaries: ''
category: 'System-Digital Signatures'!
!SecureHashAlgorithm commentStamp: '<historical>' prior: 0!
This class implements the Secure Hash Algorithm (SHA) described in the U.S. government's Secure Hash Standard (SHS). This standard is described in FIPS PUB 180-1, "SECURE HASH STANDARD", April 17, 1995.
The Secure Hash Algorithm is also described on p. 442 of 'Applied Cryptography: Protocols, Algorithms, and Source Code in C' by Bruce Scheier, Wiley, 1996.
See the comment in class DigitalSignatureAlgorithm for details on its use.
Implementation notes:
The secure hash standard was created with 32-bit hardware in mind. All arithmetic in the hash computation must be done modulo 2^32. This implementation uses ThirtyTwoBitRegister objects to simulate hardware registers; this implementation is about six times faster than using LargePositiveIntegers (measured on a Macintosh G3 Powerbook). Implementing a primitive to process each 64-byte buffer would probably speed up the computation by a factor of 20 or more.
!
Item was changed:
----- Method: SecureHashAlgorithm class>>initialize (in category 'class initialization') -----
initialize
"SecureHashAlgorithm initialize"
"For the curious, here's where these constants come from:
#(2 3 5 10) collect: [:x | ((x sqrt / 4.0) * (2.0 raisedTo: 32)) truncated hex]"
K1 := ThirtyTwoBitRegister fromInteger: 16r5A827999.
K2 := ThirtyTwoBitRegister fromInteger: 16r6ED9EBA1.
K3 := ThirtyTwoBitRegister fromInteger: 16r8F1BBCDC.
K4 := ThirtyTwoBitRegister fromInteger: 16rCA62C1D6.
+
+ TA := ThirtyTwoBitRegister fromInteger: 16r67452301.
+ TB := ThirtyTwoBitRegister fromInteger: 16rEFCDAB89.
+ TC := ThirtyTwoBitRegister fromInteger: 16r98BADCFE.
+ TD := ThirtyTwoBitRegister fromInteger: 16r10325476.
+ TE := ThirtyTwoBitRegister fromInteger: 16rC3D2E1F0.
+ (TI := Bitmap new: 5)
+ at: 1 put: 16r67452301;
+ at: 2 put: 16rEFCDAB89;
+ at: 3 put: 16r98BADCFE;
+ at: 4 put: 16r10325476;
+ at: 5 put: 16rC3D2E1F0!
- !
Item was changed:
----- Method: SecureHashAlgorithm>>expandedBlock: (in category 'private') -----
expandedBlock: aByteArray
"Convert the given 64 byte buffer into 80 32-bit registers and answer the result."
+
+ | src |
- | out src v |
- out := Array new: 80.
src := 1.
1 to: 16 do: [:i |
+ (expandBuffer at: i) loadFrom: aByteArray at: src.
- out at: i put: (ThirtyTwoBitRegister fromByteArray: aByteArray at: src).
src := src + 4].
17 to: 80 do: [:i |
+ t1
+ loadFrom: (expandBuffer at: i - 3);
+ bitXor: (expandBuffer at: i - 8);
+ bitXor: (expandBuffer at: i - 14);
+ bitXor: (expandBuffer at: i - 16);
- v := (out at: i - 3) copy.
- v bitXor: (out at: i - 8);
- bitXor: (out at: i - 14);
- bitXor: (out at: i - 16);
leftRotateBy: 1.
+ (expandBuffer at: i) loadFrom: t1 ]
- out at: i put: v].
- ^ out
!
Item was changed:
----- Method: SecureHashAlgorithm>>finalHash (in category 'private') -----
finalHash
"Concatenate the final totals to build the 160-bit integer result."
"Details: If the primitives are supported, the results are in the totals array. Otherwise, they are in the instance variables totalA through totalE."
+ | result |
+ result := ByteArray new: 20.
+ totals
+ ifNil: [ "compute final hash when not using primitives"
+ result
+ unsignedShortAt: 1 put: totalE low bigEndian: false;
+ unsignedShortAt: 3 put: totalE hi bigEndian: false;
+ unsignedShortAt: 5 put: totalD low bigEndian: false;
+ unsignedShortAt: 7 put: totalD hi bigEndian: false;
+ unsignedShortAt: 9 put: totalC low bigEndian: false;
+ unsignedShortAt: 11 put: totalC hi bigEndian: false;
+ unsignedShortAt: 13 put: totalB low bigEndian: false;
+ unsignedShortAt: 15 put: totalB hi bigEndian: false;
+ unsignedShortAt: 17 put: totalA low bigEndian: false;
+ unsignedShortAt: 19 put: totalA hi bigEndian: false ]
+ ifNotNil: [ "compute final hash when using primitives"
+ result
+ unsignedLongAt: 1 put: (totals at: 5) bigEndian: false;
+ unsignedLongAt: 5 put: (totals at: 4) bigEndian: false;
+ unsignedLongAt: 9 put: (totals at: 3) bigEndian: false;
+ unsignedLongAt: 13 put: (totals at: 2) bigEndian: false;
+ unsignedLongAt: 17 put: (totals at: 1) bigEndian: false ].
+ ^(LargePositiveInteger new: result size)
+ replaceFrom: 1
+ to: result size
+ with: result
+ startingAt: 1;
+ normalize!
- | r |
- totals ifNil: [ "compute final hash when not using primitives"
- ^ (totalA asInteger bitShift: 128) +
- (totalB asInteger bitShift: 96) +
- (totalC asInteger bitShift: 64) +
- (totalD asInteger bitShift: 32) +
- (totalE asInteger)].
-
- "compute final hash when using primitives"
- r := 0.
- 1 to: 5 do: [:i |
- r := r bitOr: ((totals at: i) bitShift: (32 * (5 - i)))].
- ^ r
- !
Item was added:
+ ----- Method: SecureHashAlgorithm>>hashFunction: (in category 'private') -----
+ hashFunction: i
+ "Compute the hash function for the i-th step of the block hash loop. We number our steps 1-80, versus the 0-79 of the standard."
+ "Details: There are four functions, one for each 20 iterations. The second and fourth are the same."
+
+ t1 loadFrom: b.
+ i <= 20 ifTrue: [
+ t2
+ loadFrom: b;
+ bitInvert;
+ bitAnd: d.
+ ^t1
+ bitAnd: c;
+ bitOr: t2 ].
+ i <= 40 ifTrue: [
+ ^t1
+ bitXor: c;
+ bitXor: d ].
+ i <= 60 ifTrue: [
+ t2
+ loadFrom: b;
+ bitOr: c;
+ bitAnd: d.
+ ^t1
+ bitAnd: c;
+ bitOr: t2 ].
+ ^t1
+ bitXor: c;
+ bitXor: d
+ !
Item was removed:
- ----- Method: SecureHashAlgorithm>>hashFunction:of:with:with: (in category 'private') -----
- hashFunction: i of: x with: y with: z
- "Compute the hash function for the i-th step of the block hash loop. We number our steps 1-80, versus the 0-79 of the standard."
- "Details: There are four functions, one for each 20 iterations. The second and fourth are the same."
-
- i <= 20 ifTrue: [^ x copy bitAnd: y; bitOr: (x copy bitInvert; bitAnd: z)].
- i <= 40 ifTrue: [^ x copy bitXor: y; bitXor: z].
- i <= 60 ifTrue: [^ x copy bitAnd: y; bitOr: (x copy bitAnd: z); bitOr: (y copy bitAnd: z)].
- ^ x copy bitXor: y; bitXor: z
- !
Item was changed:
----- Method: SecureHashAlgorithm>>hashInteger:seed: (in category 'public') -----
hashInteger: aPositiveInteger seed: seedInteger
"Hash the given positive integer. The integer to be hashed should have 512 or fewer bits. This entry point is used in the production of random numbers"
| buffer dstIndex |
"Initialize totalA through totalE to their seed values."
+ totals
+ ifNil: [
+ totalA := ThirtyTwoBitRegister
+ fromInteger: ((seedInteger bitShift: -128) bitAnd: 16rFFFFFFFF).
+ totalB := ThirtyTwoBitRegister
+ fromInteger: ((seedInteger bitShift: -96) bitAnd: 16rFFFFFFFF).
+ totalC := ThirtyTwoBitRegister
+ fromInteger: ((seedInteger bitShift: -64) bitAnd: 16rFFFFFFFF).
+ totalD := ThirtyTwoBitRegister
+ fromInteger: ((seedInteger bitShift: -32) bitAnd: 16rFFFFFFFF).
+ totalE := ThirtyTwoBitRegister
+ fromInteger: (seedInteger bitAnd: 16rFFFFFFFF) ]
+ ifNotNil: [
+ totals
+ at: 1 put: ((seedInteger bitShift: -128) bitAnd: 16rFFFFFFFF);
+ at: 2 put: ((seedInteger bitShift: -96) bitAnd: 16rFFFFFFFF);
+ at: 3 put: ((seedInteger bitShift: -64) bitAnd: 16rFFFFFFFF);
+ at: 4 put: ((seedInteger bitShift: -32) bitAnd: 16rFFFFFFFF);
+ at: 5 put: (seedInteger bitAnd: 16rFFFFFFFF) ].
- totalA := ThirtyTwoBitRegister
- fromInteger: ((seedInteger bitShift: -128) bitAnd: 16rFFFFFFFF).
- totalB := ThirtyTwoBitRegister
- fromInteger: ((seedInteger bitShift: -96) bitAnd: 16rFFFFFFFF).
- totalC := ThirtyTwoBitRegister
- fromInteger: ((seedInteger bitShift: -64) bitAnd: 16rFFFFFFFF).
- totalD := ThirtyTwoBitRegister
- fromInteger: ((seedInteger bitShift: -32) bitAnd: 16rFFFFFFFF).
- totalE := ThirtyTwoBitRegister
- fromInteger: (seedInteger bitAnd: 16rFFFFFFFF).
- self initializeTotalsArray.
-
"pad integer with zeros"
buffer := ByteArray new: 64.
dstIndex := 0.
aPositiveInteger digitLength to: 1 by: -1 do: [:i |
buffer at: (dstIndex := dstIndex + 1) put: (aPositiveInteger digitAt: i)].
"process that one block"
self processBuffer: buffer.
^ self finalHash
!
Item was changed:
----- Method: SecureHashAlgorithm>>hashStream: (in category 'public') -----
hashStream: aPositionableStream
"Hash the contents of the given stream from the current position to the end using the Secure Hash Algorithm. The SHA algorithm is defined in FIPS PUB 180-1. It is also described on p. 442 of 'Applied Cryptography: Protocols, Algorithms, and Source Code in C' by Bruce Scheier, Wiley, 1996."
"SecureHashAlgorithm new hashStream: (ReadStream on: 'foo')"
| startPosition buf bitLength |
self initializeTotals.
"(SecureHashAlgorithm new hashMessage: '') radix: 16
=> 'DA39A3EE5E6B4B0D3255BFEF95601890AFD80709'"
+ aPositionableStream atEnd ifTrue: [self processFinalBuffer: #[] bitLength: 0].
- aPositionableStream atEnd ifTrue: [self processFinalBuffer: #() bitLength: 0].
startPosition := aPositionableStream position.
+ buf := ByteArray new: 64.
[aPositionableStream atEnd] whileFalse: [
+ buf := aPositionableStream next: 64 into: buf startingAt: 1.
- buf := aPositionableStream next: 64.
(aPositionableStream atEnd not and: [buf size = 64])
ifTrue: [self processBuffer: buf]
ifFalse: [
bitLength := (aPositionableStream position - startPosition) * 8.
self processFinalBuffer: buf bitLength: bitLength]].
^ self finalHash
!
Item was added:
+ ----- Method: SecureHashAlgorithm>>initialize (in category 'initialize-release') -----
+ initialize
+
+ self primHasSecureHashPrimitive
+ ifTrue: [
+ totals := Bitmap new: 5.
+ expandBuffer := Bitmap new: 80 ]
+ ifFalse: [
+ totalA := ThirtyTwoBitRegister new.
+ totalB := ThirtyTwoBitRegister new.
+ totalC := ThirtyTwoBitRegister new.
+ totalD := ThirtyTwoBitRegister new.
+ totalE := ThirtyTwoBitRegister new.
+ expandBuffer := Array new: 80.
+ 1 to: 80 do: [ :index |
+ expandBuffer at: index put: ThirtyTwoBitRegister new ].
+ t1 := ThirtyTwoBitRegister new.
+ t2 := ThirtyTwoBitRegister new.
+ t := ThirtyTwoBitRegister new.
+ a := ThirtyTwoBitRegister new.
+ b := ThirtyTwoBitRegister new.
+ c := ThirtyTwoBitRegister new.
+ d := ThirtyTwoBitRegister new.
+ e := ThirtyTwoBitRegister new ]!
Item was changed:
----- Method: SecureHashAlgorithm>>initializeTotals (in category 'private') -----
initializeTotals
"Initialize totalA through totalE to their seed values."
+ totals
+ ifNil: [
+ "total registers for use when primitives are absent"
+ totalA loadFrom: TA.
+ totalB loadFrom: TB.
+ totalC loadFrom: TC.
+ totalD loadFrom: TD.
+ totalE loadFrom: TE ]
+ ifNotNil: [
+ totals
+ replaceFrom: 1
+ to: totals size
+ with: TI
+ startingAt: 1 ]!
- "total registers for use when primitives are absent"
- totalA := ThirtyTwoBitRegister fromInteger: 16r67452301.
- totalB := ThirtyTwoBitRegister fromInteger: 16rEFCDAB89.
- totalC := ThirtyTwoBitRegister fromInteger: 16r98BADCFE.
- totalD := ThirtyTwoBitRegister fromInteger: 16r10325476.
- totalE := ThirtyTwoBitRegister fromInteger: 16rC3D2E1F0.
- self initializeTotalsArray.
- !
Item was removed:
- ----- Method: SecureHashAlgorithm>>initializeTotalsArray (in category 'private') -----
- initializeTotalsArray
- "Initialize the totals array from the registers for use with the primitives."
-
- totals := Bitmap new: 5.
- totals at: 1 put: totalA asInteger.
- totals at: 2 put: totalB asInteger.
- totals at: 3 put: totalC asInteger.
- totals at: 4 put: totalD asInteger.
- totals at: 5 put: totalE asInteger.
- !
Item was changed:
----- Method: SecureHashAlgorithm>>processBuffer: (in category 'private') -----
processBuffer: aByteArray
"Process given 64-byte buffer, accumulating the results in totalA through totalE."
+ | tmp |
+ totals ifNotNil: [ ^self processBufferUsingPrimitives: aByteArray ].
- | a b c d e w tmp |
- self primHasSecureHashPrimitive
- ifTrue: [^ self processBufferUsingPrimitives: aByteArray]
- ifFalse: [totals := nil].
- "initialize registers a through e from the current totals"
- a := totalA copy.
- b := totalB copy.
- c := totalC copy.
- d := totalD copy.
- e := totalE copy.
-
"expand and process the buffer"
+ self expandedBlock: aByteArray.
+
+ "initialize registers a through e from the current totals"
+ a loadFrom: totalA.
+ b loadFrom: totalB.
+ c loadFrom: totalC.
+ d loadFrom: totalD.
+ e loadFrom: totalE.
- w := self expandedBlock: aByteArray.
1 to: 80 do: [:i |
+ t
+ loadFrom: a;
+ leftRotateBy: 5;
+ += (self hashFunction: i);
- tmp := (a copy leftRotateBy: 5)
- += (self hashFunction: i of: b with: c with: d);
+= e;
+ += (expandBuffer at: i);
- += (w at: i);
+= (self constantForStep: i).
+ tmp := e.
e := d.
d := c.
c := b leftRotateBy: 30.
b := a.
+ a := t.
+ t := tmp ].
- a := tmp].
"add a through e into total accumulators"
totalA += a.
totalB += b.
totalC += c.
totalD += d.
totalE += e.
!
Item was changed:
----- Method: SecureHashAlgorithm>>processBufferUsingPrimitives: (in category 'private') -----
processBufferUsingPrimitives: aByteArray
"Process given 64-byte buffer using the primitives, accumulating the results in totals."
- | w |
"expand and process the buffer"
+ self
+ primExpandBlock: aByteArray into: expandBuffer;
+ primHashBlock: expandBuffer using: totals.
- w := Bitmap new: 80.
- self primExpandBlock: aByteArray into: w.
- self primHashBlock: w using: totals.
!
Levente Uzonyi uploaded a new version of System to project The Trunk:
http://source.squeak.org/trunk/System-ul.774.mcz
==================== Summary ====================
Name: System-ul.774
Author: ul
Time: 27 October 2015, 1:44:16.212 am
UUID: 5c00c594-7a1a-4148-9b3b-bfe6ae6486e1
Ancestors: System-ul.773
Reverted some of the less efficient optimizations.
=============== Diff against System-mt.772 ===============
Item was changed:
Object subclass: #SecureHashAlgorithm
instanceVariableNames: 'totalA totalB totalC totalD totalE totals'
+ classVariableNames: 'K1 K2 K3 K4 TA TB TC TD TE TI'
- classVariableNames: 'K1 K2 K3 K4'
poolDictionaries: ''
category: 'System-Digital Signatures'!
!SecureHashAlgorithm commentStamp: '<historical>' prior: 0!
This class implements the Secure Hash Algorithm (SHA) described in the U.S. government's Secure Hash Standard (SHS). This standard is described in FIPS PUB 180-1, "SECURE HASH STANDARD", April 17, 1995.
The Secure Hash Algorithm is also described on p. 442 of 'Applied Cryptography: Protocols, Algorithms, and Source Code in C' by Bruce Scheier, Wiley, 1996.
See the comment in class DigitalSignatureAlgorithm for details on its use.
Implementation notes:
The secure hash standard was created with 32-bit hardware in mind. All arithmetic in the hash computation must be done modulo 2^32. This implementation uses ThirtyTwoBitRegister objects to simulate hardware registers; this implementation is about six times faster than using LargePositiveIntegers (measured on a Macintosh G3 Powerbook). Implementing a primitive to process each 64-byte buffer would probably speed up the computation by a factor of 20 or more.
!
Item was changed:
----- Method: SecureHashAlgorithm class>>initialize (in category 'class initialization') -----
initialize
"SecureHashAlgorithm initialize"
"For the curious, here's where these constants come from:
#(2 3 5 10) collect: [:x | ((x sqrt / 4.0) * (2.0 raisedTo: 32)) truncated hex]"
K1 := ThirtyTwoBitRegister fromInteger: 16r5A827999.
K2 := ThirtyTwoBitRegister fromInteger: 16r6ED9EBA1.
K3 := ThirtyTwoBitRegister fromInteger: 16r8F1BBCDC.
K4 := ThirtyTwoBitRegister fromInteger: 16rCA62C1D6.
+
+ TA := ThirtyTwoBitRegister fromInteger: 16r67452301.
+ TB := ThirtyTwoBitRegister fromInteger: 16rEFCDAB89.
+ TC := ThirtyTwoBitRegister fromInteger: 16r98BADCFE.
+ TD := ThirtyTwoBitRegister fromInteger: 16r10325476.
+ TE := ThirtyTwoBitRegister fromInteger: 16rC3D2E1F0.
+ (TI := Bitmap new: 5)
+ at: 1 put: 16r67452301;
+ at: 2 put: 16rEFCDAB89;
+ at: 3 put: 16r98BADCFE;
+ at: 4 put: 16r10325476;
+ at: 5 put: 16rC3D2E1F0!
- !
Item was changed:
----- Method: SecureHashAlgorithm>>expandedBlock: (in category 'private') -----
expandedBlock: aByteArray
+ "Convert the given 64 byte buffer into 80 32-bit registers and answer the result."
+
+ | out src |
- "Convert the given 64 byte buffer into 80 32-bit registers and answer the result."
- | out src v |
out := Array new: 80.
src := 1.
1 to: 16 do: [:i |
out at: i put: (ThirtyTwoBitRegister fromByteArray: aByteArray at: src).
src := src + 4].
17 to: 80 do: [:i |
+ out at: i put: (
+ (out at: i - 3) copy
+ bitXor: (out at: i - 8);
+ bitXor: (out at: i - 14);
+ bitXor: (out at: i - 16);
+ leftRotateBy: 1) ].
- v := (out at: i - 3) copy.
- v bitXor: (out at: i - 8);
- bitXor: (out at: i - 14);
- bitXor: (out at: i - 16);
- leftRotateBy: 1.
- out at: i put: v].
^ out
!
Item was changed:
----- Method: SecureHashAlgorithm>>finalHash (in category 'private') -----
finalHash
"Concatenate the final totals to build the 160-bit integer result."
"Details: If the primitives are supported, the results are in the totals array. Otherwise, they are in the instance variables totalA through totalE."
+ | result |
+ result := ByteArray new: 20.
+ totals
+ ifNil: [ "compute final hash when not using primitives"
+ result
+ unsignedShortAt: 1 put: totalE low bigEndian: false;
+ unsignedShortAt: 3 put: totalE hi bigEndian: false;
+ unsignedShortAt: 5 put: totalD low bigEndian: false;
+ unsignedShortAt: 7 put: totalD hi bigEndian: false;
+ unsignedShortAt: 9 put: totalC low bigEndian: false;
+ unsignedShortAt: 11 put: totalC hi bigEndian: false;
+ unsignedShortAt: 13 put: totalB low bigEndian: false;
+ unsignedShortAt: 15 put: totalB hi bigEndian: false;
+ unsignedShortAt: 17 put: totalA low bigEndian: false;
+ unsignedShortAt: 19 put: totalA hi bigEndian: false ]
+ ifNotNil: [ "compute final hash when using primitives"
+ result
+ unsignedLongAt: 1 put: (totals at: 5) bigEndian: false;
+ unsignedLongAt: 5 put: (totals at: 4) bigEndian: false;
+ unsignedLongAt: 9 put: (totals at: 3) bigEndian: false;
+ unsignedLongAt: 13 put: (totals at: 2) bigEndian: false;
+ unsignedLongAt: 17 put: (totals at: 1) bigEndian: false ].
+ ^(LargePositiveInteger new: result size)
+ replaceFrom: 1
+ to: result size
+ with: result
+ startingAt: 1;
+ normalize!
- | r |
- totals ifNil: [ "compute final hash when not using primitives"
- ^ (totalA asInteger bitShift: 128) +
- (totalB asInteger bitShift: 96) +
- (totalC asInteger bitShift: 64) +
- (totalD asInteger bitShift: 32) +
- (totalE asInteger)].
-
- "compute final hash when using primitives"
- r := 0.
- 1 to: 5 do: [:i |
- r := r bitOr: ((totals at: i) bitShift: (32 * (5 - i)))].
- ^ r
- !
Item was removed:
- ----- Method: SecureHashAlgorithm>>hashFunction:of:with:with: (in category 'private') -----
- hashFunction: i of: x with: y with: z
- "Compute the hash function for the i-th step of the block hash loop. We number our steps 1-80, versus the 0-79 of the standard."
- "Details: There are four functions, one for each 20 iterations. The second and fourth are the same."
-
- i <= 20 ifTrue: [^ x copy bitAnd: y; bitOr: (x copy bitInvert; bitAnd: z)].
- i <= 40 ifTrue: [^ x copy bitXor: y; bitXor: z].
- i <= 60 ifTrue: [^ x copy bitAnd: y; bitOr: (x copy bitAnd: z); bitOr: (y copy bitAnd: z)].
- ^ x copy bitXor: y; bitXor: z
- !
Item was added:
+ ----- Method: SecureHashAlgorithm>>hashFunction:of:with:with:using:and: (in category 'private') -----
+ hashFunction: i of: x with: y with: z using: t1 and: t2
+ "Compute the hash function for the i-th step of the block hash loop. We number our steps 1-80, versus the 0-79 of the standard."
+ "Details: There are four functions, one for each 20 iterations. The second and fourth are the same."
+
+ t1 loadFrom: x.
+ i <= 20 ifTrue: [
+ t2
+ loadFrom: x;
+ bitInvert;
+ bitAnd: z.
+ ^t1
+ bitAnd: y;
+ bitOr: t2 ].
+ i <= 40 ifTrue: [
+ ^t1
+ bitXor: y;
+ bitXor: z ].
+ i <= 60 ifTrue: [
+ t2
+ loadFrom: x;
+ bitOr: y;
+ bitAnd: z.
+ ^t1
+ bitAnd: y;
+ bitOr: t2 ].
+ ^t1
+ bitXor: y;
+ bitXor: z
+ !
Item was changed:
----- Method: SecureHashAlgorithm>>hashInteger:seed: (in category 'public') -----
hashInteger: aPositiveInteger seed: seedInteger
"Hash the given positive integer. The integer to be hashed should have 512 or fewer bits. This entry point is used in the production of random numbers"
| buffer dstIndex |
"Initialize totalA through totalE to their seed values."
+ totals
+ ifNil: [
+ totalA := ThirtyTwoBitRegister
+ fromInteger: ((seedInteger bitShift: -128) bitAnd: 16rFFFFFFFF).
+ totalB := ThirtyTwoBitRegister
+ fromInteger: ((seedInteger bitShift: -96) bitAnd: 16rFFFFFFFF).
+ totalC := ThirtyTwoBitRegister
+ fromInteger: ((seedInteger bitShift: -64) bitAnd: 16rFFFFFFFF).
+ totalD := ThirtyTwoBitRegister
+ fromInteger: ((seedInteger bitShift: -32) bitAnd: 16rFFFFFFFF).
+ totalE := ThirtyTwoBitRegister
+ fromInteger: (seedInteger bitAnd: 16rFFFFFFFF) ]
+ ifNotNil: [
+ totals
+ at: 1 put: ((seedInteger bitShift: -128) bitAnd: 16rFFFFFFFF);
+ at: 2 put: ((seedInteger bitShift: -96) bitAnd: 16rFFFFFFFF);
+ at: 3 put: ((seedInteger bitShift: -64) bitAnd: 16rFFFFFFFF);
+ at: 4 put: ((seedInteger bitShift: -32) bitAnd: 16rFFFFFFFF);
+ at: 5 put: (seedInteger bitAnd: 16rFFFFFFFF) ].
- totalA := ThirtyTwoBitRegister
- fromInteger: ((seedInteger bitShift: -128) bitAnd: 16rFFFFFFFF).
- totalB := ThirtyTwoBitRegister
- fromInteger: ((seedInteger bitShift: -96) bitAnd: 16rFFFFFFFF).
- totalC := ThirtyTwoBitRegister
- fromInteger: ((seedInteger bitShift: -64) bitAnd: 16rFFFFFFFF).
- totalD := ThirtyTwoBitRegister
- fromInteger: ((seedInteger bitShift: -32) bitAnd: 16rFFFFFFFF).
- totalE := ThirtyTwoBitRegister
- fromInteger: (seedInteger bitAnd: 16rFFFFFFFF).
- self initializeTotalsArray.
-
"pad integer with zeros"
buffer := ByteArray new: 64.
dstIndex := 0.
aPositiveInteger digitLength to: 1 by: -1 do: [:i |
buffer at: (dstIndex := dstIndex + 1) put: (aPositiveInteger digitAt: i)].
"process that one block"
self processBuffer: buffer.
^ self finalHash
!
Item was changed:
----- Method: SecureHashAlgorithm>>hashStream: (in category 'public') -----
hashStream: aPositionableStream
"Hash the contents of the given stream from the current position to the end using the Secure Hash Algorithm. The SHA algorithm is defined in FIPS PUB 180-1. It is also described on p. 442 of 'Applied Cryptography: Protocols, Algorithms, and Source Code in C' by Bruce Scheier, Wiley, 1996."
"SecureHashAlgorithm new hashStream: (ReadStream on: 'foo')"
| startPosition buf bitLength |
self initializeTotals.
"(SecureHashAlgorithm new hashMessage: '') radix: 16
=> 'DA39A3EE5E6B4B0D3255BFEF95601890AFD80709'"
+ aPositionableStream atEnd ifTrue: [self processFinalBuffer: #[] bitLength: 0].
- aPositionableStream atEnd ifTrue: [self processFinalBuffer: #() bitLength: 0].
startPosition := aPositionableStream position.
+ buf := ByteArray new: 64.
[aPositionableStream atEnd] whileFalse: [
+ buf := aPositionableStream next: 64 into: buf startingAt: 1.
- buf := aPositionableStream next: 64.
(aPositionableStream atEnd not and: [buf size = 64])
ifTrue: [self processBuffer: buf]
ifFalse: [
bitLength := (aPositionableStream position - startPosition) * 8.
self processFinalBuffer: buf bitLength: bitLength]].
^ self finalHash
!
Item was added:
+ ----- Method: SecureHashAlgorithm>>initialize (in category 'initialize-release') -----
+ initialize
+
+ self primHasSecureHashPrimitive
+ ifTrue: [ totals := Bitmap new: 5 ]
+ ifFalse: [
+ totalA := ThirtyTwoBitRegister new.
+ totalB := ThirtyTwoBitRegister new.
+ totalC := ThirtyTwoBitRegister new.
+ totalD := ThirtyTwoBitRegister new.
+ totalE := ThirtyTwoBitRegister new ]!
Item was changed:
----- Method: SecureHashAlgorithm>>initializeTotals (in category 'private') -----
initializeTotals
"Initialize totalA through totalE to their seed values."
+ totals
+ ifNil: [
+ "total registers for use when primitives are absent"
+ totalA loadFrom: TA.
+ totalB loadFrom: TB.
+ totalC loadFrom: TC.
+ totalD loadFrom: TD.
+ totalE loadFrom: TE ]
+ ifNotNil: [
+ totals
+ replaceFrom: 1
+ to: totals size
+ with: TI
+ startingAt: 1 ]!
- "total registers for use when primitives are absent"
- totalA := ThirtyTwoBitRegister fromInteger: 16r67452301.
- totalB := ThirtyTwoBitRegister fromInteger: 16rEFCDAB89.
- totalC := ThirtyTwoBitRegister fromInteger: 16r98BADCFE.
- totalD := ThirtyTwoBitRegister fromInteger: 16r10325476.
- totalE := ThirtyTwoBitRegister fromInteger: 16rC3D2E1F0.
- self initializeTotalsArray.
- !
Item was removed:
- ----- Method: SecureHashAlgorithm>>initializeTotalsArray (in category 'private') -----
- initializeTotalsArray
- "Initialize the totals array from the registers for use with the primitives."
-
- totals := Bitmap new: 5.
- totals at: 1 put: totalA asInteger.
- totals at: 2 put: totalB asInteger.
- totals at: 3 put: totalC asInteger.
- totals at: 4 put: totalD asInteger.
- totals at: 5 put: totalE asInteger.
- !
Item was changed:
----- Method: SecureHashAlgorithm>>processBuffer: (in category 'private') -----
processBuffer: aByteArray
"Process given 64-byte buffer, accumulating the results in totalA through totalE."
+ | a b c d e t tmp w tmp2 tmp3 |
+ totals ifNotNil: [ ^self processBufferUsingPrimitives: aByteArray ].
- | a b c d e w tmp |
- self primHasSecureHashPrimitive
- ifTrue: [^ self processBufferUsingPrimitives: aByteArray]
- ifFalse: [totals := nil].
"initialize registers a through e from the current totals"
a := totalA copy.
b := totalB copy.
c := totalC copy.
d := totalD copy.
e := totalE copy.
"expand and process the buffer"
w := self expandedBlock: aByteArray.
+ tmp := ThirtyTwoBitRegister new.
+ tmp2 := ThirtyTwoBitRegister new.
+ tmp3 := ThirtyTwoBitRegister new.
1 to: 80 do: [:i |
+ tmp
+ loadFrom: a;
+ leftRotateBy: 5;
+ += (self hashFunction: i of: b with: c with: d using: tmp2 and: tmp3);
- tmp := (a copy leftRotateBy: 5)
- += (self hashFunction: i of: b with: c with: d);
+= e;
+= (w at: i);
+= (self constantForStep: i).
+ t := e.
e := d.
d := c.
c := b leftRotateBy: 30.
b := a.
+ a := tmp.
+ tmp := t ].
- a := tmp].
"add a through e into total accumulators"
totalA += a.
totalB += b.
totalC += c.
totalD += d.
totalE += e.
!
On 23/10/15 12:07, Jose San Leandro wrote:
> Sad news.
Indeed. I'll try again next year.
There are of course several devrooms where we can propose talks, e.g.
"Containers and Process Isolation" should be interested to hear about
our long-time experience with images
Stephan
https://fosdem.org/2016/news/2015-10-22-accepted-devrooms/