tim Rowledge uploaded a new version of WebClient-Core to project The Trunk: http://source.squeak.org/trunk/WebClient-Core-ul.132.mcz
==================== Summary ====================
Name: WebClient-Core-ul.132 Author: ul Time: 11 April 2023, 6:05:30.414442 pm UUID: 7d14b070-76e7-47d2-a31d-9a2b283cefc6 Ancestors: WebClient-Core-mt.130
Speed up MD5 computation in WebUtils: - deprecated #md5HashStream: in favor of #md5HashStreamAsBytes:, which does the same, but returns the bytes in the correct order (not reversed). - avoid reading and writing words from and to ByteArrays (buffer) by changing the class of buffer between WordArray and ByteArray as needed. Hopefully endianness will be fine on all platforms (tests will fail if not). - use #readInto:startingAt:count: instead of #nextInto: to read from the stream into the buffer. This avoids a few sends and potential reallocation of the buffer. - do not convert the ByteString to ByteArray in #md5Digest:. ByteArray >> #replaceFrom:to:with:startingAt: can copy from a ByteString into the buffer.
Throughput increases by ~6x for 16k inputs.
=============== Diff against WebClient-Core-mt.130 ===============
Item was changed: ----- Method: WebUtils class>>md5Digest: (in category 'md5') ----- md5Digest: aStringOrByteArray "This creates a little endian hex string to be used with various auth methods + This is the same as htdigest (apache) uses for its md5 digest auth db. + + self md5Digest: 'foo' " - This is the same as htdigest (apache) uses for its md5 digest auth db"
+ ^(self md5HashStreamAsBytes: aStringOrByteArray readStream) hex! - ^(self md5HashStream: (ReadStream on: aStringOrByteArray asByteArray)) reversed hex!
Item was changed: ----- Method: WebUtils class>>md5HashStream: (in category 'md5') ----- md5HashStream: aStream + "This method returns the bytes of the md5 hash in revese order for backwards compatibility. Consider updating your implementation to avoid unnecessary reversal. - "self md5HashStream: (ReadStream on: 'foo')" - - | start buffer chunkSize n words hash | - hash := WordArray - with: 16r67452301 - with: 16rEFCDAB89 - with: 16r98BADCFE - with: 16r10325476. - words := WordArray new: 16. - buffer := ByteArray new: 64. - start := aStream position. - [ - chunkSize := (aStream nextInto: buffer) size. - chunkSize < 64 or: [ aStream atEnd ] ] - whileFalse: [ - 1 to: 16 do:[:i| words at: i put: (buffer unsignedLongAt: i*4-3 bigEndian: false)]. - self md5Transform: words hash: hash ]. - buffer from: chunkSize +1 to: buffer size put: 0. - chunkSize < 56 ifTrue: [ - buffer at: chunkSize + 1 put: 128. "trailing bit" - ] ifFalse:[ - "not enough room for the length, so just pad this one, then..." - chunkSize < 64 ifTrue:[buffer at: chunkSize + 1 put: 128]. - 1 to: 16 do:[:i| words at: i put: (buffer unsignedLongAt: i*4-3 bigEndian: false)]. - self md5Transform: words hash: hash. - "process one additional block of padding ending with the length" - buffer atAllPut: 0. - chunkSize = 64 ifTrue: [buffer at: 1 put: 128]. - ]. - "Fill in the final 8 bytes with the 64-bit length in bits." - n := (aStream position - start) * 8. - 7 to: 0 by: -1 do:[:i| buffer at: (buffer size - i) put: ((n bitShift: 7 - i * -8) bitAnd: 255)]. - "Final round" - 1 to: 16 do:[:i| words at: i put: (buffer unsignedLongAt: i*4-3 bigEndian: false)]. - self md5Transform: words hash: hash. + self md5HashStream: 'foo' readStream " + + self deprecated: 'Use #md5HashStreamAsBytes: and reverse the bytes in your code.'. + ^(self md5HashStreamAsBytes: aStream) reverseInPlace! - ^(ByteArray new: 16) - unsignedLongAt: 1 put: (hash at: 4) bigEndian: true; - unsignedLongAt: 5 put: (hash at: 3) bigEndian: true; - unsignedLongAt: 9 put: (hash at: 2) bigEndian: true; - unsignedLongAt: 13 put: (hash at: 1) bigEndian: true; - yourself!
Item was added: + ----- Method: WebUtils class>>md5HashStreamAsBytes: (in category 'md5') ----- + md5HashStreamAsBytes: aStream + "Return a ByteArray containing the md5 hash of aStream. + + self md5HashStreamAsBytes: 'foo' readStream " + + | start buffer chunkSize n hash | + hash := WordArray + with: 16r67452301 + with: 16rEFCDAB89 + with: 16r98BADCFE + with: 16r10325476. + buffer := ByteArray new: 64. + start := aStream position. + [ + chunkSize := aStream readInto: buffer startingAt: 1 count: 64. + chunkSize < 64 or: [ aStream atEnd ] ] + whileFalse: [ + WordArray adoptInstance: buffer. + self md5Transform: buffer hash: hash. + ByteArray adoptInstance: buffer ]. + buffer from: chunkSize +1 to: buffer size put: 0. + chunkSize < 56 ifTrue: [ + buffer at: chunkSize + 1 put: 128. "trailing bit" + ] ifFalse:[ + "not enough room for the length, so just pad this one, then..." + chunkSize < 64 ifTrue:[buffer at: chunkSize + 1 put: 128]. + WordArray adoptInstance: buffer. + self md5Transform: buffer hash: hash. + ByteArray adoptInstance: buffer. + "process one additional block of padding ending with the length" + buffer atAllPut: 0. + chunkSize = 64 ifTrue: [buffer at: 1 put: 128]. + ]. + "Fill in the final 8 bytes with the 64-bit length in bits." + n := (aStream position - start) * 8. + 7 to: 0 by: -1 do:[:i| buffer at: (buffer size - i) put: ((n bitShift: 7 - i * -8) bitAnd: 255)]. + "Final round" + WordArray adoptInstance: buffer. + self md5Transform: buffer hash: hash. + ByteArray adoptInstance: hash. "Trick to avoid reallocation." + ^hash!
Item was changed: ----- Method: WebUtils class>>webSocketHandshake:with:with: (in category 'websockets') ----- webSocketHandshake: key1 with: key2 with: data "Do the actual WebSocket handshake computation"
| bytes | bytes := ByteArray new: 16. bytes longAt: 1 put: key1 bigEndian: true. bytes longAt: 5 put: key2 bigEndian: true. bytes replaceFrom: 9 to: 16 with: data. + ^self md5HashStreamAsBytes: bytes readStream! - ^(self md5HashStream: bytes readStream) reversed!
On 2023-04-14, at 12:40 AM, commits@source.squeak.org wrote:
tim Rowledge uploaded a new version of WebClient-Core to project The Trunk: http://source.squeak.org/trunk/WebClient-Core-ul.132.mcz
==================== Summary ====================
Name: WebClient-Core-ul.132 Author: ul
Following from this I note that there are several two-plus year old versions of WebClient-core in the inbox. It looks like they have been dealt with and should be moved to the treated-inbox. Any objections ct &| mt?
tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Useful random insult:- A mental midget with the IQ of a fencepost.
Hi Tim --
Good question. Let's wait for ct to return from his holidays. To me, mt.125 and ct.125-128 are still to be integrated into Trunk. I would not move it to treated. Best, Marcel Am 14.04.2023 02:48:26 schrieb tim Rowledge tim@rowledge.org:
On 2023-04-14, at 12:40 AM, commits@source.squeak.org wrote:
tim Rowledge uploaded a new version of WebClient-Core to project The Trunk: http://source.squeak.org/trunk/WebClient-Core-ul.132.mcz
==================== Summary ====================
Name: WebClient-Core-ul.132 Author: ul
Following from this I note that there are several two-plus year old versions of WebClient-core in the inbox. It looks like they have been dealt with and should be moved to the treated-inbox. Any objections ct &| mt?
tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Useful random insult:- A mental midget with the IQ of a fencepost.
Hi all,
I have moved WebClient-Core-ct.126 as an ancestor of WebClient-Core-ct.128 to treated. WebClient-Core-ct.128 had an interesting discussion with Tobias (topa), but unfortunately, we were unable to make a final design decision on whether we want to support preauthentication; however, the current comments in the WebClient are not aligned with the discussion. Always looking forward to other views and opinions. WebClient-Core-ct.125 and WebClient-Core-ct.127 look like small fixes/refactoring that I would still deem relevant; I have not yet re-checked their implementation by myself, though. If anyone else reviews them and finds them fine, please do not hesitate to merge them! :-)
Best, Christoph
PS: Please include "christoph" or "thiede" in any messages that should not escape my attention. In theory, I still try to read every conversation on the list, but in practice, I'm months behind right now. Unfortunately, "ct" as an email filter would produce a lot (!) of false positives. Work continues on Squeak Inbox Talk to improve my awareness of relevant things on the list. ;-)
Am 2023-04-14 um 1:48 PM schrieb Marcel Taeumel via Squeak-dev:
Hi Tim --
Good question. Let's wait for ct to return from his holidays. To me, mt.125 and ct.125-128 are still to be integrated into Trunk. I would not move it to treated.
Best, Marcel Am 14.04.2023 02:48:26 schrieb tim Rowledge tim@rowledge.org:
On 2023-04-14, at 12:40 AM, commits@source.squeak.org wrote:
tim Rowledge uploaded a new version of WebClient-Core to project The Trunk: http://source.squeak.org/trunk/WebClient-Core-ul.132.mcz
==================== Summary ====================
Name: WebClient-Core-ul.132 Author: ul
Following from this I note that there are several two-plus year old versions of WebClient-core in the inbox. It looks like they have been dealt with and should be moved to the treated-inbox. Any objections ct &| mt?
tim
tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Useful random insult:- A mental midget with the IQ of a fencepost.
squeak-dev@lists.squeakfoundation.org