Squeak-dev May 2023

squeak-dev@lists.squeakfoundation.org

16 participants
109 discussions

The Trunk: WebClient-Core-ul.132.mcz
by commits＠source.squeak.org 12 May '23

12 May '23

tim Rowledge uploaded a new version of WebClient-Core to project The Trunk: http://source.squeak.org/trunk/WebClient-Core-ul.132.mcz ==================== Summary ==================== Name: WebClient-Core-ul.132 Author: ul Time: 11 April 2023, 6:05:30.414442 pm UUID: 7d14b070-76e7-47d2-a31d-9a2b283cefc6 Ancestors: WebClient-Core-mt.130 Speed up MD5 computation in WebUtils: - deprecated #md5HashStream: in favor of #md5HashStreamAsBytes:, which does the same, but returns the bytes in the correct order (not reversed). - avoid reading and writing words from and to ByteArrays (buffer) by changing the class of buffer between WordArray and ByteArray as needed. Hopefully endianness will be fine on all platforms (tests will fail if not). - use #readInto:startingAt:count: instead of #nextInto: to read from the stream into the buffer. This avoids a few sends and potential reallocation of the buffer. - do not convert the ByteString to ByteArray in #md5Digest:. ByteArray >> #replaceFrom:to:with:startingAt: can copy from a ByteString into the buffer. Throughput increases by ~6x for 16k inputs. =============== Diff against WebClient-Core-mt.130 =============== Item was changed: ----- Method: WebUtils class>>md5Digest: (in category 'md5') ----- md5Digest: aStringOrByteArray "This creates a little endian hex string to be used with various auth methods + This is the same as htdigest (apache) uses for its md5 digest auth db. + + self md5Digest: 'foo' " - This is the same as htdigest (apache) uses for its md5 digest auth db" + ^(self md5HashStreamAsBytes: aStringOrByteArray readStream) hex! - ^(self md5HashStream: (ReadStream on: aStringOrByteArray asByteArray)) reversed hex! Item was changed: ----- Method: WebUtils class>>md5HashStream: (in category 'md5') ----- md5HashStream: aStream + "This method returns the bytes of the md5 hash in revese order for backwards compatibility. Consider updating your implementation to avoid unnecessary reversal. - "self md5HashStream: (ReadStream on: 'foo')" - - | start buffer chunkSize n words hash | - hash := WordArray - with: 16r67452301 - with: 16rEFCDAB89 - with: 16r98BADCFE - with: 16r10325476. - words := WordArray new: 16. - buffer := ByteArray new: 64. - start := aStream position. - [ - chunkSize := (aStream nextInto: buffer) size. - chunkSize < 64 or: [ aStream atEnd ] ] - whileFalse: [ - 1 to: 16 do:[:i| words at: i put: (buffer unsignedLongAt: i*4-3 bigEndian: false)]. - self md5Transform: words hash: hash ]. - buffer from: chunkSize +1 to: buffer size put: 0. - chunkSize < 56 ifTrue: [ - buffer at: chunkSize + 1 put: 128. "trailing bit" - ] ifFalse:[ - "not enough room for the length, so just pad this one, then..." - chunkSize < 64 ifTrue:[buffer at: chunkSize + 1 put: 128]. - 1 to: 16 do:[:i| words at: i put: (buffer unsignedLongAt: i*4-3 bigEndian: false)]. - self md5Transform: words hash: hash. - "process one additional block of padding ending with the length" - buffer atAllPut: 0. - chunkSize = 64 ifTrue: [buffer at: 1 put: 128]. - ]. - "Fill in the final 8 bytes with the 64-bit length in bits." - n := (aStream position - start) * 8. - 7 to: 0 by: -1 do:[:i| buffer at: (buffer size - i) put: ((n bitShift: 7 - i * -8) bitAnd: 255)]. - "Final round" - 1 to: 16 do:[:i| words at: i put: (buffer unsignedLongAt: i*4-3 bigEndian: false)]. - self md5Transform: words hash: hash. + self md5HashStream: 'foo' readStream " + + self deprecated: 'Use #md5HashStreamAsBytes: and reverse the bytes in your code.'. + ^(self md5HashStreamAsBytes: aStream) reverseInPlace! - ^(ByteArray new: 16) - unsignedLongAt: 1 put: (hash at: 4) bigEndian: true; - unsignedLongAt: 5 put: (hash at: 3) bigEndian: true; - unsignedLongAt: 9 put: (hash at: 2) bigEndian: true; - unsignedLongAt: 13 put: (hash at: 1) bigEndian: true; - yourself! Item was added: + ----- Method: WebUtils class>>md5HashStreamAsBytes: (in category 'md5') ----- + md5HashStreamAsBytes: aStream + "Return a ByteArray containing the md5 hash of aStream. + + self md5HashStreamAsBytes: 'foo' readStream " + + | start buffer chunkSize n hash | + hash := WordArray + with: 16r67452301 + with: 16rEFCDAB89 + with: 16r98BADCFE + with: 16r10325476. + buffer := ByteArray new: 64. + start := aStream position. + [ + chunkSize := aStream readInto: buffer startingAt: 1 count: 64. + chunkSize < 64 or: [ aStream atEnd ] ] + whileFalse: [ + WordArray adoptInstance: buffer. + self md5Transform: buffer hash: hash. + ByteArray adoptInstance: buffer ]. + buffer from: chunkSize +1 to: buffer size put: 0. + chunkSize < 56 ifTrue: [ + buffer at: chunkSize + 1 put: 128. "trailing bit" + ] ifFalse:[ + "not enough room for the length, so just pad this one, then..." + chunkSize < 64 ifTrue:[buffer at: chunkSize + 1 put: 128]. + WordArray adoptInstance: buffer. + self md5Transform: buffer hash: hash. + ByteArray adoptInstance: buffer. + "process one additional block of padding ending with the length" + buffer atAllPut: 0. + chunkSize = 64 ifTrue: [buffer at: 1 put: 128]. + ]. + "Fill in the final 8 bytes with the 64-bit length in bits." + n := (aStream position - start) * 8. + 7 to: 0 by: -1 do:[:i| buffer at: (buffer size - i) put: ((n bitShift: 7 - i * -8) bitAnd: 255)]. + "Final round" + WordArray adoptInstance: buffer. + self md5Transform: buffer hash: hash. + ByteArray adoptInstance: hash. "Trick to avoid reallocation." + ^hash! Item was changed: ----- Method: WebUtils class>>webSocketHandshake:with:with: (in category 'websockets') ----- webSocketHandshake: key1 with: key2 with: data "Do the actual WebSocket handshake computation" | bytes | bytes := ByteArray new: 16. bytes longAt: 1 put: key1 bigEndian: true. bytes longAt: 5 put: key2 bigEndian: true. bytes replaceFrom: 9 to: 16 with: data. + ^self md5HashStreamAsBytes: bytes readStream! - ^(self md5HashStream: bytes readStream) reversed!

4 3

Please ignore | MIME & encoding test for squeak-dev HyperKitty archive 🧪
by Thiede, Christoph 12 May '23

12 May '23

This is a test. Sorry ... Sent from OWA. Ümläutß, sorry for the noisé! 😎 Tab tab tab RxMatcher>>matchAgainstLookbehind:positive:nextLink: {matching} · ct 3/6/2020 20:09 (changed) Formatted text with different attributes This is an inline image: [cid:ee098c3d-1111-42bf-8d07-ffc5c53bee47] And another one: [cid:a0584d9b-1f02-4e63-9bcb-5eefa152b54b] Attached are a third image and a changeset file. Best, Christoph

1 0

The Trunk: Kernel-mt.1513.mcz
by commits＠source.squeak.org 12 May '23

12 May '23

Marcel Taeumel uploaded a new version of Kernel to project The Trunk: http://source.squeak.org/trunk/Kernel-mt.1513.mcz ==================== Summary ==================== Name: Kernel-mt.1513 Author: mt Time: 12 May 2023, 9:47:22.555412 am UUID: da4aac4d-9677-7a43-ab46-8fe9e08f1fb0 Ancestors: Kernel-eem.1512 Fix issue with categorization of classes. Better support for default category. Do not allow empty class category. =============== Diff against Kernel-eem.1512 =============== Item was changed: ----- Method: Categorizer>>classify:under:suppressIfDefault: (in category 'classifying') ----- classify: element under: heading suppressIfDefault: aBoolean "Store the argument, element, in the category named heading. If aBoolean is true, then invoke special logic such that the classification is NOT done if the new heading is the Default and the element already had a non-Default classification -- useful for filein" | catName catIndex elemIndex realHeading | + realHeading := (heading isEmptyOrNil or: [self isSpecialCategoryName: heading]) - realHeading := (heading isNil or: [self isSpecialCategoryName: heading]) ifTrue: [Default] ifFalse: [heading asSymbol]. (catName := self categoryOfElement: element) = realHeading ifTrue: [^ self]. "done if already under that category" catName ifNotNil: [ (aBoolean and: [realHeading = Default]) ifTrue: [^ self]. "return if non-Default category already assigned in memory" self basicRemoveElement: element]. "remove if in another category" (categoryArray indexOf: realHeading) = 0 ifTrue: [self addCategory: realHeading]. catIndex := categoryArray indexOf: realHeading. elemIndex := catIndex > 1 ifTrue: [categoryStops at: catIndex - 1] ifFalse: [0]. [(elemIndex := elemIndex + 1) <= (categoryStops at: catIndex) and: [element >= (elementArray at: elemIndex)]] whileTrue. "elemIndex is now the index for inserting the element. Do the insertion before it." elementArray := elementArray copyReplaceFrom: elemIndex to: elemIndex-1 with: (Array with: element). "add one to stops for this and later categories" catIndex to: categoryArray size do: [:i | categoryStops at: i put: (categoryStops at: i) + 1]. ((categoryArray includes: Default) and: [(self listAtCategoryNamed: Default) size = 0]) ifTrue: [self removeCategory: Default]. self assertInvariant.! Item was changed: ----- Method: ClassBuilder>>name:inEnvironment:subclassOf:type:instanceVariableNames:classVariableNames:poolDictionaries:category:unsafe: (in category 'class definition') ----- name: className inEnvironment: env subclassOf: newSuper type: type instanceVariableNames: instVarString classVariableNames: classVarString poolDictionaries: poolString category: category unsafe: unsafe "Define a new class in the given environment. If unsafe is true do not run any validation checks. This facility is provided to implement important system changes." | oldClass instVars classVars copyOfOldClass newClass | environ := env. instVars := Scanner new scanFieldNames: instVarString. classVars := (Scanner new scanFieldNames: classVarString) collect: [:x | x asSymbol]. "Validate the proposed name" unsafe ifFalse:[(self validateClassName: className) ifFalse:[^nil]]. oldClass := env at: className ifAbsent:[nil]. oldClass isBehavior ifFalse: [oldClass := nil] "Already checked in #validateClassName:" ifTrue: [ copyOfOldClass := oldClass copy. copyOfOldClass superclass addSubclass: copyOfOldClass]. [ | newCategory needNew force organization oldCategory | unsafe ifFalse:[ "Run validation checks so we know that we have a good chance for recompilation" (self validateSuperclass: newSuper forSubclass: oldClass) ifFalse:[^nil]. (self validateInstvars: instVars from: oldClass forSuper: newSuper) ifFalse:[^nil]. (self validateClassvars: classVars from: oldClass forSuper: newSuper) ifFalse:[^nil]. (self validateSubclassFormat: type from: oldClass forSuper: newSuper extra: instVars size) ifFalse:[^nil]]. "See if we need a new subclass" needNew := self needsSubclassOf: newSuper type: type instanceVariables: instVars from: oldClass. needNew == nil ifTrue:[^nil]. "some error" (needNew and:[unsafe not]) ifTrue:[ "Make sure we don't redefine any dangerous classes" (self tooDangerousClasses includes: oldClass name) ifTrue:[ self error: oldClass name, ' cannot be changed'. ]. "Check if the receiver should not be redefined" (oldClass ~~ nil and:[oldClass shouldNotBeRedefined]) ifTrue:[ self notify: oldClass name asText allBold, ' should not be redefined. \Proceed to store over it.' withCRs]]. needNew ifTrue:[ "Create the new class" newClass := self newSubclassOf: newSuper type: type instanceVariables: instVars from: oldClass. newClass == nil ifTrue:[^nil]. "Some error" newClass setName: className. newClass environment: environ. ] ifFalse:[ "Reuse the old class" newClass := oldClass. ]. "Install the class variables and pool dictionaries... " force := (newClass declare: classVarString) | (newClass sharing: poolString). "... classify ..." + newCategory := category ifNotNil: [category asSymbol]. + organization := environ ifNotNil:[environ organization]. + oldClass isNil ifFalse: [oldCategory := (organization categoryOfElement: oldClass name) asSymbol]. + organization classify: newClass name under: newCategory suppressIfDefault: true. + newCategory := organization categoryOfElement: newClass name. "if default" - category ifNotNil: [ - newCategory := category asSymbol. - organization := environ ifNotNil:[environ organization]. - oldClass isNil ifFalse: [oldCategory := (organization categoryOfElement: oldClass name) asSymbol]. - organization classify: newClass name under: newCategory suppressIfDefault: true]. "... recompile ..." newClass := self recompile: force from: oldClass to: newClass mutate: false. "... export if not yet done ..." (environ at: newClass name ifAbsent:[nil]) == newClass ifFalse:[ [environ at: newClass name put: newClass] on: AttemptToWriteReadOnlyGlobal do:[:ex| ex resume: true]. environ flushClassNameCache. ]. newClass doneCompiling. "... notify interested clients ..." oldClass isNil ifTrue: [ SystemChangeNotifier uniqueInstance classAdded: newClass inCategory: newCategory. ^ newClass]. newCategory ~= oldCategory ifTrue: [SystemChangeNotifier uniqueInstance class: newClass recategorizedFrom: oldCategory to: category] ifFalse: [SystemChangeNotifier uniqueInstance classDefinitionChangedFrom: copyOfOldClass to: newClass.]. ] ensure: [copyOfOldClass ifNotNil: [copyOfOldClass superclass removeSubclass: copyOfOldClass]. Behavior flushObsoleteSubclasses. ]. ^newClass!

1 0

squeak.org research info updated for 2022
by Marcel Taeumel 09 May '23

09 May '23

Hi all -- I just updated our list of publications around Squeak/Smalltalk for 2022 and even some of 2023: https://squeak.org/research/ [https://squeak.org/research/] Please let us know if you know of other publications. Best, Marcel

1 0

Parsing large json file *occasionally* results in an Integer
by tim Rowledge 05 May '23

05 May '23

I have some large json files that get parsed; we're talking 15Mb or so. They *should* result in large arrays of arrays of dictionaries of... you get the picture. Almost always, they do. Again and again, parse fileA and it works perfectly. Except just every now and then - maybe 3 times out of several hundred, on disparate dates, in different working images - the result is 58. Not 58 items in the root array, just 58, the SmallInteger. I can't see a code path that can result in that, which certainly makes it more interesting than usual. To make it bit more intriguing the subsequent code raises a dNU when this happens and so I get a debugger to look at. I can see the 58 that has been returned and put into the temp. I can retry the "open file, read as json" and it works perfectly. Fortunately that means I can proceed from there but the mystery remains. Whilst typing this my Pi has happily read & parsed that file 100 times without error. I *think* this has only actually happened on my Big! Scary! Xeon! server and never on the Pi, Mac, or little x64 xubuntu server. So maybe it's some strange bug courtesy of those nice people that brought us the Pentium Divide Error etc. So this is a shout into the void in hope of hearing an echo back that tells me somebody else has seen this and trapped an actual error. Or whatever. Oh, should probably point out I'm using the JSON-ul.56 from http://www.squeaksource.com/PostgresV3in a 5.3 image tim -- tim Rowledge; tim(a)rowledge.org; http://www.rowledge.org/tim M$ are grinning pigs in a sea of buggy code - The Inquirer

5 11

The Trunk: Compiler-mt.491.mcz
by commits＠source.squeak.org 05 May '23

05 May '23

Marcel Taeumel uploaded a new version of Compiler to project The Trunk: http://source.squeak.org/trunk/Compiler-mt.491.mcz ==================== Summary ==================== Name: Compiler-mt.491 Author: mt Time: 5 May 2023, 3:39:26.933945 pm UUID: e15f8291-ec32-9a4a-8af3-5328323a2ca3 Ancestors: Compiler-mt.490 Allow #pvt* methods to be overridden and extended in subclasses, that is, allow "super pvtFoo" calls. For example, Cuis' VectorEngine needs this. =============== Diff against Compiler-mt.490 =============== Item was changed: ----- Method: MessageNode>>pvtCheckForPvtSelector: (in category 'private') ----- pvtCheckForPvtSelector: encoder + "If the code being compiled is trying to send a private message (e.g. 'pvtCheckForPvtSelector:') to anyone other than self (or super), then complain to encoder." - "If the code being compiled is trying to send a private message (e.g. 'pvtCheckForPvtSelector:') to anyone other than self, then complain to encoder." selector isPvtSelector ifTrue: + [receiver isSelfOrSuperPseudoVariable ifFalse: + [encoder notify: 'Private messages may only be sent to self or super']].! - [receiver isSelfPseudoVariable ifFalse: - [encoder notify: 'Private messages may only be sent to self']].! Item was added: + ----- Method: ParseNode>>isSelfOrSuperPseudoVariable (in category 'testing') ----- + isSelfOrSuperPseudoVariable + + ^ self isSelfPseudoVariable or: [self isSuperPseudoVariable]! Item was added: + ----- Method: ParseNode>>isSuperPseudoVariable (in category 'testing') ----- + isSuperPseudoVariable + "Overridden in VariableNode." + ^false! Item was added: + ----- Method: VariableNode>>isSuperPseudoVariable (in category 'testing') ----- + isSuperPseudoVariable + "Answer if this ParseNode represents the 'super' pseudo-variable." + + ^ key = 'super' or: [name = '{{super}}']!

1 0

The Trunk: Environments-eem.85.mcz
by commits＠source.squeak.org 04 May '23

04 May '23

Eliot Miranda uploaded a new version of Environments to project The Trunk: http://source.squeak.org/trunk/Environments-eem.85.mcz ==================== Summary ==================== Name: Environments-eem.85 Author: eem Time: 4 May 2023, 12:18:59.95869 pm UUID: 162eabaa-b8ca-479f-8062-8e2c4fc3aea9 Ancestors: Environments-ul.84 Neater, faster classOrTraitNamed:, which almost always is given a class name, not a metaclass or trait class name. =============== Diff against Environments-ul.84 =============== Item was changed: ----- Method: Environment>>classOrTraitNamed: (in category 'classes and traits') ----- classOrTraitNamed: aString "aString is either a class or trait name or a class or trait name followed by ' class' or 'classTrait' respectively. + Answer the class or metaclass it names, or nil if not found." - Answer the class or metaclass it names." + | spaceIndex postFix meta baseName | + spaceIndex := aString indexOf: Character space. + spaceIndex = 0 + ifTrue: + [meta := false. + baseName := aString] + ifFalse: + [postFix := aString copyFrom: spaceIndex + 1 to: aString size. + baseName := aString copyFrom: 1 to: spaceIndex - 1. + (postFix = 'class' or: [postFix = 'classTrait']) + ifTrue: [meta := true] + ifFalse: [^nil]]. + ^(Symbol lookup: baseName) ifNotNil: + [ :baseNameSymbol | + declarations + at: baseNameSymbol + ifPresent: + [ :global | + global isBehavior ifTrue: + [meta + ifFalse: [ global ] + ifTrue: [ global classSide ] ] ] ]! - | meta baseName | - (aString endsWith: ' class') - ifTrue: [ - meta := true. - baseName := aString copyFrom: 1 to: aString size - 6 ] - ifFalse: [ - (aString endsWith: ' classTrait') - ifTrue: [ - meta := true. - baseName := aString copyFrom: 1 to: aString size - 11 ] - ifFalse: [ - meta := false. - baseName := aString ] ]. - ^(Symbol lookup: baseName) ifNotNil: [ :baseNameSymbol | - declarations at: baseNameSymbol ifPresent: [ :global | - global isBehavior ifTrue: [ - meta - ifFalse: [ global ] - ifTrue: [ global classSide ] ] ] ]!

1 0

The Trunk: Files-eem.203.mcz
by commits＠source.squeak.org 03 May '23

03 May '23

Eliot Miranda uploaded a new version of Files to project The Trunk: http://source.squeak.org/trunk/Files-eem.203.mcz ==================== Summary ==================== Name: Files-eem.203 Author: eem Time: 2 May 2023, 3:03:46.035929 pm UUID: 1f6149df-859d-47aa-8926-5a0d8c368377 Ancestors: Files-mt.202 Cheaper implementation of FileDirectory class>>extensionFor:. Include error code in StandardFileStream>>primClose: (latest VMs answer a meaningful error if primitiveFileClose fails). =============== Diff against Files-mt.202 =============== Item was changed: ----- Method: FileDirectory class>>extensionFor: (in category 'name utilities') ----- + extensionFor: fileNameString + "Answer the extension of the given file name string, if any, or the empty string if none." - extensionFor: fileName - "Return the extension of given file name, if any." + | i | + i := fileNameString lastIndexOf: DirectoryClass extensionDelimiter. + ^i = 0 + ifTrue: [''] + ifFalse: [fileNameString copyFrom: i + 1 to: fileNameString size]! - | delim i | - delim := DirectoryClass extensionDelimiter. - i := fileName findLast: [:c | c = delim]. - i = 0 - ifTrue: [^ ''] - ifFalse: [^ fileName copyFrom: i + 1 to: fileName size]. - ! Item was changed: ----- Method: StandardFileStream>>primClose: (in category 'primitives') ----- primClose: id "Close this file." + <primitive: 'primitiveFileClose' module: 'FilePlugin' error: ec> - <primitive: 'primitiveFileClose' module: 'FilePlugin'> self primitiveFailed !

1 0

Monticello Update from the world menu bar
by Eliot Miranda 02 May '23

02 May '23

Hi Marcel, Hi All, minor chaos ensues if one uses the menu update when a commit dialogue for a package covered by the update has been opened but not completed. I guess the update should check for commits in progress and refuse to update until the commit has been closed. The issue is that there is an in-progress versioned working copy that doesn't inherit from the top of the update but from the tip of the imag''s pre-update version. So afterwards one has a damaged history. When I tripped over this today it didn't upload the broken verison to the repository so all I had to do was quit the image, delete the version in my package cache, repeat the update, and then repeat the commit. But it would have been nicer if the update had refused to go ahead while the uncommitted versioned working copy existed. _,,,^..^,,,_ best, Eliot

2 1