Christoph Thiede uploaded a new version of Tools to project The Trunk:
http://source.squeak.org/trunk/Tools-ct.1119.mcz
==================== Summary ====================
Name: Tools-ct.1119
Author: ct
Time: 29 January 2022, 6:52:15.107695 pm
UUID: 8d4822ff-359b-d245-b692-d3dbd9326d8c
Ancestors: Tools-mt.1118
Fixes the do-it receiver in plain code holders and eliminates duplication.
=============== Diff against Tools-mt.1118 ===============
Item was removed:
- ----- Method: Browser>>doItReceiver (in category 'accessing') -----
- doItReceiver
- "This class's classPool has been jimmied to be the classPool of the class
- being browsed. A doIt in the code pane will let the user see the value of
- the class variables."
-
- ^ self selectedClass!
Item was added:
+ ----- Method: CodeHolder>>doItReceiver (in category 'accessing') -----
+ doItReceiver
+ "If there is an instance associated with me, answer it, for true mapping of self. If not, then do what other code-bearing tools do, viz. give access to the class vars."
+
+ ^ self selectedClass!
Item was removed:
- ----- Method: MessageSet>>doItReceiver (in category 'accessing') -----
- doItReceiver
- ^ self selectedClass!
Item was removed:
- ----- Method: MethodHolder>>doItReceiver (in category 'menu') -----
- doItReceiver
- "If there is an instance associated with me, answer it, for true mapping of self. If not, then do what other code-bearing tools do, viz. give access to the class vars."
- ^ self selectedClass!
Marcel Taeumel uploaded a new version of Tools to project The Trunk:
http://source.squeak.org/trunk/Tools-mt.1118.mcz
==================== Summary ====================
Name: Tools-mt.1118
Author: mt
Time: 29 January 2022, 12:26:50.298306 pm
UUID: 60dc6fe1-c374-f941-9ea2-51a514ffe136
Ancestors: Tools-mt.1117
Adds a simple way to browse method sources and class comments that either benefit from UTF8 or suffer from Squeak's #leadingChar encoding, i.e., code-point > 255.
non-ASCII method sources: 62
non-ASCII class comments: 7
leading-char method sources: 6
leading-char class comments: 0
See method commentary for more information.
=============== Diff against Tools-mt.1117 ===============
Item was added:
+ ----- Method: Unicode class>>browseClassCommentsWithLeadingCharEncoding (in category '*Tools-Browsing') -----
+ browseClassCommentsWithLeadingCharEncoding
+ "See commentary in browseMethodsWithLeadingCharEncoding."
+
+ ^ self systemNavigation
+ browseMessageList: (Array streamContents: [:s |
+ self systemNavigation
+ allClassesDo: [:cls |
+ (cls comment asString anySatisfy: [:each | each codePoint > 255])
+ ifTrue: [s nextPut: (MethodReference class: cls selector: #Comment)] ]])
+ name: 'Class comments affected by #leadingChar'!
Item was added:
+ ----- Method: Unicode class>>browseClassCommentsWithNonAsciiEncoding (in category '*Tools-Browsing') -----
+ browseClassCommentsWithNonAsciiEncoding
+ "See commentary in browseMethodsWithNonAsciiEncoding."
+
+ ^ self systemNavigation
+ browseMessageList: (Array streamContents: [:s |
+ self systemNavigation
+ allClassesDo: [:cls | cls comment asString isAsciiString
+ ifFalse: [s nextPut: (MethodReference class: cls selector: #Comment)] ]])
+ name: 'Class comments with non-ASCII contents'!
Item was added:
+ ----- Method: Unicode class>>browseMethodsWithLeadingCharEncoding (in category '*Tools-Browsing') -----
+ browseMethodsWithLeadingCharEncoding
+ "Browse a list of methods whose sources are affected by Squeak's #leadingChar magic, which can confuse tools such as TextDiff depending on your current encoding (i.e., EncodedCharSet or LanguageEnvironment). NOTE THAT if you want to change those methods, ensure that your current encoding uses a leadingChar of 0, e.g., locale en-US and Latin1Environment. See UTF8TextConverter class >> #decodeByteString: and Unicode class >> #value:."
+
+ ^ self systemNavigation
+ browseMessageList: (self systemNavigation
+ allMethodsSelect: [:method | method getSource asString
+ anySatisfy: [:each | each codePoint > 255]])
+ name: 'Methods sources affected by #leadingChar'!
Item was added:
+ ----- Method: Unicode class>>browseMethodsWithNonAsciiEncoding (in category '*Tools-Browsing') -----
+ browseMethodsWithNonAsciiEncoding
+ "Browse a list of methods whose sources benefit from UTF8 encoding."
+
+ ^ self systemNavigation
+ browseMessageList: (self systemNavigation
+ allMethodsSelect: [:method | method getSource asString isAsciiString not])
+ name: 'Methods with non-ASCII sources'!
Marcel Taeumel uploaded a new version of Monticello to project The Trunk:
http://source.squeak.org/trunk/Monticello-mt.770.mcz
==================== Summary ====================
Name: Monticello-mt.770
Author: mt
Time: 29 January 2022, 11:01:45.317024 am
UUID: 5c68eef4-9384-ee45-b945-07e9b835c188
Ancestors: Monticello-mt.769
Complements Multilingual-mt.261
=============== Diff against Monticello-mt.769 ===============
Item was changed:
----- Method: MCMczReader>>contentsForMember: (in category 'private') -----
contentsForMember: member
^[(member contentStreamFromEncoding: 'utf8') text contents] on: InvalidUTF8
do: [:exc |
"Case of legacy encoding, presumably it is latin-1.
But if contents starts with a null character, it might be a case of WideString encoded in UTF-32BE"
| str |
+ self flag: #discuss. "mt: Isn't our mac-roman legacy more likely than latin-1?"
str := (member contentStreamFromEncoding: 'latin1') text.
exc return: ((str peek = Character null and: [ str size \\ 4 = 0 ])
ifTrue: [WideString fromByteArray: str contents asByteArray]
ifFalse: [str contents])]!
Marcel Taeumel uploaded a new version of Multilingual to project The Trunk:
http://source.squeak.org/trunk/Multilingual-mt.261.mcz
==================== Summary ====================
Name: Multilingual-mt.261
Author: mt
Time: 29 January 2022, 10:59:27.832024 am
UUID: c87a7c62-86fe-204d-b0cf-12e891df09e2
Ancestors: Multilingual-mt.260
Fixes file-in/out (.cs .st) regression. Always use UTF8 but allow for trying MacRoman on InvalidUTF8 error. Still write out that UTF8 BOM to make new file-outs again compatible with older images' file-in code.
Since 2018, we are filing-out UTF8 only - assuming that nobody played around with Locale and LanguageEnvironment because the defaults for MultiByteFileStream are derived from there. So, let's hope that "TextConverter defaultSystemConverter" answered an UTF8 converter for all (or most) file-outs since 2018. Same goes for our .sources and .changes file, which now default to UTF8 explicitely, not by accident. Regardless of any Locale's LanguageEnvironment's #systemConverterClass, which is actually useful for non-code text-file access.
Note that if there are any .cs or .st files out there with an encoding other than MacRoman or UTF8, there are ways to re-encode them before filing them in. Yet, I don't think that it's an actual issue at this point. Maybe Latin1, but this was never used as a file-out encoding.
This commit is a coherent part of a slightly larger Locale cleanup. It's almost ready. Then we can update those translations for the next release. :-)
=============== Diff against Multilingual-mt.260 ===============
Item was changed:
----- Method: MultiByteBinaryOrTextStream>>fileOutClass:andObject: (in category 'fileIn/Out') -----
fileOutClass: extraClass andObject: theObject
+
+ self nextPutUTF8BOM.
- UTF8TextConverter writeBOMOn: self.
^ super fileOutClass: extraClass andObject: theObject!
Item was added:
+ ----- Method: MultiByteBinaryOrTextStream>>nextPutUTF8BOM (in category 'fileIn/Out') -----
+ nextPutUTF8BOM
+ "Backwards compatibility. BOM was used in older file-outs to switch from mac-roman to utf-8 on file-in. Nowadays we always expect utf-8 on file-in and fall back only on InvalidUTF8 exception."
+
+ self binary.
+ UTF8TextConverter writeBOMOn: self.
+ self text.!
Item was changed:
----- Method: MultiByteBinaryOrTextStream>>setConverterForCode (in category 'fileIn/Out') -----
setConverterForCode
+ "Always use and expect UTF-8 encoding for source code so that we can share those files. Any existing BOM should be skipped automatically when decoding content. See #writeBOMOn: and #decodeByteString:."
+ self converter: UTF8TextConverter new.!
- | current |
- current := converter saveStateOf: self.
- self position: 0.
- self binary.
- ((self next: 3) = #[ 16rEF 16rBB 16rBF ]) ifTrue: [
- self converter: UTF8TextConverter new
- ] ifFalse: [
- self converter: MacRomanTextConverter new.
- ].
- converter restoreStateOf: self with: current.
- self text.
- !
Item was added:
+ ----- Method: MultiByteBinaryOrTextStream>>setConverterForOldCode (in category 'fileIn/Out') -----
+ setConverterForOldCode
+
+ self converter: MacRomanTextConverter new.!
Item was changed:
----- Method: MultiByteBinaryOrTextStream>>setEncoderForSourceCodeNamed: (in category 'fileIn/Out') -----
setEncoderForSourceCodeNamed: streamName
+ self deprecated.
+ self setConverterForCode.!
- | l |
- l := streamName asLowercase.
- " ((l endsWith: FileStream multiCs) or: [
- (l endsWith: FileStream multiSt) or: [
- (l endsWith: (FileStream multiSt, '.gz')) or: [
- (l endsWith: (FileStream multiCs, '.gz'))]]]) ifTrue: [
- self converter: UTF8TextConverter new.
- ^ self.
- ].
- "
- ((l endsWith: FileStream cs) or: [
- (l endsWith: FileStream st) or: [
- (l endsWith: (FileStream st, '.gz')) or: [
- (l endsWith: (FileStream cs, '.gz'))]]]) ifTrue: [
- self converter: MacRomanTextConverter new.
- ^ self.
- ].
-
- self converter: UTF8TextConverter new.
- !
Item was changed:
----- Method: MultiByteFileStream>>fileOutClass:andObject: (in category 'fileIn/Out') -----
fileOutClass: extraClass andObject: theObject
+
+ self nextPutUTF8BOM.
- self binary.
- UTF8TextConverter writeBOMOn: self.
- self text.
^ super fileOutClass: extraClass andObject: theObject!
Item was added:
+ ----- Method: MultiByteFileStream>>isSourceFile (in category 'private') -----
+ isSourceFile
+
+ self flag: #InvalidUTF8. "Ignore source files. Those MUST BE valid utf-8. Broken chunks cannot be fixed by changing converters in the middle of file reading."
+ (SourceFiles at: 1) ifNotNil: [self fullName = (SourceFiles at: 1) fullName ifTrue: [^ true]].
+ (SourceFiles at: 2) ifNotNil: [self fullName = (SourceFiles at: 2) fullName ifTrue: [^ true]].
+
+ ^ false!
Item was added:
+ ----- Method: MultiByteFileStream>>nextPutUTF8BOM (in category 'fileIn/Out') -----
+ nextPutUTF8BOM
+ "Backwards compatibility. BOM was used in older file-outs to switch from mac-roman to utf-8 on file-in. Nowadays we always expect utf-8 on file-in and fall back only on InvalidUTF8 exception."
+
+ self binary.
+ UTF8TextConverter writeBOMOn: self.
+ self text.!
Item was changed:
+ ----- Method: MultiByteFileStream>>setConverterForCode (in category 'fileIn/Out') -----
- ----- Method: MultiByteFileStream>>setConverterForCode (in category 'private') -----
setConverterForCode
+ "Always use and expect UTF-8 encoding for source code so that we can share those files. Any existing BOM should be skipped automatically when decoding content. See #writeBOMOn: and #decodeByteString:."
+
+ self converter: UTF8TextConverter new.!
-
- | currentPosition |
- (SourceFiles at: 2)
- ifNotNil: [self fullName = (SourceFiles at: 2) fullName ifTrue: [^ self]].
- currentPosition := self position.
- self position: 0.
- self binary.
- ((self next: 3) = #[ 16rEF 16rBB 16rBF ]) ifTrue: [
- self converter: UTF8TextConverter new
- ] ifFalse: [
- self converter: MacRomanTextConverter new.
- ].
- self position: currentPosition.
- self text.
- !
Item was added:
+ ----- Method: MultiByteFileStream>>setConverterForOldCode (in category 'fileIn/Out') -----
+ setConverterForOldCode
+
+ self isSourceFile ifTrue: [^ self].
+ self converter: MacRomanTextConverter new.!
Chris Muller uploaded a new version of Collections to project The Trunk:
http://source.squeak.org/trunk/Collections-cmm.981.mcz
==================== Summary ====================
Name: Collections-cmm.981
Author: cmm
Time: 28 January 2022, 7:28:17.843333 pm
UUID: f7d931d8-11ea-4e5e-bb7d-6e4f5b46f4b8
Ancestors: Collections-ct.980
Better API compatibility between file-based and non file-based streams.
=============== Diff against Collections-ct.980 ===============
Item was added:
+ ----- Method: Stream>>ensureOpen (in category 'file open/close') -----
+ ensureOpen
+ "API compatibility with FileBased streams."!
Item was added:
+ ----- Method: Stream>>open (in category 'file open/close') -----
+ open
+ "API compatibility with FileBased streaams."
+ ^ self!
Item was changed:
+ ----- Method: Stream>>openReadOnly (in category 'file open/close') -----
- ----- Method: Stream>>openReadOnly (in category 'accessing') -----
openReadOnly
^self!