<div id="__MailbirdStyleContent" style="font-size: 10pt;font-family: Arial;color: #000000;text-align: left" dir="ltr">
Hmm... I wonder whether those diff issues in Monticello tools originate from an inconsistency between an .mcz's snapshot.bin and snapshot.st. The .st is explicitely written in utf-8. The .bin relies on whatever that DataStream-on-RWBinaryOrTextStream does ... if contents are written binary, there is no chance that the text converter can remove that #leadingChar from the characters in, e.g., class comments or method sources. I think.<div><br><div class="mb_sig"></div>
<div>Jakob? Maybe this one is for you? =D</div><div><br></div><div>MCPatchOperation >> #sourceText</div><div>MCPatchOperation >> #fromSource<br></div><div>MCPatchOperation >> #toSource<br></div><div>MCMczWriter >> #writeSnapshot:</div><div><br></div><div>A simple test is to revert the comment of ThirtyTwoBitRegister to "ul 4/13/2015" and then use the "history" of the "System" package to compare System-mt.1294 against <working copy>. Given that you have your locale set to 'ja', your #languageEnvironment will have #leadingChar 5 and then you see this:</div><div><br></div><div><img id="e693576e-ef34-4056-b367-280dc384fccd" src="cid:c4a6ab45-68c2-4416-a72a-7e3fea74fd53" width="283" height="234"></img><br></div><div><br></div><div>Interestingly, #utf8ToSqueak in both #fromSource and #toSource solved this issue here. So, I am wondering whether the #leadingChar > 0 does only reveal the actual issue, which is an utf-8 encoding not being yet decoded to Squeak. However, one would assume that the snapshot.bin has actual Squeak instances of MCClassDefinition etc. in it, right? Hmm....</div><div><br></div><div>Best,</div><div>Marcel</div></div><blockquote class="history_container" type="cite" style="border-left-style: solid;border-width: 1px;margin-top: 20px;margin-left: 0px;padding-left: 10px;min-width: 500px">
<p style="color: #AAAAAA; margin-top: 10px;">Am 29.01.2022 12:27:05 schrieb commits@source.squeak.org <commits@source.squeak.org>:</p><div style="font-family:Arial,Helvetica,sans-serif">Marcel Taeumel uploaded a new version of Tools to project The Trunk:<br>http://source.squeak.org/trunk/Tools-mt.1118.mcz<br><br>==================== Summary ====================<br><br>Name: Tools-mt.1118<br>Author: mt<br>Time: 29 January 2022, 12:26:50.298306 pm<br>UUID: 60dc6fe1-c374-f941-9ea2-51a514ffe136<br>Ancestors: Tools-mt.1117<br><br>Adds a simple way to browse method sources and class comments that either benefit from UTF8 or suffer from Squeak's #leadingChar encoding, i.e., code-point > 255.<br><br>non-ASCII method sources: 62<br>non-ASCII class comments: 7<br>leading-char method sources: 6<br>leading-char class comments: 0<br><br>See method commentary for more information.<br><br>=============== Diff against Tools-mt.1117 ===============<br><br>Item was added:<br>+ ----- Method: Unicode class>>browseClassCommentsWithLeadingCharEncoding (in category '*Tools-Browsing') -----<br>+ browseClassCommentsWithLeadingCharEncoding<br>+ "See commentary in browseMethodsWithLeadingCharEncoding."<br>+ <br>+ ^ self systemNavigation <br>+ browseMessageList: (Array streamContents: [:s |<br>+ self systemNavigation<br>+ allClassesDo: [:cls |<br>+ (cls comment asString anySatisfy: [:each | each codePoint > 255])<br>+ ifTrue: [s nextPut: (MethodReference class: cls selector: #Comment)] ]])<br>+ name: 'Class comments affected by #leadingChar'!<br><br>Item was added:<br>+ ----- Method: Unicode class>>browseClassCommentsWithNonAsciiEncoding (in category '*Tools-Browsing') -----<br>+ browseClassCommentsWithNonAsciiEncoding<br>+ "See commentary in browseMethodsWithNonAsciiEncoding."<br>+ <br>+ ^ self systemNavigation <br>+ browseMessageList: (Array streamContents: [:s |<br>+ self systemNavigation<br>+ allClassesDo: [:cls | cls comment asString isAsciiString<br>+ ifFalse: [s nextPut: (MethodReference class: cls selector: #Comment)] ]])<br>+ name: 'Class comments with non-ASCII contents'!<br><br>Item was added:<br>+ ----- Method: Unicode class>>browseMethodsWithLeadingCharEncoding (in category '*Tools-Browsing') -----<br>+ browseMethodsWithLeadingCharEncoding<br>+ "Browse a list of methods whose sources are affected by Squeak's #leadingChar magic, which can confuse tools such as TextDiff depending on your current encoding (i.e., EncodedCharSet or LanguageEnvironment). NOTE THAT if you want to change those methods, ensure that your current encoding uses a leadingChar of 0, e.g., locale en-US and Latin1Environment. See UTF8TextConverter class >> #decodeByteString: and Unicode class >> #value:."<br>+ <br>+ ^ self systemNavigation <br>+ browseMessageList: (self systemNavigation<br>+ allMethodsSelect: [:method | method getSource asString<br>+ anySatisfy: [:each | each codePoint > 255]])<br>+ name: 'Methods sources affected by #leadingChar'!<br><br>Item was added:<br>+ ----- Method: Unicode class>>browseMethodsWithNonAsciiEncoding (in category '*Tools-Browsing') -----<br>+ browseMethodsWithNonAsciiEncoding<br>+ "Browse a list of methods whose sources benefit from UTF8 encoding."<br>+ <br>+ ^ self systemNavigation <br>+ browseMessageList: (self systemNavigation<br>+ allMethodsSelect: [:method | method getSource asString isAsciiString not])<br>+ name: 'Methods with non-ASCII sources'!<br><br><br></div></blockquote></div>