<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">Hi all<div class=""><br class=""><blockquote type="cite" class="">On 02.07.2018, at 19:29, Ron Teitelbaum <<a href="mailto:ron@usmedrec.com" class="">ron@usmedrec.com</a>> wrote:<br class=""><br class="">That is strange.<br class=""><br class="">On Squeak 4.1 <br class=""><br class="">$^ charCode -> 94<br class="">94 asCharacter -> $^<br class="">128 asCharacter -> $€ charCode -> 128 (doesn't show properly in text on email but does in squeak, see image).<br class=""><br class=""><span id="cid:ii_jj4j1oin0_1645bffb52003a3f"><Capture.PNG></span><br class=""><br class="">In other words, if I use my keyboard to type it in, it seems to be represented fine and it evaluates to charCode 94 as expected.<br class=""><br class="">But something created with 128 charCode also is represented with the same symbol and it also retains it's 128 charCode as you can see with you send charCode to the string representation that was created.<br class=""><br class="">If this was filed out it would seem that either version could have been used in the code and you wouldn't notice it. Manually changing it by typing in ^ and fileing it out again will probably fix it. An external editor changing 128 to 94 chars will also probably work.<br class=""><br class="">All the best,<br class=""></blockquote><div class=""><br class=""></div><div class="">Maybe I can shed a bit light on things here. </div><div class=""><br class=""></div><div class="">If you look at the attached image (which is one of the default fonts we use), you see that ^ and _ are present after [\] but ALSO after {|}~. This seems to be intentional so that you, should you want, can switch betwen caret/underscore and up-arrow/left-arrow printing for return/assignment and here's how it's done:</div><div class=""><br class=""></div><div class="">StrikeFont>>useLeftArrow</div><span class="Apple-tab-span" style="white-space:pre"> </span>self characterToGlyphMap.<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>characterToGlyphMap at: 96 put: 95.<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>characterToGlyphMap at: 95 put: 94</div><div class=""><br class=""></div><div class="">and</div><div class=""><br class=""></div><div class="">StrikeFont>>useUnderscore</div><span class="Apple-tab-span" style="white-space:pre"> </span>self characterToGlyphMap.<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>characterToGlyphMap at: 96 put: 129.<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>characterToGlyphMap at: 95 put: 128<div class=""><br class=""></div><div class=""><br class=""></div><div class="">There's the 128.</div><div class=""><br class=""></div><div class="">What happens here, too, is that 128 is no proper character to begin with.</div><div class=""><br class=""></div><div class="">Our characters represent unicode codepoints, which, for ByteStrings, happen to _exactly_ match the ISO 8859-1 (Latin1) encoding. (In fact, that was a design decision for Unicode to begins with; does NOT hold for UTF-8 tho).</div><div class=""><br class=""></div><div class="">In both, Unicode and ISO 8859-1, certain "character codes" are not actually characters. The control characters (<32) are intentionally undefined, as are codes between 128 and 159 (includes 128). However, 8859-1 was often combined with Ansi escape codes (aka ISO 6429), which defines the codes from 128 as Control Block C1, which Unicode subsequently adopted.</div><div class=""><br class=""></div><div class="">Long story short, Characters between 128 and 159 are inherently non-printable. Either they control output or format output, but cannot in themselves be displayed. The StrikeFonts utilize that and use those code points in fonts to relocate caret, underscore, left-arrow and right-arrow so that they can serve as substitutes when you don't want ^ _ in code but rather arrows.</div><div class=""><br class=""><div class="">=-=-=-=-=</div><div class=""><br class=""></div><div class="">That being said, I just saw that the fileList forces MacRoman encoding (deprecated since MacOS X 10.0 in 2001....) which _has_ a proper meaning for 128, namely Ä. However, the respective method probably needs an overhaul:</div><div class=""><br class=""></div><div class="">FileList>>defaultEncoderFor: aFileName<br class=""><br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>"This method just illustrates the stupidest possible implementation of encoder selection."<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>| l |<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>l := aFileName asLowercase.<br class="">"<span class="Apple-tab-span" style="white-space:pre"> </span>((l endsWith: FileStream multiCs) or: [<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>l endsWith: FileStream multiSt]) ifTrue: [<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>^ UTF8TextConverter new.<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>].<br class="">"<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>((l endsWith: FileStream cs) or: [<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>l endsWith: FileStream st]) ifTrue: [<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>^ MacRomanTextConverter new.<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>].<br class=""><br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>^ Latin1TextConverter new.<br class=""><br class="">=-=-=-=-=-=</div><div class=""><br class=""></div><div class="">Indeed, the file x.cs contains an 128 at the indicated position. Which is in the middle of a binary SmartRefStream-dump. Maybe we must change the fileIn logic to make the stream binary when it encounters a smartrefstream? that would certainly help. </div><div class=""><br class=""></div><div class="">Best regards</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>-Tobias</div><div class=""><br class=""></div><div class=""><br class=""><div class=""><br class=""></div><img apple-inline="yes" id="A14259D6-31F7-48A7-A05E-9CA3D72C8664" height="10" width="996" apple-width="yes" apple-height="yes" src="cid:26DD849D-5E02-4CF0-A417-1316E4E0FA22@fritz.box" class=""><br class=""><blockquote type="cite" class=""><br class="">Ron Teitelbaum<br class=""><br class=""><br class="">On Mon, Jul 2, 2018 at 12:23 PM, Eliot Miranda <<a href="mailto:eliot.miranda@gmail.com" class="">eliot.miranda@gmail.com</a>> wrote:<br class=""> <br class="">Hi Subbu,<br class=""><br class="">> On Jul 2, 2018, at 7:24 AM, K K Subbu <kksubbu.ml@gmail.com> wrote:<br class="">> <br class="">> Hi,<br class="">> <br class="">> I need help in tracing a bug (see attached picture) which triggered a MNU while trying to view a .cs file in FileTool. I traced the problem to peek on aStream returning a nil because a wrong character code was being returned in generated.<br class="">> <br class="">> In the attached picture, aStream isBinary is false and the basicNext returns the correct $^ character which gets stored in character1 local var. But an inspector displays it as Character 128. In the same inspector window $^ shows the correct character code as 94.<br class="">> <br class="">> This is on Squeak5.2alpha-64b-Linux-18127. What is happening here?<br class=""><br class="">No idea. Do you have a test case?<br class=""><br class="">> Has anyone seen this type of behavior before?<br class="">> <br class="">> <br class="">> Thanks in advance .. Subbu<br class="">> <strangeCharBug.png><br class=""><br class=""></blockquote><br class=""></div></div></body></html>