<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto">Hi Christoph, Hi All,<div dir="ltr"><br><blockquote type="cite">On Mar 17, 2020, at 3:51 PM, Thiede, Christoph <Christoph.Thiede@student.hpi.uni-potsdam.de> wrote:<br><br></blockquote></div><blockquote type="cite"><div dir="ltr"><div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<p>Hi all! :-)</p>
<p>After some recent fun with the Unicode class, I found out that its data is quite out of date (for example, the comments do not even "know" that code points can be longer than 4 bytes. Younger characters such as 😺❤🤓 are not categorized correctly, etc. ...).
Luckily, there is already some logic to fetch the latest data from <a href="http://www.unicode.org" class="OWAAutoLink" id="LPlnk135036" previewremoved="true">www.unicode.org</a>. I'm currently reworking this logic because it's not completely automated yet
and has some slips, but so long, I have one general question for you:</p></div></div></blockquote><div><br></div>And consequently I have a couple of questions for you. In the Spur VM Characters are immediate (they are like SmallInteger and exist in oops (object-oriented pointers) as tagged values). In the 32-bit variant Characters are 30-bit unsigned integers. In the 64-bit variant they are also 30-bit unsigned integers, but could easily be extended to be up to 61-bit unsigned integers.<div><br></div><div>Q1, can you arrange that the Unicode support does not break in initialization on the 32-bit variant? It may be that the 32-bit variant cannot represent code points beyond 30 bits in size, but we should try to ensure that initialization still runs to completion even if it fails to initialize information relating to code points beyond 30 bits in size.</div><div><br></div><div>Q2, how many bits should the 64-bit variant VM support for immediate Characters?</div><div><br></div><div>Then something to consider is that it is conceptually possible to support something like WideCharacter, which would represent code points outside of the immediate Character range on the 32-bit variant, analogous to LargePositiveInteger beyond SmallInteger maxVal. This can be made to work seamlessly, just as it does currently with integers, and with Floats where SmallFloat64 is only used on 64-bits.</div><div><br></div><div>It has implications in a few parts of the system:</div><div>- failure code for WideString (VeryWideString?) at:[put:] primitives that would have to manage overflow into/access from WideCharacter instances</div><div>- ImageSegment and other (un)pickling systems that need to convert to/from a bit-specific “wire” protocol/representation</div><div>- 32-bit <=> 64-bit image conversion </div><div><br></div><div>All this is easily doable (because we have models of doing it for Float and Integer general instances). But we need good specifications so we can implement the right thing from the get-go.</div><div><br><blockquote type="cite"><div dir="ltr"><div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<p>At the moment, we have 30 class variables each for one Unicode category number. These class vars map in alphabetical order to the integers from 0 to: 29. Is this tedious structure really necessary? For different purposes, I would like to get the category
name of a specific code point from a client. The current design makes this impossible without writing additional mappings.</p>
<p>Tl;dr: I would like to propose to drop these class variables and use Symbols instead. They are comparable like integers, and as they are flyweights, this should not be a performance issue either. Of course, #<span>generalCategoryOf: will have to keep returning
numbers, but we could deprecate it and use a new #generalTagOf: in the future. Furthermore, this would also allow us to deal with later added category names (though I don't know whether this will ever happen).</span></p>
<div id="Signature">
<div id="divtagdefaultwrapper" dir="ltr" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;">
<div name="divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:; margin:0">
<div>
<div class="_rp_T4" id="Item.MessagePartBody">
<div class="_rp_U4 ms-font-weight-regular ms-font-color-neutralDark rpHighlightAllClass rpHighlightBodyClass" id="Item.MessageUniqueBody" style="font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont">
<div dir="ltr">
<div id="divtagdefaultwrapper"><font face="Calibri,Helvetica,sans-serif,EmojiFont,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe UI Symbol,Android Emoji,EmojiSymbols">
<div id="Signature">
<div style="margin:0px"><font style="font-family:Calibri,Arial,Helvetica,sans-serif,serif,EmojiFont"></font></div>
</div>
</font></div>
</div>
</div>
</div>
<div class="_rp_T4" id="Item.MessagePartBody"><br>
</div>
<div class="_rp_T4" id="Item.MessagePartBody">Examples:</div>
</div>
</div>
</div>
</div>
<blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;">
<div>
<div dir="ltr" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;">
<div name="divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:; margin:0">
<div>
<div class="_rp_T4" id="Item.MessagePartBody"><span>Unicode generalTagOf: $a asUnicode. "#Ll"</span></div>
</div>
</div>
</div>
</div>
<div>
<div dir="ltr" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;">
<div name="divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:; margin:0">
<div>
<div class="_rp_T4" id="Item.MessagePartBody"><span><br>
</span></div>
</div>
</div>
</div>
</div>
<div>
<div dir="ltr" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;">
<div name="divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:; margin:0">
<div>
<div class="_rp_T4" id="Item.MessagePartBody">
<div>Unicode class >> isLetterCode: charCode</div>
</div>
</div>
</div>
</div>
</div>
<div>
<div dir="ltr" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;">
<div name="divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:; margin:0">
<div>
<div class="_rp_T4" id="Item.MessagePartBody">
<div><span style="font-size: 12pt; white-space: pre;"> ^ (</span><span style="font-size: 12pt;">self generalTagOf: charCode) first = $L</span></div>
</div>
</div>
</div>
</div>
</div>
<div>
<div dir="ltr" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;">
<div name="divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:; margin:0">
<div>
<div class="_rp_T4" id="Item.MessagePartBody"><br>
</div>
</div>
</div>
</div>
</div>
<div>
<div dir="ltr" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;">
<div name="divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:; margin:0">
<div>
<div class="_rp_T4" id="Item.MessagePartBody">
<div>Unicode class >> isAlphaNumericCode: charCode</div>
</div>
</div>
</div>
</div>
</div>
<div>
<div dir="ltr" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;">
<div name="divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:; margin:0">
<div>
<div class="_rp_T4" id="Item.MessagePartBody">
<div><span style="white-space: pre;"> </span>| tag|</div>
</div>
</div>
</div>
</div>
</div>
<div>
<div dir="ltr" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;">
<div name="divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:; margin:0">
<div>
<div class="_rp_T4" id="Item.MessagePartBody">
<div><span style="white-space: pre;"> ^ (</span>tag := self generalCategoryOf: charCode<span style="font-size: 12pt;">) first = $L</span></div>
<div><span style="font-size: 12pt;"> or: [tag</span><span style="font-size: 12pt;"> = #Nd]</span></div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<div>
<div dir="ltr" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;">
<div name="divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:; margin:0">
<div>
<div class="_rp_T4" id="Item.MessagePartBody">
<div></div>
</div>
</div>
</div>
</div>
</div>
<div>
<div dir="ltr" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, EmojiFont, "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;">
<div name="divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:; margin:0">
<div>
<div class="_rp_T4" id="Item.MessagePartBody">
<div><br>
</div>
How do you think about this proposal? Please let me know and I will go ahead! :D</div>
<div class="_rp_T4" id="Item.MessagePartBody"><br>
</div>
<div class="_rp_T4" id="Item.MessagePartBody">Best,</div>
<div class="_rp_T4" id="Item.MessagePartBody">Christoph</div></div></div></div></div></div></div></blockquote><div><br></div><div>Best, Eliot<br><div dir="ltr"><span style="background-color: rgba(255, 255, 255, 0);">_,,,^..^,,,_ (phone)</span></div></div></div></body></html>