<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"><html><head><meta content="text/html;charset=UTF-8" http-equiv="Content-Type"></head><body ><div style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt;"><div><div style="font-family: Verdana, Arial, Helvetica, sans-serif;font-size: 10.0pt;"><div><div style="font-family: Verdana, Arial, Helvetica, sans-serif;font-size: 10.0pt;"><div>Hi folks,<br></div><div><br></div><div>TL;DR in XTreams-Parsing do I need to add support to account for the "combining mark" as described in this regex tutorial here: <a target="_blank" href="https://www.regular-expressions.info/unicode.html">https://www.regular-expressions.info/unicode.html</a><br></div><div><br></div></div></div></div></div><blockquote style="border: 1px solid rgb(204, 204, 204); padding: 7px; background-color: rgb(245, 245, 245);"><div><div>The Unicode code point U+0300 (grave accent) is a "combining mark"<br></div><div>Any code point that is not a combining mark can be followed by any number of combining marks.<br></div><div>This sequence, like U+0061 U+0300 above, is displayed as a single grapheme on the screen.<br></div><div><br></div></div></blockquote><div><div style="font-family: Verdana, Arial, Helvetica, sans-serif;font-size: 10.0pt;"><div><div style="font-family: Verdana, Arial, Helvetica, sans-serif;font-size: 10.0pt;"><div>per: <a href="https://www.compart.com/en/unicode/U+00E0" target="_blank">https://www.compart.com/en/unicode/U+00E0</a> “à” can be encoded several ways:<br></div><div><br></div><table style="margin: 0.0px;padding: 0.0px;border: 0.0px;outline: 0.0px;font-size: 20.0px;vertical-align: baseline;background: 0.0px 0.0px rgb(255,255,255);width: 958.0px;border-collapse: collapse;min-width: 100.0%;border-spacing: 0.0px;color: rgb(55,54,55);font-family: "Source Sans Pro", Roboto, "San Francisco", "Segoe UI", sans-serif;font-style: normal;font-weight: 300;letter-spacing: normal;orphans: 2;text-transform: none;white-space: normal;widows: 2;word-spacing: 0.0px;" class="x_-1469354899data-table"><tbody style="margin: 0.0px;padding: 0.0px;border: 0.0px;outline: 0.0px;font-size: 20.0px;vertical-align: baseline;background: 0.0px 0.0px;"><tr style="margin: 0.0px;padding: 0.0px;border: 0.0px;outline: 0.0px;font-size: 20.0px;vertical-align: baseline;background: 0.0px 0.0px;"><td style="margin: 0.0px;padding: 0.6em 1.25em;border: 0.0px;outline: 0.0px;font-size: 20.0px;vertical-align: top;background: 0.0px 0.0px;height: auto;line-height: 20.0px;font-weight: 400;width: 250.0px;" class="x_-1469354899label x_-1469354899first-column">UTF-8 Encoding:<br></td><td style="margin: 0.0px;padding: 0.6em 1.25em;border: 0.0px;outline: 0.0px;font-size: 20.0px;vertical-align: top;background: 0.0px 0.0px;height: auto;line-height: 20.0px;" class="x_-1469354899second-column"><code style="margin: 0.0px;padding: 0.0px;border: 0.0px;outline: 0.0px;font-size: 20.0px;vertical-align: baseline;background: 0.0px 0.0px;color: rgb(0,85,136);font-family: "Source Code Pro", monospace;font-weight: 300;">0xC3 0xA0</code><br></td></tr><tr style="margin: 0.0px;padding: 0.0px;border: 0.0px;outline: 0.0px;font-size: 20.0px;vertical-align: baseline;background: 0.0px 0.0px;"><td style="margin: 0.0px;padding: 0.6em 1.25em;border: 0.0px;outline: 0.0px;font-size: 20.0px;vertical-align: top;background: 0.0px 0.0px rgb(242,242,242);height: auto;line-height: 20.0px;font-weight: 400;width: 250.0px;" class="x_-1469354899label x_-1469354899first-column">UTF-16 Encoding:<br></td><td style="margin: 0.0px;padding: 0.6em 1.25em;border: 0.0px;outline: 0.0px;font-size: 20.0px;vertical-align: top;background: 0.0px 0.0px rgb(242,242,242);height: auto;line-height: 20.0px;" class="x_-1469354899second-column"><code style="margin: 0.0px;padding: 0.0px;border: 0.0px;outline: 0.0px;font-size: 20.0px;vertical-align: baseline;background: 0.0px 0.0px;color: rgb(0,85,136);font-family: "Source Code Pro", monospace;font-weight: 300;">0x00E0</code><br></td></tr><tr style="margin: 0.0px;padding: 0.0px;border: 0.0px;outline: 0.0px;font-size: 20.0px;vertical-align: baseline;background: 0.0px 0.0px;"><td style="margin: 0.0px;padding: 0.6em 1.25em;border: 0.0px;outline: 0.0px;font-size: 20.0px;vertical-align: top;background: 0.0px 0.0px;height: auto;line-height: 20.0px;font-weight: 400;width: 250.0px;" class="x_-1469354899label x_-1469354899first-column">UTF-32 Encoding:<br></td><td style="margin: 0.0px;padding: 0.6em 1.25em;border: 0.0px;outline: 0.0px;font-size: 20.0px;vertical-align: top;background: 0.0px 0.0px;height: auto;line-height: 20.0px;" class="x_-1469354899second-column"><code style="margin: 0.0px;padding: 0.0px;border: 0.0px;outline: 0.0px;font-size: 20.0px;vertical-align: baseline;background: 0.0px 0.0px;color: rgb(0,85,136);font-family: "Source Code Pro", monospace;font-weight: 300;">0x000000E0</code><br></td></tr></tbody></table><div><br></div><div><br></div><div>I assume that the sequence 0xC3 0xA0 is the combination the regex dude refers to.</div><div><br></div><div>Here are some relevant Printit (values render correctly in Squeak with unifont installed, not so much in the browser where I print them)<br></div></div></div></div></div><blockquote style="border: 1px solid rgb(204, 204, 204); padding: 7px; background-color: rgb(245, 245, 245);"><div><div><div style="font-family: Verdana, Arial, Helvetica, sans-serif;font-size: 10.0pt;"><div><div style="font-family: Verdana, Arial, Helvetica, sans-serif;font-size: 10.0pt;"><div>Character codePoint:224 <br></div></div></div></div></div><div>Character value: 16r0000E0 <br></div><div><br></div><div>Character value: 16r0061<br></div><div>Character value: 16r0300 <br></div><div>(Character value: 16r0061) asString , (Character value: 16r0300) asString<br></div><div><br></div><div>Escape: backslash character: character hexes: hexes<br></div></div><div><action: 'Escape' arguments: #( 1 2 3 )><br></div><div><br></div><div>backslash = '\' ifTrue:<br></div><div>[character = $s ifTrue: [^Character space].<br></div><div>character = $t ifTrue: [^Character tab].<br></div><div>character = $n ifTrue: [^Character cr].<br></div><div>character = $r ifTrue: [^Character lf].<br></div><div>character = $x ifTrue: [^('16r', (String withAll: hexes)) asNumber asCharacter]].<br></div><div><div>^character<br></div><div>Character value: 16r00C3<br></div><div>Character value: 16r00A0<br></div><div><div style="font-family: Verdana, Arial, Helvetica, sans-serif;font-size: 10.0pt;"><div><div style="font-family: Verdana, Arial, Helvetica, sans-serif;font-size: 10.0pt;"><div>(Character value: 16r00C1) asString , (Character value: 16r00A0) asString<br></div></div></div></div></div></div></blockquote><div><div style="font-family: Verdana, Arial, Helvetica, sans-serif;font-size: 10.0pt;"><div><div style="font-family: Verdana, Arial, Helvetica, sans-serif;font-size: 10.0pt;"><div><br></div></div></div></div></div><div>The reason I ask is that just as Character does not (nor should it?) support combining marks<br></div><div>Neither does XTreams-Parsing...from the PEG grammar and the relevant callback I have the following rules:<br></div><div><br></div><blockquote style="border: 1px solid rgb(204, 204, 204); padding: 7px; background-color: rgb(245, 245, 245);"><div><div>Escape <- BACKSLASH [x] [0-9A-F]{6} / BACKSLASH [nrts\-\\\[\]\''\"] / EscapeError<br></div><div>EscapeError <- BACKSLASH .<br></div><div><br></div></div></blockquote><div>with callback:<br></div><blockquote style="border: 1px solid rgb(204, 204, 204); padding: 7px; background-color: rgb(245, 245, 245);"><div><div>Escape: backslash character: character hexes: hexes<br></div><div><action: 'Escape' arguments: #( 1 2 3 )><br></div><div><br></div><div>backslash = '\' ifTrue:<br></div><div>[character = $s ifTrue: [^Character space].<br></div><div>character = $t ifTrue: [^Character tab].<br></div><div>character = $n ifTrue: [^Character cr].<br></div><div>character = $r ifTrue: [^Character lf].<br></div><div>character = $x ifTrue: [^('16r', (String withAll: hexes)) asNumber asCharacter]].<br></div><div>^character<br></div></div></blockquote><div>which you can see does not support capture of the pair 0x00C3 0x00A0 to return “à”<br></div><div><br></div><div>I am strongly leaning towards ignoring the pairs and assuming that all characters such as above are part of the extension.<br></div><div><br></div><div>Thoughts appreciated.<br></div><div><br></div><div>thx<br></div><div><br></div><div><br></div><div><br></div><div><br></div></div><br></body></html>