On 2013-02-12, at 00:27, Yoshiki Ohshima <Yoshiki.Ohshima at acm.org> wrote:

> On Sun, Feb 10, 2013 at 6:52 AM, David T. Lewis <lewis at mail.msen.com> wrote:
>> Hi Yoshiki,
>> 
>> Thank you for explaining this.
>> 
>> I am still confused on one point. Can you say if the #zapSelectionWithCompositionWith:
>> method should be changed? Here is the change that I was proposing:
>> 
>> zapSelectionWithCompositionWith: aString
>>        "Deselect, and replace the selection text by aString.
>>         Remember the resulting selectionInterval in UndoInterval and otherInterval.
>>         Do not set up for undo."
>> 
>>        | stream newString aText beforeChar |
>>        wasComposition := false.
>>        ((aString isEmpty or: [(beforeChar := self charBefore) isNil]) or: [
>>                aString size = 1 and: [(Unicode isComposition: aString first) not]]) ifTrue: [
>>                        ^ self zapSelectionWith: (Text string: aString emphasis: emphasisHere)].
>> 
>>        stream := UnicodeCompositionStream on: (String new: 16).
>>        stream nextPut: beforeChar.
>>        stream nextPutAll: aString.
>>        newString := stream contents.
>>        aText := Text string: newString emphasis: emphasisHere.
>>        wasComposition := true.
>>        self markBlock < self pointBlock
>>                ifTrue: [ self setMark: self markBlock stringIndex - 1.
>>                                self zapSelectionWith: aText.
>>                                self setMark: self markBlock stringIndex + 1]
>>                ifFalse: [ self setPoint: self pointBlock stringIndex - 1.
>>                                self zapSelectionWith: aText.
>>                                self setPoint: self pointBlock stringIndex + 1]
>> 
>> 
>> But I do not know if this is correct, or if we should leave the method
>> as it is now:
>> 
>>        ( ... )
>>        Text := Text string: newString emphasis: emphasisHere.
>>        self markBlock < self pointBlock
>>                ifTrue: [self setMark: self markBlock stringIndex - 1]
>>                ifFalse: [self setPoint: self  pointBlock stringIndex - 1].
>> 
>>        wasComposition := true.
>>        self zapSelectionWith: aText.
> 
> My memory on this is murky but I think what I wrote was not perfect.
> At one point, OLPC XO had two theories on inputting accented
> characters and it did gave us a "naked" composition character on its
> own.  In that case, that composition character has to be tested with
> the "beforeChar" and may have to be combined with it.  So, beforeChar
> may be replaced with the new string.  So, the code supported both
> ways, if IIRC.
> 
> I need to test things to see whether we need to fiddle with the off-by-one...
> 
> -- 
> -- Yoshiki

Ah yes, the XO by default doesn't use "dead keys" for accented characters (where you press the accent first, which does not produce a character, then you press the char and it makes an accented one) but instead "composes" the accent on top of the char (that is, you press the base char first, and then the accent, which gets applied to the previous char).

The way this works (I think) is that the keyboard directly creates Unicode Combining Marks which other software (like GTK) even stores as two unicode code points. E.g., when I get a title string that was entered in a different app we have to run #composeAccents to make the string display correctly in Etoys (Squeak's Unicode rendering does not deal with composite marks yet).

http://www.unicode.org/faq/char_combmark.html

- Bert -

PS: If you want to pull your hairs out, try to read about correct Unicode string comparisons, or sorting ...