On 2013-08-07, at 04:54, tim Rowledge <tim at rowledge.org> wrote:

> After more peering at dark corners of code, some possible light glimmers appear. I hope they're not an oncoming train…
> 
> So far as I can work out, Squeak converts incoming characters from the keyboard and clipboard according to the locale setting returned from the locale plugin (or goes for 'en' if that goes wrong). For now I'll assume that having a locale that needs non-ascii characters will result in at least some strings being WideString objects and that they will get displayed nicely, according to the fonts installed. So we have either latin-1 ascii BytesStrings or unicode WideStrings. Correct so far?

Yes, although we do not usually have fonts installed that provide more than Latin-1. The mechanism is there but largely unused (there is only a handful of WideString instances in our image). Installing wide fonts isn't exactly trivial either, IIRC for TTFs all glyphs outside Latin-1 get stripped unless you know which line in which method to change.

> Currently Scratch expects the translation dictionary files it uses to have utf8 encoded strings for the not-english translations, so a utf8 class of some sort will be required in order to stuff them and translate to WideString.

This is one of the system/squeak boundaries. The PO files are utf8 encoded, you should change that reader to return proper WideStrings (just by a strategically placed utf8ToSqueak send).

> Similarly the project file format expects either Strings (as in the old Squeak latin-1 ascii) or UTF8 strings, so a conversion either way is still needed. I think that utf8 can be restricted to only existing as an intermediary buffer object.

Sounds about right.

> I don't suppose the code currently in Squeak handles right-to-left languages? 


It does not, which is why I suggested to keep using Scratch's text rendering.

- Bert -