Well I think the unicode value has been in there in the mac vm since march of 2002.

Andreas suggested

Proposal: utf32Code in sqKeyboardEvent Date: March 18, 2005 11:41:12 PM PST (CA)

Folks,

Since I'm not sure all of you have followed the discussion about scroll events (which then went into keyboard events) here's my proposal again: I would like to add a utf32Code field to sqKeyboardEvent which serves the purpose to provide a UTF-32 value if the VM provides extended input services (the Mac VM already does; others will surely follow). The utf32Code field only needs to be filled in for EventKeyChar events (e.g., neither for EventKeyDown or EventKeyUp since those are used for reporting keys not characters).

John is already playing with this but I want to make sure there is general agreement on this issue. Ian, Tim? Any extra comments?

Cheers,

• Andreas

I could be wrong but after this discussion utf32Code was never used by the Windows VM, well other than setting it to zero?

However in working with Sophie we found some shortcoming, and I had suggest this change (below) but was told Windows doesn't support it.

Subject: sq.h changes, grabbed reserved field in sqKeyboardEvent Date: March 29, 2006 1:59:49 PM PST (CA)

In our work with Sophie we became aware of a shortcoming in how keyboard events are created and what data they contain and what data is needed to transform keyboard command on various obscure European keyboards into meaningful interactions within Tweak and other important applications like Sophie.

In the past on the macintosh and we believe on Windows the data coming up as a result of a keystroke was

key down charCode: a magic number mapping to a key on the keyboard, different between mac/windows pressCode: 0

key char charCode the macroman value of the key pressed pressCode the utf-32 value of the key pressed

key up charCode: a magic number mapping to a key on the keyboard, different between mac/windows pressCode: 0

The suggestion was:

Now the results will be.

key down charCode: a magic number mapping to a key on the keyboard, different between mac/windows pressCode the utf-32 value of the key pressed additionalData the macroman value of the key pressed

key char charCode the macroman value of the key pressed pressCode the utf-32 value of the key pressed additionalData a magic number mapping to a key on the keyboard, different between mac/windows

key up charCode: a magic number mapping to a key on the keyboard, different between mac/windows pressCode the utf-32 value of the key pressed additionalData the macroman value of the key pressed

This preserves the existing behaviour, but adds the ability to better support non-usa keyboards and character sets. I will make this change btw weekend baring a more elegant solution.

/* keyboard input event */ typedef struct sqKeyboardEvent { int type; /* EventTypeKeyboard */ unsigned int timeStamp; /* time stamp */ int charCode; /* character code in Mac Roman encoding */ int pressCode; /* press code; any of EventKeyXXX */ int modifiers; /* combination of xxxKeyBit */ int utf32Code; /* UTF-32 unicode value */ int additionCharacterData; /* Additional data about charcter for platform specific tweak dispatching */ int windowIndex; /* host window structure */ } sqKeyboardEvent;

On May 15, 2007, at 12:09 PM, Bert Freudenberg wrote:

On May 15, 2007, at 20:51 , Chris Petsos wrote:

...

We are trying to make Squeak work with Unicode...Yoshiki Ohshima has been very helpful till now and his suggestion was to let all the data that the platform (e.g. Windows, Unix etc...) is sending, enter the Squeak VM as is and deal with them there (in the image side) with InputInterpreter classes.

Yeah, I know, for some strange historical reason the Japanese folks like this approach ;)

I much prefer the very simple solution that John put into the Mac VM - add a char32 field to each KeyStroke event containing the unicode value. Nice and simple.

...

Another clue is that Squeak in Windows uses ANSI versions of Windows functions which support only 8-bit characters

That's highly irrelevant to the Unix VM. And for filenames we'd most likely should use UTF-8 which is 8-bit and actually supported by the Unix VM right now.

• Bert -

-- ======================================================================== === John M. McIntosh johnmci@smalltalkconsulting.com Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ======================================================================== ===

Hello,

...

We are trying to make Squeak work with Unicode...Yoshiki Ohshima has been very helpful till now and his suggestion was to let all the data that the platform (e.g. Windows, Unix etc...) is sending, enter the Squeak VM as is and deal with them there (in the image side) with InputInterpreter classes.

Yeah, I know, for some strange historical reason the Japanese folks like this approach ;)

I much prefer the very simple solution that John put into the Mac VM

• add a char32 field to each KeyStroke event containing the unicode

value. Nice and simple.

If we can replace all VMs, that would do. Of course, the image can check the VM version so that they can install proper "interpreter" (including non-interpreting one for later VMs.)

It should be better to use pre-composed chars for these chars, but I'm not entirely sure if it makes sense for all different input methods.

-- Yoshiki

On May 21, 2007, at 21:47 , Chris Petsos wrote:

...

Squeak's TTF importer discards code points > 255. Here is a hack around this:

http://lists.squeakfoundation.org/pipermail/squeak-dev/2006-June/ 105184.html

When i load the package i get this Syntax Error... "Postscript:" self inform: 'If keyboard input stops working now, please disable the 'unicodeInput'Nothing more expected -> preference'

Oops, that looks like a typo ... should have been "unicodeInput".

And the keyboard input stops indeed...

It's probably best if you hand-pick the TT-related changes. The changeset also hacks the event handling which might be incompatible with what you did.

- Bert -

On Wednesday 16 May 2007 12:39 am, Bert Freudenberg wrote:

On May 15, 2007, at 20:51 , Chris Petsos wrote:

...

We are trying to make Squeak work with Unicode...Yoshiki Ohshima has been very helpful till now and his suggestion was to let all the data that the platform (e.g. Windows, Unix etc...) is sending, enter the Squeak VM as is and deal with them there (in the image side) with InputInterpreter classes.

Yeah, I know, for some strange historical reason the Japanese folks like this approach ;)

This woud be a preferred approach for Indic languages too. The ability to switch layouts through hotkeys is crucial because of multiplicity of languages (over 23!). I could start a paragraph in Kannada layout, switch to US layout for English and arabic numerals, use Devanagari for Hindi and then revert to Kannada, all in a single sentence.

Currently, lack of Indic support in Squeak is a showstopper for public schools in my area.

Regards .. Subbu

Hi Chris -

Chris Petsos wrote:

Follow these to make your VM (in Windows) unicode capable... +/* Force Unicode WM_CHAR */ +SetWindowLongW(stWindow,GWL_WNDPROC,GetWindowLong(stWindow,GWL_WNDPROC)); +SetWindowLongW(consoleWindow,GWL_WNDPROC,GetWindowLong(consoleWindow,GWL_WN DPROC));

Have you tried handling WM_UNICHAR instead of tricking the VM into Unicode mode? If you do what you did above some other messages come in with Unicode args too and while this may be desirable at some point I'm not sure if we want this right away. For the time being I'd feel safer if we would handle WM_UNICHAR explicitly. Also, WM_UNICHAR uses (according to the docs) UTF-32 instead of UTF-16 as in your code. This might come in handy at some point.

What do you say we start maintaining a Unicode version of the Squeak VM for Windows and Unix...

I'm happy to fold the fixes into the mainstream version.

Cheers, - Andreas

On Mon, 2007-05-21 at 23:32 -0700, Andreas Raab wrote:

Have you tried handling WM_UNICHAR instead of tricking the VM into Unicode mode? If you do what you did above some other messages come in with Unicode args too and while this may be desirable at some point I'm not sure if we want this right away. For the time being I'd feel safer if we would handle WM_UNICHAR explicitly. Also, WM_UNICHAR uses (according to the docs) UTF-32 instead of UTF-16 as in your code. This might come in handy at some point.

OK...i've been trying to use the WM_UNICHAR solution but i haven't managed to achieve it yet... Does the window have to be registered to receive Unicode with

+SetWindowLongW(stWindow,GWL_WNDPROC,GetWindowLong(stWindow,GWL_WNDPROC)); +SetWindowLongW(consoleWindow,GWL_WNDPROC,GetWindowLong(consoleWindow,GWL_WNDPROC));

again in this case? ... i suppose not. What i did is to add the following to the relevant switch statement

switch(msg->message) { case WM_KEYDOWN: case WM_SYSKEYDOWN: if(virtCode) keyCode = virtCode; ................................. case WM_UNICHAR: // HERE printf("Oops! Unicode char!\n"); break; case WM_CHAR: case WM_SYSCHAR: /* Note: VK_RETURN is recorded as virtual key ONLY */ if(keyCode == 13) return 1; pressCode = EventKeyChar; break;

Also, i defined WM_UNICHAR with

#ifndef WM_UNICHAR #define WM_UNICHAR 0x109 #endif

I also defined those as a workaround while googling

#define UNICODE #define _UNICODE

But, printf("Oops! Unicode char!\n");

is never invoked...

Has anyone implemented this solution??? Some guidelines would be most welcome...

Sorry if this is a simple C-related thing... i'm not a very good C programmer...but i want to become one! :-)

Thanks in advance...

Christos.