In our work with Sophie we became aware of a shortcoming in how keyboard events are created and what data they contain and what data is needed to transform keyboard command on various obscure European keyboards into meaningful interactions within Tweak and other important applications like Sophie.
In the past on the macintosh and we believe on Windows the data coming up as a result of a keystroke was
key down charCode: a magic number mapping to a key on the keyboard, different between mac/windows pressCode: 0
key char charCode the macroman value of the key pressed pressCode the utf-32 value of the key pressed
key up charCode: a magic number mapping to a key on the keyboard, different between mac/windows pressCode: 0
Now the results will be.
key down charCode: a magic number mapping to a key on the keyboard, different between mac/windows pressCode the utf-32 value of the key pressed additionalData the macroman value of the key pressed
key char charCode the macroman value of the key pressed pressCode the utf-32 value of the key pressed additionalData a magic number mapping to a key on the keyboard, different between mac/windows
key up charCode: a magic number mapping to a key on the keyboard, different between mac/windows pressCode the utf-32 value of the key pressed additionalData the macroman value of the key pressed
This preserves the existing behaviour, but adds the ability to better support non-usa keyboards and character sets. I will make this change btw weekend baring a more elegant solution.
/* keyboard input event */ typedef struct sqKeyboardEvent { int type; /* EventTypeKeyboard */ unsigned int timeStamp; /* time stamp */ int charCode; /* character code in Mac Roman encoding */ int pressCode; /* press code; any of EventKeyXXX */ int modifiers; /* combination of xxxKeyBit */ int utf32Code; /* UTF-32 unicode value */ int additionCharacterData; /* Additional data about charcter for platform specific tweak dispatching */ int windowIndex; /* host window structure */ } sqKeyboardEvent;
-- ======================================================================== === John M. McIntosh johnmci@smalltalkconsulting.com 1-800-477-2659 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ======================================================================== ===
John -
John M McIntosh wrote:
In our work with Sophie we became aware of a shortcoming in how keyboard events are created and what data they contain and what data is needed to transform keyboard command on various obscure European keyboards into meaningful interactions within Tweak and other important applications like Sophie.
What exactly is the shortcoming that you are referring to?
In the past on the macintosh and we believe on Windows the data coming up as a result of a keystroke was
key down charCode: a magic number mapping to a key on the keyboard, different between mac/windows pressCode the utf-32 value of the key pressed additionalData the macroman value of the key pressed
This won't work at all on Windows. There is no way for me to generate either a utf32 or a macroman value for "the key pressed". All that's reported is a virtual key code which is arbitrarily assigned by Windows, e.g., one of the VK_XXX constants.
key char charCode the macroman value of the key pressed pressCode the utf-32 value of the key pressed additionalData a magic number mapping to a key on the keyboard, different between mac/windows
And this won't work either on Windows, in particular not for international keyboards. When keys are composed we get sequences like "accent key down, accent key up, character key down, (composed) character input, character key up". This happens on the Mac as well, for example try pressing option-E first (you get keydown/keyup) and then e (which gives you accent-e). So there is not necessarily a correspondence between a key and its ascii value.
This preserves the existing behaviour, but adds the ability to better support non-usa keyboards and character sets.
Could you please first state the problem before proposing a solution ;-)
Cheers, - Andreas
Andreas Raab wrote:
Could you please first state the problem before proposing a solution ;-)
Andreas Raab wrote:
77 is VK_M which Microsoft describes as "The M key" (see for example http://msdn.microsoft.com/library/default.asp?url=/library/en-us/wceddk/html... there is another -longer- list of virtual key codes for Windows but I can't find it right now).
OK, that's the information I needed :-) I'm not quite sure about the consequences for some of the weirder keyboard layouts though:
From a user's point of view (what the user thinks he pressed) ctrl-shift-VK_1 on "normal" keyboards would mean ctrl-!, on French ones it would mean ctrl-1 .
Michael
Michael Rueger wrote:
I'm not quite sure about the consequences for some of the weirder keyboard layouts though:
From a user's point of view (what the user thinks he pressed) ctrl-shift-VK_1 on "normal" keyboards would mean ctrl-!, on French ones it would mean ctrl-1 .
Ah, keyboard layouts. What a pain. Can we please (please, please, please!) not require VMs to deal with those? It's a pain and it's a mess and I'd rather have some image code deal with it. So, instead of extending keyboard events, how about simply reporting a "keyboard layout identifier"[*] that the image can use to map keys reported by the VM into whatever it wants to?
[*] On Windows, that would be an identifier from GetKeyboardLayoutName (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winui/winui...)
Cheers, - Andreas
So why can't you use ToUnicode() to translate the virtual key code or the scan key code into the unicode character?
Sure I can return 42 the magic number from LMGetKbdType() meaning you are using as Estonia keyboard, or maybe 78 for Devanagari keyboard in the Indian Language Kit, however it's unlikely I or any other developer will want to code the keyboard to ascii/macroman/unicode whatever translation tables in the image.
On 29-Mar-06, at 4:16 PM, Andreas Raab wrote:
Michael Rueger wrote:
I'm not quite sure about the consequences for some of the weirder keyboard layouts though:
From a user's point of view (what the user thinks he pressed) ctrl-shift-VK_1 on "normal" keyboards would mean ctrl-!, on French ones it would mean ctrl-1 .
Ah, keyboard layouts. What a pain. Can we please (please, please, please!) not require VMs to deal with those? It's a pain and it's a mess and I'd rather have some image code deal with it. So, instead of extending keyboard events, how about simply reporting a "keyboard layout identifier"[*] that the image can use to map keys reported by the VM into whatever it wants to?
[*] On Windows, that would be an identifier from GetKeyboardLayoutName (http://msdn.microsoft.com/library/ default.asp?url=/library/en-us/winui/winui/windowsuserinterface/ userinput/keyboardinput/keyboardinputreference/ keyboardinputfunctions/getkeyboardlayoutname.asp)
Cheers,
- Andreas
-- ======================================================================== === John M. McIntosh johnmci@smalltalkconsulting.com 1-800-477-2659 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ======================================================================== ===
Hi John -
So why can't you use ToUnicode() to translate the virtual key code or the scan key code into the unicode character?
I wish it were that simple but it wouldn't work - not for the problem statement that Mike posted (that's why I was asking for it).
The problem statement itself (which is perfectly valid from a user perspective) is actually remarkably ill-defined from a system perspective: It asks the key mapper to take three keys as input (Ctrl-Shift-1) and generate 2 keys output (Ctrl-! or somesuch). The reason this is ill-defined is that that's not how key mappers work (at least the one's I know about) - all they care about is the combination of all keys. Which means that ToUnicode will happily respond to Ctrl-Shift-1 saying "oh, sorry, there is no Unicode character for that combination".
And, IIRC (but I need to verify this) ToUnicode is the precise manual equivalent of what Windows does anyway when it translates WM_KEYDOWN into WM_CHAR. Which means that all you need is to look at what characters get generated already to find out how ToUnicode would respond. Which in the case of our Ctrl-Shift-1 example is not at all (I tested this with various keyboard layouts which supports the theory).
So for the plain Unicode mapping I can use ToUnicode() - in fact I am using it already (via WM_KEYDOWN -> WM_CHAR translation) and report it in the utf32 field for the key stroke event. But I *can't* use ToUnicode() to solve the problem we were discussing - to map the input while making arbitrary distinctions between Ctrl and Shift in the Windows key mapper (not unless we start masking out ctrl state which immediately leads to other issues - that is exactly what I mean by "it's a pain").
If you want such a distinction (and that's why the problem statement is so important) you need to do it manually to begin with. And I'd rather have that code in the image.
BTW, just out of interest: Does the Mac make a distinction between certain modifiers when it maps keyboard input? I'd be quite surprised if you'd get "Ctrl-!" out of mapping Ctrl-Shift-1 as input (but what do I know ;-)
Sure I can return 42 the magic number from LMGetKbdType() meaning you are using as Estonia keyboard, or maybe 78 for Devanagari keyboard in the Indian Language Kit, however it's unlikely I or any other developer will want to code the keyboard to ascii/macroman/unicode whatever translation tables in the image.
That's one good thing about Open Source: It only takes one programmer to fix that problem and if he's out there the Internet can find him.
Cheers, - Andreas
John,
Sure I can return 42 the magic number from LMGetKbdType() meaning you are using as Estonia keyboard, or maybe 78 for Devanagari keyboard in the Indian Language Kit, however it's unlikely I or any other developer will want to code the keyboard to ascii/macroman/unicode whatever translation tables in the image.
And, why is that? That is how it works right now in Squeak 3.8.
If the VM is too clever but not take the Asian language input method into account, I'm afraid that there will be no easy way to get the functionality back. On current Windows and some older versions of Mac VM, people can input Japanese, Korean, Chinese, etc. into Squeakland (Squeak 3.8 based). But the last patched version of Unix VM (3.6-3 or such), we needed to disable all the conversions implemented in the VM after all.
-- Yoshiki
So what is the problem with the current version of the Mac VM not allowing people to input Japanese, Korean, Chinese?
On 30-Mar-06, at 4:22 AM, Yoshiki Ohshima wrote:
John,
Sure I can return 42 the magic number from LMGetKbdType() meaning you are using as Estonia keyboard, or maybe 78 for Devanagari keyboard in the Indian Language Kit, however it's unlikely I or any other developer will want to code the keyboard to ascii/macroman/unicode whatever translation tables in the image.
And, why is that? That is how it works right now in Squeak 3.8.
If the VM is too clever but not take the Asian language input method into account, I'm afraid that there will be no easy way to get the functionality back. On current Windows and some older versions of Mac VM, people can input Japanese, Korean, Chinese, etc. into Squeakland (Squeak 3.8 based). But the last patched version of Unix VM (3.6-3 or such), we needed to disable all the conversions implemented in the VM after all.
-- Yoshiki
-- ======================================================================== === John M. McIntosh johnmci@smalltalkconsulting.com 1-800-477-2659 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ======================================================================== ===
Oh, ya, I was following the unix vm for clues about how to implement the key down/up/char logic and noticed it was broken.
I'll note that Tetsuya Hayashi tetha@st.rim.or.jp supplied patches and testing to confirm the mac carbon vm worked correctly for Japanese input back in april of 2004.
Also note all this work was re-factored on the conversion to unix file names and I have some concerns that I might have made a mistake or two since I'm unable to understand how to fully test to confirm everything works as expected, beyond careful reading.
On 1-Apr-06, at 6:24 PM, Yoshiki Ohshima wrote:
John,
So what is the problem with the current version of the Mac VM not allowing people to input Japanese, Korean, Chinese?
It turned out that Squeak 3.8.11beta5U allows it (great!). The problem was the Unix VM...
-- Yoshiki
-- ======================================================================== === John M. McIntosh johnmci@smalltalkconsulting.com 1-800-477-2659 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ======================================================================== ===
Here is another run at the fence:
John M McIntosh wrote:
Sure I can return 42 the magic number from LMGetKbdType() meaning you are using as Estonia keyboard, or maybe 78 for Devanagari keyboard in the Indian Language Kit, however it's unlikely I or any other developer will want to code the keyboard to ascii/macroman/unicode whatever translation tables in the image.
What we could do for trying to solve the problem Mike described, is to expose the functionality of ToUnicode or equivalent. E.g., instead of having the VM doing all the mapping, allow the image request information about "what would the result of mapping this key combination be" and basically use the information we get out of the keyboard events (meta state and key code). That way, if you wanted to map Ctrl-Shift-1 to Ctrl-! you'd filter out the Ctrl key, map Shift-1 to "!" and are done.
The issue here is that I've got no idea how hard it is to support this on other platforms...
Cheers, - Andreas
k, so on the key up/down can we pass up the unicode value? At least on the mac at that point I know the unicode values in relationship to what the keyboard interaction was. On 31-Mar-06, at 12:15 PM, Andreas Raab wrote:
Here is another run at the fence:
John M McIntosh wrote:
Sure I can return 42 the magic number from LMGetKbdType() meaning you are using as Estonia keyboard, or maybe 78 for Devanagari keyboard in the Indian Language Kit, however it's unlikely I or any other developer will want to code the keyboard to ascii/macroman/unicode whatever translation tables in the image.
What we could do for trying to solve the problem Mike described, is to expose the functionality of ToUnicode or equivalent. E.g., instead of having the VM doing all the mapping, allow the image request information about "what would the result of mapping this key combination be" and basically use the information we get out of the keyboard events (meta state and key code). That way, if you wanted to map Ctrl-Shift-1 to Ctrl-! you'd filter out the Ctrl key, map Shift-1 to "!" and are done.
The issue here is that I've got no idea how hard it is to support this on other platforms...
Cheers,
- Andreas
-- ======================================================================== === John M. McIntosh johnmci@smalltalkconsulting.com 1-800-477-2659 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ======================================================================== ===
John M McIntosh wrote:
k, so on the key up/down can we pass up the unicode value? At least on the mac at that point I know the unicode values in relationship to what the keyboard interaction was.
Well ... to be honest, I'd rather not do that unless we have a good reason to do it. Mostly, because people might get used to the idea to grab a utf char out of a key down event where in reality that mapping is inappropriate (unless you know precisely what you're doing and where). I agree that it can be helpful to know the utf value (if one exist) but I'm not even sure we can do this easily cross-platform (you still haven't answered my question about Ctrl-Shift-1 on Macs from an earlier email).
So if you could explain to me just once more which problem you're trying to solve with sticking the utf char into these event's I'd greatly appreciate it. I might appear daft here but I really don't want us to go down a slippery path that'll lead to a system which works in 98% of all cases but where the important ones are in the remaining 2%.
Cheers, - Andreas
On 31-Mar-06, at 12:15 PM, Andreas Raab wrote:
Here is another run at the fence:
John M McIntosh wrote:
Sure I can return 42 the magic number from LMGetKbdType() meaning you are using as Estonia keyboard, or maybe 78 for Devanagari keyboard in the Indian Language Kit, however it's unlikely I or any other developer will want to code the keyboard to ascii/macroman/unicode whatever translation tables in the image.
What we could do for trying to solve the problem Mike described, is to expose the functionality of ToUnicode or equivalent. E.g., instead of having the VM doing all the mapping, allow the image request information about "what would the result of mapping this key combination be" and basically use the information we get out of the keyboard events (meta state and key code). That way, if you wanted to map Ctrl-Shift-1 to Ctrl-! you'd filter out the Ctrl key, map Shift-1 to "!" and are done.
The issue here is that I've got no idea how hard it is to support this on other platforms...
Cheers,
- Andreas
--
John M. McIntosh johnmci@smalltalkconsulting.com 1-800-477-2659 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================
Before discussing, let alone debating, mechanism perhaps we could agree on policy and expected outcome?
i.e. What should be the end result of whatever vm and image code is involved?
Once that is decided we can make sensible progress on the rest. If it takes each vm doing something different and one or more platform specific classes handling key code translation in some case and not in others, who cares? The end result for the user is all that matters to anyone but us chickens. *We* just need to worry if we can understand the mechanism in a year or twos time when some changes are needed.
tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Semolina - a system of signalling with pudding.
Hi Tim -
Before discussing, let alone debating, mechanism perhaps we could agree on policy and expected outcome?
i.e. What should be the end result of whatever vm and image code is involved?
I think you're asking the same question that I asked, namely for the problem statement.
Cheers, - Andreas
Once that is decided we can make sensible progress on the rest. If it takes each vm doing something different and one or more platform specific classes handling key code translation in some case and not in others, who cares? The end result for the user is all that matters to anyone but us chickens. *We* just need to worry if we can understand the mechanism in a year or twos time when some changes are needed.
tim
tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Semolina - a system of signalling with pudding.
This was the original problem statement
77 is VK_M which Microsoft describes as "The M key" (see for example http://msdn.microsoft.com/library/default.asp?url=/library/en-us/ wceddk/html/wceddkVirtual_Key_Codes.asp there is another -longer- list of virtual key codes for Windows but I can't find it right now).
OK, that's the information I needed :-) I'm not quite sure about the consequences for some of the weirder keyboard layouts though: From a user's point of view (what the user thinks he pressed) ctrl-shift-VK_1 on "normal" keyboards would mean ctrl-!, on French ones it would mean ctrl-1 . Michael
On 31-Mar-06, at 2:42 PM, Andreas Raab wrote:
Hi Tim -
Before discussing, let alone debating, mechanism perhaps we could agree on policy and expected outcome? i.e. What should be the end result of whatever vm and image code is involved?
I think you're asking the same question that I asked, namely for the problem statement.
Cheers,
- Andreas
Once that is decided we can make sensible progress on the rest. If it takes each vm doing something different and one or more platform specific classes handling key code translation in some case and not in others, who cares? The end result for the user is all that matters to anyone but us chickens. *We* just need to worry if we can understand the mechanism in a year or twos time when some changes are needed. tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Semolina - a system of signalling with pudding.
-- ======================================================================== === John M. McIntosh johnmci@smalltalkconsulting.com 1-800-477-2659 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ======================================================================== ===
On 31-Mar-06, at 2:42 PM, Andreas Raab wrote:
I think you're asking the same question that I asked, namely for the problem statement.
Not really; what I would like is a clear statement of what is wanted at the end. Well, maybe. Some people define a problem statement as a complaint of what isn't right, I suppose "It does this wrong and I want it to do that instead" is a reasonable stretch.
After whatever processing the vm and some optional image code does is complete, what information should be passed up to the portable side of the event system? Under what conditions? What parts are nice but optional and what are utterly crucial?
Pressing 'm' pretty much works now. I think. Exactly what events *must* go to the portable event system for this?
What if you're on a japanese keyboard and have to do ctl-shift-alt- meta-RMS- and then type 247 before releasing any?
What if your machine has a totally raw keyboard and tells you "key 7 was pushed. key 42 was pushed. key 42 was released. key 17 was pushed. key 7 was released. key 17 was released"
What about an OS that tells you "character 'M' is here", ie you would have to infer the shift key being pressed from the character being a capital M.
Those are all plausible inputs to the best of my knowledge. What I'd like to see is an agreement about what the eventual outputs should be.
tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Klingon Code Warrior:- 10) "This code is a piece of crap! You have no honor!"
vm-dev@lists.squeakfoundation.org
-
Andreas Raab -
John M McIntosh -
Michael Rueger -
tim Rowledge -
Yoshiki Ohshima