Accented Characters do not input

[christianclavet]

Hi, I had this problem since a long time. My keyboard use a key to input the accent then with another key it define the accented character.

This has never worked before in IRRlicht, but last week I tried something and would like to propose this for a patch. (Unless you think there is a better method to implement support for this)

I modified the source file named "CGUIEditBox.cpp"

and changed the method inputchar with this:

Code: Select all

void CGUIEditBox::inputChar(wchar_t c)
{
	static bool accentmod = false;
	static bool accentmod1 = false;
	static bool accentmod2 = false;
	static bool accentmod3 = false;

	if (!IsEnabled)
		return;

	if (c != 0)
	{
		if (Text.size() < Max || Max == 0)
		{
			core::stringw s;
		
			if (MarkBegin != MarkEnd)
			{
				// replace marked text
				const s32 realmbgn = MarkBegin < MarkEnd ? MarkBegin : MarkEnd;
				const s32 realmend = MarkBegin < MarkEnd ? MarkEnd : MarkBegin;

				if (c==*L"`") accentmod=true;
				if (c==168) accentmod1=true; // ¨ badly detected, quick fix 
				if (c==*L"^") accentmod2=true;
				if (c==184) accentmod3=true; // ¸ badly detected, quick fix
				s = Text.subString(0, realmbgn);
				if (!accentmod && !accentmod1 && !accentmod2 && !accentmod3)
				{
					s.append(c);
					CursorPos = realmbgn+1;
				} else CursorPos = realmbgn;
				s.append( Text.subString(realmend, Text.size()-realmend) );
				
				Text = s;
				
			}
			else
			{
				// add new character
				// Christian Clavet, check here for the accent key.
				if (accentmod) 
				{  
					if (c==*L"a")	c=*L"à";
					if (c==*L"A") 	c=*L"À";
					if (c==*L"e") 	c=*L"è";
					if (c==*L"E")	c=*L"È";
					if (c==*L"i") 	c=*L"ì";
					if (c==*L"I")	c=*L"Ì";
					if (c==*L"o") 	c=*L"ò";
					if (c==*L"O")	c=*L"Ò";
					if (c==*L"u") 	c=*L"ù";
					if (c==*L"U")	c=*L"Ù";
					accentmod=false;
	
				} else
				if (accentmod1)
				{  
					if (c==*L"a")	c=*L"ä";
					if (c==*L"A") 	c=*L"Ä";
					if (c==*L"e") 	c=*L"ë";
					if (c==*L"E")	c=*L"Ë";
					if (c==*L"i") 	c=*L"ï";
					if (c==*L"I")	c=*L"Ï";
					if (c==*L"o") 	c=*L"ö";
					if (c==*L"O")	c=*L"Ö";
					if (c==*L"u") 	c=*L"ü";
					if (c==*L"U")	c=*L"Ü";
					accentmod1=false;
										
				} else
				if (accentmod2) 
				{  
					if (c==*L"a")	c=*L"â";
					if (c==*L"A") 	c=*L"Â";
					if (c==*L"e") 	c=*L"ê";
					if (c==*L"E")	c=*L"Ê";
					if (c==*L"i") 	c=*L"î";
					if (c==*L"I")	c=*L"Î";
					if (c==*L"o") 	c=*L"ô";
					if (c==*L"O")	c=*L"Ô";
					if (c==*L"u") 	c=*L"û";
					if (c==*L"U")	c=*L"Û";
					accentmod2=false;

				} else
				if (accentmod3)
				{  
					if (c==*L"c")	c=*L"ç";
					if (c==*L"C")	c=*L"Ç";
					accentmod3=false;
				} 
				if (c==*L"`") accentmod=true;
				if (c==168) accentmod1=true; // ¨ badly detected, quick fix 
				if (c==*L"^") accentmod2=true;
				if (c==184) accentmod3=true; // ¸ badly detected, quick fix
				if (!accentmod && !accentmod1 && !accentmod2 && !accentmod3)
				{
					s = Text.subString(0, CursorPos);
					s.append(c);
					s.append( Text.subString(CursorPos, Text.size()-CursorPos) );
					Text = s;
					++CursorPos;
				}
			}

			BlinkStartTime = os::Timer::getTime();
			setTextMarkers(0, 0);
		}
	}
	breakText();
	sendGuiEvent(EGET_EDITBOX_CHANGED);
	calculateScrollPos();
}

I'm using TortoiseSVN and here is my attempt to do a patch from revision 2528:

Code: Select all

Index: CGUIEditBox.cpp
===================================================================
--- CGUIEditBox.cpp	(revision 2900)
+++ CGUIEditBox.cpp	(working copy)
@@ -1255,9 +1255,13 @@
 	return (s32)BrokenTextPositions.size() - 1;
 }
 
-
 void CGUIEditBox::inputChar(wchar_t c)
 {
+	static bool accentmod = false;
+	static bool accentmod1 = false;
+	static bool accentmod2 = false;
+	static bool accentmod3 = false;
+
 	if (!IsEnabled)
 		return;
 
@@ -1266,27 +1270,95 @@
 		if (Text.size() < Max || Max == 0)
 		{
 			core::stringw s;
-
+		
 			if (MarkBegin != MarkEnd)
 			{
 				// replace marked text
 				const s32 realmbgn = MarkBegin < MarkEnd ? MarkBegin : MarkEnd;
 				const s32 realmend = MarkBegin < MarkEnd ? MarkEnd : MarkBegin;
 
+				if (c==*L"`") accentmod=true;
+				if (c==168) accentmod1=true; // ¨ badly detected, quick fix 
+				if (c==*L"^") accentmod2=true;
+				if (c==184) accentmod3=true; // ¸ badly detected, quick fix
 				s = Text.subString(0, realmbgn);
-				s.append(c);
+				if (!accentmod && !accentmod1 && !accentmod2 && !accentmod3)
+				{
+					s.append(c);
+					CursorPos = realmbgn+1;
+				} else CursorPos = realmbgn;
 				s.append( Text.subString(realmend, Text.size()-realmend) );
+				
 				Text = s;
-				CursorPos = realmbgn+1;
+				
 			}
 			else
 			{
 				// add new character
-				s = Text.subString(0, CursorPos);
-				s.append(c);
-				s.append( Text.subString(CursorPos, Text.size()-CursorPos) );
-				Text = s;
-				++CursorPos;
+				// support for accent key method.
+				if (accentmod) 
+				{  
+					if (c==*L"a")	c=*L"à";
+					if (c==*L"A") 	c=*L"À";
+					if (c==*L"e") 	c=*L"è";
+					if (c==*L"E")	c=*L"È";
+					if (c==*L"i") 	c=*L"ì";
+					if (c==*L"I")	c=*L"Ì";
+					if (c==*L"o") 	c=*L"ò";
+					if (c==*L"O")	c=*L"Ò";
+					if (c==*L"u") 	c=*L"ù";
+					if (c==*L"U")	c=*L"Ù";
+					accentmod=false;
+	
+				} else
+				if (accentmod1)
+				{  
+					if (c==*L"a")	c=*L"ä";
+					if (c==*L"A") 	c=*L"Ä";
+					if (c==*L"e") 	c=*L"ë";
+					if (c==*L"E")	c=*L"Ë";
+					if (c==*L"i") 	c=*L"ï";
+					if (c==*L"I")	c=*L"Ï";
+					if (c==*L"o") 	c=*L"ö";
+					if (c==*L"O")	c=*L"Ö";
+					if (c==*L"u") 	c=*L"ü";
+					if (c==*L"U")	c=*L"Ü";
+					accentmod1=false;
+										
+				} else
+				if (accentmod2) 
+				{  
+					if (c==*L"a")	c=*L"â";
+					if (c==*L"A") 	c=*L"Â";
+					if (c==*L"e") 	c=*L"ê";
+					if (c==*L"E")	c=*L"Ê";
+					if (c==*L"i") 	c=*L"î";
+					if (c==*L"I")	c=*L"Î";
+					if (c==*L"o") 	c=*L"ô";
+					if (c==*L"O")	c=*L"Ô";
+					if (c==*L"u") 	c=*L"û";
+					if (c==*L"U")	c=*L"Û";
+					accentmod2=false;
+
+				} else
+				if (accentmod3)
+				{  
+					if (c==*L"c")	c=*L"ç";
+					if (c==*L"C")	c=*L"Ç";
+					accentmod3=false;
+				} 
+				if (c==*L"`") accentmod=true;
+				if (c==168) accentmod1=true; // ¨ badly detected, quick fix 
+				if (c==*L"^") accentmod2=true;
+				if (c==184) accentmod3=true; // ¸ badly detected, quick fix
+				if (!accentmod && !accentmod1 && !accentmod2 && !accentmod3)
+				{
+					s = Text.subString(0, CursorPos);
+					s.append(c);
+					s.append( Text.subString(CursorPos, Text.size()-CursorPos) );
+					Text = s;
+					++CursorPos;
+				}
 			}
 
 			BlinkStartTime = os::Timer::getTime();
@@ -1298,7 +1370,6 @@
 	calculateScrollPos();
 }
 
-
 void CGUIEditBox::calculateScrollPos()
 {
 	if (!AutoScroll)

Is there a tutorial to apply it back? I don't know how to apply to patch back to check it out.

[CuteAlien]

Thanks, for your work. We will have to patch accented character input, but this is something the OS should (and can) do. Which means it has to be done in the CIrrDevices because there is more than one way to input those accented characters - depending on your selected keyboard layout. This is a slightly bigger work probably (and might need some decision how/if we should handle Windows 98, which caused last time I tried to fix that more trouble than all other OS together).

[christianclavet]

Thanks. I'm very happy that I could propose a working solution for IRRlicht even if it's not the solution that was required. It's my first time to propose something since I understand a tiny little bit more of C++ and IRRlicht source.

Are the other OS are working with accented character input? (Linux, MACOS?)

I see the WM_KEYUP inside CirrDeviceWin32. I'll need to review my notes, because the code here is really different on what I'm used to.

I'll check here where it get it accented key. Probably this in line 231.

Code: Select all

			WORD KeyAsc=0;
			GetKeyboardState(allKeys);
			ToAscii((UINT)wParam,(UINT)lParam,allKeys,&KeyAsc,0);

Found the virtual key table on the site:

http://msdn.microsoft.com/en-us/library ... S.85).aspx
(note: the URL was hiding the message, it's plain text)

Perhaps it's the OEM keys.
Looking at the table:

On my french canadian keyboard:
VK_OEM_2 is "/ or ?" on the english keyboard and "é or É" on mine (Output is ok, since we only need to press it once)
VK_OEM_4 is "[{" on the english keyboard and "^" accent on mine. Actually print the accent.
VK_OEM_6 is "]}" on the english keyboad and "¸ or ¨" accents on mine. Actually print the accent.
VK_OEM_8 is single quote, double quote on the english keyboard but is "`" accent on my keyboard. Actually print the accent.

From this virtual table we could process the accent and then output the accented character when the next key is pressed (voyels), but first there would be a need to find the "accent" symbol output so the other keyboards won't be affected. I don't know how my current C++ skills are, but I could look into it.

[CuteAlien]

You might take a look at the solution I tried last time for international keyboard input. I don't know anymore if it handled accented characters on Windows correctly (I wrote it for cyrillic character input) and found no time for further testing so far, but maybe it works: http://irrlicht.sourceforge.net/phpBB2/ ... highlight=

There's also a Linux solution, but that one isn't OK, there is a better way on Linux which I just avoided last time because it needed more engine changes (and I wasn't in the team back then).

That stuff is all rather high on my priority, I really hope I can find some time to improve Irrlicht on that before 1.7 release. Any feedback if that stuff also helps with accented characters is very welcome :-)

[CuteAlien]

Oh wow - I just spend a few hours on this and it turns out to be a lot more tricky than I had hoped for. Accented characters are called "dead keys" on Windows and so far I found no way handling them correctly with WM_KEYDOWN messages. There are WM_DEADCHAR message, but we get them after WM_KEYDOWN. WM_CHAR would handle them (and as we call TranslateMessage already we should get that), but mixing those two isn't trivial in a lot of cases. Just using WM_CHAR also won't work (not good for games).

I have more and more the feeling that something like your solution - moved to the IrrDevice - might turn out to be better than many others. But probably it starts breaking apart once we add more languages.

Another solution would probably be separating text-input from the other input (other = using keyboard as game-input-device). Or maybe I find some ideas when going one level deeper using window hooks. Or maybe I can peek for later messages, as one of the main problems I have is that I would need the info from WM_CHAR and co. before WM_KEYDOWN.

Anyway... what I realize for now is - this will take some more time.

[Dorth]

Can't we provide WM_CHAR as well as WM_KEYDOWN, allowing people to choose which input they like? As such, in a game, I could store my key bindings with WM_KEYDOWN codes and use WM_CHAR for input.

[CuteAlien]

I think we will have to make that decission for editboxes already in the engine. And it's not just WM_CHAR but there is also WM_SYSCHAR and I've already found some article by an editor-programmer that the combinations between those two are sometimes rather unexpected. Also we have to care that the solution is done in a way that works multiplatform, so we can't just use for example WM_CHAR in editboxes and they stop working on linux.

I would prefer to find a solution that can put all key-input in one fat event instead of splitting it up. But I'm not sure so far if that is even possible with the current Window system as there is no obvious connection between WM_KEYDOWN and the other messages like WM_CHAR (we'll it is certainly always possible if we code the whole key-translations ourself - for every language....yeah). I'm not sure yet if a connection could be made with buffering or peeking at messages or if there is another way.

So in short - yes, using WM_CHAR is a way, just still hoping there is a better one.

[christianclavet]

Hi, I have'nt had the time to test with the keycodes. I'm sorry to hear that it will not work because of the deadkeys.

Is that happening only on Win32 or it's working correctly on other platforms?

We currently need this only for text input on the GUI. I don't even think I'd put this on the filerequester as It's bad practice to put accented character on filenames.

For the moment, my solution works for French, English. I wonder how it would work for spanish or other languages using accented character input.
If we need to support all language we will also need a font system that handle all of the characters too. (The current set is perfect for my need)

[CuteAlien]

We have the same problem on Linux. But I haven't found time yet to read the X11 documentation on that, so right now I don't even know how hard it will be to solve it there.

Sorry, for now I suppose you will have to live with your patch.
Maybe we're lucky and someone invests more time and we got some more ideas here.

[Dorth]

Possibly looking at how wxWidget pulls it off might help?

[CuteAlien]

Possibly looking at how wxWidget pulls it off might help?

Maybe, but then again they are not in the same situation. They don't have to care about the keyboard as an input-device for games where getting each key-press is important foremost. They can just use WM_CHAR.

[Dorth]

Umm, actually, they offer multiple way to get the key, it's basic code, it's unicode and also support mouse and joystick events (maybe other).

[CuteAlien]

This is by the way the main problem: Getting input-characters is basically completely seperate from getting keys. And at least in Windows there is no obvious connection between the two - those are different messages and so far I know of no way to combine them. You can't just say - get me the key and if it is a char get me that one also. Which would be the perfect solution. I think it is possible to do this on X11, but I'm not sure yet.

So we have a few possibilities right now:
1. Find a way to get the connection between keys and inputs so we can create one event for that. This is clearly what I would prefer, so I will try some more to find a solution for that. But it might turn out to be plain impossible (or at least without a major overhead), in which case one of the other solutions might be necessary.
2. Split input-chars from key-presses completely like Windows does it. I don't like it, but maybe we have no other choice.
3. Keep it like now, but add dead-key support in Irrlicht-code the way christianclavet started doing it. We can also do that in the devices already. Basically coding the same language-support ourself which operating-systems do otherwise. But this looks very complex to do correct, so most likely it is not really a viable solution. It would have to care about the currently set language-keyboard-handles on each system and have translation tables for each of them to work correctly and I don't think we should do that. (we could do that if we find operating-system functions which do that for us - which is basically what I would expect from a good OS API... I mean they _have_ to do that internally anyway). On the other hand - offering that for just a few common language-settings might be some workaround that would make most users already happy...

And yes - looking at other solutions makes certainly sense. Whenever I find time for that. Anyone else doing that giving me hints for a good solution is also very much appreciated :-)

[CuteAlien]

Thanks to tonic it seems we have a solution now: http://sourceforge.net/tracker/index.ph ... tid=540678

What I had missed was that the functions do actually work when the TranslateMessage is kicked out of the message-loop.

I have added a somewhat changed version of that patch now to svn. It's not yet in the 1.7 release branch because that got branched a few minutes before I added that patch *sigh*. But it's in trunk.

edit: Well, at least it's a solution for Windows. Linux is still open :-(