MultiByte Language support pack for Irrlicht

[zgock]

hi.

I made some patches Irrlicht supports Muitybyte Language,
Justlike Japanese,Korean,Chinese etc.

http://www.zgock-lab.net/irrlicht/IrrTT.zip

supported features.
1)support Display Multibyte Charactor with Truetype fonts.
(fixed some bugs my Truetype extension and marged in Irrlicht)
(using Freetype2 library)
2)support "ImputMethod" in Entering Charactors to Editbox.
(current Win32 only. Linux not yet(in development...).)
3)Support Multibyte filenames in CGUIFileOpenDialog.
4)Support Multibyte clipboard data

i hope niko interested that and marge into Irrlicht 0.10(for all asians..)

/////////////////////////////////////////////////////////////////////////
Irrlicht a gogo:
http://www.zgock-lab.net/irrlicht/
Japanese Tutorial page for Irrlicht

[hybrid]

Cool
I'm also working on the Linux Input methods. I've finished the step from currently used KeySym-character mapping to using XLookupString similar to IrrlichtNX. But I'm still struggling with XwcLookupString which would support i18n.
I will try your code and prepare a complete patch later on. Maybe you could tell about the current state of your Linux development?

[niko]

Cool, thanks for posting. I'll definately take a look at it and try to add it into the next version.

[hybrid]

Ok, I've added a first patch to fix some issues with keyboard input using Linux. Works good for me. It's not using the Unicode version of the XLookupString method - I did not get it to work yet
I wll try to continue to work on it, though. There are some references I still have to check. Will also try to merge with the Windows variant from this thread.

[zgock]

update IrrTT.zip
http://www.zgock-lab.net/irrlicht/IrrTT.zip

1)include missing file(CGUIEnvironment.h)
2)fixed mipmap probrem in DirectX

>hybrid
my Linux development is 10%ish. your work should be more progress.
i feel best solution is i spent my power to Win32, and marge em finally.

here is example XIM on Xlib.
that is using XmbLookupString. but almost close to XwcLookupString.
i hope that helps you.

http://home.catv.ne.jp/pp/ginoue/im/xim-e.html

[hybrid]

Thanks zgock! I will check that in detail, but seems to be very useful. Good examples for pure X11 apps are becoming rare.

[_Dum0nde]

I'm really interested by this patch and I'm wondering if you were going to update to 0.10? Or else, how could I get it to work (the demo doesn't work now)? Do I need to recompile Irrlicht after modifying its source?

Thank you very much zgock!

[Phan Thai Trung]

Patch for 0.12:

I made a simple patch version just in order to show (unicode/asia) truetype font in IrrLicht 0.12.0. Files you have to modify are:

1/ irrlicht.h
2/ IGUIEnvironment.h
3/ CGUIEnvironment.h
4/ CGUIEnvironment.cpp

Please follow these steps:

Code: Select all

0/ COPY CGuiTTFont.* TO IrrLicht source directory.
1/ IN irrlicht.h, INSERT THIS LINE:
   #include "../CGUITTFont.h" // Truetype Font
2/ 
FIND IN IGUIEnvironment.h:
	virtual IGUIFont* getFont(const c8* filename) = 0;
INSERT AFTER THAT:
	//! returns the Truetype font
	virtual IGUIFont* getFont(const c8* filename,u32 fontsize) = 0; // Truetype Font

3/ INSERT THIS LINE IN IGUIEnvironment.h:
#include "CGUITTFont.h" // Truetype Font

FIND IN THAT FILE:
	//! returns the font
	virtual IGUIFont* getFont(const c8* filename);
INSERT AFTER THAT:
	//! returns the Truetype font
	virtual IGUIFont* getFont(const c8* filename,u32 fontsize); // Truetype Font

FIND IN THAT FILE:
	struct SFont
	{
		core::stringc Filename;
		IGUIFont* Font;

		bool operator < (const SFont& other) const
		{
			return (Filename < other.Filename);
		}
	};

INSERT:

	// Truetype Face
	struct STTFace
	{
		core::stringc Filename;
		CGUITTFace* Face;

		bool operator < (const STTFace& other) const
		{
			return (Filename < other.Filename);
		}
	};

	// Truetype Font
	struct STTFont
	{
		core::stringc Filename;
		u32 size;
		CGUITTFont* Font;

		bool operator < (const STTFont& other) const
		{
			if (Filename != other.Filename){
				return (Filename < other.Filename);
			} else {
				return (size < other.size);
			}
		}
	};

	core::array<STTFace> Faces; // Truetype Face
	core::array<STTFont> TTFonts; // Truetype Font

4/ FIND IN CGUIEnvironment.cpp:
#include "CGUIFont.h"

INSERT:
#include "CGUITTFont.h" // Truetype Font

FIND FUNCTION 
IGUIFont* CGUIEnvironment::getFont(const c8* filename)

INSERT AFTER THAT FUNCTION:

//! returns the Truetype font
IGUIFont* CGUIEnvironment::getFont(const c8* filename,u32 fontsize)
{
	// search existing font

	STTFace f;
	STTFont tf;

	if (!filename)
		filename = "";

	f.Filename = filename;
	f.Filename.make_lower();


	s32 index = Faces.binary_search(f);
	if (index == -1){
		f.Face = new CGUITTFace();
		if (f.Face->load(f.Filename.c_str())){
			Faces.push_back(f);
			index = Faces.binary_search(f);
		} else {
			f.Face->drop();
			return	0;
		}
	}
	STTFace *face = &Faces[index];

	tf.Filename = face->Filename;
	tf.size = fontsize;
	index = TTFonts.binary_search(tf);
	if (index != -1){
		return TTFonts[index].Font;
	}

    // not existing yet. try to load font.

	CGUITTFont* font = new CGUITTFont(Driver);
	if (!font->attach(face->Face,fontsize))
	{
		font->drop();
		return 0;
	}

	// add to fonts.

	tf.Font = font;
	TTFonts.push_back(tf);

	return font;
}

Follow instruction of zgock to build Irrlicht with freetype lib. Note that if you are using freetype-2.1.10 and Visual C++:
If you got this error while linking:
freetype2110_d.lib(ftinit.obj) : error LNK2001: unresolved external
symbol _otv_module_class
Then comment this line FT_USE_MODULE(otv_module_class) in the ftmodule.h file to solve problem.

[nasedo]

Good work~ Very useful.

[tumitoto7]

I managed to have a clean compile and build the irrlicht.DLL as Phan Thai Trung suggested (unresolved external issue was indeed, happened). Anyway, after I built the DLL, I try to recompile what zgock did on his example JPtester but I am somehow getting compiler errors:

c:\download\irrFonts\IrrTT\JPtester\main.cpp(35): error C2501: 'font' : missing storage-class or type specifiers
c:\download\irrFonts\IrrTT\JPtester\main.cpp(35): error C2501: 'font2' : missing storage-class or type specifiers
c:\download\irrFonts\IrrTT\JPtester\main.cpp(191): error C2059: syntax error : ')'
c:\download\irrFonts\IrrTT\JPtester\main.cpp(192): error C2059: syntax error : ')'
c:\download\irrFonts\IrrTT\JPtester\main.cpp(195): error C2059: syntax error : ')'
c:\download\irrFonts\IrrTT\JPtester\main.cpp(196): error C2059: syntax error : ')'
c:\download\irrFonts\IrrTT\JPtester\main.cpp(199): error C2059: syntax error : ')'
c:\download\irrFonts\IrrTT\JPtester\main.cpp(200): error C2059: syntax error : ')'
c:\download\irrFonts\IrrTT\JPtester\main.cpp(205): error C2664: 'irr::gui::IGUISkin::setFont' : cannot convert parameter 1 from 'int *' to 'irr::gui::IGUIFont *'
Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or function-style cast
c:\download\irrFonts\IrrTT\JPtester\main.cpp(206): error C2227: left of '->AntiAlias' must point to class/struct/union
type is 'int *'
c:\download\irrFonts\IrrTT\JPtester\main.cpp(209): error C2227: left of '->AntiAlias' must point to class/struct/union
type is 'int *'
c:\download\irrFonts\IrrTT\JPtester\main.cpp(220): error C2227: left of '->AntiAlias' must point to class/struct/union
type is 'int *'
c:\download\irrFonts\IrrTT\JPtester\main.cpp(221): error C2227: left of '->TransParency' must point to class/struct/union
type is 'int *'
c:\download\irrFonts\IrrTT\JPtester\main.cpp(243): error C2227: left of '->draw' must point to class/struct/union
type is 'int *'
c:\download\irrFonts\IrrTT\JPtester\main.cpp(245): error C2227: left of '->draw' must point to class/struct/union
type is 'int *'

It looks like certain data types are no longer compatible anymore. Does anyone able to come up with a demo with this patch from Phan Thai Trung that you can cut/paste Asian languages on an edit box? Thanks in advance.

[tumitoto7]

Ok, I found out why I was getting compiler error on the previous post, I missed to update the irrlicht.h to include the CGUITTFont.h. After fixing that. I am now getting a linker error (from Jptester came with the irrTT.zip):

JPtester error LNK2019: unresolved external symbol "__declspec(dllimport) class irr::IrrlichtDevice * __cdecl irr::createDevice(enum irr::video::E_DRIVER_TYPE,class irr::core::dimension2d<int> const &,unsigned int,bool,bool,bool,class irr::IEventReceiver *,unsigned short const *)" (__imp_?createDevice@irr@@YAPAVIrrlichtDevice@1@W4E_DRIVER_TYPE@video@1@ABV?$dimension2d@H@core@1@I_N22PAVIEventReceiver@1@PBG@Z) referenced in function _main

I tried to replace the CIrrDeviceWin32.cpp with the one provided by Zgock (into irrlicht version 0.12 from irrTT.zip) but it will create an compiler error:

c:\download\irrlicht-0.12.0\irrlicht-0.12.0\source\source\Irrlicht\CIrrDeviceWin32.cpp(319): error C2511: 'irr::CIrrDeviceWin32::CIrrDeviceWin32(irr::video::E_DRIVER_TYPE,const irr::core::dimension2d<T> &,irr::u32,bool,bool,bool,irr::IEventReceiver *,const wchar_t *)' : overloaded member function not found in 'irr::CIrrDeviceWin32'
with
[
T=irr::s32
]

Any help will be appreciated!! Niko, do you plan to add the multi-language support in 0.13? ohhhh...I need it!! thanks!!!

[solehome]

my way to use it in irrlicht-1.1:
1) download and build freetype-2.2.1
2) copy CGUITTFont.cpp and CGUITTFont.h to "..\irrlicht-1.1\source"
3) compare all the other .cpp and .h files in IrrTT with their correspondences in irrlicht-1.1 by using a file-compare tool(I use Beyond.Compare). Update the correspondence files in irrlicht-1.1 to match irrTT. Pay attention to the "font" concerned codes.
4) add CGUITTFont.cpp and CGUITTFont.h to your compiler IDE(I use VS 2005). add reference to freetype's .h and .lib files.
5) build irrlicht. if encounter error, please check the step 3).
6) try the JPtester example in irrTT.
7) JPtester runs ok in my machine, but throws an error when exits in debug mode. I check it, found the reason:
--- Error Reason ----
in "bool CGUITTFont::attach(CGUITTFace *Face,u32 size)"
the member array "Glyphs" of CGUITTFont is allocated a continuous memeory buffer by using "reallocate" method of array. The elements of Glyphs are not initialized sufficiently, e.g., the pointers "tex", "tex16" and "image" are not set to NULL and the debug name is not set to "CGUITTGlyph". Furthermore, because class "CGUITTGlyph" has virtual funcions, so the virtual function pointers(__vfptr) of each element in Glyphs should be initialized. Without the constructor of CGUITTGlyph, we have to set the __vfptr manually. My solution is shown as below:
--- My Naive Solution---

Code: Select all

...
void CGUITTGlyph::init()
{
	#ifdef _DEBUG
	setDebugName("CGUITTGlyph");
	#endif

	tex = NULL;
	tex16 = NULL;
	image = NULL;
}
...
bool CGUITTFont::attach(CGUITTFace *Face,u32 size)
{
	if (!Driver)
		return false;

	if (attached){
		tt_face->drop();
	}
	tt_face = Face;
	tt_face->grab();

	this->size = size;

                
	CGUITTGlyph tmp;  // for access the __vfptr of CGUITTGlyph 
	int*tmp2;              

	Glyphs.reallocate(tt_face->face->num_glyphs);
	Glyphs.set_used(tt_face->face->num_glyphs);
	for (int i = 0;i < tt_face->face->num_glyphs;i++){
		Glyphs[i].Driver = Driver;
		Glyphs[i].size = size;
		Glyphs[i].face = &(tt_face->face);
		Glyphs[i].cached = false;
		tmp2 = (int*)&(Glyphs[i]); 
		*tmp2 = *(int*)(&tmp);// the __vfptr is arranged in the front of the memory block by VC++
		Glyphs[i].init(); // initialize the other memebers

//		Glyphs[i].cache((wchar_t)i + 1);
	}
	attached = true;
	return	true;
}
...

NOTE:
i) You have to comment out "fonts->drop();" in the end of JPtester to make it work.
ii) I only test it in VS 2005, winxp sp2. maybe it only works in windows platform.
iii) There are definitely much room to optimize.
iv) If the solution does not work for you, please wait for the officially new version which must be cleaner and correct.
--- Hope ----
Hope the admin of Irrlicht add officially surpport to Unicode Text input and render.
Localized font will make a 3D engine cute and kind to many users around the world and make the users express themselves freely.

[irrlichtmania]

Yes,Yes,Yes UNICODE, you are 100% right.
There is a lot of people that are waiting for this ... Germans, Spanish, Italians, Portuguese and so on ....
This is very important to the engine. Because this is essensial to get quality projects.

[nhsimon]

my way to use it in irrlicht-1.1:
1) download and build freetype-2.2.1
2) copy CGUITTFont.cpp and CGUITTFont.h to "..\irrlicht-1.1\source"
3) compare all the other .cpp and .h files in IrrTT with their correspondences in irrlicht-1.1 by using a file-compare tool(I use Beyond.Compare). Update the correspondence files in irrlicht-1.1 to match irrTT. Pay attention to the "font" concerned codes.
4) add CGUITTFont.cpp and CGUITTFont.h to your compiler IDE(I use VS 2005). add reference to freetype's .h and .lib files.
5) build irrlicht. if encounter error, please check the step 3).
6) try the JPtester example in irrTT.
7) JPtester runs ok in my machine, but throws an error when exits in debug mode. I check it, found the reason:
--- Error Reason ----
in "bool CGUITTFont::attach(CGUITTFace *Face,u32 size)"
the member array "Glyphs" of CGUITTFont is allocated a continuous memeory buffer by using "reallocate" method of array. The elements of Glyphs are not initialized sufficiently, e.g., the pointers "tex", "tex16" and "image" are not set to NULL and the debug name is not set to "CGUITTGlyph". Furthermore, because class "CGUITTGlyph" has virtual funcions, so the virtual function pointers(__vfptr) of each element in Glyphs should be initialized. Without the constructor of CGUITTGlyph, we have to set the __vfptr manually. My solution is shown as below:
--- My Naive Solution---

Code: Select all

...
void CGUITTGlyph::init()
{
	#ifdef _DEBUG
	setDebugName("CGUITTGlyph");
	#endif

	tex = NULL;
	tex16 = NULL;
	image = NULL;
}
...
bool CGUITTFont::attach(CGUITTFace *Face,u32 size)
{
	if (!Driver)
		return false;

	if (attached){
		tt_face->drop();
	}
	tt_face = Face;
	tt_face->grab();

	this->size = size;

                
	CGUITTGlyph tmp;  // for access the __vfptr of CGUITTGlyph 
	int*tmp2;              

	Glyphs.reallocate(tt_face->face->num_glyphs);
	Glyphs.set_used(tt_face->face->num_glyphs);
	for (int i = 0;i < tt_face->face->num_glyphs;i++){
		Glyphs[i].Driver = Driver;
		Glyphs[i].size = size;
		Glyphs[i].face = &(tt_face->face);
		Glyphs[i].cached = false;
		tmp2 = (int*)&(Glyphs[i]); 
		*tmp2 = *(int*)(&tmp);// the __vfptr is arranged in the front of the memory block by VC++
		Glyphs[i].init(); // initialize the other memebers

//		Glyphs[i].cache((wchar_t)i + 1);
	}
	attached = true;
	return	true;
}
...

NOTE:
i) You have to comment out "fonts->drop();" in the end of JPtester to make it work.
ii) I only test it in VS 2005, winxp sp2. maybe it only works in windows platform.
iii) There are definitely much room to optimize.
iv) If the solution does not work for you, please wait for the officially new version which must be cleaner and correct.
--- Hope ----
Hope the admin of Irrlicht add officially surpport to Unicode Text input and render.
Localized font will make a 3D engine cute and kind to many users around the world and make the users express themselves freely.

can you send me the source??nhsimon_li@21cn.com

[MadHyde]

I merged irrTT to Irrlicht1.2.
This includes zgock and solehome's code.
For all of Multibyte Language users:)
http://etwas.wolfish.org/files/IrrTT-1.2-win32.zip

There is one problem in relation to this.
In IME mode, CompositionWindow always appears in left upper.
But, I want to put Window inside GUI element(near caret)
Does nobody know a good method?