utf8 helpers

[hendu]

http://sourceforge.net/p/irrlicht/patches/293/

Data tends to be in utf-8, platforms act natively in wchar. This is a much smaller implementation than Nalin's irrUString, yet it should be useful to many platforms.

[christianclavet]

This look really interesting. Will extended characters will be stripped or converted (accented characters, etc)?

Also Irrlicht using mostly LATIN1 type fonts for display, displaying one of wchar will display correctly?

[hendu]

I personally use it this way:

Data is in utf8 on disk. The TTF class takes stringw in its draw method. So on loading I convert from utf8 to wchar.

If you use bitmap fonts you need to have one with your symbols. All valid extended chars are converted.

[christianclavet]

This is really nice!

[Nadro]

Patch applied into trunk.

[christianclavet]

Wow Thanks! That's a great news. I'll use this version for my project now!
EDIT: Strange, it compile but I get a linker error.

Code: Select all

>XML_Manager.obj : error LNK2001: symbole externe non résolu "void __cdecl irr::core::utf8ToWchar(char const *,wchar_t *,unsigned __int64)" (?utf8ToWchar@core@irr@@YAXPBDPA_W_K@Z)

I'm still using MSVC Express 2008. Something is missing in the IRR project? The library compiled, but using this function with the compiled library give this error.
irrString.h should contain some link to the new function?

[Nadro]

I have to add export defines to the new functions in irrString.h. I'll fix it today.

[hendu]

Yep, forgot about shared builds.

https://github.com/clbr/seirr/commit/97 ... 3a4f6629e0

[christianclavet]

Thanks Hendu, I added manually the new commands and recompiled the DLL then used it again with my project and it seem to work. Compile, link. Have to check how to use the command properly now.

I have a crash every time I use it:

I define this:

Code: Select all

 
//xml is from irrXML
core::stringc input=xml->getAttributeValue("str");
core::stringw output=L""
core::utf8ToWchar(input.c_str(),output,input.size());

I am assuming that I would get the converted string in the output variable. But got some kind of memory allocation error that create a crash. Each time the debugger point me to the core::utf8ToWchar() command.

[hendu]

You're passing a stringw where wchar_t* is expected, and you're claiming the output buffer is input.size() bytes long, when it is 2/4 bytes long. So of course it crashes trying to write N bytes to a 2/4-sized buffer

Code: Select all

wchar_t out[200];
utf8ToWchar("test", out, 200);

[christianclavet]

Thanks Hendu. Now it work just fine! Got my accents back!
That is a great addition to Irrlicht since we can read/write strings encoded in UTF8 now!

[Nadro]

Problem with missing exports is fixed in trunk.