Irrlicht i18n (Unicode, FreeType, etc.)

[MadHyde]

Hi,
Thank you for your great work, Nalin

I brought the report about CGUITTFont from Japanese user.
Anonymous918 said

I loaded two fonts. these are same name but different size.
And drew all at the same time.
In this case, The font position shifted.
This issue seems to be related to reused font texture.

Test code

[Nalin]

Okay. I fixed that problem. The download link in the OP has been updated with a newer version.

The issue is that I was caching FT_Face objects. When you loaded up a larger font, it was overwriting some of the fields in the FT_Face. This caused errors when calculating font position.

I added a couple FreeType methods to set the size of the face before performing size calculations. It works around this issue for now. Ideally, I want to determine if it is better to cache the FT_Face or create a new one for each font size. I will do that when I re-write the loading/drawing code to improve performance.

---
EDIT:

Actually, I've been thinking about how to improve the drawing code. Obviously, the solution is to batch the drawing calls. The normal IGUIFont uses a single bitmap and uses the draw2DImageBatch() function of the video driver.

I've been leaning towards a solution where I create a blank ITexture to hold all the rendered glyphs. Whenever a new glyph is loaded, I manually add it to the ITexture and update a map to store the index of the glyph. It would render as fast as the built-in IGUIFont class, at the expense of a slower loading of glyphs.

Alternatively, I could go for an easier yet slower solution where I batch render each individual letter in the string (ie, draw all the a's, then b's, then c's, and so on.) Potentially less error-prone as the first method, but it would be slower as each letter is its own texture and would require a state change to draw.

Does anybody have any opinions?

[hybrid]

Texture changes are one of the most expensive calls. So the fastest solutions should be obvious. Moreover, rendering the same character at all places seems also quite complex and simply odd...

[Nalin]

Texture changes are one of the most expensive calls. So the fastest solutions should be obvious. Moreover, rendering the same character at all places seems also quite complex and simply odd...

No, I was saying that if you took a text string, "This is a test" and tried to render it, it would render the T, then the h, then both i's, then the three s's, then the a, then both t's, and lastly the e. That would cut down on texture switches, as each letter is its own separate texture. It currently just renders each letter one at a time using a draw2DImage() call.

The more redundant characters you are rendering, the better that solution would work. My first idea would be slow for the initial creation of the texture, but it would be as fast as the bitmapped font on rendering. Would it be faster to create a whole new texture than to modify an existing one? That is how it currently works. If you try to draw a character that isn't loaded, it queries FreeType for a bitmap image of the character and creates a new texture. If modifying an existing texture is quicker, then it will be all-around quicker. If not, I could always keep an IImage in memory and re-create the texture when a new glyph is added to it.

Really, those are the only two solutions I could think up. In your experience, do you know of any other solutions that could be used?

[hybrid]

Well, updating the texture (just as overwriting it) is also expensive. So you should batch that as well. You could create regions of maybe abput 100 glyphs around the currently chosen one. This would speed up generation (at least the CPU/GPU part) and only require a decent CPU at first. Also it would require maybe a little more memory, but both should be ok (and maybe even configurable). But it's probably best to just try some methods and see how they perform.

[MadHyde]

Okay. I fixed that problem. The download link in the OP has been updated with a newer version.

Thanks Nalin. It works fine! and I think your first improving idea is better.

BTW,

just wanted to try it but irrUString.h doesn't compile.

Code: Select all

..\..\Libs\irrlicht-1.7.1\include\irrUString.h||In member function `irr::core::ustring16<TAlloc>& irr::core::ustring16<TAlloc>::replace(irr::uchar32_t, irr::uchar32_t)':|
..\..\Libs\irrlicht-1.7.1\include\irrUString.h|1892|error: expected `;' before "a"|
..\..\Libs\irrlicht-1.7.1\include\irrUString.h|1893|error: `a' was not declared in this scope|
||=== Build finished: 2 errors, 3 warnings ===|

Which compiler are you using?
In the meantime, try this:
Possibly fixed irrUString.h.

I saw this error on Ubuntu 9.10, when using code:blocks 8.02 with gcc 4.4.1. Both the OP and this test ver.
Unfortunately, I don't know how to fix.

[Nalin]

I saw this error on Ubuntu 9.10, when using code:blocks 8.02 with gcc 4.4.1. Both the OP and this test ver.
Unfortunately, I don't know how to fix.

I installed the latest g++ version and confirmed the error. I fixed it and uploaded a new version of irrUString.h. You can get it from the download link in the OP.

[MadHyde]

Thank you again

[Nalin]

Here is a test version of the new CGUITTFont class that makes use of the draw2DImageBatch() function to improve rendering speed.

Download:
http://irrlicht.suckerfreegames.com/tes ... _batch.zip

There are two new member functions in CGUITTFont:
void setBatchLoadSize(u32 batch_size)
void setMaxPageTextureSize(const core::dimension2du& texture_size)

The first function adjusts how many glyphs to batch at once. It defaults to 100. What this means is that if you try to load glyph 225, it will load glyphs 175 to 275, for a total of 100 glyphs.

The second function sets the size of the glyph page texture. By default, the class tries to make the texture large enough to store 400 glyphs on it. This lets you override the default behavior.

I'll work on profiling and optimizing it later. I just want to make sure I haven't broken anything as I've made a bunch of changes to the class.

If it draws a bunch of blocky opaque squares instead of text, use DirectX instead of OpenGL. I seem to have stumbled upon some sort of OpenGL texture creation bug. The forceGlyphUpdate() function was added to work around this bug until it gets fixed in Irrlicht. That will force the textures to be re-created. If you suffer from blocky textures, you will want to call that function after the blocky texture has been drawn at least once.

[Nalin]

Alright, I'm releasing my new version of CGUITTFont now. It uses batched drawing like the normal IGUIFont class.

You can download it via the link in the OP.

Compared to the test version in my last post, this new one fixes some bugs that I found as well as pre-loads the first 127 ascii characters.

Because of the change in how the class draws the glyphs, the grayscale glyph mode won't look correct in the EDT_SOFTWARE renderer, as that renderer doesn't support color blending on the alpha channel. In the future, I may get around to adding that feature back in, but for now, if you need a software renderer that displays grayscale fonts correctly, just use the EDT_BURNINGSVIDEO renderer.

Old CGUITTFont:


New CGUITTFont:


I get a 165 frames/second increase with the new class. I call yield() every loop. The second image also shows the full font texture as created by the class.

________
EDIT:

Fixed a small memory leak when a font was destructed.
Updated the link in the OP.

[Nalin]

New version of irrUString.

• Added function: empty(). Returns true if the string is empty.
• Added function: find_raw(const ustring&). Searches for text and returns the true position in the array that it is located.
• Added function: replace(const ustring&, const ustring&). Replaces all instances of a string with another.
• Fixed some bugs that could result in heap corruption.

Also, I've got the very first work in progress of my string table class finished.
Download

Currently, you can load strings in from an XML file, retrieve them for use in your program, and replace parameters. I still need to create methods for organizing multiple string tables.

How to use:

Code: Select all

// Set a username for use later on.
core::ustring user_name("Bob");

// Create a new string table from spanish.xml.
irr::st::CStringTable* stringtable = irr::st::loadStringTable("spanish.xml", device->getFileSystem());

// Set it as our active string table.  This lets us use the shorthand st::_T() function to retrieve a value from the string table.
irr::st::setActiveStringTable(stringtable);

// Assume that Greeting results in:
//     ¡Hola, {{%1}}!  ¿Cómo estás?
// The following code would then return:
//     ¡Hola, Bob!  ¿Cómo estás?
core::ustring greeting = st::_T("Greeting").param(user_name);

The zip file also includes an example.xml that illustrates how to create an XML string table.

[Nalin]

Okay, I finished my string table class. Download links plus detailed instructions have been added to the OP.

[zet.dp.ua]

Good work! Dictionary class is VERY useful in the game development process. This is "must have" class for every game developer.

[Nalin]

Good work! Dictionary class is VERY useful in the game development process. This is "must have" class for every game developer.

Thanks. I modeled it somewhat after Qt's QString class in regards to how you can chain parameters, while trying to make it a separate component to Irrlicht and my ustring class. If anybody has any suggestions for it, let me know. That said...

---

I've uploaded some new changes.

ustring:

• unicode::determineUnicodeBOM() function added. It scans a char array for any unicode byte order marks and returns what it found.
• Re-arranged code to change the way new strings are created.
• Removed the generic pointer constructor and replaced it with specific char, uchar8_t, uchar16_t, and uchar32_t constructors.
• findLastChar and findLastCharNotInList work properly now.
• loadDataStream() method added. It takes a char* and calls determineUnicodeBOM() to check for a byte order mark, then calls the appropriate loading function based on the returned BOM. The char* constructor also calls this method. It makes loading unicode files extremely easy.
• Fixed some bugs.

CGUITTFont:

• Added a workaround for a bug in driver->removeTexture() when using OpenGL.

CStringTable:

• Added a helper function to CStringTableUString to assist in getting the referenced string.
• getStringTable() and getMap() added to CStringTableManager.
• getActiveStringTable() function added.

[zet.dp.ua]

One suggestion: it would be better to minimize the additional info in the xml file. <entry id="My string">My string</entry>, <s id="My string">My string</s> could be enough.

Integrating string table with irrlicht's text node can really save a lot of time.
Don't write any possibly localizable text in the code!!!