Textures compression tool (BINARY RELEASED)

[REDDemon]

Current content:
1) KTX loader for Irrlicht
2) KTX loader worked also with FVF's Patch by Nadro
3) KTX loader for pure OpenGL applications (use GLEW and read from a standard input stream std::istream)
4) KTX exporter: tool for creating KTX textures (has texture compression and is easily pluggable with more functionalities)

Info about compressed textures:
Compressed textures are faster in loading (especially with the KTX file format wich has all in a nutshell) and they requires less memory (from 1/4 to 1/6 only required compared to uncompressed textures). On old videocards compressed textures improves also rendering speed due to reduced bandwith usage (and better caching).
Compressed textures are used on every AAA game title and are very usefull on mobile devices wich have usually less memory available. Note that also download time for your title is reduced because your package will be more little with compressed textures. So most bottlenecks can be simply solved by using compressed Textures.

KTX exporter:
Download link:
https://sourceforge.net/projects/fixeengine/files/

Screenshots:



Video:
http://www.youtube.com/watch?v=oZs21xda9lY (watch the Tutorial-Video on Youtube)

Notes:
It is written using Irrlicht, POCO, GLEW and OpenGL (+stl and std library of course)

Instructions:
Open the needed files (File->Open), select one of them and open the compression window (Export->compressedRGB). Then press "Start Compression". The output of the compression will be putted in the "KTX" folder. If you want to clear the cache of recently opened files you have just to clear the content of the "cache" folder.


KTX loader for Irrlicht:

Download link:
https://sourceforge.net/projects/fixeen ... 0Irrlicht/

Screenshots:


Instructions:
there's a readme.txt file in the download link that explains what to do: anyway is very simple: you have to add few files to Irrlicht and recompile it. Then you have just to compile the example.



Try KTX loader also with Nadro's patch:

get Test code there:
http://irrlicht.sourceforge.net/forum/v ... 7&start=75

screenshot:




KTX loader for pure OpenGL applications: (the same from my engine)
Download link:
Not related to Irrlicht

Screenshots:


Instructions:
Just provide a reference to a input stream (std::istream) wich can come from any kind of stream (internet connection, compressed/cyphred archive, opening a file for reading) and make sure you have an active OpenGL context. It returns the name of the GL texture that you can read directly with OpenGL


About KTX textures:

Info:
KTX textures are designed to provide fast and efficient loading for OpenGL textures. With a KTX texture is possible storing all kind of info usefull for setting up 1 texture unit. (all texture types are supported by KTX loaders: 1D, 2D, 3D, cubemaps. 1D & 2D arrays. compressed textures. floating point textures. luminance textures etc..)
With Irrlicht using a 512x512 RGB texture requires up to 1MB of video memory + time for decoding it from a image file (that can be PNG, JPG etc.)
the same texture compressed as DXT1 texture requires only 172KB (the same amount of memory is required on Hard disk and on Video memory.. very nice) of memory and is several times faster in loading.

Specifications link:
http://www.khronos.org/opengles/sdk/too ... rmat_spec/

[REDDemon]

Seriously no one interested? I thought that there was lot of people asking for compressed textures (wich is the only texture format wich can save video memory) in Irrlicht probably I was wrong

[christianclavet]

Probably nobody had the time to reply...
Is there a big patch to be able to load back thoses textures?

[ACE247]

Oh wow! Nice work REDDemon!
As christianclavet said would be super If we could load these textures compressed into video memory again.

[REDDemon]

Thanks! yes there will be definitely a loader for Irrlicht. I'm trying to make it working alongside standar material system so that it can be used with shaders also. (not really a patch to SMaterial but a custom IMaterialRenderer with its corresponding services wich is also more easy for every user to add to its current project without need to recompile irrlicht). The image will be loaded directly to GPU without the need to create an intermediary IImage. (but with the chance to get the KTX file from a zipped archive, as side note creating a IImage is not possible since the decompression algorithms are all propietary. Actually i'm just using OpenGL API for use textures.).

Since someone started speaking to port Irrlicht to newest drivers (wich always support compressed textures) if there will be a official API change I'll be glad to write a real patch to current API. For now bugs are much more easy to catch with IMaterialRenderer than cacthin them inside a patch to current API.

[fmx]

Nice work
Will you release your texture tool as a separate binary (or source) soon?
As well as irrlicht, I want to experiment with KTX textures with vanilla GL and other engines too

[Virion]

I'm currently working on an iOS (non-irrlicht) game and I realize textures which can be sent directly to the GPU is really good. I used a lot of PNG files (a lot of animations for my characters) in that game and it crashes very frequently. After that I changed all the PNG files to PVR files (requires POT resolution) and it no longer crash. But the PVR files are very huge: around 2MB fixed-size for a 1024x1024 image of RGBA4444 and 4MB size for the same 1024x1024 image but in RGBA8888. I can tell you this kind of format is really good especially for AAA games or mobile games. I'm glad that you have made it very small file size and memory footprint and yet retain good quality. I will try this out when I back to irrlicht again. Good job.

[REDDemon]

thanks glad to hear appreciation. I finally found a way to "hack" Irrlicht API for using the textures. do not even need for a custom IMaterialRenderer, so no API changes at all and minimum user effort to have it working without a Patch. I think that 1 max 2 weeks everything is ready. The only required thing is to recompile Irrlicht but I think its worth.

[REDDemon]

Ok I have released the first version of the loader. In the download page there are the simple instructions for get it working. You can try it to play with until also the exporter is ready for the release.

Inside there is 1 test file (compressed BGR texture of 512x512 in size).

I want to hear your opinions about that . I had 1 design problem wich is the following

CNullDriver::addTexture(ITexture) is a private method. So I was not able to add directly my texture with that method. The things wich is actually done is dropping/grabbin the video drivers (very wrong!!!) from the texture. If the drivers grab the texture is possible that both won't be deleted. The example works fine anyway.. Any suggestion to improve the loader and to solve that issue?

Anyway the exporter will be ready soon

[Nadro]

Really good work! KTX support will be very usefull for Irrlicht.

[REDDemon]

Thx! Your FVF is even more usefull! You did a great work!

FVF worked very nice with compressed Textures. Now Irrlicht have both FVF and compressed textures.

The KTX loader needed no changes at all. Only the "custom scene node example" required few changes but worked at the first run.

[fmx]

Thanks for releasing, I'm new to the whole concept of texture compression so your work is coming in useful

I had a quick look at the specification and your irrlicht loader
at first glance it looks like KTX is nothing but a container for compressed DXTx texture data which gets used by glCompressedTexImage2D directly?

Somebody correct me if I'm wrong:
- Texture filesize reduction is primarily due to DXT compression?
- DDS texture format uses DXT compression too, right?
- DX8/9 supports loading of DXT compressed textures?
- Irrlicht doesn't already allow importing DDS textures with GL or DX drivers?!

I dont have anything against your KTX loader REDDemon, but I get the feeling maybe irrlicht needs a proper .DDS texture loader first (for both GL and DX)...

[Nadro]

Hehe thanks Returning to the problem of texture compression I'm also interesting in a native support of compressed texture in Irrlicht. As I know the mainly problem is a good interface of an IImage class. I think will be enough if we will add these extension for our existing IImage class:

Code: Select all

class CImage
{
public:
...
        bool compress(E_COLOR_FORMAT format) // Where ECF_DXT1, ECT_DXT3, ECF_DXT5, ECF_KTX will be support.
        {
                // Here we can write code for compression a RAW data into a compressed data; at start we don't need this method worked)
        }
 
        bool decompress()
        {
                // Like a compress we don't need this method worked at start
        }
 
        bool isCompressed()
        {
                return IsCompressed;
        }
...
private:
        bool IsCompressed; // if true, many CImage methods aren't active, we can manipulate only uncompressed images.
...
};

What do You think about this idea? I think that this is the simplest and really functional way for support compressed textures in Irrlicht. Next step will be add support for create Device dependent textures from a compressed images, but this isn't hard. I'm interesting what other developers think about it? Will we should integrate it at first in shader-pipeline branch and next in a trunk, or for trunk first?

When IImage interface will be ready (it doesn't require too much of time if we will leave compress and decompress methods empty at first) we'll integrate REDDemon's KTX image loader and avaiable on a forum DDS loader with an engine. I can do it in a short time, but I need green light from an other devs

Somebody correct me if I'm wrong:
- Texture filesize reduction is primarily due to DXT compression?
- DDS texture format uses DXT compression too, right?
- DX8/9 supports loading of DXT compressed textures?
- Irrlicht doesn't already allow importing DDS textures with GL or DX drivers?!

I dont have anything against your KTX loader REDDemon, but I get the feeling maybe irrlicht needs a proper .DDS texture loader first (for both GL and DX)...

You're right, but I think that we should integrate both DDS and KTX. The most important part -> file loaders is done, so we don't have to write too many internal code. On a forum is avaiable a patch for DDS support, so in similar way we can add KTX support in OpenGL driver, but at first we must accept or prepare better concept of an IImage class.

[REDDemon]

at first glance it looks like KTX is nothing but a container for compressed DXTx texture data which gets used by glCompressedTexImage2D directly?

Yes. It can contain every kind of texture. (not limited to DXT compressed textures, there are also other compression standards). You just need the right file and it can be loaded... As sad before KTX is almost the same of DDS except that KTX is for OpenGL instead of DX. so your statement "nothing but a container for compressed DXT data" works also for DDS textures.

And. Actually the KTX loader can also be used for texture compression (indirectly). You just need to set the correct parameters inside the header file and video drivers will do the rest. (of course that's slower than loading raw compressed data and maybe not all drivers will support that while already compressed DXT data is widely supported on almost all videocards and that's why I wrote a TextureExporter).

If you give me the time to read DX documentation is probable that the KTX texture can be loadedd also in DX (just need to translate all GL parameters into DX equivalents.). And the same works for DDS textures. DDS files can be loaded into OpenGL with some modification. But since there is already a DDS loader (never known about that) is probably better using 2 different loaders. I will write in that case a plugin to my tool for converting KTX files to DDS and vice-versa. (not seems to hard looking at DDS files' spec.

@Nadro:

You Idea works to me (with that interface my compression tool becomes a bit obsolete since compression will work also inside from Irrlicht but using a GUI tool can be also faster so maybe is better having 2 compression ways for Irrlicht).

so use a IImage as alias for a compressed internal texture? just give me a nice API and I'll add the missing implementation .

what about mipmap generation API and texture creation flags?

We should have some control also in mipmap generation. In my loader I'm implementing now this

Code: Select all

 
/**
        \param ktx_stream: you get this by opening a KTX file with irrlicht (also from inside a compressed archive)
        \param start_level : 0 is the biggest mipmap level. You can specify to start using a more little level
                 (if you need to save more memory, or increase performance on target platform).
                Occlusion queries can be used to see if a mipmap level is rendered or not. If it is not
                rendered you can just reload the texture without that mipmap level (explained well
                in a GPU gems chapter).
        \param num_of_mipmaps: 0 is default value and leave the decision of how many mipmaps to 
                create to the KTX loader. You can specify the number of mipmap levels to be used startin
                from "start_level" (very usefull for texture ATLASES wich needs a truncated mip pyramid
                for avoid visible artifacts). 1 means only 1 miplevel (usefull for GUI elements wich don't
                need mipmap pyramid). Supported only from OpenGL 1.2 and above
*/
loadTexture(io::IReadFile* ktx_stream, u32 start_level = 0, u32 num_of_mipmaps = 0)
 

wich is intended to provide users full control on mipmap pyramid. Probably we have to sacrifice a bit of this flexibility. But just give me good
hooks in the API and I'll do the rest

[CuteAlien]

One note about texture-compression. There was once a working patch years ago for OpenGL: http://irrlicht.sourceforge.net/forum/v ... hp?t=10488
(no idea why the poster is now known only a guest... he wasn't back then).

Bitplane mentioned once to me that there had been some concern because of a texture-compression patent. But last I've checked it the patent only covers the implementation of the decompression (the part that is now disabled by define in Irrlicht), while just loading dds and passing it on compressed to the graphic card drivers is fine (the driver vendors are paying the patent fees... sucks for opensource drivers, but unfortunately with current hardware there is not much of a way around this to get support for compressed textures). Note that I am not a lawyer, so I might be wrong - also I usually do not care that much about patents, but this one is somewhat famous because the patent-owners are aggressively protecting it.