How to load textures faster?

[blackMasoon]

Well... it is not a game and I need to be able to pause the sequence on each frame I want. Does any lib from this forum support that?

[CuteAlien]

Pausing should always be simple - just don't update to the next frame.

[HerrAlmanack]

Pausing should always be simple - just don't update to the next frame.

IMPORTANT: however, if you use a delta time based system that uses a main logic tik() method. make sure you pass the correct d time and not ALL the time that passed while the player was paused.

[hendu]

FWIW, libjpeg-turbo is some 30% faster. Perhaps will help your bottleneck.

[Lonesome Ducky]

Well, the first and most obvious (not the best, mind you) step would be to use an uncompressed format. CPU cycles will be saved since you don't have to uncompress the file first. The problem being that even 1000 bmps could easily send you past a gigabyte.

[hendu]

@Ducky

So you have a few SSDs in a raid?

IO speed is much more of a bottleneck than cpu decompression in my experience.

[blackMasoon]

I did a little test and here are the results:

loading 640 JPG files & creating textures out of them takes 32 139 ms
loading 640 BMP files & creating textures out of them takes 26 700 ms.

One JPG file size is ~30 kB
One BMP file size is ~1.1 mB


Looks like that technique with uncompressed files is not a good idea..

[Lonesome Ducky]

Huh, well I'll eat my words . I guess at a certain compression rate it is far quicker to decompress than read the raw data.

[blackMasoon]

I only need to display those images. Not make textures of them... Any idea how could I simplify that process? Perhaps display them as a top layer with something else than irrlicht?

[hybrid]

Unless you use one of the SW renderers, you can only display textures. since you don't display all at once, it would be a good idea to reuse textures and copy the new image to the texture, instead of creating textures for all images. Also, as I said before, certain setting scan influence the processing time. Please give more details about the image size, the loading and display process, etc.

[blackMasoon]

Ok so here is how do I do it. At the begining I init the img's and load all textures:

Code: Select all

 
 
        cam_rect[0] = core::rect<irr::s32>(10,10,273,223);
        cam_rect[1] = core::rect<irr::s32>(283,10,556,223);
        cam_rect[2] = core::rect<irr::s32>(566,10,839,223);
        cam_rect[3] = core::rect<irr::s32>(566,230,839,440);
        
        
        for(int i = 0; i < 4; ++i)
        {
                cam_img[i] = device->getGUIEnvironment()->addImage(cam_rect[i]);
                cam_img[i]->setScaleImage(true);
        }
 
        for(int i = 0; i < filepath.size(); ++i)
        {
                tex.push_back( driver->getTexture( filepath[i].c_str()  ) );
                tex2.push_back( driver->getTexture( filepath2[i].c_str()  ) );
                tex3.push_back( driver->getTexture( filepath3[i].c_str()  ) );
                tex4.push_back( driver->getTexture( filepath4[i].c_str()  ) );
        }
 

I use 4 array lists because there are 4 views to display. Later in the main loop i do sth like this:

Code: Select all

 
                cam_img[0]->setImage(tex[frame]);
                cam_img[1]->setImage(tex2[frame]);
                cam_img[2]->setImage(tex3[frame]);
                cam_img[3]->setImage(tex3[frame]);
 

[omaremad]

If mipmapping is not needed, disable it before you load the textures, mipmapping is the true bottle neck in texture creation see the shader example for how to disable mipmaps

[blackMasoon]

Lol... disabling mipmaps made it even slower... Is there any other simple API to draw images on an other layer of HUD which cooperates with irrlicht?

[hybrid]

Well, you still didn't answer most of my other questions. But if disabling mipmaps makes *loading* slower, you're either doing something completely wrong, or you measure something different.
Talking about ways to render the images I'd suggest to use the pure draw2DImage calls instead of GUI elements. But of course this won't change loading times.

[devsh]

Howabout this.. you don't need more than 25 fps
you have 640 images which load in 32 seconds
50msec for 1 image
out of that 25% is decompression
and 75% data copying and texture creation
irrlicht is to blame for that as it first reads from texture into software texture and then copy from software texture to opengl/directx one
a frames at 25 fps take 1000/25= 40msec

I'd advise to try png's or tga's (faster decomp, still good copmression ratio, and png is lossless)
Then I'd advise to set the default texture loading format (which is R8G8B8A8 in irrlicht, and should be same as your texture format, jpeg==r8g8b8 no alpha)
This should cut your data allocated by irrlicht by 25% == that 75% of time ofr copying and texture creation shrinks to 57.25%
disabling mipmapping should help.
At this point you should get 41.125msec to load

Don't load all images at once!!!!
The images as bmps take up 700mb, but irrlicht has two copies SO 700mb in the RAM and 700mb in the VRAM! If you do that irrlicht will run out of gfx memory on your card and make everything slow.

All media players need to buffer when skipping to frame, you need to allow 150ms to prebuffer at least 3 images.

You need to stall for half a second in order to buffer 12 frames.

You can load from file into irrlicht software texture (IImage) in another thread, you don't need dual core cpu. Even Pentium 4 with Hyper-Threading gives exactly 190-200% performance as expected(I tested it with my grass generator).

Your IImage loading thread should load images (and dispose of them) all the time and tell you what image it's loading (how far it's buffered and only sleep if it has buffered 5 seconds of video (you only need to use thread sync if you skip to frame)). This should take care of decompressing and copying the data around to RAM.

Your display thread should wait for 12 images to be loaded and then load them as ITextures by using createTextureFromData (or something like that which makes a texture out of IImage data). YOU SHOULD DELETE IImages as soon as you made textures out of them, but obviously in the IImage loading and delete thread so you dont waste your render thread's time.

IF YOUR LOADING IS FALLING BEHIND THE PLAYING, DROP FRAMES!!! INSTEAD OF PLAYING ALL PLAY AT 20FPS or 18FPS BY NOT DISPLAYING SOME FRAMES!
(also if you are falling behind don't delete the already loaded IImages or ITextures)

Then your render loop should look like this:

int current_frame;
u32 beginTime;
while (current_frame<END_FRAME)
{
beginTime = timer->getTime();
device->run();
IImage* currentImage = LoadingThreadData.loadedIImagesPointers[current_frame];
if (!currentImage)//not loaded yet
{
//tell the loading thread to skip to
LoadingThreadData.setSkip(++current_frame);
continue;
}
ITexture* texture = driver->createTextureFromData(currentIImage);
driver->beginScene();//DO NOT CLEAR Z BUFFER OR CLEAR BACKBUFFER, ALL TAKES TIME
material.setTexture(0,texture);
//~MAKE SURE MATERIAL HAS ZBUFFERCHECK==PASS ALWAYS OTHERWISE YOU WILL HAVE TO CLEAR Z BUFFER
driver->setMaterial(material);
driver->renderTriangleList();//or something like that, render triangles of a screen quad directly, don't call gui->draw2DImage or driver->draw2DImage too many overhead calls such as matrix state changes etc.
driver->endScene();
if (timer->getTime()-beginTime<1000/24) //still have time left
driver->deleteTexture(texture);
if (timer->getTime()-beginTime<1000/24) //still have time left
sleep(timer->getTime()-beginTime<1000/24);
}

P.S. I Offer My programming services for a small fee (email me devsh.graphicsprogramming AT gmail dot com).