Weird D3D9 behavior with textures

[lumlum]

Hello,

I'm fairly sure i've ran into an anomaly with the D3D9 driver. When I run the following code (which basically does nothing but loading/removing textures) I get very different results when it comes to memory with D3D9 and OpenGL.
If I load/draw a big texture for a few frames then remove it and load/draw a much smaller one there is some kind of remote memory usage with D3D9. If the small texture uses like 10Mo of ram, and the big one like 250Mo or something, then when the code goes back to drawing the small one, even tho I called removeAllTextures(), the application still uses about 150Mo when it should be back to ~25Mo or something. Obviously all textures should be removed at some point after calling removeAllTextures() but once the big texture has been loaded and drawn multiple times, the memory usage will never go back down. And If I only draw the small one then the big one then do not draw the small one again and just keep calling only removeAllTextures everyframe, the memory usage still won't go back to 25Mo. It looks like no matter what I do, once i've loaded and drawn the big texture a few times I won't be able to have the same behavior as OpenGL unless I drop and recreate the device.

When using OpenGL, there is no such issue I can load/remove textures in every possible way, once i can removeAllTextures, every bit of memory is free'ed.
Same thing seems to happen for all texture sizes, I just used a very big one in order to be able to easily notice the difference.
Same issue with both 1.8.3 and trunk version, however the memory usage is smaller with the trunk one (ie : like 120Mo instead of 170Mo) but still a lot higher than the OpenGL driver.
Note that it's not a proper memory leak, the memory usage does not grow indefinitely, there seems to be some kind of cap, I've tried to use the following code but as an infinite loop (the code would alternate everyframe between the 2 textures) and after several minutes, the memory usage was still the same when drawing the small texture (150Mo or something), it's way higher than expected but it does not grow forever. It looked like it depended on the maximum size of the biggest texture drawn or something.

In the end I wouldn't say it is too much of an issue (besides the fact that your application might use more memory with the D3D9 driver than the OpenGL one) nor that I know where it comes from or how it behaves exactly or even if it's a bug but still I thought it would be worth posting because when you call removeAllTextures you expect the memory to actually be free'ed and not to have 150Mo+ to still be remotely used.
The only way i've found so far to reset the memory usage was to drop/recreate the D3D9 device.
I did not post in the bug section because i'm not sure if it is the normal behavior or not but to me it does not seem normal at all. If it is then i'd really like, if possible, to get some sort of explanation because this has been bothering me for a while now.

I used the following BMPs :

Small texture : http://technologie-f-mauriac.jimdo.com/ ... 1395577376

Big texture : http://ilab.engr.utk.edu/ilabdocs/Epilo ... %20dpi.bmp

Code: Select all

#include <irrlicht.h>
 
using namespace irr;
 
using namespace core;
using namespace scene;
using namespace video;
using namespace io;
using namespace gui;
 
#ifdef _MSC_VER
#pragma comment(lib, "Irrlicht.lib")
#pragma comment(linker, "/subsystem:windows /ENTRY:mainCRTStartup")
#endif
 
 
 
int main()
{
 
    IrrlichtDevice *device =
        createDevice(video::EDT_DIRECT3D9, dimension2d<u32>(640, 480));
 
    if (!device)
        return 1;
 
    IVideoDriver* driver = device->getVideoDriver();
    ISceneManager* smgr = device->getSceneManager();
    IGUIEnvironment* guienv = device->getGUIEnvironment();
 
    int j = 0;
    ITexture *t;
    while(device->run())
    {
        if(j<20){
            driver->removeAllTextures();
            t = driver->getTexture("pathToSmallTexture");
            j++;}
        else if(j<30){
            driver->removeAllTextures();
            t = driver->getTexture("pathToBigTexture");
            j++;}
        else{
            driver->removeAllTextures();
            t = driver->getTexture("pathToSmallTexture");}
 
        driver->beginScene(true, true, SColor(255,100,101,140));
 
        driver->draw2DImage(t, vector2d<s32>(0, 0));
 
        driver->endScene();
    }
 
 
    device->drop();
 
    return 0;
}

[CuteAlien]

Which Irrlicht version are you using? There has been a memory leak in D3D9 in trunk for a few weeks, but i got fixed 1-2 days ago. So maybe you just have to check-out the new version. Don't know about 1.8.3, but it could be you have to wait a little bit until the memory is released. Not sure about the details right now, but I remember there was some timer involved in releasing textures sometimes.

[lumlum]

Thanks for answering quickly, yes I know there was a texture memory leak that got fixed a few days ago, at first i thought it was the issue so i made sure i had the right version. I actually tested that little code with 1.8.3, rev5104 and rev5267(latest) and none of them have the memory leak you're talking about but all have that weird behavior I explained in the original post.

I've done some more testing and I got the following results :

If at some point I keep removing all textures then loading/drawing the small one every frame with the following code then eventually, after a few minutes, the memory usage will go back to the expected ~20Moish which is much better :

Code: Select all

 
    #include <irrlicht.h>
     
    using namespace irr;
     
    using namespace core;
    using namespace scene;
    using namespace video;
    using namespace io;
    using namespace gui;
     
    #ifdef _MSC_VER
    #pragma comment(lib, "Irrlicht.lib")
    #pragma comment(linker, "/subsystem:windows /ENTRY:mainCRTStartup")
    #endif
     
     
     
    int main()
    {
     
        IrrlichtDevice *device =
            createDevice(video::EDT_DIRECT3D9, dimension2d<u32>(640, 480));
     
        if (!device)
            return 1;
     
        IVideoDriver* driver = device->getVideoDriver();
        ISceneManager* smgr = device->getSceneManager();
        IGUIEnvironment* guienv = device->getGUIEnvironment();
     
        int j = 0;
        ITexture *t;
        while(device->run())
        {
            if(j<20){
                driver->removeAllTextures();
                t = driver->getTexture("smallTexture.bmp");
                j++;}
            else if(j<30){
                driver->removeAllTextures();
                t = driver->getTexture("bigTexture.bmp");
                j++;}
            else if(j==30){
                driver->removeAllTextures();
                t = driver->getTexture("smallTexture.bmp");
                j++;
            }
            else{
                driver->removeAllTextures();
                t = driver->getTexture("smallTexture.bmp");}
     
            driver->beginScene(true, true, SColor(255,100,101,140));
 
            driver->draw2DImage(t, vector2d<s32>(0, 0));
     
            driver->endScene();
        }
     
     
        device->drop();
     
        return 0;
    }

However if I just call removeAllTextures once then load the small texture once and keep drawing it normally everyframe with the following code then some memory is never free'ed and it remains at 100Mo+ instead of ~20Mo :

Code: Select all

 
    #include <irrlicht.h>
     
    using namespace irr;
     
    using namespace core;
    using namespace scene;
    using namespace video;
    using namespace io;
    using namespace gui;
     
    #ifdef _MSC_VER
    #pragma comment(lib, "Irrlicht.lib")
    #pragma comment(linker, "/subsystem:windows /ENTRY:mainCRTStartup")
    #endif
     
     
     
    int main()
    {
     
        IrrlichtDevice *device =
            createDevice(video::EDT_DIRECT3D9, dimension2d<u32>(640, 480));
     
        if (!device)
            return 1;
     
        IVideoDriver* driver = device->getVideoDriver();
        ISceneManager* smgr = device->getSceneManager();
        IGUIEnvironment* guienv = device->getGUIEnvironment();
     
        int j = 0;
        ITexture *t;
        while(device->run())
        {
            if(j<20){
                driver->removeAllTextures();
                t = driver->getTexture("smallTexture.bmp");
                j++;}
            else if(j<30){
                driver->removeAllTextures();
                t = driver->getTexture("bigTexture.bmp");
                j++;}
            else if(j==30){
                driver->removeAllTextures();
                t = driver->getTexture("smallTexture.bmp");
                j++;
            }
            else{
                /*driver->removeAllTextures();
                t = driver->getTexture("smallTexture.bmp");*/}
     
            driver->beginScene(true, true, SColor(255,100,101,140));
 
            driver->draw2DImage(t, vector2d<s32>(0, 0));
     
            driver->endScene();
        }
     
     
        device->drop();
     
        return 0;
    }

You should try these simple samples when you got some time, you'd easily see what i'm talking about.

[mongoose7]

How do you know it is not a system problem. For example, Unix and Windows are different in that Unix *never* gives back memory although internally it marks it as free. Windows, I know, seems to allocate memory as memory objects so it can often be returned to the system and the task size is thereby reduced. But I don't know how this is controlled. For example, if you allocate a lot of small items, it would be wasteful to make each an object as you waste up to 4K for each and maybe you could run out of handles? As you see, I don't really know how it all works, but couldn't this be what you are seeing?

[CuteAlien]

OK, no timers involved, that's just for meshbuffers, not for textures.
But I get very different results (on Windows 7). D3D goes back to 11MB immediately, while OpenGL stay at around 50 MB. Also the app runs a lot slower in OpenGL (over 300% of the time compared to d3d9). So ... interesting test-case.
edit: Slower would be explained if D3D would not release... so only OpenGL would reload the textures. But why has it less memory then?
edit2: Slowness in OpenGL comes from COpenGLCoreTexture::uploadTexture. OGL and D3D both remove all textures and reload from disk. But for some reason uploading textures to the card is slower in opengl than loading the texture from disk.
edit3: OK, this is really weird. I don't get such different results when using other textures. Then the speed is the same for OGL an d3d (also huge textures, just different ones). Thought OpenGL still takes more memory - I thought it no longer would do that...

[Mel]

I've always seen that sort of behavior in D3D (I don't use GL often, so i can't tell :/) Somehow, it has to do with how the memory is handled by the Direct3D runtime, not Irrlicht. It shadows the hardware texture with its own copy in system memory and doesn't necesarily gets deleted when it is requested to do so (same goes to when it is used for the first time, textures never get uploaded to the GPU until their first usage) Most of the times, they will keep there until some LRU routine hits them and get removed, or perhaps, the memory is needed again. Looks like D3D does some memory pooling on its own and doesn't release the memory for a texture to avoid excessive memory fragmentation.