xml based postprocessing framework

[Granyte]

the shaders would have to be optimised but the framework is quite well done

[tbw]

930 fps in mode 0 (no antialiasing)
730 fps in mde 1 (fxaa aplied)
450 in mode 2 (four cascading effects with about 30 passes)

ok, its not a speed machine but also not so bad...
but however, I'm grateful for any hints,

...these things really have to be optimized

is a little bit general I think.

[ACE247]

930 fps in mode 0 (no antialiasing)
730 fps in mde 1 (fxaa aplied)
450 in mode 2 (four cascading effects with about 30 passes)

See, what did I say?

"If its not done perfect...." And that's quite fast.
Okay i must admit it though, its not quite perfect. The shaders can use a lot of tuning, but for the most part irrlicht itself needs tuning.

[Granyte]

While we are on it i modified the framework to handle mrt to allow depth write on in a single pass and maybe even defered rendering i might post my sources if you guys are interested in it

[ACE247]

Anything with single pass depth and the possibility of deferred, makes me happy

[tbw]

I modified the RenderDepth function in the following way to enable mrt rendering

Code: Select all

void CPostProcessManager::renderDepth(const video::SColor& defaultDepth)
{   
    if (DepthPassNodes.size())
    {
        // animate and render the camera to ensure correct depth and normal information
        scene::ICameraSceneNode* camera = Device->getSceneManager()->getActiveCamera();
        if (camera)
        {
            camera->OnRegisterSceneNode(); 
            camera->OnAnimate(Device->getTimer()->getTime()); 
            camera->render(); 
            DepthMaterial->setPixelShaderConstant("MaxDistance", camera->getFarValue());
        }
        // set depth and normal render target texture 
        // as multi render target 
        core::array<video::IRenderTarget> mrt;
        mrt.push_back(video::IRenderTarget(RenderTargetMap["rttDepth"]));
        mrt.push_back(video::IRenderTarget(RenderTargetMap["rttNormal"]));
        Device->getVideoDriver()->setRenderTarget(mrt, true, true, defaultDepth);
 
        // render all nodes that are stored in the depth pass array
        for(u32 i=0; i<DepthPassNodes.size(); i++)
        {
            // get the scene node from the array
            scene::ISceneNode* node = DepthPassNodes[i];
            
            if (node->isVisible())
            {
                // save the scene node materials
                core::array<video::E_MATERIAL_TYPE> bufferMaterialList(node->getMaterialCount());
                bufferMaterialList.set_used(0);
                
                for(u32 j=0; j<node->getMaterialCount(); j++)
                    bufferMaterialList.push_back(node->getMaterial(j).MaterialType);
 
                // apply the depth material
                node->setMaterialType(DepthMaterial->getMaterialType());
                
                // animate the node
                node->OnAnimate(Device->getTimer()->getTime());
 
                // render the node
                node->render();
 
                // reset the scene node to the original material
                for(u32 j=0; j<node->getMaterialCount(); j++)
                    node->getMaterial(j).MaterialType = bufferMaterialList[j];
            }
        }
    }
}

using this vertex shader

Code: Select all

//Matrices
float4x4 WorldMatrix;
float4x4 ViewMatrix;
float4x4 ProjMatrix;
 
struct VS_INPUT
{
    float4 Pos    : POSITION0;
    float3 Normal : NORMAL0;
    float2 Tex    : TEXCOORD0;
};
 
struct VS_OUTPUT
{
    float4 Pos         : POSITION; // Vertexposition
    float4 WorldPos    : TEXCOORD0; // World Position
    float4 WorldNormal : TEXCOORD1; // World Normal
    float  Depth       : TEXCOORD2; // Depth    
};
 
VS_OUTPUT main (VS_INPUT input)
{
    VS_OUTPUT output;
 
    // default positions
    output.Pos = mul(mul(mul(input.Pos, WorldMatrix), ViewMatrix), ProjMatrix);
    
    // world positions
    output.WorldPos = mul(input.Pos, WorldMatrix);
    
    // world normal
    output.WorldNormal = 0.5*mul(input.Normal, WorldMatrix)+0.5;
 
    // depth
    output.Depth = output.Pos.z;
 
    return output;
}

and this pixel shader (glsl equivalent)

Code: Select all

niform float MaxDistance;
 
struct MRT_OUT
{
    float4 rt0 : COLOR0;
    float4 rt1 : COLOR1;
};
 
MRT_OUT main(float4 worldPos    : TEXCOORD0, 
             float4 worldNormal : TEXCOORD1, 
             float  depth       : TEXCOORD2)
{
    MRT_OUT output;
    
    // put the worldpos, calculated in vertex shader, into rgb of the first rt
    output.rt0.rgb = worldPos.xyz;
    
    // put the depth, calculated in vertex shader, into alpha of the first rt
    output.rt0.a = depth/MaxDistance;
 
    // put normal into the second rt
    output.rt1 = worldNormal;
    
    return output;
}
 

but as before with a second render pass for depth,normals and world coords...

[Granyte]

that's a nice aproach my modification alow the user to define MRTS in the RTT definition i would say the only issue right now is that the mrt get binded all the way to the shader
so this mean that the shaders would have to be rewrittten because there will be something binded to texture 1 or 2 if you created a rtt with that many textures in it

modifications to PostProcessManager.h

Code: Select all

core::array<video::IRenderTarget> AuxBuffer[2];
    
 
    // additional render target textures (defined in rtt.xml)
    core::map<core::stringw, core::array<video::IRenderTarget>> RenderTargetMap;

modifications to PostProcessManager.cpp
line 62

Code: Select all

void CPostProcessManager::SwapAuxBuffers()
{
    // swap the in and out buffers
    core::array<video::IRenderTarget> tmp = RenderTargetMap["auxIn"];
    RenderTargetMap["auxIn"] = RenderTargetMap["auxOut"];
    RenderTargetMap["auxOut"] = tmp;
}

line 87

Code: Select all

void CPostProcessManager::render(E_POSTPROCESS_EFFECT effect)
{
    if (effect<EPPE_COUNT)
    {       
        // first swap the in and out buffers
        SwapAuxBuffers();
 
        // run through the effect chain
        for (u32 i=0; i<EffectChain[effect].size(); i++)
        {
            // retrieve the post process
            IPostProcess* postProcess = EffectChain[effect][i];
            
            // bind input buffer 
            if( !postProcess->getRenderSource().empty())
            {
                 core::array<video::IRenderTarget> a = RenderTargetMap[postProcess->getRenderSource()];
                 for (int i =0; i< a.size();i++)
                 {
                 postProcess->getMaterial().setTexture(i, a[i].RenderTexture);
                 }
            }
            
 
                
 
 
            
            
            // bind output buffer
            if( !postProcess->getRenderTarget().empty())
                Device->getVideoDriver()->setRenderTarget(RenderTargetMap[postProcess->getRenderTarget()],true,true,video::SColor(40,255,255,255));
 
            // render the post process
            postProcess->render();
            
 
        }
    }
}

line 128

Code: Select all

void CPostProcessManager::update()
{
    // render the scene into the framebuffer after postprocessing
    core::array<video::IRenderTarget> a = RenderTargetMap["auxOut"];
    RenderToScreen->getMaterial().setTexture(0, a[0].RenderTexture);
    Device->getVideoDriver()->setRenderTarget(video::ERT_FRAME_BUFFER, true, true);
    RenderToScreen->render();
}

line 196

Code: Select all

void CPostProcessManager::loadRTTConfig()
{
    // create a xml reader
    io::IrrXMLReader* xmlReader = io::createIrrXMLReader("config/rtt.xml");
 
    // we'll be looking for the rendertarget tag in the xml
    const core::stringw renderTargetTag(L"RenderTarget");
 
    while(xmlReader && xmlReader->read())
    {    
        switch(xmlReader->getNodeType())
        {    
        case io::EXN_ELEMENT:
            {
                // we are in the setup section and we find a rendertarget to parse
                if (renderTargetTag.equals_ignore_case(xmlReader->getNodeName()))
                {
                    // get the rtt parameters
                    core::stringw id = xmlReader->getAttributeValueSafe("id");
                    u32 width = (u32) xmlReader->getAttributeValueAsInt("width");
                    u32 height = (u32) xmlReader->getAttributeValueAsInt("height");
                    f32 scale = (f32) xmlReader->getAttributeValueAsFloat("scale");
                    video::ECOLOR_FORMAT colorFormat[4];
                    u32 count = (u32) xmlReader->getAttributeValueAsInt("MRT");
                    colorFormat[0] = (video::ECOLOR_FORMAT) xmlReader->getAttributeValueAsInt("colorFormat");
                    colorFormat[1] = (video::ECOLOR_FORMAT) xmlReader->getAttributeValueAsInt("colorFormat1");
                    colorFormat[2] = (video::ECOLOR_FORMAT) xmlReader->getAttributeValueAsInt("colorFormat2");
                    colorFormat[3] = (video::ECOLOR_FORMAT) xmlReader->getAttributeValueAsInt("colorFormat3");
                    // set width and height of the rtt
                    if (scale > 0.0f)
                    {
                        width =  (u32) (scale * Device->getVideoDriver()->getScreenSize().Width);
                        height =  (u32) (scale * Device->getVideoDriver()->getScreenSize().Height);
                    }
                    if (width==0 || height==0)
                    {
                        width=Device->getVideoDriver()->getScreenSize().Width;
                        height=Device->getVideoDriver()->getScreenSize().Height;
                    }
 
                    core::array<video::IRenderTarget> a;
                    // add the rendertarget with its properties and store it in the render target map
                    if (count >1)
                    for(u32 i=0; i<count; i++)
                    {
                    core::stringw lid = id;
                    lid+=i;
                    video::ITexture* texture = Device->getVideoDriver()->addRenderTargetTexture(core::dimension2d<u32>(width, height), lid, colorFormat[i]);    
                    a.push_back( video::IRenderTarget(texture));
                    }
                    else
                    {
                    video::ITexture* texture = Device->getVideoDriver()->addRenderTargetTexture(core::dimension2d<u32>(width, height), id, colorFormat[0]); 
                    a.push_back( video::IRenderTarget(texture));
                    }
                    RenderTargetMap[id] =a;
                }
            }
            break;
        }
    }
    delete xmlReader;    
}

i also added this because some of my nodes render to a texture during thier animate pass or something so i though it could be usefull

Code: Select all

void CPostProcessManager::RollbackRTT()
{
    Device->getVideoDriver()->setRenderTarget(RenderTargetMap["auxOut"]);
 
}

MRT are then declared this way in the rtt.xml

Code: Select all

<RenderTarget id="auxIn" colorFormat="3" scale="1.0" MRT="2" colorFormat1="9"/>
    <RenderTarget id="auxOut" colorFormat="3" scale="1.0" MRT="2"colorFormat1="9"/>
    <RenderTarget id="coulds" colorFormat="3" scale="1.0" MRT="2"colorFormat1="6"/>

and they get named accordign to thier ID and a number so textures from auxIn will be named "auXin0" "auXin1" etc etc etc.

[tbw]

Very nice!
Your approach is the definitly the better one....

[Granyte]

The weakness i see in my technique is that it require some shader modification when the user pushes the number of texture per mrt unless we push the number of texture required for the famework to 8 and put all the texture that remain fix to the texture layer 4 and higher.(as irrlicht suport only 4 rtt we could limit the number of texture per mrt to 4)

the other weakness i see is that the depth write relie exclusively on the user unless we write a merge depth shader that would write the rttdepth to the second texture of the mrt.

the final weakness is that every shader will need to copie the content of the mrt to the other mrt

[ACE247]

I tried compiling the latest download zip of the Postprocess framework using Linux. And no matter what version of irrlicht I use, even pre 1.6.1.
I get this:

Code: Select all

irrlicht-1.8/include/irrMap.h: In member function ‘irr::core::map<const irr::core::string<char>, irr::video::ITexture*>::AccessClass& irr::core::map<const irr::core::string<char>, irr::video::ITexture*>::AccessClass::operator=(const irr::core::map<const irr::core::string<char>, irr::video::ITexture*>::AccessClass&)’:
 
irrlicht-1.8/include/irrMap.h:635:8: error: non-static reference member ‘irr::core::map<const irr::core::string<char>, irr::video::ITexture*>& irr::core::map<const irr::core::string<char>, irr::video::ITexture*>::AccessClass::Tree’, can’t use default assignment operator
 
irrlicht-1.8/include/irrMap.h:635:8: error: non-static reference member ‘const irr::core::string<char>& irr::core::map<const irr::core::string<char>, irr::video::ITexture*>::AccessClass::Key’, can’t use default assignment operator
 
PostProcessManager.cpp: In member function ‘void CPostProcessManager::SwapAuxBuffers()’:
 
PostProcessManager.cpp:53:53: note: synthesized method ‘irr::core::map<const irr::core::string<char>, irr::video::ITexture*>::AccessClass& irr::core::map<const irr::core::string<char>, irr::video::ITexture*>::AccessClass::operator=(const irr::core::map<const irr::core::string<char>, irr::video::ITexture*>::AccessClass&)’ first required here

I really have no clue how someone managed to compile this in anyway.

Also have to ask, which Source of postprocess framework are you using? I see that yours appears quite different(guessed by line numbers in granyte's post) from from the version i'm able to obtain

[Granyte]

wich compiler are you trying to use ?

[torleif]

I'm unsure about if this is my fault or well, what it could be...
Anyhow, the problem I am having is that whenever I try to include this into my project I get the following error:

Code: Select all

C:\Users\Kristoffer\Desktop\ChannelIrrlicht\include\irrMap.h|488|error: non-static reference member 'irr::core::map<irr::core::string<wchar_t, irr::core::irrAllocator<wchar_t> >, irr::video::ITexture*>& irr::core::map<irr::core::string<wchar_t, irr::core::irrAllocator<wchar_t> >, irr::video::ITexture*>::AccessClass::Tree', can't use default assignment operator|

Unsure of why this could be :/ I am trying to run this on mingw on Code::Blocks... Any ideas?

It seems gcc 4.2 doesn't like the default operator and passes the AccessClass into the = operator, instead of the pointer in the SwapAuxBuffers() function. You can get around this by changing the SwapAuxBuffers in PostProcessManager.cpp like so:

Code: Select all

void CPostProcessManager::SwapAuxBuffers()
{
    video::ITexture* texin = RenderTargetMap["auxIn"];
    video::ITexture* texout = RenderTargetMap["auxOut"];
    RenderTargetMap["auxIn"] = texout;
    RenderTargetMap["auxOut"] = texin;
}
 

It's possible it's a bug in irrlicht.

[ACE247]

Yes that certainly works. However, some of the shaders also do not want to work correctly, but thats be due to something else. I'll look into it.

[torleif]

The more I play with this framework the more I love it. This is a prime example of well written, natural, intuitive code. I big thanks to tbw.

Yes that certainly works. However, some of the shaders also do not want to work correctly, but thats be due to something else. I'll look into it.

Like other people in this thread I've ran into some problems with the OpenGL shaders, as I'm using an embedded graphics processor too. I've fixed the following shaders as they have minor issues with them (incorrect casts etc:)

• EPPE_SEPIA
• EPPE_GRAY_SCALE
• EPPE_DESATURATE
• VBLUR
• HBLUR

The following are still yet to be fixed:

• Pencil Scetch
• Scratched (compiles but doesn't work)

A note with this download is some of the shaders I've replaced with christianclavet's fix list.

Download the OpenGL shader fixes here

[Granyte]

the lastes download is suposed to have all the shaders fixed