Real Motion Blur PS problem

[stefbuet]

Hi,

I'm trying to create a real motion blur pixel shader effect in glsl, following the GPU gems III book (chapter 27) as a post process effect using depth buffer.
However, I can't get what I would like to.

I first draw into an RTT the mesh depth map with a black/white gradient.
Then I draw the standart scene (no shader) into another RTT.
Then with the motion blur shader I calculate each pixel position, and velocity.
With the same shader, I blur the scene with directionnal bluring.

The depth RTT is OK, but the final result is bad, here is the result :
thumbnails : standart and depth RTTs:



I don't know where is the problem

Here is the code I created to [try to] make this effect:

depth_vs.vert:

Code: Select all

uniform mat4 mWorldViewProj;
uniform float MaxD;

void main() {
	vec4 tPos = ftransform();
	gl_Position = tPos;
	gl_TexCoord[0] = vec4(MaxD, tPos.y, tPos.z, tPos.w);
	gl_TexCoord[1].xy = gl_MultiTexCoord0.xy;
}

depth_ps.frag:

Code: Select all

void main() {
	vec4 vInfo = gl_TexCoord[0];
	float depth = vInfo.z / vInfo.x;
      gl_FragColor = vec4(depth, depth, depth, 0.0);
}

motionBlur_vs.vert:

Code: Select all

varying vec2 vTexCoord;

void main() {
    vTexCoord = gl_MultiTexCoord0.xy;
	gl_Position = ftransform();
}

motionBlur_ps.frag:

Code: Select all

uniform sampler2D depthTexture;
uniform sampler2D sceneTexture;
uniform mat4 mPreviousWorldViewProj;
uniform mat4 mWorldViewProjInversed;
varying vec2 vTexCoord;

void main() {
    
    //calculate pixel position
    float zOverW=texture2D(depthTexture, vTexCoord);
    vec4 H=float4(vTexCoord.x*2-1,(1-vTexCoord.y)*2-1, zOverW,1);
    vec4 D=H*mWorldViewProjInversed;
    vec4 worldPos=D/D.w;

    //calculate pixel velocity
    vec4 currentPos=H;
    vec4 previousPos=worldPos*mPreviousWorldViewProj;
    previousPos/=previousPos.w;
    vec2 velocity=(currentPos-previousPos)/2.f;
	
	float n=sqrt(pow(velocity.x,2)+pow(velocity.y,2));
	//gl_FragColor=vec4(H,H,H,H);

    //performing motion blur
    vec4 color=texture2D(sceneTexture, vTexCoord);
    vTexCoord.xy+=velocity;
    for(int i=1; i<4; ++i, vTexCoord.xy+=velocity) {
        vec4 currentColor=texture2D(sceneTexture, vTexCoord);
        color+=currentColor;
    }
    vec4 finalColor=color/4.f;

    //update the data
    gl_FragColor = texture2D(depthTexture,vTexCoord);

   
}

And the main file to use all shaders:
(you will need a mesh.3df and tex.png to run this exemple...)

main.cpp:

Code: Select all

/*
  Name: Real motion blur pixel shader
  Author: Stefbuet
  Date: 12/03/10
*/

#include <cstdlib>
#include <iostream>
#include <IRR/irrlicht.h>

using namespace irr;
using namespace scene;
using namespace video;

IrrlichtDevice *device;
IVideoDriver *driver;
ISceneManager *smgr;
IGPUProgrammingServices *gpu;

s32 depthShader;
s32 motionBlurShader;

IMeshSceneNode *node;
ICameraSceneNode *cam;

core::matrix4 worldViewProj, lastWorldViewProj, worldViewProjInv;
f32 FarLink;

ITexture *depthBufferRTT, *sceneRTT;

class depthShaderCB : public IShaderConstantSetCallBack {
    public:
    virtual void OnSetConstants(IMaterialRendererServices *services, s32 userData) {
	
		services->setVertexShaderConstant("mWorldViewProj", worldViewProj.pointer(), 16);
		
		FarLink=500.0;
		services->setVertexShaderConstant("MaxD", &FarLink, 1);
	}
};

class motionBlurShaderCB : public IShaderConstantSetCallBack {
    public:
    virtual void OnSetConstants(IMaterialRendererServices *services, s32 userData) {
    
        FarLink=500.0;
		services->setVertexShaderConstant("MaxD", &FarLink, 1);
		
		services->setVertexShaderConstant("mWorldViewProj", worldViewProj.pointer(), 16);
    
        worldViewProjInv=worldViewProj;
        worldViewProjInv.makeInverse();
        services->setPixelShaderConstant("mWorldViewProjInversed", worldViewProjInv.pointer(), 16);
         
        services->setPixelShaderConstant("mPreviousWorldViewProj", lastWorldViewProj.pointer(), 16);
        
        int depthAddress=1, sceneAddress=2;
        services->setPixelShaderConstant("depthTexture", (float*)(&depthAddress),1);
        services->setPixelShaderConstant("sceneTexture", (float*)(&sceneAddress),1);
        
       
    }
};

void updateMatrix() {
    //update the world matrix projection view for depth shader pass:
    worldViewProj = driver->getTransform(video::ETS_PROJECTION);			
    worldViewProj *= driver->getTransform(video::ETS_VIEW);
    worldViewProj *= driver->getTransform(video::ETS_WORLD);
}

depthShaderCB *depthCB;
motionBlurShaderCB *motionBlurCB;

int main(int argc, char *argv[]) {
    
    IrrlichtDevice *device =createDevice( video::EDT_OPENGL, core::dimension2d<u32>(512,512), 16,false, false, false, 0);
    driver = device->getVideoDriver();
    smgr = device->getSceneManager();
    gpu=driver->getGPUProgrammingServices();
    
    cam=smgr->addCameraSceneNodeFPS();
    cam->setPosition(core::vector3df(400,200,400));
    
    node=smgr->addMeshSceneNode(smgr->getMesh("mesh.3ds"));
    node->setMaterialFlag(EMF_LIGHTING, false);
    node->setMaterialTexture(0, driver->getTexture("tex.png"));
    
    depthBufferRTT=driver->addRenderTargetTexture(core::dimension2d<u32>(512,512),"test", ECF_A8R8G8B8);
    sceneRTT=driver->addRenderTargetTexture(core::dimension2d<u32>(512,512),"test2", ECF_A8R8G8B8);
    
    depthCB=new depthShaderCB();
    depthShader=gpu->addHighLevelShaderMaterialFromFiles("depth_vs.vert", "vertexMain", EVST_VS_1_1,"depth_ps.frag", "pixelMain", EPST_PS_1_1,depthCB, EMT_SOLID);
   
    motionBlurCB=new motionBlurShaderCB();
    motionBlurShader=gpu->addHighLevelShaderMaterialFromFiles("motionBlur_vs.vert", "vertexMain", EVST_VS_1_1,"motionBlur_ps.frag", "pixelMain", EPST_PS_1_1,depthCB, EMT_SOLID);
   
   
    while(device->run()) {
    
        driver->beginScene(true, true, SColor(0xff999999));
        
        
        
        //update the world matrix projection view for depth shader pass and keep the old one to process motion blur:
        lastWorldViewProj=worldViewProj;
        updateMatrix();
        
        //render a depth texture from the object:
        driver->setRenderTarget(depthBufferRTT, true, true, SColor(0xffffffff));
        node->setMaterialType((E_MATERIAL_TYPE)depthShader);
        smgr->drawAll();
        driver->setRenderTarget(0, false, false);
        
        
        
        //render the normal scene to an RTT texture to get its pixels in motion blur shader:
        node->setMaterialType(EMT_SOLID);
        //node->setMaterialTexture(0, driver->getTexture("tex.png"));
        driver->setRenderTarget(sceneRTT, true, true, SColor(0xffffffff));
        smgr->drawAll();
        driver->setRenderTarget(0, false, false, SColor(0xffffffff));
        
        //render the object with motion blur:
        node->setMaterialTexture(1, depthBufferRTT);
        node->setMaterialTexture(2, sceneRTT);
        node->setMaterialType((E_MATERIAL_TYPE)motionBlurShader);
        
        
        smgr->drawAll();
        
        driver->draw2DImage(sceneRTT, core::rect<s32>(0,0,128,128), core::rect<s32>(0,0,512,512));
        driver->draw2DImage(depthBufferRTT, core::rect<s32>(128+1,0,128+128+1,128), core::rect<s32>(0,0,512,512));
        
        
        driver->endScene();
    
    }
    
    return EXIT_SUCCESS;
}

Alright, do you know why the final mesh is all brown colored ?

Stef.

[stefbuet]

Well, the mesh was brown because variables weren't sent to the shader. In fact I put the depth shader callback as argument in the motion blur shader creation call instead of the motion blur one.

So now it's okay, however the mesh is... Well, not as what I exepted!
I added a skydome with the depth/motion blur shaders to see the effect more easyly, but there is still no effect...



I'm not sure, but I think I should do the bluring in another pass:
I calculate the depth RTT in a first shader.
I calculate the vertex velocity and render it to another RTT in a sec shader.
Finaly I blur the scene with a third shader sending him the velocity RTT...

Any advice ?

[stefbuet]

Alright.
I'm now trying, before doing velocity calculation, to do a shader for post proc which is able to make some directionnal blur with a velocity texture, to the current scene render.
Here is a first try, with a const speed everywhere. Next step is to make this blurred from a velocity texture...



Althrought, I'm not really OK with vertex velocity calculation...
Last tests I did where not so great... I had trouble with Irrlicht matrices.

[Lonesome Ducky]

I'm working on a motion blur shader too, although I'm doing an image based accumulation type thing. Pretty interesting to see a more advanced method being done. I'll try taking a look at some of the documentation on this type and try to help you, but I can't promise anything cuz shaders usually go right over my head

[Brainsaw]

This is cool. Would be nice if you got it working and created an extension that others can use (me for example ). I guess it would add some thrill to my game.

[stefbuet]

I would like to compute pixel velocity.
I mean vertex velocity and then it would be interpolated to the pixel shader, then into an RTT for the bluring shader.

However, I can't manage to compute that vertex velocity.
I just pass as parameter the world view current and previous matrix to get vertex velocity, like this:

VS:

Code: Select all

uniform mat4 previousProj;
uniform mat4 currentProj;
varying vec4 velocity;

void main() {

    //calculate vertex velocity, will be interpolated to the PS
    vec4 currentPos=currentProj*gl_Vertex;
    vec4 previousPos=previousProj*gl_Vertex;
    velocity=currentPos-previousPos;

    gl_Position=ftransform();


}

PS:

Code: Select all

varying vec4 velocity;

void main() {

    vec4 v=velocity;
    vec4 color=vec4(velocity.xy,0,0);
    gl_FragColor=color;

}

What's wrong ?
velocity=vec(0,0,0,0)....

[sudi]

take a look at this article: http://victorporof.wordpress.com/2010/0 ... tion-blur/

[stefbuet]

I've looked at this article. It's using the gpu gems III way, in fact it's exactly the same code. It would have helped if the source were released, but it's not the case. anyway thx. I looked for almost 1 week everywere and found motion blur from directX9 SDK, a Ogre based motion blur shader, and a pdf paper about doing motion blur. All of them are using 2 differents way to do it, but pretty similar.

Now the little thing which bother me is that I just can't calculate the vertex velocity in my shader, and it should be really easy to do. I give as parameter to the shader the current WorldViewProj, the previous frame WorldViewProj matrices, and then calculate the position, the velocity of each vertex...
The velocity map returned is just black, with blinking red colors (x) and green (y) and I can see the mesh on this texture when it's blinking... but that's definitly not what I exept.

I have to do some tests, but it seems that my Irrlicht matrix are broken

[sudi]

no they are not...but when u transform by invers WorldProjView you will get modelspace and that doesn't change. use InversProjView

[heda]

Sorry to dig up such an old post, but I am experiencing exactly the same problems. Did you, or anyone else, manage to get the GPU Gems 3, Chap 27 motion blur shader to work with Irrlicht? I am very curious how! Thanks!

[stefbuet]

I don't have the code here BUT:
I computed the old 3D world pixel position with a depth buffer using the XEffect method (viewport ray computation, see XEffect thread with water shader) and then projected it into screen space using the previous projection matrix.
Then I blurred the pixel taking into account the delta of 2D screen space pixel positions. The XEffect method to compute the 3D position isn't that precise so sometimes it blurs the scene a little bit strangely when too close, otherwise it's ok.