[SOLVED] screen space decals

[arnir]

hello, Iam trying to implement screen space decals system.
here is paper Iam using as reference: http://www.slideshare.net/blindrenderer ... e-14699854

main principle is:
rasterize unit cube on the place where decal should be
reconstruct view pixel position using depth texture
and then multiply it by inverse world-view matrix, to get back to the model space of unit cube, whereX and Y of this position can then be instantly used as UV coordinates for texture.

simple but i still can't get it right
texture is stretching while I moving camera and Iam sure that inverse world matrix is somehow bad.

before I start showing the code, I want to know something:

is it true that:
M * v = v'
and
Minv * v' = v
????
in other words:
if I multiply vector v by matrix M, I got vector v'
if I mutiply vector v' by inverse matrix M, I should get v again?

if is it true, why this test doesn't work?

Code: Select all

matrix4 projection = m_pDevice->getVideoDriver()->getTransform(ETS_PROJECTION);
matrix4 view = m_pDevice->getVideoDriver()->getTransform(ETS_VIEW);
matrix4 model = m_pDevice->getVideoDriver()->getTransform(ETS_WORLD);
 
matrix4 mvp = model * view * projection;
 
vector3df test(150, 100, 50);
printf("initial position: %f %f %f\n", test.X, test.Y, test.Z);
 
mvp.transformVect(test);
printf("after mpv: %f %f %f\n", test.X, test.Y, test.Z);
 
core::matrix4 inv_mvp;
 
if (!mvp.getInverse(inv_mvp))
{
    printf("getInverse fail\n");
}
 
inv_mvp.transformVect(test);
printf("after inverse mpv: %f %f %f\n", test.X, test.Y, test.Z);

the code outputs this:

Code: Select all

initial position: 150.000000 100.000000 50.000000
after mpv: 68496.304688 -356109.750000 1220350.125000
after inverse mpv: 1372.228882 -3210.327148 1.000000

[smso]

The transformVect() function is not the same as "multiplication of vector by a matrix". Think about this:
matrix4 is 4x4 and vector3df is 3x1 (or 1x3 ?)...

Regards,
smso

[hendu]

No, the function does correspond to the multiplication (the fourth component of a 4-wide vector is simply ignored), and OP's case should indeed work.

[smso]

No, the function does correspond to the multiplication (the fourth component of a 4-wide vector is simply ignored), and OP's case should indeed work.

Sorry, my fault.
I looked at the source code "matix4.h" and confirm hendu's saying.

Regards,
smso

[arnir]

ok, forget about math above, here's the code:

decal.vert

Code: Select all

uniform vec3 VertexFarLeftUp;
 
varying vec4 ScreenPos;
varying vec3 FarLeftUp;
varying float FarRightUpX;
varying float FarLeftDownY;
 
void main()
{
    ScreenPos = ftransform();
    gl_Position = ScreenPos;
    FarLeftUp = VertexFarLeftUp;
    FarLeftDownY = -FarLeftUp.y;
    FarRightUpX = -FarLeftUp.x;
}

decal.frag

Code: Select all

uniform sampler2D firstPassTexture;
uniform sampler2D decalTexture;
 
uniform mat4 InvWorldView;
 
varying vec4 ScreenPos;
varying vec3 FarLeftUp;
varying float FarRightUpX;
varying float FarLeftDownY;
 
 
void main()
{
    vec4 projCoord = ScreenPos / ScreenPos.w;
    projCoord += vec4(1.0);
    projCoord *= 0.5;
    projCoord.xy = clamp(projCoord.xy, 0.001, 0.999);
 
    //get depth
    float vDepth = texture2D(firstPassTexture, projCoord.xy).a;
 
    //reconstruct view pixel position
    vec3 vProjPos = vec3(mix(FarLeftUp.x, FarRightUpX, projCoord.x),
                        mix(FarLeftDownY, FarLeftUp.y, projCoord.y),
                        FarLeftUp.z);
    vec3 scenePosView = vec3(vProjPos * vDepth);
 
    vec4 position = InvWorldView * vec4(scenePosView,1.0);
    
    vec2 uv = position.xy;
    uv += 0.5;
 
    gl_FragColor = texture2D(decalTexture, uv);
}
 

decal shader callback

Code: Select all

class CDecalCallback : public IShaderConstantSetCallBack
{
public:
    CDecalCallback(IEngineDevice *device)
    {
        m_pDevice = device;
    }
 
    virtual void OnSetConstants(irr::video::IMaterialRendererServices* services, irr::s32 userData)
    {
        int tex0 = 0;
        int tex1 = 1;
 
        services->setPixelShaderConstant("firstPassTexture", (const float*)&tex0, 1);
        services->setPixelShaderConstant("decalTexture", (const float*)&tex1, 1);
 
        irr::scene::ICameraSceneNode* cam = m_pDevice->getSceneManager()->getActiveCamera();
        irr::core::matrix4 viewMat = m_pDevice->getVideoDriver()->getTransform(ETS_VIEW);
        irr::core::matrix4 projMat = m_pDevice->getVideoDriver()->getTransform(ETS_PROJECTION);
 
 
        irr::core::vector3df farLeftUp= cam->getViewFrustum()->getFarLeftUp();
        viewMat.transformVect(farLeftUp);
        services->setVertexShaderConstant("VertexFarLeftUp", (irr::f32*)&farLeftUp, 3);
 
 
        core::matrix4 worldViewProj(m_pDevice->getVideoDriver()->getTransform(video::ETS_PROJECTION));
        worldViewProj *= m_pDevice->getVideoDriver()->getTransform(video::ETS_VIEW);
        worldViewProj *= m_pDevice->getVideoDriver()->getTransform(video::ETS_WORLD);
 
        core::matrix4 finalMat(worldViewProj.getTransposed());
 
        matrix4 invFinalMat;
        finalMat.getInverse(invFinalMat);
 
        services->setPixelShaderConstant("InvWorldView", invFinalMat.pointer(), 16);
    }
 
private:
    IEngineDevice *m_pDevice;
};

example usage:

Code: Select all

 
ISceneNode *DecalBox;
s32 Decal;
 
Decal = driver->getGPUProgrammingServices()->addHighLevelShaderMaterialFromFiles(
        "./data/shaders/decal.vert", "main", EVST_VS_1_1,
        "./data/shaders/decal.frag", "main", EPST_PS_1_1,
        new CDecalCallback(device),
        EMT_TRANSPARENT_ALPHA_CHANNEL);
 
//must be unit cube
DecalBox = device->getSceneManager()->addCubeSceneNode(1.0f);
DecalBox->setMaterialFlag(irr::video::EMF_BACK_FACE_CULLING, false);
DecalBox->setMaterialFlag(irr::video::EMF_FRONT_FACE_CULLING, true);
DecalBox->setMaterialType((E_MATERIAL_TYPE)Decal);
 
//set texture containing depth
DecalBox->getMaterial(0).setTexture(0, firstPass);
 
//set decal texture
DecalBox->getMaterial(0).setTexture(1, decalTexture);

result:
(stretching while I look around)


result when I remove line

Code: Select all

worldViewProj *= m_pDevice->getVideoDriver()->getTransform(video::ETS_VIEW);

from callback code. I know it's bad idea but take a look now:


looking good, BUT as expected due missing view inverse matrix, decal keep following me while I look around (but follow geometry "under the decal" - it's good!)

any hints how to fix this?

I've tried many things around changing order of: making inverse matrix and doing transpose but no success

[hendu]

I pasted your code into a test app, and got correct results (same vector before multiply, and after inversion).

Your code looks correct, so perhaps it's your version of irr. Old version?

[arnir]

If I made the test outside game loop, it works too. it's because world, view and projection matrices are not used, they are all zeroed. but when I try it after smgr->drawAll(), in this time all matrices are corectly set by active camera, the test will fail

[hendu]

My test was with properly set matrices. If they had been identities, the middle vector would also have been the same; it wasn't.

[arnir]

I've pasted my test after: smgr->drawAll() to terrain rendering irrlicht example, and in all versions the test fail :-/

[smso]

@arnir:

... (but follow geometry "under the decal" - it's good!)...

Then you should update invFinalMat only when firing the gun,
i.e. calculate invFinalMat outside the shader callback and send it to the shader callback at the proper moment.

Regards,
smso

[arnir]

yeah I fix it!
Irrlicht and everything was all right, it was my mistake. I didn't realized that if you deproject view-space position to the 2D model space (interested only in X Y, to be used as texture coordinates), you lost all transformations that you made in world-space (rotation!). and what happen if you look at 2D, xy plane from a side? look at the previous screenshots (the first one)

[arnir]

for those who are interested in:
if you want to make it right, you cannot use the same cube(scene node) all the time, you must provide cube with verticies, that if shader deproject these verticies to the model space, XY plane will be the one you're facing to, when you creating the decal.