[SOLVED] RenderTargetTexture strange Y-flip

[phixuannhat]

I'm testing RTT feature of Irrlicht to do some post-possessing fx and encounter this strange behavior of RTT.

This is normal render with code:

Code: Select all

driver->beginScene();

driver->draw2DImage(BG, rect<s32>(0,0,renderSize.Width,renderSize.Height), rect<s32>(0,0,1024,1024));
smgr->drawAll();

driver->endScene();

And this is when RTT is used, notice the background is flipped while the model is not

Code: Select all

driver->beginScene();

driver->setRenderTarget(scr);
driver->draw2DImage(BG, rect<s32>(0,0,renderSize.Width,renderSize.Height), rect<s32>(0,0,1024,1024));
smgr->drawAll();

scrQuad.setTexture(scr);
scrQuad.render();

driver->endScene();

Below is how I implement the ScreenQuad class, modified from slavik262's ScreenQuad in http://irrlicht.sourceforge.net/phpBB2/ ... processing

Code: Select all

class ScreenQuad
{
	IVideoDriver *driver;

	S3DVertex vertices[4];
	u16 indices[6];
	SMaterial mat;

public:
	ScreenQuad(IVideoDriver *_driver)
	{
		driver = _driver;

		vertices[0] = S3DVertex(vector3df(-1, 1, 0), vector3df(0, 0, 1), SColor(255,255,255,255), vector2df(0, 0));
		vertices[1] = S3DVertex(vector3df(1, 1, 0), vector3df(0, 0, 1), SColor(255,255,255,255), vector2df(1, 0));
		vertices[2] = S3DVertex(vector3df(1, -1, 0), vector3df(0, 0, 1), SColor(255,255,255,255), vector2df(1, 1));
		vertices[3] = S3DVertex(vector3df(-1, -1, 0), vector3df(0, 0, 1), SColor(255,255,255,255), vector2df(0, 1));

		indices[0] = 0;
		indices[1] = 1;
		indices[2] = 3;
		indices[3] = 1;
		indices[4] = 2;
		indices[5] = 3;

		mat.BackfaceCulling = false;
		mat.ZWriteEnable = false;
		mat.Lighting = false;
	}

	inline void setTexture(ITexture *_tex, int id=0)
	{
		mat.TextureLayer[id].Texture = _tex;
		mat.TextureLayer[id].TextureWrapU = ETC_CLAMP_TO_EDGE;
		mat.TextureLayer[id].TextureWrapV = ETC_CLAMP_TO_EDGE;
	}

	inline void setMaterialType(E_MATERIAL_TYPE mt)
	{
		mat.MaterialType = mt;
	}

	void render(ITexture *rt=0)
	{
		matrix4 oldWorld = driver->getTransform(ETS_WORLD);
		matrix4 oldView = driver->getTransform(ETS_VIEW);
		matrix4 oldProj = driver->getTransform(ETS_PROJECTION);

		driver->setRenderTarget(rt);
		driver->setMaterial(mat);

		driver->setTransform(ETS_WORLD, IdentityMatrix);
		driver->setTransform(ETS_VIEW, IdentityMatrix);
		driver->setTransform(ETS_PROJECTION, IdentityMatrix);
		driver->drawIndexedTriangleList(vertices, 4, indices, 2);

		driver->setTransform(ETS_WORLD, oldWorld);
		driver->setTransform(ETS_VIEW, oldView);
		driver->setTransform(ETS_PROJECTION, oldProj);
	}

};

anyone encounter this before? please advise, thanks

[slavik262]

That's quite strange. At first glance, I don't know why it's causing problems.

On a side note though, some of the stuff in the quad class isn't necessary. If you're basing it off an old version of my quad class, some of the code was in there because I was having problems and wasn't sure what the root cause is. For example, every time you set a texture, you don't need to set the wrap properties to CLAMP. This only needs to be done once (in the constructor). Similarly, getting the current matrices (oldWorld, oldView, oldProj) and re-applying them after isn't necessary. That was just the product of me trying to fix some bug.

Also, if you're not aware, putting a function in the class body (as opposed to a .cpp file elsewhere) automatically inlines it.

If you're curious, he're my current ScreenQuad class.

Code: Select all

namespace Graphics
{
	class ScreenQuad
	{
	public:
		ScreenQuad()
		{
			vd = GlobalContext::DeviceContext.GetVideoDriver();

			const SColor white(255, 255, 255, 255);
			vertices[0].Pos = vector3df(-1, 1, 0);
			vertices[0].TCoords = vector2df(0, 0);
			vertices[0].Color = white;
			vertices[1].Pos = vector3df(1, 1, 0);
			vertices[1].TCoords = vector2df(1, 0);
			vertices[1].Color = white;			
			vertices[2].Pos = vector3df(1, -1, 0);
			vertices[2].TCoords = vector2df(1, 1);
			vertices[2].Color = white;
			vertices[3].Pos = vector3df(-1, -1, 0);
			vertices[3].TCoords = vector2df(0, 1);
			vertices[3].Color = white;
			indices[0] = 0;
			indices[1] = 1;
			indices[2] = 3;
			indices[3] = 1;
			indices[4] = 2;
			indices[5] = 3;

			mat.BackfaceCulling = false;
			mat.Lighting = false;
			mat.ZBuffer = video::ECFN_ALWAYS;
			mat.ZWriteEnable = false;
			for(u32 c = 0; c < MATERIAL_MAX_TEXTURES; c++)
			{
				mat.TextureLayer[c].TextureWrapU = video::ETC_CLAMP;
				mat.TextureLayer[c].TextureWrapV = video::ETC_CLAMP;
			}
		}

		SMaterial& GetMaterial() { return mat; }

		//Set the texture to render with the quad
		void SetTexture(ITexture* __restrict tex, u32 layer = 0)
		{
			mat.TextureLayer[layer].Texture = tex;
		}

		ITexture* GetTexture(u32 layer = 0) { return mat.TextureLayer[layer].Texture; }

		void SetMaterialType(E_MATERIAL_TYPE mt) { mat.MaterialType = mt; }

		void Render(bool setRTToFrameBuff = true)
		{
			if(setRTToFrameBuff)
				vd->setRenderTarget(video::ERT_FRAME_BUFFER);
			vd->setMaterial(mat);
			vd->setTransform(ETS_WORLD, core::IdentityMatrix);
			vd->setTransform(ETS_VIEW, core::IdentityMatrix);
			vd->setTransform(ETS_PROJECTION, core::IdentityMatrix);
			vd->drawIndexedTriangleList(vertices, 4, indices, 2);
		}

		void Render(ITexture* rt)
		{
			vd->setRenderTarget(rt);
			vd->setMaterial(mat);
			vd->setTransform(ETS_WORLD, core::IdentityMatrix);
			vd->setTransform(ETS_VIEW, core::IdentityMatrix);
			vd->setTransform(ETS_PROJECTION, core::IdentityMatrix);
			vd->drawIndexedTriangleList(vertices, 4, indices, 2);
		}

	protected:
		S3DVertex vertices[4];
		u16 indices[6];
		SMaterial mat;

		IVideoDriver* vd;
	};

} //end namespace graphics

[phixuannhat]

The problem is still there when I use your current ScreenQuad. However, when I change the driver from OpenGL to DX9, all is fixed. As I prefer using OpenGL, any thoughts on how fix this bug on OpenGL?

Btw, I searched around forum and found that others had some kind of flipped texture with RTT before when using Irrlicht 1.4 or so. But I'm using the latest (1.7.1), so I think their bugs are diff from mine.

[BlindSide]

This is a well known issue. RTT in OpenGL are flipped, so just flip your texcoords in the shaders (1.0 - texcoord.y) when you're working with them.

[Mel]

Is there any way of solving that inside the Open GL driver?

[hybrid]

Not in general. For texture access in software we've solved this by copying the texture (and thereby flipping it). But for direct access on the GPU, you will have to keep this in mind.

[phixuannhat]

This is a well known issue. RTT in OpenGL are flipped, so just flip your texcoords in the shaders (1.0 - texcoord.y) when you're working with them.

Then both the model and background are flipped together, but in this case, only the background is flipped. That's why I find it strange

[phixuannhat]

More observations:

- When calling another draw2DImage to draw the RT, both model and bg appear normal (non-flipped)

- I set all material types as EMT_SOLID (for both the model and ScreenQuad) to eliminate bugs, if any, from shaders (as I'm using IrrCg for shaders). The bg's still flipped while the model isnt.

[hybrid]

We have had some problems with the 2d methods not properly setting or resetting the rttflag. Thisis used inside the draw2d methods to flip RTTs. Maybe try the same code with a more recent SVN version, could already be fixed.

[sudi]

well a really easy fix for this is after rendering to a rtt or rendering from a rtt...ah anyway before you use draw2d methods or the gui call

Code: Select all

driver->setMaterial(driver->get2DMaterial());

That fixes all problems.

[phixuannhat]

The problem is fixed using Sudi's tip and fliped uv coord when driver is OGL. Thank you all