Projection DirectX and OpenGL

[andytheb]

Hi,

I'm trying to get head tracking to work with Irrlicht (see Johnny Lee's Wii remote head tracking on youtube).
When figuring out the projection matrix I read that OpenGl and DirectX have different conventions:

OpenGL has the viewing direction in negative z direction (right handed) and projects into a left handed canonical view volume with ranges ( [-1,1], [-1,1],[-1,1] )
( see http://www.ugrad.cs.ubc.ca/~cs314/notes/ogl_vvol.html )

while DirectX has the viewing direction in positive z direction (left handed) and project into a left handed canonical view volume with ranges ( [-1,1],[-1,1],[0,1] )
( see http://www.codeguru.com/cpp/misc/misc/m ... c10123__3/ )

Irrlicht seems to use DirectX style projection matrices and accounts for the z-Flip for OpenGL in COpenGLDriver::setTransform:

Code: Select all

case ETS_PROJECTION:
		createGLMatrix(glmat, mat);
		// flip z to compensate OpenGLs right-hand coordinate system
		glmat[12] *= -1.0f;
		glMatrixMode(GL_PROJECTION);
		glLoadMatrixf(glmat);
		break;

1. The matrix element glMat[12] being (0,3) is 0 for the normal frustum projection (see links above). So this does not flip anything. Is this really correct?

2. I've tried to generate a customized frustum projection matrix, from both the DirectX and OpenGL convention, but neither works as expected on OpenGl under linux (nvidia driver).
So far I have come up with:

Code: Select all

matrix4 make_projection_matrix( float left, float right,
			float bottom, float top,
			float nearval, float farval )
{
   float x, y, a, b, c, d;
   float m[16];

   x = (2.0*nearval) / (right-left);
   y = (2.0*nearval) / (top-bottom);
   a = (right+left) / (right-left);
   b = (top+bottom) / (top-bottom);
   c = (farval+nearval) / ( farval-nearval);
   d = -(2.0*farval*nearval) / (farval-nearval);

   matrix4 M;

   m[0] = x;     
   m[1] = 0.0F;  
   m[2] = 0.0F;  
   m[3] = 0.0F;  
   
   m[4] = 0.0F;  
   m[5] = y;     
   m[6] = 0.0F;  
   m[7] = 0.0F;  

   m[8] = a;      
   m[9] = b;      
   m[10] = c;       
   m[11] = 1.0F; // originally -1.0F  

   m[12] = 0.0F;
   m[13] = 0.0F;
   m[14] = d;
   m[15] = 0.0F;
   
   M.setM(m);
   return M;
}

That gives the right x and y values from camera.getViewFrustum() (that uses ??? convention) but the z-Values are not as expected.
What strikes me, is that the OpenGL specification doesn't work at all, not even with a manual z-Flip. Does Irrlicht do anything else strange with the matrix?

3. (For the future)
Suppose I want a customized frustum projection that works both for OpenGl and DirectX, what am I to do? Is it possible for Irrlicht to translate between the two?

thanks in advance,
Andy

[vitek]

It seems to me that if you're trying to do head tracking, you would want to modify the view matrix, not the projection.

Travis

[andytheb]

Simply changing the view ( camera position and target ) works, but it does not reflect the true geometry.
If you look at your screen (being the "projection plane" in the virtual world) from an angle, it's plane is not perpendicular to the viewing direction. That's why the frustrum needs to be changed accordingly.

There is already some code due to Johnny Lee
( http://www.cs.cmu.edu/~johnny/projects/wii/ )
and I have confirmed for myself, that the frustum needs to be updated like this:

Code: Select all

matrix4 projectionMatrix = make_projection_matrix( nearPlane*( -0.5 * screenWidth - cameraPosition.X ) / d_perp,
    						       nearPlane*( 0.5 * screenWidth - cameraPosition.X ) / d_perp,
    						       nearPlane*( -0.5 * screenHeight - cameraPosition.Y ) / d_perp ,
    						       nearPlane*( 0.5 * screenHeight - cameraPosition.Y  ) / d_perp,
    						       nearPlane, farPlane);

/* this method is adapted from mesalib and is in accordance with all information about opengl frustum projection matrices I could find on the net */
matrix4 make_projection_matrix( float left, float right,
			float bottom, float top,
			float nearval, float farval )
{
   float x, y, a, b, c, d;
   float m[16];

   x = (2.0*nearval) / (right-left);
   y = (2.0*nearval) / (top-bottom);
   a = (right+left) / (right-left);
   b = (top+bottom) / (top-bottom);
   c = -(farval+nearval) / ( farval-nearval);
   d = -(2.0*farval*nearval) / (farval-nearval);

  matrix4 M;

   m[0] = x;     
   m[1] = 0.0F;  
   m[2] = 0.0F;  
   m[3] = 0.0F;  
   
   m[4] = 0.0F;  
   m[5] = y;     
   m[6] = 0.0F;  
   m[7] = 0.0F;  

   m[8] = a;      
   m[9] = b;      
   m[10] = c;       
   m[11] = -1.0F;   

   m[12] = 0.0F;
   m[13] = 0.0F;
   m[14] = d;
   m[15] = 0.0F;
   
   M.setM(m);
   return M;
}

Even though make_projection_matrix is taken from mesalib it simply does not yield what would I would expected. Nothing is displayed at all (even changing the viewing direction from z to -z ).

Is Irrlicht messing with the projection matrix in any form before setting it in COpenGlDriver::setTransform(...) ???

Andy

[andytheb]

Ok, I finally managed to get it working correctly, at least when the screen normal is perpendicular to the z axis, and the virtual screen is centered around
( screenWidth / 2, screenHeight / 2).

The effect is really impressive, so check out my quick and dirty code release:
http://www.burtzlaff.de/trackhead.html

The projection matrix was determined by trial and error and is still not really perfect.
It has the following form:

Code: Select all

matrix4 make_projection_matrix( float left, float right,
			float bottom, float top,
			float nearval, float farval )
{
   float x, y, a, b, c, d;
   float m[16];

   x = (2.0*nearval) / (right-left);
   y = (2.0*nearval) / (top-bottom);
   a = (right+left) / (right-left);
   b = (top+bottom) / (top-bottom);
   c = -(farval+nearval) / ( farval-nearval);
   d = -(2.0*farval*nearval) / (farval-nearval);

   matrix4 M;

   m[0] = x;     
   m[1] = 0.0F;  
   m[2] = 0.0F;  
   m[3] = 0.0F;  
   
   m[4] = 0.0F;  
   m[5] = y;     
   m[6] = 0.0F;  
   m[7] = 0.0F;  

   m[8] = a;      
   m[9] = b;      
   m[10] = -c;   // original Opengl +c     
   m[11] = 1.0F; // original Opengl -1.0F  

   m[12] = 0.0F;
   m[13] = 0.0F;
   m[14] = d;
   m[15] = 0.0F;
   
   M.setM(m);
   return M;
}

When using the matrix originally derived for opengl the z axis seems to be strangely "invert", meaning that the depth (at least) is not calculated correctly. Can anybody elaborate on what Irrlicht actually does with the projection matrix?

Andy

[tonic]

I don't know if this helps you at all, but I also had a problem with the projection matrix when setting up custom 2d orthogonal view. I describe the problem and a workaround here:
http://irrlicht.sourceforge.net/phpBB2/ ... 329#151329

[cheeks]

Hi, thanks for that I was trying to translate that matrix with no luck

Do u happen to find a way to solve the Z invert thing?

Thanks

[FreakNigh]

Huh I actually came across your project when I was putting together mine... Didn't realize you already had it all done. You need to make a video for it. Cept I use just the head without any markers / dots on it and my class / lib is more designed for rapid and simple use.

This is left handed frustum builder I used which is alittle different from yours.

Code: Select all

float* CvIrrCamController::create_glfrustumLH(float left, float right, float bottom, float top, float nearZ, float farZ, float *m)
{
	float x = (2.0F*nearZ) / (right-left);
	float y = (2.0F*nearZ) / (top-bottom);
	float a = (right+left) / (right-left);
	float b = (top+bottom) / (top-bottom);
	float c = -(farZ+nearZ) / ( farZ-nearZ);
	float d = -(2.0F*farZ*nearZ) / (farZ-nearZ);

#define M(row,col)  m[col*4+row]
	M(0,0) = x;     M(0,1) = 0.0F;  M(0,2) = a;      M(0,3) = 0.0F;
	M(1,0) = 0.0F;  M(1,1) = y;     M(1,2) = b;      M(1,3) = 0.0F;
	M(2,0) = 0.0F;  M(2,1) = 0.0F;  M(2,2) = -c;      M(2,3) = d;
	M(3,0) = 0.0F;  M(3,1) = 0.0F;  M(3,2) = 1.0F;  M(3,3) = 0.0F;
#undef M
	
	return m;
}

[kparamas]

Hi andytheb,
Please let me know what is the value of d_perp.

The example code uses OpenCV. Is there a code that uses Nintendo Wiimotes for headtracking.
I am having hard time setting up the projection matrix while using Irrlicht.
I have a working code in OpenGL.

Thanks,
Guru

Simply changing the view ( camera position and target ) works, but it does not reflect the true geometry.
If you look at your screen (being the "projection plane" in the virtual world) from an angle, it's plane is not perpendicular to the viewing direction. That's why the frustrum needs to be changed accordingly.

There is already some code due to Johnny Lee
( http://www.cs.cmu.edu/~johnny/projects/wii/ )
and I have confirmed for myself, that the frustum needs to be updated like this:

Code: Select all

matrix4 projectionMatrix = make_projection_matrix( nearPlane*( -0.5 * screenWidth - cameraPosition.X ) / d_perp,
    						       nearPlane*( 0.5 * screenWidth - cameraPosition.X ) / d_perp,
    						       nearPlane*( -0.5 * screenHeight - cameraPosition.Y ) / d_perp ,
    						       nearPlane*( 0.5 * screenHeight - cameraPosition.Y  ) / d_perp,
    						       nearPlane, farPlane);

/* this method is adapted from mesalib and is in accordance with all information about opengl frustum projection matrices I could find on the net */
matrix4 make_projection_matrix( float left, float right,
			float bottom, float top,
			float nearval, float farval )
{
   float x, y, a, b, c, d;
   float m[16];

   x = (2.0*nearval) / (right-left);
   y = (2.0*nearval) / (top-bottom);
   a = (right+left) / (right-left);
   b = (top+bottom) / (top-bottom);
   c = -(farval+nearval) / ( farval-nearval);
   d = -(2.0*farval*nearval) / (farval-nearval);

  matrix4 M;

   m[0] = x;     
   m[1] = 0.0F;  
   m[2] = 0.0F;  
   m[3] = 0.0F;  
   
   m[4] = 0.0F;  
   m[5] = y;     
   m[6] = 0.0F;  
   m[7] = 0.0F;  

   m[8] = a;      
   m[9] = b;      
   m[10] = c;       
   m[11] = -1.0F;   

   m[12] = 0.0F;
   m[13] = 0.0F;
   m[14] = d;
   m[15] = 0.0F;
   
   M.setM(m);
   return M;
}

Even though make_projection_matrix is taken from mesalib it simply does not yield what would I would expected. Nothing is displayed at all (even changing the viewing direction from z to -z ).

Is Irrlicht messing with the projection matrix in any form before setting it in COpenGlDriver::setTransform(...) ???

Andy

[devsh]

I used to have problems in setting my custom proj matrix in ogl (i think it applies to view too and any other matrices)

when you set it using the driver->setTransform you need to pass a matrix with inverted X, just simply matrix.setTransform(irr::core::vector3df(-matrix.getTransform.X,matrix.getTransform.Y, matrix.getTransform.Z))

but when passing to a shader you need to pass a matrix with not inverted X