Why does the clipping plane change the projection matrix?

[Guest]

Hi,

today I had a very strange problem with my z-buffer. Normally I use OpenGL, and it seems as if it used a 24bit z-buffer. Now I tried DirectX, which must use a 16bit z-buffer if I tell it to do so. 24bit were enough, 16bit resulted in problems (strange artefacs)...

So I wanted to change the near clipping plane, so that the 16 bits are used for the more "important" part of the scene, not for the centimeters in front of the camera.

But when I call setNearValue(), also the projection matrix is changed! But why? I want the same view as before, I want only to change the near clipping plane!

Is it a bug? I can not understand why the projection matrix is dependent on the near clipping plane... I thought the clipping planes are just interesting for the z-buffer?!

Thanks!

[nutpor]

Just set near clip plane to any value above 1.0f and it would be fine.

[Guest]

Sorry, but you misunterstood me.
Of course I can change the near clipping plane value. But this changes the projection matrix and this is exactly want I do _not_ want to do.

When I change it for example to 2, it looks as if I am looking through binoculars. This is because the projection matrix is changed.

The question was: _Why_ is the projection matrix changed? Near clipping plane and projection matrix should be independent?!

[Baal Cadar]

This is just correct behaviour. Nearclipplane, farclipplane and FOV angles determine the projection matrix. The projection matrix transforms the camera's view frustum into a cuboid (usually a cube), the canonical view volume. And this frustum of course depends on the clip planes, since these are its front and back borders.

By the way, DirectX can use 24bit zbuffer too, but don't ask me how to change this in irrlicht. I don't know.

[Andi|xng]

OK, but there must be also a possibility to change only the near clipping plane for the z-buffer without modifying the projection matrix?

Let's say I want only to display objects that are between 3 and 1000 units away. How can I do that without changing the projection matrix?

[Baal Cadar]

No, there is no way. Read my last post. Changing the near clip plane inherently and inevitably changes the projection matrix. The near clip distance/plane is used to calculate the projection matrix.

Actually the projection matrix decides over the zbuffer resolution, meaning if their was some miraculous way to change one without the other (there isn't), it wouldn't help either.

I wonder what the problem is. Why is it bad that your projection matrix gets changed? As long as you leave the FOV the same, this should have the desired effect.

[Andi|xng]

I wonder what the problem is. Why is it bad that your projection matrix gets changed? As long as you leave the FOV the same, this should have the desired effect.

Here is my code (Java, but very easy to understand though):

Code: Select all

//1 m nearplane
nativeCamera.setFarValue(500);
nativeCamera.setNearValue(1f);
nativeCamera.setFOV(1.256636f);

Result:

Code: Select all

//5 m nearplane
nativeCamera.setFarValue(500);
nativeCamera.setNearValue(5f);
nativeCamera.setFOV(1.256636f);

Result:


The same FOV, but its like looking through binoculars.
Perhaps I can restore the original view by decreasing (shouldn't it be increasing? But decreasing works, increasing zooms even more in) the FOV-value? I tried 0.2 and it looks much better, but how is the exact formular to get the same view as in nearplane==1?

Thank you very much

[Baal Cadar]

Sorry, Andi|xng. I don't know why this is happening. Only thing I can think of is, that the scene is very small compared to the near clip distance. Is this the case for you? How many units is the camera away from the, uhm, Anstoßpunkt? If the scene is rather small unit-wise, it could be more appropriate to set the far clip plane nearer, much nearer, instead of setting the near clip plane farer away. This will improve zbuffer resolution too, as near-clip/far-clip ratio is important.

[Andi|xng]

OK, let me explain it.

First a used "1 m = 1 unit" in my game. But then there were lighting problems. I described them already in another thread, I can also show you screenshots if you like.

So I switched to "1 m = 100 units". This worked quite ok with the lighting (some errors though, but not as much as with the old scaling factor).

Then the results were good, but only on my computer, because it seems to use 24bit z-buffer. On another computer 16bit z-buffer was used and the results were ugly (I can also show screenshots if you like), even when the far plane was as near as possible. The problem was, that the z-buffer used its precision for the area very near to the camera, namely 1 centimeter in my case. So I wanted to set it to 1 m = 100 units. But setNearPlane(100) resulted in a "extreme binocular view", even with the same FOV.

It would be great if I could set my scaling back to "1 m = 100 units" to avoid the lighting problems. But then the binocular effect happens.
At the moment I am trying to find a formula to decrease the FOV to a value, that looks like the old nearPlane=1...

Stadium metrics: The field is 40x30 m, the camera is about 30 m away from the kickoff position. Hast Du noch irgendeine andere Idee?

[Baal Cadar]

With this scale in use I don't know why this happens. You say Java, so this is not Irrlicht 0.14? For I only looked it up in this version's source. Didn't find anything suspicious. You can look up how the projection matrix is built yourself it's in irr::core::matrix4::buildProjectionMatrixPerspectiveFovLH. CCameraSceneNode calls this after each change of one of the depending params (FOVy, aspect ratio, near and far plane)

You should dig into the implementation. Maybe debug into irrlicht, to see what happens.

[Andi|xng]

I could solve the problem by changing the FOV value to a value that I found with experiments (no formula).

Thanks for your help in this thread!

[grey lantern (as guest)]

About the 16 bit zbuffer - the ONLY time I have seen irrlicht looking good on my PC running a 16 bit zbuffer is when the models in view are very small and the near/far clipplanes are close to each other (ie the resolution is enough to stop z-fighting/artifacts).

This seems to plague a lot of irrlicht stuff I try out (demos from this forum etc). 24bit works fine. As you say though if the "other" system fallsback to 16 bit the problem is there again - Did NIKO ever attempt a fix for this problem or is it just that with a 16bit buffer we really do have to keep things relatively small and close?

Anyway I customised irrlicht.dll to provide 24bit buffer without stencil buffer (it was probably the request for SB that made the fallback happen) - this is for DX(8) only as that is all I have in my custom version. I just added code to check for 24bit minus stencil (unless requested) and then fallback if all else fails.

Changed this:

Code: Select all

	// check stencil buffer compatibility
	if (StencilBuffer)
	{
		present.AutoDepthStencilFormat = D3DFMT_D24S8;
		if(FAILED(pID3D->CheckDeviceFormat(D3DADAPTER_DEFAULT, devtype,
			present.BackBufferFormat, D3DUSAGE_DEPTHSTENCIL, 
			D3DRTYPE_SURFACE, D3DFMT_D24S8)))
		{
			os::Printer::log("Device does not support stencilbuffer, disabling stencil buffer.", ELL_WARNING);
			StencilBuffer = false;
		}
		else
		if(FAILED(pID3D->CheckDepthStencilMatch(D3DADAPTER_DEFAULT, devtype,
			present.BackBufferFormat, present.BackBufferFormat, D3DFMT_D24S8)))
		{
			os::Printer::log("Depth-stencil format is not compatible with display format, disabling stencil buffer.", ELL_WARNING);
			StencilBuffer = false;
		} 		
	}
	if (!StencilBuffer)
		present.AutoDepthStencilFormat = D3DFMT_D16;

to this:

Code: Select all

	// check stencil buffer compatibility
	if (StencilBuffer)
	{
		present.AutoDepthStencilFormat = D3DFMT_D24S8;
		if(FAILED(pID3D->CheckDeviceFormat(D3DADAPTER_DEFAULT, devtype,
			present.BackBufferFormat, D3DUSAGE_DEPTHSTENCIL, 
			D3DRTYPE_SURFACE, D3DFMT_D24S8)))
		{
			os::Printer::log("Device does not support stencilbuffer D3DFMT_D24S8, disabling stencil buffer and trying D24X8.", ELL_WARNING);
			StencilBuffer = false;
		}
		else if(FAILED(pID3D->CheckDepthStencilMatch(D3DADAPTER_DEFAULT, devtype,
			present.BackBufferFormat, present.BackBufferFormat, D3DFMT_D24S8)))
		{
			os::Printer::log("Depth-stencil format is not compatible with display format, disabling stencil buffer and trying D24X8.", ELL_WARNING);
			StencilBuffer = false;
		} 	
		else
		{
			os::Printer::log("Device supports D3DFMT_D24S8 Format, Using (32bit) 24bit buffer + 8 bit stencil.", ELL_WARNING);
		}
	}

	// Start Added Code - grey lantern
	if (!StencilBuffer)
	{
		present.AutoDepthStencilFormat = D3DFMT_D24X8;
		if(FAILED(pID3D->CheckDeviceFormat(D3DADAPTER_DEFAULT, devtype,
			present.BackBufferFormat, D3DUSAGE_DEPTHSTENCIL, 
			D3DRTYPE_SURFACE, D3DFMT_D24X8)))
		{
			os::Printer::log("Device does not support D3DFMT_D24X8 Format, using 16bit buffer.", ELL_WARNING);
			StencilBuffer = false;
			present.AutoDepthStencilFormat = D3DFMT_D16;
		}
		else if(FAILED(pID3D->CheckDepthStencilMatch(D3DADAPTER_DEFAULT, devtype,
			present.BackBufferFormat, present.BackBufferFormat, D3DFMT_D24X8)))
		{
			os::Printer::log("Device does not support D3DFMT_D24X8 Format, using 16bit buffer.", ELL_WARNING);
			StencilBuffer = false;
			present.AutoDepthStencilFormat = D3DFMT_D16;
		} 
		else
		{
			os::Printer::log("Device supports D3DFMT_D24X8 Format, Using (32bit) 24bit buffer with NO stencil.", ELL_WARNING);
		}
	}

	if (gOveridewith16bitbuffer)
	{
		StencilBuffer = false;
		present.AutoDepthStencilFormat = D3DFMT_D16;
		os::Printer::log("Overiding ZBuffer format with explicit 16Bit!", ELL_WARNING);
	}

Even though this is working around and may of no use - it may be of use to someone if they are using D3D8.1 and want that extra option. Of course you have to add the flags/code to the interface - the last part is just a "hacky flag" I have to test overiding with 16bit buffers.

[centipede]

Folks, Irrlichts way of calculating the projection matrix is plain wrong.

The matrix per se is alright, but don't forget that the frustum perspective matrix transforms the space so that a given set of coordinates (Left,Right,Top,Bottom) on the zNear plane is mapped to -1..1 of the viewing cube.

BUT it is still up to Irrlicht to calculate the correct Left,Right,Top,Bottom coordinates that would lie on the zNear-plane. If zNear==1 then everything works smoothly, but if that is changed, things get bad. Say you have aspect=1 and FOV=90 degrees. In the function buildProjectionMatrixPerspectiveFovLH of matrix4.h that would give a w and h of 1. So the coordinates (-1,-1,zNear) and (1,1,zNear) are mapped to (-1,-1,-1) and (1,1,1). But if zNear==5 you all of a sudden have mapped (-1,-1,5) onto the same coordinates as (-1,-1,1) would be if zNear was its default value. Resulting in a very narrow scope of vision.

Simple fix:

Code: Select all

	inline void matrix4::buildProjectionMatrixPerspectiveFovLH(f32 fieldOfViewRadians, f32 aspectRatio, f32 zNear, f32 zFar)

	{

		f32 h = (f32)(cos(fieldOfViewRadians/2) / sin(fieldOfViewRadians/2));

		f32 w = h / aspectRatio;

// ADD THESE LINES
//////////////////////

		w *= zNear;
		h *= zNear;

Hope I'm correct in this (it works fine here with a near-plane of 0.1)
Regards

[Baal Cadar]

Now this makes sense.
I looked at the source too, but didn't spot this.

[Jolindien]

Or even better, just replace 2*zNear/w by 2.0f/w ... and 2*zNear/h by 2.0f/h.

We should report this bug so that it is corrected in the next versions.