getScreenCoordinatesFrom3DPosition values?

[rubixcuber]

I was trying to use getScreenCoordinatesFrom3DPosition to handle some mouse interaction and I was going crazy trying to get it to work so I made a simple test project to try to get a handle on it...

Here's the whole thing basically:

Code: Select all

        driver->beginScene(true, true, SColor(0,48,48,48));
 
        vector3df target = camera->getTarget(); // (0,0,100)
        vector3df pos = camera->getAbsolutePosition();  // (0,0,0)
 
        vector3df test(-1.0, -1.0, 0.0);
        vector3df test2(-1.0, -1.0, 1.0);
        vector3df test3(0, 0, 1.0);
        vector3df test4(-1.0, -1.0, 0.5);
        vector3df test5(-1.0, -1.0, 1.5);
 
 
        vector2di res;
 
        res = collision->getScreenCoordinatesFrom3DPosition(test); // (-17,913) 
        res = collision->getScreenCoordinatesFrom3DPosition(test2);// (-17,913) Shouldn't one of these two be (0,768)?
        res = collision->getScreenCoordinatesFrom3DPosition(test3);// (512, 384) good
        res = collision->getScreenCoordinatesFrom3DPosition(test4);// (-545, 1441) Uh?
        res = collision->getScreenCoordinatesFrom3DPosition(test5);// (-160, 736) Close, I guess?
 

Maybe I'm missing something simple?

[CuteAlien]

I'm not sure if it makes sense to use this for points which are not inside the visible area (aka the view frustum). Basically the answer is - they don't project on the screen. Your first point is even on the same plane as the camera itself, the camera can't look there. That's kinda like trying to look at your own eye - or trying shoot a picture of of a point which is on top of your camera.

Instead of 0 you should use the distance to the near-plane. And the values for x-y - when they are 0,0 they will be at the center in your case (camera just looking along z-axis). For other values it depends on the depth. You can get the corner values from the view-frustum (see Wikipedia: https://en.wikipedia.org/wiki/Viewing_frustum). You get that with camera->getViewFrustum(). If you check docs (http://irrlicht.sourceforge.net/docu/st ... ustum.html) you see it has functions like getNearLeftUp () and getFarLeftUp (). Those will be 0,0 in screen-coordinates. While getNearRightDown () and getFarRightDown() will give you the right-bottom corner at different depths.

[rubixcuber]

Forgive me, but I feel like I'm missing something that is going on still.

Basically I was using an implementation of draw2dImage that I built based on the one by Lonesome Ducky with support for rotation, scale, skew, etc.

And even in their original code it uses drawIndexedTriangleList with all of the vertices having a Z of 1.0 which show up on the screen just fine.

I thought I could add a quick mouse interaction check by taking the vertices being sent to drawIndexedTriangleList and using getScreenCoordinatesFrom3DPosition to work back to matching screen positions. Given that it's got rotation and skew and such I thought it would be nice just to get it easily that way rather than redundantly applying all of the transformations again to the pixel coordinates supplied as well just to see where those end up.

But if I draw something in the upper left corner using draw2dImage, the vertex passed to drawIndexedTriangleList is -1.0, 1.0, 1.0 yet getScreenCoordinatesFrom3DPosition says (-17,913)

Also... the nearLeftUp from the view frustrum is -0.968, 0.726, 0.999

So I guess is there any way to make these two systems work together? I could scale the vertex positions by the size of the near rectangle on the view frustrum and get the right numbers it would seem, but I guess I don't understand why a quad drawn at a y of 1.0 is at the top of the screen, yet the top of the near frustrum is not even 0.75. If I draw a triangle at -1.0, 1.0, 0.999 to honor the Z of the view frustrum it's still right at the top, yet getScreenCoordinatesFrom3DPosition puts us up at -145 like it's taking into account the aspect ratio.

[CuteAlien]

Phew, I try to answer - but don't blame me when I get something wrong - partly guessing, partly looked at driver-sources trying to figure out what's going on myself.

And even in their original code it uses drawIndexedTriangleList with all of the vertices having a Z of 1.0 which show up on the screen just fine.

First of all Z = 1.0 might be the usual value for the near-plane. So that might show up (unlike 0.5) even in 3D projections with default settings for the near-far planes.
But you seem to be talking about 2D drawing - I didn't except that in my post above. 2D drawing functions are using an orthographic projection. And those don't have a view frustum which looks like the one in the usual 3D projections. Also near-plane in this case should be at 0 (everything in front of the camera should work for orthographic projections).

But if I draw something in the upper left corner using draw2dImage, the vertex passed to drawIndexedTriangleList is -1.0, 1.0, 1.0 yet getScreenCoordinatesFrom3DPosition says (-17,913)

That's because those coordinates still go through a model-view matrix. And that matrix is done in a way that it projects 2d-points on X from -1 to 1 to the screen-coordinats 0 to screenwidth. You can multiply the vectors by the model-view matrix to get the real coordinates in 3D.

Also... the nearLeftUp from the view frustrum is -0.968, 0.726, 0.999

My guess is some kind of rounding-errors. But not sure. I'm not exactly certain how the frustum for orthographic projections looks like. My guess would be some cube with infinite end or end at farplane distance (not sure if orthographic projections also clip at far-plane or not - probably they do). I would have to write some test-code for figuring that out.

[MartinVee]

Just my two cents, I can confirm that an orthographic projection clips at the far plane.

[rubixcuber]

Well, thank you for the help and for looking into it. I've just went with manually computing the screen coordinates by matching all of the transformations and got that working just fine.

I realized along the way that part of my troubles were because at the same time I had the 5,5 pixel offset in the cursor position due to Windows that I found mentioned in another thread which is part of why combined with the screen position from 3D stuff I was getting very confused with the numbers. I haven't upgraded to the latest version of Irrlicht 1.8 yet because I've got custom engine changes and been lazy but that might be a good reason to since you mentioned that might have been fixed now.