New method proposal: ISceneNode::getScreenSpace()

[Mel]

It Gets the node's screen space.

This represents the quad that englobes the whole area of the rendered object in screen space. Useful for instance to pick lod levels. Smaller objects in screen are provided smaller detail, or to pick light's bounding area in screen, for a deferred renderer, and change on a per type of scene node basis. Or even other type of effects. It could be calculated just after the visibility of an object has been determined. it wouldn't be really a huuuge overhead and the bounding boxes/spheres could be used to calculate it.

Questions? tomatoes?...

[hendu]

It is a matrix multiply per node, and not needed in many applications, so it should be calculated on-demand rather than automatically. Good idea otherwise.

[Mel]

It could perhaps speed up also the scene node picking with the mouse, instead of a 3D ray-box test, it would only be necesary a 2D test. The point is that the nodes aren't really aware of their impact on screen, so it could be a way to know it before any rendering happens. Also, it could be very well disabled for nodes not needing it certainly.

[CuteAlien]

Hm, could be a general function so people can then use it for any 3d vector. And I guess maybe in camera? Or in viewfrustum class.

I seem to have something like that in my tools, but as standalone function:

Code: Select all

 
// result in range -1 to 1
static bool worldSpaceToProjectionPos(irr::core::vector2df& result, const irr::core::vector3df & worldPos, const irr::core::matrix4& viewProjMat)
{
    irr::f32 transformedPos[4] = { worldPos.X, worldPos.Y, worldPos.Z, 1.0f };
 
    viewProjMat.multiplyWith1x4Matrix(transformedPos);
 
    if (transformedPos[3] < 0)
        return false;
 
    const irr::f32 zDiv = transformedPos[3] == 0.0f ? 1.0f : 1.f / transformedPos[3];
    result.X = transformedPos[0] * zDiv;
    result.Y = transformedPos[1] * zDiv;
 
    return true;
}
 

(badly documented, the transformedPos[3] probably checks if it's behind the camera)

It's used as follows:

Code: Select all

 
matrix4 viewMat = camera->getViewMatrix();
matrix4 viewProjMat = camera->getProjectionMatrix() * viewMat;
irr::core::vector2df screen;
const irr::core::vector3df worldPos;
worldSpaceToProjectionPos(screen, worldPos, viewProjMat) )
 

(Note: Would need some more testing before I put it in Irrlicht)

edit: Thought about it a little bit more - and I guess my solution isn't doing what you need. As for example all corners of the node could be outside the screen-space and the object would still be partly inside. I guess what you would need is to clip the transformed bounding-box against the view frustum and then project the corner points of that clipped object to screenspace. Or maybe somehow projection first and then clipping. Anyway... not so trivial anymore.

[Mel]

Stay on view space then. Clipping an AABB against an axis aligned plane is trivial. You can transform the bounding volumes corners into view space, obtain an AABB in that space, clip it against near/far planes so you don't get values off range, and transform this into projection space. This last projection has an AABB that can be trivially clipped again, and its X/Y coordinates have exactly the quad we're looking for.

in a nutshell:
-Pick the bounding volume
-Transform it into view space (this is still not projective)
-Grab this transformed volume AABB and clip it against the near and far planes (if its Z values are outside of the near/far distances of the camera, clamp to them), If the box is completely outside of these planes, we may as well stop, as it is not visible at all. (i.e, totally behind the camera, or totally beyond the far plane)
-Transform into projection space (this is projective, but as we clipped in the previous step, its values are right)
-This secondly transformed volume has another AABB whose X and Y components are exactly what we're looking for. If their ranges are beyond -1,1, clamp to them.

And we're done. Optionally, convert those into 0-Width/0-Height ranges

PRO: Simple and accurate, on a per node basis and hardware API agnostic, as the 0,1 /-1,1 range problem is avoided in the view space.
CON: double the matrix products

[CuteAlien]

Clipping a box against planes is not that trivial (you get non-box geometry). At least I have no routines for that currently and would have to spend some time writing those. I guess with your recent work on CSG you might have done some work in that area, so if you got code to share for clipping solid objects against planes I won't complain :-) I guess there's one simplification at least that those 2 clipping planes can't overlap.

[Mel]

I mean clipping AABB's. If you want to "clip" a AABB against an axis aligned plane, it is the same as clamping the box coordinates to minimal or maximal values. The frustum planes become axis aligned after the projection

It could be something like this, suggestions accepted. Sorry for the long code, but it is easy to understand. It is not corrected, it may not be directly compilable, but i think the idea is transmited

Code: Select all

 
 
        rectf ISceneNode::getScreenSpace()
        {
            matrix4 view = camera->getViewMatrix();
            matrix4 projection = camera->getProjectionMatrix();
 
            f32 near = camera->getNearValue();
            f32 far = camera->getFarValue();
 
            //This is a box in world space, picked up from any node
            aabb box = getBoundingVolume();
            vector3df[8] corners;
 
            //build our points array
            corners[0] = vector3df(box.minEdge.X, box.minEdge.X, box.minEdge.X);
            corners[1] = vector3df(box.minEdge.X, box.minEdge.X, box.maxEdge.X);
            corners[2] = vector3df(box.minEdge.X, box.maxEdge.X, box.minEdge.X);
            corners[3] = vector3df(box.minEdge.X, box.maxEdge.X, box.maxEdge.X);
            corners[4] = vector3df(box.maxEdge.X, box.minEdge.X, box.minEdge.X);
            corners[5] = vector3df(box.maxEdge.X, box.minEdge.X, box.maxEdge.X);
            corners[6] = vector3df(box.maxEdge.X, box.maxEdge.X, box.minEdge.X);
            corners[7] = vector3df(box.maxEdge.X, box.maxEdge.X, box.maxEdge.X);
 
            //This is a new box in viewspace. the Z plane is aligned to this box, thus, clipping against it is clamping the box edges
            aabb viewBox;
 
            view.transformVect(corners[0]);//NOT PROJECTIVE, AKA, this transform doesn't perform the homogeneization division
            viewBox.reset(corners[0]);
 
            //We transform the corners to view space and obtain a new box
            for (u32 i = 1; i < 8; ++i)
            {
                view.transformVect(corners[i]); //NOT PROJECTIVE, AKA, this transform doesn't perform the homogeneization division
                viewBox.insert(corners[i]);
            }
 
            //If the box is outside the near plane or the far plane, we may return quick
            if (viewBox.maxEdge.Z < near || viewBox.minEdge.Z > far) 
                return rectf(0, 0, 0, 0);
 
            //We restrict the Z values, i.e. "clip" against the near plane
            if (viewBox.minEdge.Z < near)
                viewBox.minEdge.Z = near;
 
            //And clip against the far plane
            if (viewBox.minEdge.Z > far)
                viewBox.minEdge.Z = far;
 
            //our box is ready for the projection. build our points array again 
            corners[0] = vector3df(viewBox.minEdge.X, viewBox.minEdge.X, viewBox.minEdge.X);
            corners[1] = vector3df(viewBox.minEdge.X, viewBox.minEdge.X, viewBox.maxEdge.X);
            corners[2] = vector3df(viewBox.minEdge.X, viewBox.maxEdge.X, viewBox.minEdge.X);
            corners[3] = vector3df(viewBox.minEdge.X, viewBox.maxEdge.X, viewBox.maxEdge.X);
            corners[4] = vector3df(viewBox.maxEdge.X, viewBox.minEdge.X, viewBox.minEdge.X);
            corners[5] = vector3df(viewBox.maxEdge.X, viewBox.minEdge.X, viewBox.maxEdge.X);
            corners[6] = vector3df(viewBox.maxEdge.X, viewBox.maxEdge.X, viewBox.minEdge.X);
            corners[7] = vector3df(viewBox.maxEdge.X, viewBox.maxEdge.X, viewBox.maxEdge.X);
 
            //Reusing the first box
            projection.transformVect(corners[0]);//PROJECTIVE, this transform performs the homogeneization division so the result is in the range -1,1!
            box.reset(corners[0]);
 
            //We transform the corners to projection space and obtain a new box
            for (u32 i = 1; i < 8; ++i)
            {
                view.transformVect(corners[i]); //PROJECTIVE, this transform performs the homogeneization division so the result is in the range -1,1!
                box.insert(corners[i]);
            }
 
            //If the box is outside the projected volume, it is not visible
            if (box.maxEdge.X < -1 || box.minEdge.X > 1 || box.maxEdge.Y < -1 || box.minEdge.Y > 1)
                return rectf(0, 0, 0, 0);
 
            //"clipping" again...
            if (box.minEdge.X < -1)
                box.minEdge.X = -1;
 
            if (box.maxEdge.X > 1)
                box.maxEdge.X = 1;
 
            if (box.minEdge.Y < -1)
                box.minEdge.Y = -1;
 
            if (box.maxEdge.Y > 1)
                box.maxEdge.Y = 1;
 
            //Back to range 01
 
            box.minEdge = 0.5*box.minEdge + vector3df(0.5, 0.5, 0.5);
            box.maxEdge = 0.5*box.minEdge + vector3df(0.5, 0.5, 0.5);
 
            //Return
            return rectf(box.minEdge.X, box.minEdge.Y, box.maxEdge.X, box.maxEdge.Y);
        }
 
 

Suggestions accepted for a faster approach!

[REDDemon]

I think this is usefull and in most games allows to do pick-click without resort to ray-tracing or occlusion queries (ok you pick a quad instead ofa shape, but better than nothing). If we do check the screenspace only on request, I think there's no need to optimize it (As long as it is correct), of course someone may want to call it every frame for some reason ^^ in that casee I may have few ideas (accepting SIMD? => you can do 2/3 instructions clamp i think let me check documentation)

[CuteAlien]

@Mel: Hm, yeah - that could work. Have to think more about it and write some test if it works. Maybe belongs rather in the camera or frustum class (camera-function get's a aabbox and transformation and returns the rect).
Btw. the aabbox has a nice getEdges function to get corners :-) (edit: noticing now you don't use Irrlicht classes but some other aabb class).

[CuteAlien]

I tried getting it to work, but no success so far. Maybe missing z-div. Or maybe messed up something while trying to clean it up (and get it compiling...).

If someone wants to continue and try fix it, that's the current (non-working) state (it's not really on my todo, I was just giving it a quick shot because it sounded nice to have, but won't go on for now):

Code: Select all

 
#include <irrlicht.h>
 
using namespace irr;
using namespace core;
 
irr::core::rectf getScreenSpace(irr::scene::ICameraSceneNode* camera, const irr::core::aabbox3df& bbox, const irr::core::matrix4& boxTransform)
{
    // Get transformed corners of box
    vector3df corners[8];
    bbox.getEdges(corners);
    for (u32 i = 1; i < 8; ++i)
    {
        boxTransform.transformVect(corners[i]);
    }
    
    
    matrix4 view = camera->getViewMatrix();
    matrix4 projection = camera->getProjectionMatrix();
 
    //This is a new box in viewspace. the Z plane is aligned to this box, thus, clipping against it is clamping the box edges
    aabbox3df viewBox;
 
    view.transformVect(corners[0]);//NOT PROJECTIVE, AKA, this transform doesn't perform the homogeneization division
    viewBox.reset(corners[0]);
 
    //We transform the corners to view space and obtain a new box
    for (u32 i = 1; i < 8; ++i)
    {
        view.transformVect(corners[i]); //NOT PROJECTIVE, AKA, this transform doesn't perform the homogeneization division
        viewBox.addInternalPoint(corners[i]);
    }
 
    f32 near = camera->getNearValue();
    f32 far = camera->getFarValue();
    
    //If the box is outside the near plane or the far plane, we may return quick
    if (viewBox.MaxEdge.Z < near || viewBox.MinEdge.Z > far)
        return rectf(0, 0, 0, 0);
 
    //We restrict the Z values, i.e. "clip" against the near plane
    if (viewBox.MinEdge.Z < near)
        viewBox.MinEdge.Z = near;
 
    //And clip against the far plane
    if (viewBox.MinEdge.Z > far)
        viewBox.MinEdge.Z = far;
 
    //our box is ready for the projection. build our points array again
    viewBox.getEdges(corners);
 
    irr::core::aabbox3df pbox;
    projection.transformVect(corners[0]);//PROJECTIVE, this transform performs the homogeneization division so the result is in the range -1,1!
    pbox.reset(corners[0]);
 
    //We transform the corners to projection space and obtain a new box
    for (u32 i = 1; i < 8; ++i)
    {
        view.transformVect(corners[i]); //PROJECTIVE, this transform performs the homogeneization division so the result is in the range -1,1!
        pbox.addInternalPoint(corners[i]);
    }
 
    //If the box is outside the projected volume, it is not visible
    if (pbox.MaxEdge.X < -1 || pbox.MinEdge.X > 1 || pbox.MaxEdge.Y < -1 || pbox.MinEdge.Y > 1)
        return rectf(0, 0, 0, 0);
 
    //"clipping" again...
    if (pbox.MinEdge.X < -1)
        pbox.MinEdge.X = -1;
 
    if (pbox.MaxEdge.X > 1)
        pbox.MaxEdge.X = 1;
 
    if (pbox.MinEdge.Y < -1)
        pbox.MinEdge.Y = -1;
 
    if (pbox.MaxEdge.Y > 1)
        pbox.MaxEdge.Y = 1;
 
    //Back to range 01
 
    pbox.MinEdge = 0.5*pbox.MinEdge + vector3df(0.5, 0.5, 0.5);
    pbox.MaxEdge = 0.5*pbox.MinEdge + vector3df(0.5, 0.5, 0.5);
 
    //Return
    return rectf(pbox.MinEdge.X, pbox.MinEdge.Y, pbox.MaxEdge.X, pbox.MaxEdge.Y);
}
 
int main()
{
    irr::core::dimension2d<irr::u32> dimScreen(800,600);
    
    IrrlichtDevice *  Device = createDevice(irr::video::EDT_OPENGL, dimScreen);
    if (!Device)
        return false;
   
    scene::ISceneManager* smgr = Device->getSceneManager();
    video::IVideoDriver * videoDriver = Device->getVideoDriver ();
   
    irr::scene::ISceneNode * node = smgr->addCubeSceneNode (30.0f, 0, -1, 
                            core::vector3df(0, 20, 100),
                            core::vector3df(45, 45, 45),
                            core::vector3df(1.0f, 1.0f, 1.0f));
 
    smgr->addLightSceneNode(0, core::vector3df(0, 0, 0),
                            video::SColorf(1.0f, 1.0f, 1.0f),
                            100.0f);
    irr::scene::ICameraSceneNode * camera = smgr->addCameraSceneNodeFPS(0, 20.f, 0.1f );
 
    while ( Device->run() )
    {
        if ( Device->isWindowActive() )
        {
            videoDriver->beginScene(true, true);
 
            smgr->drawAll();
 
            videoDriver->setTransform(video::ETS_WORLD, core::matrix4());
 
            irr::core::rectf r = getScreenSpace(camera, node->getBoundingBox(), node->getAbsoluteTransformation());
            irr::core::recti sr( (irr::s32)(r.UpperLeftCorner.X*dimScreen.Width),  (irr::s32)(r.UpperLeftCorner.Y*dimScreen.Height)
                                , (irr::s32)(r.LowerRightCorner.X*dimScreen.Width),  (irr::s32)(r.LowerRightCorner.Y*dimScreen.Height) );
            
            videoDriver->draw2DRectangleOutline(sr, video::SColor(255, 255, 0, 0));
 
            videoDriver->endScene();
        }
 
        Device->sleep( 5 );
    }
 
    Device->closeDevice();
    Device->drop();
    Device = NULL;
 
    return 0;
}
 

[Mel]

I'll try to come up with something. Thinking about it, it can't be too costly, a skinned node is much more complex, in comparison

[Virion]

Just to point out that Unreal Engine actually uses this method for LOD (instead of camera-to-mesh distance). I know this coz' I was teaching UDK in university few years back.

[REDDemon]

I think it is missing the W component on vertices, when doing vertex transformation on GPU we use 3 coordinates and the 4th is automatically assumed to be 1. If W is 0 infact vertices are threated like directions not positions.

At the end of MVP chain, W is 1, but its value change and is used to get correct other valuesò.

Actually I think it should use vector4df starting with w = 1, then do all the transformations and when finally you do perspective divide w should become 1 again. Not sure if there are other bugs.

in the irrlicht matrix class when you do a vertex translation transform it is assumed that w = 1 for vertices infact

Code: Select all

 
template <class T>
inline void CMatrix4<T>::transformVect( vector3df& out, const vector3df& in) const
{
    out.X = in.X*M[0] + in.Y*M[4] + in.Z*M[8] + M[12];
    out.Y = in.X*M[1] + in.Y*M[5] + in.Z*M[9] + M[13];
    out.Z = in.X*M[2] + in.Y*M[6] + in.Z*M[10] + M[14];
}
 

the irrlicht's code for vector transformation assumes the vector has W = 1 (otherwise the result would be wrong).

[CuteAlien]

@REDDemon: Yeah, that's what I meant with z-div. I got that in my worldSpaceToProjectionPos code above and noticed it's missing here. Feel free to experiment - just copy code above over any Irrlicht example and you can play around with it. I couldn't code for a few weeks so my list of urgent stuff (where I promised to work on it weeks ago) is already a little overflowing so not going to continue with this one right now unless I have a working solution which I can just add to the engine.

@Virion: Replacing camera-to-object distance by camera-to-near-plane distance is also very high on my todo (thanks to the nice example of things going wrong in an incredible way which MartinVee posted recently: http://irrlicht.sourceforge.net/forum/v ... =1&t=51598) . Only reason I haven't changed it yet is that I'm thinking about a larger change to allow users to do customized sorting (and I never find time to code that...).

[REDDemon]

Sorry CuteAlien I had barely the time to look at Mel's code.