Incoherent animation of skinned meshes

christianclavet · Post by **christianclavet** » Sun Jul 27, 2008 5:46 pm

full.metal.coder, I don't see the patch on the sourceforge site, could you submit it (the corrected patch by Rogerborg)?. I don't think the DEV will take care of it if it's only in a forum thread.

Here is the link to the sourceforge page:
http://sourceforge.net/tracker/?group_i ... tid=540678

Hybrid, that patch as been checked by rogerborg, and the new functionnality is activated only via some toggle. By default it should not be activated and the code will operate exactly as before. The risk of putting the source unstable is very minimal. I would propose that If luke as the time, he check this because the patch when activated will apply directly on the animated meshes.

This patch will help IRRlicht supporting external physic engines with animated meshes.

rogerborg · Post by **rogerborg** » Sun Jul 27, 2008 9:09 pm

To be clear, all I did was found a few things that needed fixing and fixed them, and verified that a very simple unit test doesn't explode.

I wouldn't say that I'd actually reviewed it, since I'm not entirely clear on what the purpose of it is. For one thing, it doesn't seem to fix IAnimatedMesh::getMesh(s32 frame,...), at least not for the test case that I tried. Is it intended to do so?

full.metal.coder · Post by **full.metal.coder** » Sun Jul 27, 2008 9:27 pm

What do you mean by "fixing getMesh()" ?
AFAIK this method is supposed to update an internal mesh according to the frame parameter and return a pointer to it. Most animated meshes, and this include skinned ones actually return a pointer to themselves. If I am not mistaken then the method DOES work as expected... What use were you trying to make of that method anyway?

The purpose of it is to cache the mesh data (mesh buffers) at node level to make sure animation is performed in OnAnimate() and not in render() and to make this data available to (among other possible things) a physics engine. At the expense of some memory usage, turning on the skin cache can significantly improve performance when using the same mesh for several nodes, especially if you do extra processing based on mesh data (eg physics). Indeed, instead of doing the whole animation stuff at least two times (in my case) per node it will do it at most once.

dlangdev · Post by **dlangdev** » Sun Jul 27, 2008 9:40 pm

note though that irrlicht has several games depending on that feature.

you're swimming against the current here, tho i like the feature you're presenting to them.

plus, these people r conservatives. think bout the catholic church, very resistant to change.

full.metal.coder · Post by **full.metal.coder** » Sun Jul 27, 2008 9:45 pm

note though that irrlicht has several games depending on that feature.

you're swimming against the current here, tho i like the feature you're presenting to them.

???

If you're talking about the skin cache remember that it is optional and disabled by default (in the patch) so there's no possible backward compat problem.

If you're talking about something else would you mind making yourself a little clearer?

rogerborg · Post by **rogerborg** » Sun Jul 27, 2008 10:02 pm

full.metal.coder wrote:What do you mean by "fixing getMesh()" ?

The mesh returned by getMesh(s32 frame,...) from a skinned animated mesh does not match the visual mesh.

Compare and contrast the behaviour with an MD2 and a B3D: note that the B3D collision mesh seems to lag several frames behind the visual mesh.

Code: Select all

#include <irrlicht.h>

using namespace irr;
using namespace core;
using namespace scene;
using namespace video;
using namespace io;
using namespace gui;

#ifdef _IRR_WINDOWS_
#pragma comment(lib, "Irrlicht.lib")
#endif

int main()
{
    IrrlichtDevice *device =
        createDevice( video::EDT_OPENGL, dimension2d<s32>(640, 480), 32,
            false, false, false, 0);

    IVideoDriver* driver = device->getVideoDriver();
    ISceneManager* smgr = device->getSceneManager();

    IAnimatedMesh* mesh = smgr->getMesh("../../media/sydney.md2");
    IAnimatedMeshSceneNode* node = smgr->addAnimatedMeshSceneNode( mesh );
    node->setPosition(vector3df(-5, 5, 0));
    node->setMaterialFlag(EMF_LIGHTING, false);
    node->setMD2Animation ( scene::EMAT_STAND );
    node->setAnimationSpeed(5);
    node->setMaterialTexture( 0, driver->getTexture("../../media/sydney.bmp") );
    node->setScale(vector3df(.15f, .15f, .15f));

    mesh = smgr->getMesh("../../media/ninja.b3d");
    node = smgr->addAnimatedMeshSceneNode( mesh );
    node->setPosition(vector3df(5, 0, 0));
    node->setMaterialFlag(EMF_LIGHTING, false);
    node->setAnimationSpeed(2);

    ICameraSceneNode * camera = smgr->addCameraSceneNodeFPS(0, 30, 20);
    camera->setPosition(vector3df(0,5,-10));
    camera->setID(0);

    ISceneCollisionManager * collMgr = smgr->getSceneCollisionManager();

    while(device->run())
    {
        driver->beginScene(true, true, SColor(255,100,101,140));
        smgr->drawAll();
    
        line3df ray(camera->getAbsolutePosition(), 
            (camera->getTarget() - camera->getAbsolutePosition()).normalize() * 1000.f);

        ISceneNode * collisionNode = collMgr->getSceneNodeFromRayBB(ray, !0);

        if(collisionNode)
        {
            IAnimatedMeshSceneNode * animNode = (IAnimatedMeshSceneNode *)collisionNode;
            ITriangleSelector * selector = 
                smgr->createTriangleSelector(animNode->getMesh()->getMesh((s32)animNode->getFrameNr()),
                                             animNode);

            vector3df hitPoint;
            triangle3df hitTriangle;
            if(collMgr->getCollisionPoint(ray, selector, hitPoint, hitTriangle))
            {
                SMaterial triangleMaterial;
                triangleMaterial.Lighting = false;

                driver->setTransform(video::ETS_WORLD, core::matrix4());
                driver->setMaterial(triangleMaterial);
                driver->draw3DTriangle(hitTriangle, video::SColor(0,255,0,0));
            }

            selector->drop();
        }

        driver->endScene();
    }

    device->drop();

    return 0;
}

full.metal.coder · Post by **full.metal.coder** » Mon Jul 28, 2008 12:36 am

The problem actually lies in the triangle selector (or in SSkinMeshBuffer, depending on your point of view) and more particularly in the getPosition() function which DOES NOT take the transformation of the mesh buffer into account while the rendering function DOES... The simplest (and probably most consistent) fix would be to change SSkinMeshBuffer::getPosition() to transform the vertex position by the current transformation.

And indeed it has nothing to do with the skin cache.

edit : the fix is gonna be a little trickier as getPosition() returns a reference (which can be either const or non-const)... turns out the easiest way to fix this will be to forget about the getPosition() and niceness of virtual functions to use dynamic_cast and manually compute the transformed vertex positions in CTriangleSelector...

full.metal.coder · Post by **full.metal.coder** » Mon Jul 28, 2008 12:25 pm

Strangely enough the skin cache allows to get rid of this bug WITHOUT doing any change related to the transformation (the ninja mesh probably does not make use of them but it would be sensible to fix SSkinMeshBuffer or CTriangleSelector anyway) :

Code: Select all


#include <irrlicht.h>

using namespace irr;
using namespace core;
using namespace scene;
using namespace video;
using namespace io;
using namespace gui;

int main(int argc, char **argv)
{
    IrrlichtDevice *device =
        createDevice( video::EDT_OPENGL, dimension2d<s32>(640, 480), 32,
            false, false, false, 0);

    IVideoDriver* driver = device->getVideoDriver();
    ISceneManager* smgr = device->getSceneManager();

    IAnimatedMesh* mesh = smgr->getMesh("../trunk/media/sydney.md2");
    IAnimatedMeshSceneNode* node = smgr->addAnimatedMeshSceneNode( mesh );
    node->setPosition(vector3df(-5, 5, 0));
    node->setMaterialFlag(EMF_LIGHTING, false);
    node->setMD2Animation ( scene::EMAT_STAND );
    node->setAnimationSpeed(10);
    node->setMaterialTexture( 0, driver->getTexture("../trunk/media/sydney.bmp") );
    node->setScale(vector3df(.15f, .15f, .15f));
    
    SMesh upToDateMesh;

    mesh = smgr->getMesh("../trunk/media/ninja.b3d");
    node = smgr->addAnimatedMeshSceneNode( mesh );
    //node->setPosition(vector3df(5, 0, 0));
    node->setMaterialFlag(EMF_LIGHTING, false);
    node->setAnimationSpeed(1);
    node->setUseSkinCache(true);

    ICameraSceneNode * camera = smgr->addCameraSceneNodeFPS(0, 30, 20);
    camera->setPosition(vector3df(0,5,-10));
    camera->setID(0);

    ISceneCollisionManager * collMgr = smgr->getSceneCollisionManager();

    while(device->run())
    {
        driver->beginScene(true, true, SColor(255,100,101,140));
        smgr->drawAll();
        
        if ( !upToDateMesh.getMeshBufferCount() )
        {
        	array<SSkinMeshBuffer*> *cache = node->getSkinCache();
    		
    		for ( int i = 0; i < cache->size(); ++i )
    			upToDateMesh.addMeshBuffer((*cache)[i]);
    		
    	}
        
   
        line3df ray(camera->getAbsolutePosition(),
            (camera->getTarget() - camera->getAbsolutePosition()).normalize() * 1000.f);

        ISceneNode * collisionNode = collMgr->getSceneNodeFromRayBB(ray, !0);

        if(collisionNode)
        {
            IAnimatedMeshSceneNode * animNode = (IAnimatedMeshSceneNode *)collisionNode;
            ITriangleSelector * selector = smgr->createTriangleSelector(&upToDateMesh, animNode);

            vector3df hitPoint;
            triangle3df hitTriangle;
            if(collMgr->getCollisionPoint(ray, selector, hitPoint, hitTriangle))
            {
                SMaterial triangleMaterial;
                triangleMaterial.Lighting = false;

                driver->setTransform(video::ETS_WORLD, core::matrix4());
                driver->setMaterial(triangleMaterial);
                driver->draw3DTriangle(hitTriangle, video::SColor(0,255,0,0));
            }

            selector->drop();
        }

        driver->endScene();
    }

    device->drop();

    return 0;
}

The initialization of the SMesh is a bit tricky due to triangle selector not accepting an array of meshbuffers and skin cache being initialized on first OnAnimate call but it works well (Note that you could just as well create the SMesh inside the loop and fill it at every step but I'm a bit of a perfectionnist...

).

Luke · Post by **Luke** » Mon Jul 28, 2008 2:15 pm

full.metal.coder:

hey, thanks for working on this. this really needs to get fixed.

with your patch:
mainly, you are changing more then you need to, I already coded skinnedMesh to animate to a node in future. just make the SkinningBuffers pointer in the mesh point to the node buffer before animating and drawing. (then set it back) didn't expose this but, so I can see why you missed it. and plus I'd create the nodes buffers on setMesh instead of being created on the first OnAnimate.

and I've already coded new 16bit/32bit meshbuffers (which are working nicely

), with split index and vertex buffers. so I cannot really use this patch. but I'll work on storing the animation data pre-node. with the new meshbuffers, it means only the vertex buffer needs to be stored

and btw transformation of meshbuffers is only used with some rare .x meshes, so it's not the problem with any other meshes. I'll work on a fix for this too. but it's a tricky problem.

would be nice if meshbuffer's could transform (without breaking stuff) instead of having to skin them.

full.metal.coder · Post by **full.metal.coder** » Mon Jul 28, 2008 3:31 pm

and I've already coded new 16bit/32bit meshbuffers (which are working nicely Smile ), with split index and vertex buffers. so I cannot really use this patch. but I'll work on storing the animation data pre-node. with the new meshbuffers, it means only the vertex buffer needs to be stored Smile

They're not on SVN yet, are they? I'm syncing my working copy at least once a day (well trying to sync as commit rate is not always fast enough for my svn up to do something ) so I would have noticed it.

I assume that if you've create new mesh buffers you've also updated meshes to use them so applying the patch would be trivial : just change the type of mesh buffers stored in CAnimatedMeshSceneNode and you're done. Also the suggested improvements to my patch are nice but as for populating the skin cache on set mesh this is not necessarily the best thing to do as the skin cache will never be enabled before the set mesh so it will either result in cache still not being initialized or being initiliazed even if the skin cache is not enabled. Probably better to initialize in setUseSkinCache().

btw good to know transformations are not used. It would have been a pain in the butt to support that because my physics/graphics interface would have become messy and way slower (turns out physics accesses mesh data more often than graphics so it really needs to be ready for reading without any extra transformation).

Luke · Post by **Luke** » Tue Jul 29, 2008 1:05 am

no not yet, I'll start committing them soon.

well yeah, few other changes to, like no need to copy over the index data to the buffers.

and yeah setUseSkinCache() is best, setMesh() should call setUseSkinCache(SkinCacheUsed) so changing meshes doesn't break it.

and yeah, loading an animated mesh to your physics is going to kill your fps. you thought of using ragdolls instead or something. I do that in my game, that way I can use hardware skinning.

full.metal.coder · Post by **full.metal.coder** » Tue Jul 29, 2008 11:26 am

Using animated mesh for physics don't kill fps that much because I share the data and thus the skinning is only done once per frame and both physics and graphics use the same data. What causes the real drop is the actual physics when moving characters (almost) realistically in the environment.

Another thing that may be of interest :

ISceneManager::drawAll() could be split into several parts (while still being available of course) so that I would be able to perform animation, then physics then actual drawing. It would be fairly easy to do and give much more flexibility to game designers. Do you want a patch or can you do it?