Animation Blending broken in Irrlicht 1.8.0

[christianclavet]

Hi! Was there any major changes in the way animation on skinned meshes are played back?

I get some weird effects now depending on the meshes I use...

On one character, it walk now backward, and at idle pause, suddenly, we see it "turning" around show the idle (correctly) for the pause then "turn around again" to walk backward.

On another character, the walk is ok, but some part of the bone animations flip over in the attack stances.

Look like some bone rotations/transformation give odd behavior. Anybody seen that with Irrlicht 1.8.0. I saw somewhere that they were changes in the matrices transformations, could it be the cause?

Here is what I'm using to "align" to another character:

Code: Select all

void DynamicObject::lookAt(vector3df pos)
{
    vector3df offsetVector = pos - node->getPosition();
 
    vector3df rot = (-offsetVector).getHorizontalAngle();
 
    rot.X=0;
    rot.Z=0;
 
    // Will rotate the node only if it still "alive"
    if (properties.life>0)
        node->setRotation(rot);
}

[fmx]

which model format are you having problems with?
and have you tried loading them in the new MeshViewer sample yet?

i haven't thoroughly checked changes in 1.8 release yet, but the problem doesn't necessarily have to be with matrices, it could also be changes in loaders or changes in how animation frameloops are set (function parameters or type changes etc)

worst comes to worst, just upload a sample model file which is causing problems and i'll have a look at it (and so can the devs and others)

[christianclavet]

For the skinned meshes, I'm using B3D. Checked the movement of the object is ok (Node is moving along the directionnal vector properly). It's the mesh that is not rendered properly (look like bones get weird info)

Here is a link to the mesh. (The mesh is really small: 2 unit)

Here is the link to the file: http://www.clavet.org/files/models/goblin_small.zip 874K

The node is moving following a directional vector, but NOW (Irrlicht 1.8.0) the mesh animate the model in inverse direction. In 1.7.1, it was animating it in the proper direction. The only thing I could think of is that something strange that happening with the bones transformations.

The file contain multiple animations. and:

Idle animation - Play correctly and orientation seem ok (first frames in the animation).
Walk animation and all other animation - Play correctly but the mesh is rendered oriented backward (all the frames after the idle animation).

I see a quick rotation when the node is going from any animation to the idle animation.
Another thing to note: I'm using animation blending.

EDIT: Found out the problem!!!!
The animation blending is broken!!! I disabled it and now the models are animating correctly!
Here how I was applying the animation blending:
That was in my refresh loop:

Code: Select all

if (this->objectType==OBJECT_TYPE_NPC || this->objectType==OBJECT_TYPE_PLAYER)
        ((IAnimatedMeshSceneNode*)this->getNode())->setTransitionTime(0.15f);   

This is now commented off and there no problem of the alignment. But I've lost animation blending now. So there something in there that screw off the bone animation.

[Mel]

there haven't been any changes to the animation system for a long time, and although i haven't updated my applications to 1.8, i can tell i am using the same code. the panda x exporter works well, the only difference is that i skin through the video card. the animation blending works as expected. I might as well upgrade to 1.8, but i am pretty confident that the animation blending works, this error comes from long ago, and never got solved really.

something i have noticed too is that my models have extra empty joints, without influences attached to them, is that something expected too, or some exporter issue also?

[christianclavet]

Hi, Mel.

This worked 100% under Irrlicht 1.7.1. The DEV might have not changed the particular code from the animation system but that is surely using math/transformations (for the bones), So I suspect that code that changed in the math/matrices that is being used by the animation system is returning wrong values when the animation blending is activated.

For the model, this is not coming from the exporter but but the "rig" inside your animation software. I got also extra bones that have no influences, the names tell me it's coming from the 3DS Biped rig I'm using. With or without it was working well with 1.7.1 not in 1.8.0.

There is something that affect the bone transformations, the skinning is still ok and the animation play as it should (exepts some flipping occuring in the reference of the bones transformation)-- This is not 100% accurate as it come from my observation of the problem so far, not by looking at the code.

If you have some time to check, load my model and try to activate the animation blending and make the node move, you'll see that it walk backward. Once the animation blending was not activated, the animation played perfectly.

[fmx]

Glad to hear you found the problem, at least now we know what needs to be fixed for future versions of irrlicht

"empty bones" are needed for correct skeletal animation sometimes because they might have keyframes.

It's actually quite simple to manually remove bones you dont need (if you are sure they dont have any keyframes or skin-vertices attached to them):
all you have to do is to reassign the children and parents for each bone to 'remove' the bone from the heirarchy, then run through all the skin vertices to adjust the joint influence indexes, though this step is best done during loadtime when bone names are available for each vertex

this should NOT be done automatically however, because sometimes empty bones are desirable, for example to act as attachment-points for other models (swords, heads, turrets, wheels, etc)

[Mel]

I am sure the empty bones have nothing to do to the skinned meshes, because i check the influences of the bones, and the ones with vertices attached are those which come first, besides, they are the ones which have the eported bone names. I use my own custom rig, and i use many extra helpers and IK chains, but their names aren't exported, so, it must be something else.

What got changed was some quaternion specific code, and there is a define to enable the old behavior. take a look at that

http://irrlicht.sourceforge.net/docu/qu ... ource.html

[christianclavet]

Thanks! I've tried to use this:

Code: Select all

// Between Irrlicht 1.7 and Irrlicht 1.8 the quaternion-matrix conversions got fixed.
// This define disables all involved functions completely to allow finding all places 
// where the wrong conversions had been in use.
#define IRR_TEST_BROKEN_QUATERNION_USE 1

It was set to 0 before.

But now Irrlicht completely fail to compile.

Get theses errors: (Sorry the output is in french):

1>c:\users\public\projets\irrlicht-1.8\source\irrlicht\CAttributeImpl.h(1421) : error C2440: '<function-style-cast>' : impossible de convertir de 'irr::core::matrix4' en 'irr::core::quaternion'
1> Aucun constructeur n'a pu prendre le type de source, ou la résolution de la surcharge du constructeur était ambiguë
1>c:\users\public\projets\irrlicht-1.8\source\irrlicht\CAttributeImpl.h(1444) : error C2660: 'irr::core::quaternion::getMatrix' : la fonction ne prend pas 0 arguments
1>.\CXMeshFileLoader.cpp(1883) : error C2440: '<function-style-cast>' : impossible de convertir de 'irr::core::CMatrix4<T>' en 'irr::core::quaternion'
1> with
1> [
1> T=irr::f32
1> ]
1> Aucun constructeur n'a pu prendre le type de source, ou la résolution de la surcharge du constructeur était ambiguë
1>.\COgreMeshFileLoader.cpp(832) : error C2440: '<function-style-cast>' : impossible de convertir de 'irr::core::CMatrix4<T>' en 'irr::core::quaternion'
1> with
1> [
1> T=irr::f32
1> ]
1> Aucun constructeur n'a pu prendre le type de source, ou la résolution de la surcharge du constructeur était ambiguë
1>.\CMS3DMeshFileLoader.cpp(506) : error C2440: '<function-style-cast>' : impossible de convertir de 'irr::core::CMatrix4<T>' en 'irr::core::quaternion'
1> with
1> [
1> T=irr::f32
1> ]
1> Aucun constructeur n'a pu prendre le type de source, ou la résolution de la surcharge du constructeur était ambiguë
1>.\CColladaFileLoader.cpp(897) : error C2660: 'irr::core::quaternion::getMatrix' : la fonction ne prend pas 0 arguments
1>.\CColladaFileLoader.cpp(1047) : error C2660: 'irr::core::quaternion::getMatrix' : la fonction ne prend pas 0 arguments
>Irrlicht - 7 erreur(s), 0 avertissement(s)
========== Génération : 0 a réussi, 1 a échoué, 0 mis à jour, 0 a été ignoré ==========

Humm, I don't think we can fall back now, and test how it was working on the "old" behavior. Anyway, if the dev can check, I've still have the link to my model here so they should be able to replicate the problem.

EDIT:!!! Looking at the comment, this behavior is normal! It pointed me where the wrong conversion were used. So... If I'm not mistaken, there would be other place to fix in the animation blending to use the fixed conversion... Too bad for me that my math skills are so poor...

[christianclavet]

Sorry to use another post:

I've decided to dig in the source and find where the Animation blending is doing it stuff. There was changes there, and the comments mention that it.

From line 170 in CSkinnedMesh.cpp

// IRR_TEST_BROKEN_QUATERNION_USE: TODO - switched to getMatrix_transposed instead of getMatrix for downward compatibility.
// Not tested so far if this was correct or wrong before quaternion fix!
joint->Animatedrotation.getMatrix_transposed(joint->LocalAnimatedMatrix);

CSkinnedMesh.cpp:

From line 87...

Code: Select all

//--------------------------------------------------------------------------
//          Keyframe Animation
//--------------------------------------------------------------------------
 
 
//! Animates this mesh's joints based on frame input
//! blend: {0-old position, 1-New position}
void CSkinnedMesh::animateMesh(f32 frame, f32 blend)
{
    if (!HasAnimation || LastAnimatedFrame==frame)
        return;
 
    LastAnimatedFrame=frame;
    SkinnedLastFrame=false;
 
    if (blend<=0.f)
        return; //No need to animate
 
    for (u32 i=0; i<AllJoints.size(); ++i)
    {
        //The joints can be animated here with no input from their
        //parents, but for setAnimationMode extra checks are needed
        //to their parents
        SJoint *joint = AllJoints[i];
 
        const core::vector3df oldPosition = joint->Animatedposition;
        const core::vector3df oldScale = joint->Animatedscale;
        const core::quaternion oldRotation = joint->Animatedrotation;
 
        core::vector3df position = oldPosition;
        core::vector3df scale = oldScale;
        core::quaternion rotation = oldRotation;
 
        getFrameData(frame, joint,
                position, joint->positionHint,
                scale, joint->scaleHint,
                rotation, joint->rotationHint);
 
        if (blend==1.0f)
        {
            //No blending needed
            joint->Animatedposition = position;
            joint->Animatedscale = scale;
            joint->Animatedrotation = rotation;
        }
        else
        {
            //Blend animation
            joint->Animatedposition = core::lerp(oldPosition, position, blend);
            joint->Animatedscale = core::lerp(oldScale, scale, blend);
            joint->Animatedrotation.slerp(oldRotation, rotation, blend);
        }
    }
 
    //Note:
    //LocalAnimatedMatrix needs to be built at some point, but this function may be called lots of times for
    //one render (to play two animations at the same time) LocalAnimatedMatrix only needs to be built once.
    //a call to buildAllLocalAnimatedMatrices is needed before skinning the mesh, and before the user gets the joints to move
 
    //----------------
    // Temp!
    buildAllLocalAnimatedMatrices();
    //-----------------
 
    updateBoundingBox();
}
 
 
void CSkinnedMesh::buildAllLocalAnimatedMatrices()
{
    for (u32 i=0; i<AllJoints.size(); ++i)
    {
        SJoint *joint = AllJoints[i];
 
        //Could be faster:
 
        if (joint->UseAnimationFrom &&
            (joint->UseAnimationFrom->PositionKeys.size() ||
             joint->UseAnimationFrom->ScaleKeys.size() ||
             joint->UseAnimationFrom->RotationKeys.size() ))
        {
            joint->GlobalSkinningSpace=false;
 
            // IRR_TEST_BROKEN_QUATERNION_USE: TODO - switched to getMatrix_transposed instead of getMatrix for downward compatibility. 
            //                                 Not tested so far if this was correct or wrong before quaternion fix!
            joint->Animatedrotation.getMatrix_transposed(joint->LocalAnimatedMatrix);
 
            // --- joint->LocalAnimatedMatrix *= joint->Animatedrotation.getMatrix() ---
            f32 *m1 = joint->LocalAnimatedMatrix.pointer();
            core::vector3df &Pos = joint->Animatedposition;
            m1[0] += Pos.X*m1[3];
            m1[1] += Pos.Y*m1[3];
            m1[2] += Pos.Z*m1[3];
            m1[4] += Pos.X*m1[7];
            m1[5] += Pos.Y*m1[7];
            m1[6] += Pos.Z*m1[7];
            m1[8] += Pos.X*m1[11];
            m1[9] += Pos.Y*m1[11];
            m1[10] += Pos.Z*m1[11];
            m1[12] += Pos.X*m1[15];
            m1[13] += Pos.Y*m1[15];
            m1[14] += Pos.Z*m1[15];
            // -----------------------------------
 
            if (joint->ScaleKeys.size())
            {
                /*
                core::matrix4 scaleMatrix;
                scaleMatrix.setScale(joint->Animatedscale);
                joint->LocalAnimatedMatrix *= scaleMatrix;
                */
 
                // -------- joint->LocalAnimatedMatrix *= scaleMatrix -----------------
                core::matrix4& mat = joint->LocalAnimatedMatrix;
                mat[0] *= joint->Animatedscale.X;
                mat[1] *= joint->Animatedscale.X;
                mat[2] *= joint->Animatedscale.X;
                mat[3] *= joint->Animatedscale.X;
                mat[4] *= joint->Animatedscale.Y;
                mat[5] *= joint->Animatedscale.Y;
                mat[6] *= joint->Animatedscale.Y;
                mat[7] *= joint->Animatedscale.Y;
                mat[8] *= joint->Animatedscale.Z;
                mat[9] *= joint->Animatedscale.Z;
                mat[10] *= joint->Animatedscale.Z;
                mat[11] *= joint->Animatedscale.Z;
                // -----------------------------------
            }
        }
        else
        {
            joint->LocalAnimatedMatrix=joint->LocalMatrix;
        }
    }
    SkinnedLastFrame=false;
}

[fmx]

I think the blending problem can be solved soon now because its clear the changes to the quaternion class were quite major since last release

i'm curious about the empty bone problem, because it sounds like the loaders are creating null joint nodes for no apparent reason.
Does it happen with ALL your animated models, or only sometimes?

for example, would this Goblin model you uploaded produce any empty bones with irrlicht?

u3d reports these bones in the file


are those all the bones which SHOULD be in the model?
i used u3d because i dont have any other means of checking B3D files, .X files can be investigated using only notepad

[Mel]

Yes, those are all. I have tested it against the SVN 4307 (the version i use currently for my games). Seems that the extra joints that appear in my models are exported with Panda. My X models are binary though ^^U And seems that no key is exported for them, so they never get animated, nor processed.
http://i49.tinypic.com/21j213q.jpg

[CuteAlien]

I can explain the IRR_TEST_BROKEN_QUATERNION_USE stuff. This define is only there to find problems in _your_ code - and it is meant to break compiling. Basically all the places where compiling in your code breaks when this define is enabled have to be changed in your code to work with the new Irrlicht. So it is only there to help finding the places that have to be updated. If your own code still compiles when this is set then you don't need to change anything. If it fails compiling then you have to change some lines in your code. Please try this first.

What got changed are the quaternion-matrix conversions. The problem was that we mixed up left-hand and right-hand rotations in the past.

In Irrlicht this place with the comment basically was changed in a way that it continues to use the old calculations... so it shouldn't have broken stuff. But could certainly be I messed up something :-(

edit: To clarify what to do when you get compile-errors in your code:
- The lines that break when you pass a matrix to a quaternion - pass the tranposed matrix instead.
- The lines that break on quaternion::getMatrix - use quaternion::getMatrix_transposed instead.
After all such lines have been changed you can reset IRR_TEST_BROKEN_QUATERNION_USE and it should behave like in the past.

[christianclavet]

Hi. Thanks for the clarified info. I'll try it back when I get home.

By the way, I'm not using any Matrix or quaternion operation directly for the moment. (At least, I don't think so)

For the skeleton, I think these are not used (removed them and check them back in IRB with no changes):
GoblinLPlatform
GoblinRPlatform
GoblinLIKTarget
GoblinRIKTarget

Theses 4 bones are the IK goals of the rig. All the other bones are needed.

As for the problem, I was able to replicate it 100% all over the weekend:
- No animation blending: - Goblin walk normally
- Animation blending activated: - Goblin do the moonwalk
- Animation blending activated under 1.7.1: Clean movement, and walk normally

I'm using this mesh, as it's the one that give me the more evident results, I've got minor issues with some of the models I'm using (As another example: a magician, that broke his back in a reverse back split to send a spell )

EDIT: Followed the instructions. The application build without any errors. At runtime, the gobling still do the moonwalk..

[fmx]

I was checking out the quaternion stuff today, mainly for physics not for skeleton blending, and everything seems to be fine in 1.8

i think the problem is that DX-style matrices haven't been used everywhere throughout irrlicht like they should have been, because there is usually very little need (if any) to ever request for transposed matrices.
only times when transposed matrices are needed is when dealing with other libraries that use GL-style matrices, and within the GL drivers internally

so there shouldn't be any need to use transposed matrices anywhere else in irrlicht, the stuff that does need it should be edited to work with DX-style matrix calculations (where possible) to avoid this kind of problem happening in the future

i'm not sure about skeletal blending yet, but my guess is that skinned model loaders are making assumptions that matrices stored in the files will be either column or row major by default, but that isn't always the case

how about allowing programmers to pass a flag when loading models to tell irrlicht if the loaded files are expected to have DX or GL style matrices?

so something like

Code: Select all

virtual IAnimatedMesh* irr::scene::ISceneManager::getMesh ( const io::path & filename, bool assumeRowMajorMatrices )

default value for assumeRowMajorMatrices can still be determined by loader, but allowing some kind of manual control over the loading process during runtime would be very useful

[CuteAlien]

@christianclavet: Can you write an example for me to reproduce the problem? Otherwise this will take probably rather long to find as I never did any animation blending, so figuring out how to reproduce this means I first have to learn how to write such an application at all.