Animation system with bones and Kinect

Tekkai · Post by **Tekkai** » Sat Apr 30, 2011 3:17 pm

Seems that things starts to work... I just tried what you said mongoose7 but I realized that I don't have to do the -Z stuff....

Now I have one arm working propertly, I'll start to work with the whole skeleton.

By the way, I attach now the images I said posts ago, in case they are useful to other users

Descrition of the mesh used

T Position

ForkPosition

Problem with depth

Tekkai · Post by **Tekkai** » Sun May 01, 2011 6:13 pm

New update with bad news... The forearm stuff works pretty well by I think it's just casuallity.

I'm trying to work with the rest of the body and I didn't have the results I was expecting.

What I'm trying to do now is to understand how the animation systems works. To do that, what I have done is take control of the hip bone, like this:

this->setJointMode(EJUOR_CONTROL);
this->animateJoints();

this->getJointNode("rhip")->setRotation(...);

What I pretend to do now, is to put the model in the same position that if I never toke the control of the bone, so I'm getting the initial rotation of the bones, and then try to apply it again to the hip bone so I get the "same".

Here I have my hierarchy (from 3DMax)

and what I'm doing is:

Code: Select all

//Class def:   std::vector< core::quaternion > lInitialRotations;


//First time I read geometry:
if ( firstTime )
{
    this->setJointMode(EJUOR_CONTROL);

    lInitialRotations.push_back(core::quaternion(this->getJointNode("neck"       )->getRotation())); //0
    lInitialRotations.push_back(core::quaternion(this->getJointNode("torso"      )->getRotation())); //1
    lInitialRotations.push_back(core::quaternion(this->getJointNode("pelvis"     )->getRotation())); //2
    lInitialRotations.push_back(core::quaternion(this->getJointNode("root"       )->getRotation())); //3

    lInitialRotations.push_back(core::quaternion(this->getJointNode("lshoulder"  )->getRotation())); //4
    lInitialRotations.push_back(core::quaternion(this->getJointNode("lelbow"     )->getRotation())); //5
    lInitialRotations.push_back(core::quaternion(this->getJointNode("lhip"       )->getRotation())); //6
    lInitialRotations.push_back(core::quaternion(this->getJointNode("lknee"      )->getRotation())); //7

    lInitialRotations.push_back(core::quaternion(this->getJointNode("rshoulder"  )->getRotation())); //8
    lInitialRotations.push_back(core::quaternion(this->getJointNode("relbow"     )->getRotation())); //9
    lInitialRotations.push_back(core::quaternion(this->getJointNode("rhip"       )->getRotation())); //10
    lInitialRotations.push_back(core::quaternion(this->getJointNode("rknee"      )->getRotation())); //11

}

//Moving the mesh

    this->animateJoints();

    core::quaternion qrhip      = lInitialRotations[10];
    core::quaternion qtorso     = lInitialRotations[1];
    core::quaternion qpelvis    = lInitialRotations[2];
    core::quaternion qroot      = lInitialRotations[3];

    core::quaternion rotAccum = qrhip * qpelvis * qtorso * qroot;
    //core::quaternion rotAccum = qroot * qtorso * qpelvis * qrhip;

    
    core::vector3df  lRotationV;
    rotAccum.toEuler(lRotationV);
    
    core::vector3df lRotationVDEG;
    lRotationVDEG = lRotationV * core::RADTODEG;
    
    this->getJointNode("rhip")->setRotation( lRotationVDEG );

As far as I can understand, that should put me the mesh in the original position, but it's not. I know that this seems stupid, but I have to understand why is not working if I really want to connect kinect with irrlicht. I tried both rotAccum lines, to see if there was a problem with the order of multiplying the quaternions, but didn't work.

I also tried to work with this->getJointNode("rhip")->setSkinningSpace( scene::EBSS_GLOBAL ); but without any result...

Any thoughts please?

ChaiRuiPeng · Post by **ChaiRuiPeng** » Sun May 01, 2011 6:20 pm

why are you setting EUJOR_CONTROL, then animating the joints with

Code: Select all

animateJoints()

? i might be mistaken but i though if you picked EUJOR_CONTROL then you don't have to call animateJoints(), since that is setting it to the predfined animation...

mongoose7 · Post by **mongoose7** » Mon May 02, 2011 2:03 am

To Tekkai:

You have a lot of detail here and I don't know whether anyone is going to pick through it. So what about keeping it simple.

The skeleton you exported from Max, does it have rotations? If so, I would try to eliminate them. Then you can eliminate all the "getRotation()" code.

Now, what is your problem?

Tekkai · Post by **Tekkai** » Mon May 02, 2011 8:42 am

@ChaiRuiPeng

Ok, I remove it, I thought it was necessary but I just check and "nothing happens". So I'm in the same trouble... Thanks for the try though

@mongoose7

I don't have control about the rotations of 3DMax. My mesh has a biped, and the rotations are done by default. If I "remove the rotations" my mesh get twisted. I know that the code seems difficult, but in concept the only thing I'm doing is get the rotations of all the bones, and apply the original rotation to the rhip bone.. that shoud work right?

Thx everybody

mongoose7 · Post by **mongoose7** » Mon May 02, 2011 8:56 am

To answer that I would have to know the flow of your program. But why do you call animateJoints? Why do you combine all the rotations: qrhip, qpelvis, qtorso, qroot. They're not even in the right order. And Irrlicht should proagate the rotations itself - you should just set up the rotation from one joint to the next.

You are starting from a place I've never been to and are simply getting further away by the minute. Bon Voyage!

Tekkai · Post by **Tekkai** » Mon May 02, 2011 9:40 am

I just stopped to use animateJoints(), I think I was confused about the use of it.

I combine the rotations because I'm not sure if Irrlicht is propagating the rotations from the root bone. If you look at the image I attached from 3DMax it should be in the right order, aren't they? Maybe I'm confused about that

So, if irr propagate the rotations, that

Code: Select all

core::quaternion rotAccum = qrhip;

should work right? (I'm using the same code above)..

mongoose7 · Post by **mongoose7** » Mon May 02, 2011 10:39 am

OK I see that the Max model is weird. Everybody else in the whole wide world has hips connected to pelvis, not torso. The root bone should be the pelvis, the hips should come of the root bone.

If you can go back into Max and start again, make the skeleton with translations and no rotations. I'm sorry, I am not a modeller so I can't offer you any suggestions.

Alternatively, with the skeleton you have, set the joint rotations to the identity after loading. This will put the bones out of place but, with the skeleton display turned on, you should still be able to see the skeleton inside the mesh. Try not doing any animation, including not loading any, as the animation code uses quaternions and the loading code uses Euler angles (hey, I didn't design the library).

Get everything set up nicely like this first. Then you can tackle the animation.

Tekkai · Post by **Tekkai** » Mon May 02, 2011 10:50 am

Hey mongoose7, thanks for your help. I'm not modeller too, so it's hard to me to work with the model. But I have I'm working hard on understand where is the problem. Here are some more results:

If I do that:

Code: Select all

this->getJointNode("rhip")->setRotation( this->getJointNode("rhip")->getRotation() );

it works!! Awesome!! Then the problem is between the conversion euler->quaternion->euler.

So I'm investigating further more where is the problem, I think is when I'm creating the quaternion from the euler angles given by GetRotation().

Tekkai · Post by **Tekkai** » Mon May 02, 2011 12:06 pm

Ok, I know that will sound stupid, but was all about the FUC**** conversions between radians and degrees... seems that some classes works on radians and others in degrees... that's my final aproach and seems to work... let's try to connect it again with Kinect

Code: Select all

    //First time
    this->setJointMode( EJUOR_CONTROL );

    std::vector< core::CMyQuaternion > lInitialRotations;

    lInitialRotations.push_back(core::CMyQuaternion(this->getJointNode("neck"       )->getRotation()*core::DEGTORAD)); //0
    lInitialRotations.push_back(core::CMyQuaternion(this->getJointNode("torso"      )->getRotation()*core::DEGTORAD)); //1
    lInitialRotations.push_back(core::CMyQuaternion(this->getJointNode("pelvis"     )->getRotation()*core::DEGTORAD)); //2
    lInitialRotations.push_back(core::CMyQuaternion(this->getJointNode("root"       )->getRotation()*core::DEGTORAD)); //3

    lInitialRotations.push_back(core::CMyQuaternion(this->getJointNode("lshoulder"  )->getRotation()*core::DEGTORAD)); //4
    lInitialRotations.push_back(core::CMyQuaternion(this->getJointNode("lelbow"     )->getRotation()*core::DEGTORAD)); //5
    lInitialRotations.push_back(core::CMyQuaternion(this->getJointNode("lhip"       )->getRotation()*core::DEGTORAD)); //6
    lInitialRotations.push_back(core::CMyQuaternion(this->getJointNode("lknee"      )->getRotation()*core::DEGTORAD)); //7

    lInitialRotations.push_back(core::CMyQuaternion(this->getJointNode("rshoulder"  )->getRotation()*core::DEGTORAD)); //8
    lInitialRotations.push_back(core::CMyQuaternion(this->getJointNode("relbow"     )->getRotation()*core::DEGTORAD)); //9
    lInitialRotations.push_back(core::CMyQuaternion(this->getJointNode("rhip"       )->getRotation()*core::DEGTORAD)); //10
    lInitialRotations.push_back(core::CMyQuaternion(this->getJointNode("rknee"      )->getRotation()*core::DEGTORAD)); //11

//Moving the bones

    core::CMyQuaternion rotAccum = lInitialRotations[10];

    core::vector3df lRotationV;
    core::vector3df lRotationVDEG;
    
    lRotationV = rotAccum.ConvertToEuler();
    lRotationVDEG = lRotationV * core::RADTODEG;
    
    this->getJointNode("rhip")->setRotation( lRotationVDEG );

By the way, I reimplement the conversion to euler and from euler because I saw a post saying there was a bug on it, and here is my approach (get from http://www.euclideanspace.com/maths/geo ... /index.htm)

That works if you are doing inheritance from core::quaternion

Code: Select all

     
     CMyQuaternion::CMyQuaternion(vector3df eul)
    {
        // Assuming the angles are in radians.
        f32 c1 = cos(eul.X*0.5f);
        f32 s1 = sin(eul.X*0.5f);
        f32 c2 = cos(eul.Y*0.5f);
        f32 s2 = sin(eul.Y*0.5f);
        f32 c3 = cos(eul.Z*0.5f);
        f32 s3 = sin(eul.Z*0.5f);
        f32 c1c2 = c1*c2;
        f32 s1s2 = s1*s2;
        
        this->W =c1c2*c3 - s1s2*s3;
        this->X =c1c2*s3 + s1s2*c3;
        this->Y =s1*c2*c3 + c1*s2*s3;
        this->Z =c1*s2*c3 - s1*c2*s3;
    }    

    vector3df CMyQuaternion::ConvertToEuler(void)
    {
        f32 sqw = this->W*this->W;
        f32 sqx = this->X*this->X;
        f32 sqy = this->Y*this->Y;
        f32 sqz = this->Z*this->Z;

        f32 heading = 0;
        f32 attitude = 0;
        f32 bank = 0;

        f32 unit = sqx + sqy + sqz + sqw; // if normalised is one, otherwise is correction factor
        f32 test = this->X*this->Y + this->Z*this->W;
        
        // singularity at north pole
        if (test > 0.499*unit) 
        { 
            heading = 2 * atan2(this->X,this->W);
            attitude = HALF_PI;
            bank = 0;
            return vector3df(heading,attitude,bank);
        }
        // singularity at south pole
        if (test < -0.499*unit) 
        { 
            heading = -2 * atan2(this->X,this->W);
            attitude = -HALF_PI;
            bank = 0;
            return vector3df(heading,attitude,bank);
        }
        
        heading = atan2( 2*this->Y*this->W-2*this->X*this->Z , sqx - sqy - sqz + sqw);
        attitude = asin(2*test/unit);
        bank = atan2(2*this->X*this->W-2*this->Y*this->Z , -sqx + sqy - sqz + sqw);
        
        return vector3df(heading,attitude,bank);
    }

mongoose7 · Post by **mongoose7** » Mon May 02, 2011 12:06 pm

OK. Good luck!