Is there a way to get animation data from a mesh and insert it into another mesh?

[SomeGuyWithAComputer]

I'm adding wearable clothes to the npcs in my game. I export the rigged skinned character models to b3d and then use those in Irrlicht as an IAnimatedMesh.

For clothes, I use the same exact bone structure as the respective character model and export that as a b3d file and it gets loaded into Irrlicht as an IAnimatedMesh. I then position that in the right spot on the npc and use a loop that takes all the bone rotations and positions from the npc mesh and copies that to the bones on each clothing mesh. The trick to make this work is to do

Code: Select all

bone->setAnimationMode(EBAM_UNANIMATED);

on each clothing bone and then

Code: Select all

node->setJointMode(EJUOR_CONTROL);

on the ISceneNode for the clothing on spawn.

This works but it's slow. It turns out rendering a mesh with the

Code: Select all

EJUOR_CONTROL

mode is relatively slow. I optimized the crap out of my bone matching system before realizing this. Even if I make the clothing single-mesh, full-body only and not separate them into pants, shirt, boots, helmet, whatever it's STILL too slow.

There is however another way. A faster way. If I instead, in Blender, assign the same exact animations that's on my npc model's armature onto the clothing armature, then I make my code simply make the animation frame of the clothing equal to the animation frame of the npc model at all times, this works great and is really fast. The code to do this is really simple one-liner that runs on every frame:

Code: Select all

clothing->getNode()->setCurrentFrame(getNode()->getFrameNr());

This is almost a good solution except now every time I change any character animations, I would have to make that change to every clothing model ever. This is highly undesirable. There has to be a better way.

My idea: what if I found a way to extract the animation data from my main npc mesh and somehow inserted it into the clothing meshes? That way it copies all the animations from 1 source and I don't have to worry about clothing always having the same animations as the npc model.

Is such a thing possible in Irrlicht? Where would I even begin? Would I be better off reverse engineering the Blender b3d exporter to figure out how to process this data and then make it in c++ so my game can do it while loading resources or is there a way involving Irrlicht that this can be done easier? Maybe there's a way with getMeshBuffer for example.

[CuteAlien]

Do you use skinned meshes (skeletal animations)? ISkinnedMesh has a function useAnimationFrom which allows you do use the animation from another model. If I remember right you will have to clone that other mesh so each node still needs it's own mesh (skinned meshes save animations in their mesh instead of in the nodes using them).

In the past I had changed a .b3d exporter to only export animations so those were separate from the mesh, not sure if current exporters support splitting like that (I think they do, but haven't needed this in a while myself).

[SomeGuyWithAComputer]

I know of skinned meshes but I haven't really figured out how to set them up yet.

It's my understanding that first you do something like

Code: Select all

ISkinnedMesh *m = sceneManager->createSkinnedMesh();
m->addMeshBuffer();

then you somehow transfer the mesh buffer data from the IAnimatedMesh, and then you somehow convert the joints from the IAnimatedMesh to SkinnedMesh::SJoint objects and add them with m->AddJoint(). But then you have to probably reconstruct Blender-like rotation and position keyframes with ISkinnedMesh::addPositionKey and ISkinnedMesh::addRotationKey instead of setting them directly? Except there are also weight functions, so I have to somehow get the weights of each vertex and apply it to each bone too?

I haven't even figured out where to start on figuring out SkinnedMesh, but its not like I expected this to be easy so it is what it is. Is there an example of a irrlicht SkinnedMesh somewhere?

[Noiecity]

Oh, a real problem I always face is that it's difficult to automate tasks if you don't find the right algorithm. In theory, what you describe is correct. I don't usually use bones, and I've simply tried, for example, when equipping my character with a helmet, creating a specific animation for that character, or creating different versions of the same model to improve performance (loading or releasing resources correctly).
In fact, I have to adapt each armor and clothing for each character, since a Gnome is not the same as a common human. An algorithm could be created that runs in Irrlicht before entering the main loop, and once in the main loop, I have my new geometry already fixed. I also save it in some type of file... but what you describe is a significant problem to solve, practically creating a mini AI that automates that task (copying the animation of the clothing from one character to another and adapting it so that it is consistent).

In my case, I export to b3d using Blender, convert it to md2 so as not to use bones (using Fragmotion), and once it's md2, I'm left with an animated character with many keyframes as if it were a single animation... I separate the frames using qME and even name them, and I can call them using that same name using the Irrlicht API... I get much better performance. I don't know why bones cause so much latency. They take away about 200-400 fps when running at 600 fps (when there are many bones or animated characters), while md2 takes away about 40 or 70 at most... o wait, i know why a bit:

md2 is a binary format, .x is a text-based format - that's likely why it's smaller. At least for this case. I don't know that much about .md2 really as I haven't worked with it yet, but I just checked the source and if I understood it correct on a quick view it saves the whole mesh per frame. And then interpolates between those frames. So indeed no bone-based animation. Which has advantages sometimes (when meshes are small and the animation is short and fixed). Thought with enough long animations it's probably going to be way larger.

And Irrlicht doesn't really have unified those animation systems aside from the interface. But in theory it would be possible to create a frame based animation like that from a bone animation system to cache the frames. Which could be quite useful for optimization (I thought about that a few times already) in some cases (small, fixed animations for small meshes).

But well, something for the far future I suppose. Unless someone else works on that.

Bit tired right now, but I guess setFrameLoop could have another parameter to avoid resetting the loop. As workaround I think you can remember last frame before changing it with getFrameNr() and after changing you can set it again with setCurrentFrame (and don't ask me why those 2 function names aren't symmetric... I'm pretty sure they are about the same thing).

viewtopic.php?t=53041
https://gamedev.net/forums/topic/533347 ... animation/
PS: I remember that the DX9 SDK had examples of how to use SkinMesh with shaders. It might be useful to you. I remember changing the size of a part of the body using that or something like that, but I don't remember exactly (I guess it could be used for your purpose of “adapting” your animation to a different geometry)

Edit: I don't know if this will help, but you can upload the Irrlicht engine in a .zip file, upload it to ChatGPT, and have it analyze the project code so that it can give you some options once it has the full context. It may give you a complete solution, or it may help you move forward with what you want to do.

[CuteAlien]

Phew, no example. I only did this once over 15 years ago and Irrlicht changed since then :-(
But I looked it up and it was a bit different than I remembered. Also I might not have done the exact same thing you did. Maybe I'm misunderstanding.

But - you can use animation data with the mentioned useAnimationFrom from one mesh to another.

My use-case back then was the following: I had several models with the same bone-system. All in b3d format. So different human meshes - but same bones. And I had animations exported for those bones - also as b3d format. Loading the human models was just done the usual way, like example 09 which loads a skinned mesh and then uses addAnimatedMeshSceneNode to create the node.

Then I loaded animation b3d, also the usual way. And then for the animated human model-nodes I used the animation from the animation mesh like this:

Code: Select all

irr::scene::ISkinnedMesh* skinnedMesh = static_cast<irr::scene::ISkinnedMesh*>(mModelNode->getMesh());
skinnedMesh->useAnimationFrom( animationMesh );

But one problem - the animation isn't saved in the node but in the mesh. So as soon as I had more than one node per mesh (using same human mesh twice etc), then I needed one mesh-copy per node. Back then I wrote a function to create copies of animated meshes for that (createSkinnedMeshCopy - you can find it here: viewtopic.php?p=256162). But possibly that's no longer needed (that was back in Irrlicht 1.6) and it's maybe possible to simply call clone() now on the mesh you want animated more than once. Then use that copy in addAnimatedMeshSceneNode. So... step 1 is load mesh. Step 2 is create copy. Step 3 is addAnimatedMeshSceneNode with mesh-copy.

I don't know if this works with your clothes. You have to try.

Noiecity's solution is avoiding skeletal animations. Which can be faster, but won't allow applying animation from one mesh to another. But each mesh has to have all animations.

[SomeGuyWithAComputer]

Ok so maybe I can make a SkinnedMesh based off of the AnimatedMesh that's too slow and it'll work the same as before except faster. I'm going to try that thing where a loop gets all the bone rotations and positions from the npc mesh and copies that to the bones on the skinned mesh clothes because if that could work without the several milliseconds of lag per mesh I'm getting when it draws a joint-modified animated mesh, that would be the most versatile option.

In order to find out if this is even going to work the way I need it to, I need to actually create a skinned mesh first. I haven't exactly figured out how to transfer bones and weights from an AnimatedMesh to a SkinnedMesh but I did come up with this and it does at least not crash when run.

Code: Select all

IAnimatedMesh *m_objMesh = functionThatLoadsMeshFromDisk();	//not the actual function name just added for context
m_skinnedMesh = irrsmgr->createSkinnedMesh();			//create skinned mesh on the ISkinnedMesh member object
SSkinMeshBuffer * buffer = m_skinnedMesh->addMeshBuffer();
buffer->append(m_objMesh->getMeshBuffer(0));
m_skinnedMesh->finalize();					//loaders should call this after populating the mesh

Unless I'm missing something, it seems joints can only be fetched from IAnimatedMeshSceneNodes. Perhaps all the bones and weights are contained in the mesh buffer. Anyway, next is the challenge of figuring out how to get a skinned mesh into the scene. I tried the obvious:

Code: Select all

ISceneNode *n = irrsmgr->addAnimatedMeshSceneNode(m_skinnedMesh);

If all I do is put that node in the scene, it doesn't crash but it also doesn't show up. There isn't a ISkinnedMeshNode. I haven't yet tried doing this with a IMeshSceneNode but I didn't think I would ever have to use IMeshSceneNodes so I have to restructure some stuff for that first.

Is this a good approach?

[CuteAlien]

If you have bones and weights then you already have a skinned mesh.

[SomeGuyWithAComputer]

Yeah thats right. I ended getting that to work but also I can just do this one-liner.

Code: Select all

m_skinnedMesh = (ISkinnedMesh*)m_objMesh;

Then when I add it to the scene with irrsmgr->addAnimatedMeshSceneNode(m_skinnedMesh); it shows up.

Now I get to find out if there was a point to all this by trying to animate it without using the normal animation system and hope and pray its actually faster.

I made a loop that takes a list of parent bones and sets the skinned mesh bones to match like so:

Code: Select all

for (int u = 0; u < m_clothingBones.at(i).getNumBoneMatchKeys(); u++)
{
	IBoneSceneNode *pbone = m_clothingBones.at(i).getBoneMatchKeyAt(u).parentBone;
	ISkinnedMesh::SJoint *sbone = m_clothingBones.at(i).getBoneMatchKeyAt(u).clothingJoint;
	sbone->LocalAnimatedMatrix = pbone->getRelativeTransformation();	//set each SJoint to equal the position of the corresponding bone of the parent object
}
clothing->getSkinnedMesh()->skinMesh();	//I presume that running ISkinnedMesh::skinMesh() is supposed to apply joint changes to the mesh buffer if set up correctly

This doesn't result in any movement on the clothing mesh. There must be more steps besides changing the joint positions and rotations and then running skinMesh(). I presume that this works by editing the mesh itself and not just an instance of the mesh so I would have to give each instantiated thing its own copy. This would be fine if it would work.

I tried it with the IAnimatedSceneMode::setJointMode set to EJUOR_CONTROL and EJUOR_NONE. Note that ANY time my clothing mesh is drawn during ISceneManager::drawAll() while joint mode is set to EJUOR_CONTROL, it adds around 2ms per frame per mesh whether any joint changes happened or not. Using a skinned mesh, I presume, should bypass that and instead of using animation frames it edits the mesh that is referenced by the scene node. I somehow some way need to find a way to have a different animation on the upper body and a different animation on the lower body of my character mesh and clothing meshes with something besides 2ms of lag per frame per mesh.

Also, how do skinnedmesh rotation keys and position keys work? They take a SBone as a parameter but the obvious thing of adding each SBone as a position key and a rotation key results in the same old problem where nothing ever changes when I modify the rotations or positions of any Sbone. Perhaps manually setting bone positions in skinned meshes instead of using a non-blendable animation exported from blender isn't its intended use, but it should still be possible to set the animation to frame 1 and then modify all the bones at frame 1 only to get the level of control I need over the skeleton to do the type of animation blending I need.

[CuteAlien]

I don't get yet what you do exactly, but doing separate animation for top and bottom is indeed hard to solve in Irrlicht. The animation system is simply a bit too old for stuff like that.

My info which I figure out some months ago:

animateMesh updates the joints. skinMesh updates the meshbuffers aka SSkinMeshBuffer. So SSkinMeshBuffer changes each frame for every node using it. So having one instance per node is probably the best way (unless nodes do the exact same animation, it might not do it twice then... I hope).
There is no "original" unchanged mesh copy around to which stuff is applied... at least not directly. Instead each SWeight holds a copy of an original meshbuffer vertex with StaticPos, StaticNormal (so you could figure out the original meshbuffer from that, but that would be some work). Don't ask me why it was coded like that, I do not know, guy who coded was was gone before I started with Irrlicht.

So basically - SSkinMeshBuffer is updated by the weights which know the original position and normals. And weights have indices to the vertices they have to update so they know which vertex they have to update in the SSkinMeshBuffer. So if more than one weight updates a vertex then it's important it scales it's importance down correspondingly as the sum of all weights affecting a vertex has to be 1 for that to work out.

CAnimatedMeshSceneNode might help with the interpolation handling between 2 keys, but I haven't figured out yet in detail how that works. It always seems to create IBoneSceneNodes (JointChildSceneNodes). I thought those were only helpers, but maybe they are used for more? (those might cost some time so mentioning this).

Note also that animated meshes are indeed expensive if they are large - as each vertex is updated each frame. And it's done on CPU. So this is a case where every vertex costs time.

Use some profiler like "Very Sleepy" to find out what exactly is slow in your case.

[SomeGuyWithAComputer]

I used the microsecond timer capability of std::chrono to find out the unacceptable slowness happens within ISceneManager::drawAll and only IAnimatedMeshes with their joint mode set to EJUOR_CONTROL causes this lag when being drawn with drawAll. If there was an obvious way to change Irrlicht to handle IAnimatedMesh nodes set to EJUOR_CONTROL 99% faster I'm sure it would already be part of irrlicht (but i guess thats the route i'll try if I never solve all my ISkinnedMesh problems). I can't really make my meshes any simpler, they're already 500-1500 verticies. I think it's still worth going the ISkinnedMesh route. I did actually find a way to:
1) modify the SJoint rotations of ISkinnedMesh directly in a way that actually shows visible changes
and
2) do it really really fast.

There is one caveat to this. I can modify the SJoint rotations all right, I can't get them to do the rotations I want them to. I'll explain how I did this first.

When spawning the ISkinnedMesh into the scene, first I loop through all the SJoints. for each SJoint, I do:

Code: Select all

//get rid of any existing keyframes that might interfere with the way i manipulate bone rotations by modifying keyframes
sbone->RotationKeys.clear();
sbone->PositionKeys.clear();//nothing different happens if i dont also do with with positions, so i might as well do it anyway

//set up the new blank keyframes
((ISkinnedMesh*)mesh)->addPositionKey(sbone);
((ISkinnedMesh*)mesh)->addRotationKey(sbone);

((ISkinnedMesh*)mesh)->addPositionKey(sbone);
((ISkinnedMesh*)mesh)->addRotationKey(sbone);

((ISkinnedMesh*)mesh)->addPositionKey(sbone);
((ISkinnedMesh*)mesh)->addRotationKey(sbone);

This clears any existing Blender animation keyframes so that I can abuse the keyframe system to insert my custom rotations based on another mesh's bones. I found that it doesn't work at all unless I create 3 keyframes, set the 3 keyframes per bone to frames 0, 1 and 2 and run setCurrentFrame(1) on the node once per frame.

Now in my update function, for every frame, I grab rotations from the bones on the other mesh that I want to set this SkinnedMesh node to and set each keyframe on each SJoint to that. On each SJoint I want to match to a specific bone, I do this:

Code: Select all

IBoneSceneNode *pbone = getTheCorrectBone(args);
sbone->PositionKeys[0].frame = 1;
sbone->PositionKeys[0].position = pbone->getPosition();
sbone->RotationKeys[0].frame = 1;
sbone->RotationKeys[0].rotation = core::quaternion(pbone->getRelativeTransformation());

sbone->PositionKeys[1].frame = 1;
sbone->PositionKeys[1].position = pbone->getPosition();
sbone->RotationKeys[1].frame = 1;
sbone->RotationKeys[1].rotation = core::quaternion(pbone->getRelativeTransformation());

sbone->PositionKeys[2].frame = 1;
sbone->PositionKeys[2].position = pbone->getPosition();
sbone->RotationKeys[2].frame = 1;
sbone->RotationKeys[2].rotation = core::quaternion(pbone->getRelativeTransformation());

This results in actual movement on the ISkinnedMesh that syncs up with what the target mesh is doing. There's just 1 teensy little problem: all the rotations are completely borked and make no sense. I've tried using absolute transformations which also doesn't work. So now I'm stuck on this. Here's a picture.



This seems like it could be solvable. I might have to make a video to show how the movement of it really does sync up with the animation on the character mesh and looks like its kind of close to being correct. I just can't figure out what format the rotation keys are supposed to be in or what I can do to adjust them to be correct.

[CuteAlien]

Phew, diving into the animation system isn't a quick thing :-( I'm still not understanding why you can't work with useAnimationFrom and try to implement it yourself. You have the same bones and want to use the same animations on another mesh? Or am I missing something? Why do you need EJUOR_CONTROL?

And please give Very Sleepy (http://www.codersnotes.com/sleepy/) a shot for profiling. Compile your project in release but with debug data. Then it's super simple to use (just start it and click the running exe, that's all).

And there might be optimizations possible in the animation system. If you have me a test-case I can easily use I can also take a look.

[SomeGuyWithAComputer]

My character meshes have full-body running and walking animations which I consider primary animations. They also have full-body gun shooting/reload/miscellaneous animations which I consider secondary animations.

Primary animations and secondary animations need to be able to be played entirely independently of each other.

You cannot do that in Irrlicht without doing a bunch of whacky stuff, at least not without splitting the character meshes into a top half and a bottom half which is going to come with its own issues and problems to overcome.

useAnimationFrom applies an animation to an entire mesh and all bones. This approach is not compatible with what I'm doing. I need to be able to apply that animation to only some of the bones of my choosing.

EJUOR_CONTROL, the only way to directly control the bones on an IAnimatedMeshSceneNode that has an IAnimatedMesh and therefore the only way to implement primary and secondary animations on IAnimatedMeshSceneNode objects that contain IAnimatedMesh meshes, is too slow. Even if I use the EJUOR_CONTROL method on an IAnimatedMeshSceneNode that has a ISkinnedMesh, it's just as slow.

It's possible to use keyframes to fenangle ISkinnedMesh meshes that are attached to IAnimatedMeshSceneNodes so that you can directly control them similar to how I'm doing it with nodes that have an IAnimatedMesh on them instead of an ISkinnedMesh. Doing this is FAST but the only method I have found to do this (keyframe manipulation shenanigans) seems to EITHER require the rotations to be in some weird format that I haven't been able to figure out OR there are additional steps that have to be done before this can work predictably hat I haven't found out about yet.

If the ISkinnedMesh keyframe system is just bugged, not implemented or otherwise cannot actually be used this way despite my initial success, then I guess I'll start hacking away at the IAnimatedMeshSceneNode source code and trying to improve whatever in drawAll() is so slow if there's just no other way. I'm so close that it just seems like I'm ONE last problem away from getting this to work with ISkinnedMesh. I just have to crack the code on why the keyframe rotations don't behave in a way that makes sense when I make new ones and set my own rotation values.

[CuteAlien]

I think one case where EJUOR_CONTROL was pretty bad was deep hierarchical bones because it calculated the transformation matrix for that a lot. At least I faintly remember someone mentioning that once. Otherwise on a quick look I don't see where it slows down - CSkinnedMesh::transferJointsToMesh shouldn't take that much time I think. Thought I'm not sure why it doesn't just set LocalAnimatedMatrix to getRelativeTransformation of the node (maybe didn't trust it or was working different in the past).

Maybe I'm missing some memory allocation going on somewhere. Sadly I have no example so far for working with EJUOR_CONTROL - one of the things I've not done yet myself so I can't test.

About rotation - quaternion has a constructor directly using euler angles. So you should be able to pass node->getRotation() instead of getRelativeTransformation() (which can include scale and position and while it ignores the latter it has to remove scale again expensivly and that part is a bit tricky as rotation 180° and scale -1 are kinda the same as no rotation so you have some information loss there).

Maybe adding a bool to SJoint to allow disabling certain joints would help? (so only parts of the animations are used). Thought it's probably risky as if there are several joints affecting same vertex all have to be disabled or weights no longer sum up to 1 (and then things mess up).

[CuteAlien]

I wrote me some quick test for EJUOR_CONTROL: https://github.com/mzeilfelder/irr-play ... ontrol.cpp
Turns out meshes have no clone() so my old function to copy them was still needed.
But didn't noticed any unexpected speed problems, maybe compare what you do different.
The most expensive part is simply the animating of the vertices - which is what I'd expect (especially as it happens on CPU).
We could maybe make recalculating the bounding-box optional,that could save 10-15% and in most cases won't matter (bit expensive has it also has to check every vertex).

So please run Very Sleepy on your application - if skinMesh/skinJoint are not the expensive ones, then you do something wrong.

[Noiecity]

I wrote me some quick test for EJUOR_CONTROL: https://github.com/mzeilfelder/irr-play ... ontrol.cpp
Turns out meshes have no clone() so my old function to copy them was still needed.
But didn't noticed any unexpected speed problems, maybe compare what you do different.
The most expensive part is simply the animating of the vertices - which is what I'd expect (especially as it happens on CPU).
We could maybe make recalculating the bounding-box optional,that could save 10-15% and in most cases won't matter (bit expensive has it also has to check every vertex).

So please run Very Sleepy on your application - if skinMesh/skinJoint are not the expensive ones, then you do something wrong.

Sometimes CPU consumption measurements are misleading, meaning they don't always show actual consumption, since the cycles are too fast to measure accurately, especially when you have an operating system running in the background(Waiting for messages from the program to the operating system and verifying correct memory accesses in each cycle, kernel...). I know this is irrelevant, but you can also consider your CPU temperature over a long period of iterations and compare. Consider that it is difficult to measure time in less than 16.6 ms. Now imagine when a mathematical calculation using an arithmetic-logic unit or ALU, which can consume very little CPU, However, I have seen that it misleadingly shows higher consumption than the actual consumption when trying to calculate an average (out of 800 cycles, only 1 was 20%, for example, the other 799 were 0%. When measuring, you can conclude inaccurately and somewhat randomly that you actually consumed an average of 5%, which is really misleading).

P.S.: By the way, thanks for mentioning Sleepy. I didn't know about it, and it was something I needed to measure CPU consumption. It's quite accurate when it comes to seeing how much a function consumes(when consumption can actually be measured approximately)