IAnimatedMesh named animations

[bitplane]

I've been thinking about this for a couple of days and can't decide on the best solution. Currently, we have a few different animated mesh formats, each with their own interface which exists mostly to set animations:

• IAnimatedMeshMD2: Interface exists only to get the MD2 specific animations. Could be removed if IAnimatedMesh had named animations
  IAnimatedMeshMD3: Similar, but exposes joint (MD3 tag) information too. If IAnimatedMesh had joints then we might be able to remove this too.
  ISkinnedMesh: Exposes joints and skinning options, yet mostly serves as a builder class for skinned meshes. Could reworked into a mesh creator only.

For named animations, at the very least we'd need:

• IAnimatedMesh::getAnimationCount()
  IAnimatedMesh::getAnimationName(index)
  IAnimatedMesh::findAnimation(name) // maybe some other name?
  IAnimatedMeshSceneNode::setAnimation(index)
  IAnimatedMeshSceneNode::setAnimation(name)

Now, the actual animation data is the tricky part. Different animated mesh (file) formats support different features, future mesh formats may support an arbitrary set of animation-specific data, for example:

• Name
  Start and end frames of the entire range
  The type of loop, (none, cycle, ping-pong, play X times)
  The looping part of the animation (pick yourself up, then stand normally)
  The relative frames per second of this animation compared to others

I can see a few ways to solve this:

1. Complete IAttributes
Animation data is passed directly to the animated mesh type to build a frame
pros: Flexible, easy to serialize, unlimited number of input and output parameters, can easily edit internal animations using IAttributes
cons: Slow as IAttributes are used each frame, potential caching issues.
API change:

• IAttributes* IAnimatedMesh::getAnimationData(index)
  IAnimatedMesh::getMesh(IAttributes *animationData, timeSinceStart)
  IAnimatedMeshSceneNode::setAnimation(IAttributes* animationData)

2. Hidden IAttributes
Animation data is passed in via IAttributes, but mesh types hold this internally in their own implementation-specific format.
pros: Still quite flexible, fast access to mesh data, cacheable.
cons: Requires write access to IAnimatedMesh, harder to serialize.
API change:

• IAttributes* IAnimatedMesh::getAnimationData(index)
  IAnimatedMesh::getMesh(animationIndex, detailLevel, timeSinceStart)
  s32 IAnimatedMesh::addAnimation(IAttributes *animationData)

3. Struct
Animation data struct containing all possible members.
pros: Fast, self-documenting, cacheable
cons: Impossible to extend from outside the engine, bloats API over time as more members are added
API change:

• struct IAnimatedMesh::SAnimationData { startFrame, endFrame, relativeFPS, loopMode, etc }
  SAnimationData& IAnimatedMesh::getAnimationData(index)
  IAnimatedMesh::getMesh(SAnimationData data, detailLevel, timeSinceStart)
  IAnimatedMeshSceneNode::setAnimation(SAnimationData data)

So, which way do you think is best, or can you think of a better way? Are there any problems I've missed?

[bitplane]

Oh, here's another idea:

4, Everything internal
Users don't get to mess with the animation data at all, they just ask for the frame loop current frame based on the animation name or number. Mesh writers can get the animation data in the form of IAttributes, but it's held internally in some other format and read-only.

pros: compatible with all the other methods!
cons: Can't think of any (prove me wrong!)

API change:

IAnimatedMesh::getFrameLoop(animationNumber, timeSinceStart, outFrame, outFrameLoopStart, outFrameLoopEnd)
IAttributes* IAnimatedMesh::getAnimationData(index)

[3DModelerMan]

I've implemented a wrapper around the animation system already that works like that. It really works well for controlling animations. Especially if you have a getCurrentAnimationName function that returns the current animation's name. That way you can check to make sure you don't try to play the animation again, and get it stuck on the first frame alot easier than before. Another thing I did, was I stored animations in an SAnimation structure, and also made animations able to be seperated from the animated mesh scene node.

[Edit]
Right now I'm also implementing a priority system for animation sequences. Animation blending would be nice after the named animation sequences. Are we also going to be able to split sequences from frame numbers?

[bitplane]

I'm beginning to think that "frame numbers" and "FPS" don't make sense as part of the public API, they're a kind of artefact left over from when SAnimatedMesh and IAnimatedMeshMD2 were the only mesh types.

"frames per second" implies a linear playback speed. Consider an animation that starts slow and speeds up on a curve, at any moment in time it's running at a different FPS. Also consider setting different animations to each body part, ones that run at different speeds. How many FPS is the animation running at?

"Frames" themselves are also implementation-specific details, consider a dynamic animated mesh, say if the water surface node was actually an animated mesh (like it should be). A sensible representation here would be an interval from 0.0 to 1.0 rather than integer frame numbers.

The only concepts that really apply to all possible animated meshes are animation type, animation speed and amount of time passed since the animation started.

Speed can be simulated by adjusting the time, with "animation speed" being a factor (i.e. 2.0 for double speed) and belonging to the node, the adjusted time can be passed to the mesh.

Type of animation is a tricky one if it's still going to be possible to pass manual "frame loops", which it should. Not sure about the best way to go about doing this.

[bitplane]

Okay, slight update after discussion on IRC with Yoran:

CAnimatedMeshSceneNode::AnimationSpeed becomes a factor rather than being explicitly linked to FPS.
IMesh* IAnimatedMesh::getMesh(float timeInSeconds, int animationID, float levelOfDetail), to get a mesh at some specific time. CAnimatedMeshSceneNode passes timePassed*animationSpeed to timeInSeconds.
IAnimatedMesh::getAnimationCount and IAnimatedMesh::getAnimationName, IAnimatedMesh::findAnimation(name) for named animations.
IAnimatedMesh::getAnimationData(id, IAttributes*) to retrieve information about a named animation, the attribs are specific to this particular animation format.
IAnimatedMesh::get/set/addAnimation(IAttributes*) to edit the internal animations, a bit annoying to need to mutate the mesh, but I can't see another way to do it while retaining backward compatibility:

When CAnimatedMeshSceneNode calls setMesh(), mesh->addAnimation is called and the animation ID is saved. When the user calls setFrameLoop, the start and end frames are modified and passed to setAnimation(id, attribs). Animation serialization will work for any future mesh format via getAnimationData, cubic spline keyframed, feathered, mixed animations or whatever.

Next, bounding boxes also don't make sense for an animated mesh, you really mean "an animated mesh at a particular time". IAnimatedMesh::getBoundingBox(time, animationID) should be used here instead. This solves the problem of needing to create the mesh to know whether it should be culled or not, animated meshes can create a bunch of bboxes at keyframes and just interpolate.

Still thinking... others please speak up so I don't feel like I'm talking to myself

[hybrid]

Wouldn't milliseconds be a better scale for the time? And if findAnimation() does not return an animation (what type is animation anyway, IAttribute?) it might be better called getAnnimationIndex.
The bbox thing is a very important point. Will that change the base interface everywhere to getBoundingBox(time)? Because we cannot access the current time in the register process for those nodes otherwise.

[bitplane]

Wouldn't milliseconds be a better scale for the time?

Yep, you're right. If ms are used everywhere else it's probably best to go with that.

And if findAnimation() does not return an animation (what type is animation anyway, IAttribute?) it might be better called getAnnimationIndex.

If we can pass IAttributes to getMesh and have a potentially slow conversion to the internal animation type, then this could return IAttributes*. If not, we'll have to make IAnimatedMesh mutable by design, so each node adds a new animation to the IAnimatedMesh. This is the part I'm struggling with; mutable is bad, slow is bad, inflexible is bad, complex is bad. Which is worst?

The bbox thing is a very important point. Will that change the base interface everywhere to getBoundingBox(time)? Because we cannot access the current time in the register process for those nodes otherwise.

IAnimatedMeshSceneNode::getBoundingBox could check the ISceneManager's time, but yeah, this is ugly... it would be better to have it take a time, which is a huge API change

[shadowslair]

Ok, just my 2 cents:

1) The IAnimAttributes idea may do the job, not sure though. What I did is sth similar to 3dModellerMan - I have a simple struct, holding the animation name, start, end frame and speed. The .md2 format may load such predefined structs array when created setting some flag like: myMd2AnimNode->setFlag(scene::EAF_USE_Q2ANIM_SEQS, true); and the predefined sequences will be added or removed or sth similar. Then the user will simply call the animation by name (as usual for q2 .md2)- calling "run", attack" etc. which will match the new API. May be wrong though- haven`t used .md2 or .md3 for years.

2) When I`m animating stuff, I always have some frame rate in mind- usually 30 frames per second, then when calling the animation or changing its speed is easy too- most of the animators work this way- feels natural and at least it may be somewhat confusing for me to use the time in seconds or some factor.

3) About the bounding boxes created at the keyframes and being interpolated- I thought about this some time ago- sounds really temptating to interpolate the two keyframe bbox corners, which will be faster but it proved to be misleading. Here`s a very simple case where such an idea creates artifacts(in green): http://pics.data.bg/categories/1/%D0%BE ... tmap_image because it assumes that the inbetweens (mid keyframes) produce smaller bounding volume, which is wrong obviously. Even if the user uses the build-in frustum culling (frustum aabb vs node aabb) this will usually be really hardly noticeable, but still the character hand may be inside the view, but the character will be culled for the case above. Another solution may be to go through the whole animation at runtime and find the bbox with max extents and use it always, but this isnt very good solution, cause if there`s just one frame where the node generates say two or three times bigger bbox (could be even much bigger) this may result in false positives, no to mention that it may turn totally wrong if we`re animating manually. And all this is about culling- for the build-in raycasting or collision a proper bbox is always needed. So, recalculating the bbox with even thousands of vertices still may be worth it.

PS: you`re not talking to yourself- we`re all listening, just being quiet "in the shadows", cause this is more or less an important decision to make, so it rather needs a bit of thinking, then talking.

[bitplane]

The .md2 format may load such predefined structs array when created setting some flag like: myMd2AnimNode->setFlag(scene::EAF_USE_Q2ANIM_SEQS, true); and the predefined sequences will be added or removed or sth similar.

Flags like this and extra fields would mean noise in the API for each type of supported mesh, and no way to support new or complex types of animation data without recompiling the library. Using a generic IAttributes with a few documented fields for each supported mesh type makes it much more flexible.

2) When I`m animating stuff, I always have some frame rate in mind- usually 30 frames per second, then when calling the animation or changing its speed is easy too- most of the animators work this way- feels natural and at least it may be somewhat confusing for me to use the time in seconds or some factor.

To keep backward compatibility with this current method, we can just have the default animation's frame-rate be 1 frame per second, so when you do IAnimatedMeshSceneNode::setAnimationSpeed(30) it's just the same as 30fps. Frames-per-second doesn't make sense when the keyframe interpolation is anything other than linear, so it's just not compatible with today's (and tomorrow's) mesh formats.

3) About the bounding boxes created at the keyframes and being interpolated- I thought about this some time ago- sounds really temptating to interpolate the two keyframe bbox corners, which will be faster but it proved to be misleading.

Hmm yes you're right. A better implementation might just pick the largest of the two bounding boxes, then clip this against a box containing spheres around the joint positions, where the radius of a sphere is the distance between the joint and the most distant vertex under its influence. Actually skinning the mesh is very expensive, more so than actually rendering it unless it contains many, many buffers.

[xDan]

I don't really follow all of this, but would just like to add: it would be nice to be possible to get the exact bounding box at a given time. Exact as in it precisely matches the "furthest" vertices, and is not too large either.

Since the BB is useful for more than just culling. e.g. I might want to extract the BB for a given idle animation frame, and use it to position a character precisely on the ground.

(but maybe getting it at key frames is enough? I'm probably misunderstanding this..)

Also in my framework I have transition time as part of the animation data struct.

[3DModelerMan]

Yeah, I have transition time in my animation structure too. Would it be difficult to make morph targets mix with skelatal animations? I would assume you just apply the morph before you skin the vertices to the bones. Another thing I was wondering about is Dual Quaternion Skinning. It fixes alot of deformation problems with regular skinning.

[pc0de]

others please speak up so I don't feel like I'm talking to myself

animation != bikeshed

I'm currently in the middle of updating irrb to export skeletal animations which will also include an .irrmesh DTD update. When finished, I'll post it for review/modifications.

The reason I mention that here is that as a part of the research and preparation for this I looked at how other formats store animation data. Most notably Collada & Ogre's XML file formats. For the record, both of these store timing data in terms of "time" vs "frame/fps". I'm glad to see that you are considering this because in my head it just makes sense for the reasons you mentioned in your 3rd post.

I wasn't going to bring this up until after the 1st pass of irrb skeletal animation was completed, but have you given any thought to all things animable under a single animation system? In other words, an animation system that targets basic types (int, float, vector, matrix, scolor, rect, position, dimension, etc.). Having that would go along way for providing uniform animation support for nodes, skeletal, morph, texture, color, gui, and just about any other custom target under a single animation system.

[bitplane]

I'm currently in the middle of updating irrb to export skeletal animations which will also include an .irrmesh DTD update. When finished, I'll post it for review/modifications.

Excellent news! I'm sure I speak for everyone when I say we're all looking forward to this

Do you plan to implement skinning and morph targets at the same time? If so, as part of ISkinnedMesh?

I wasn't going to bring this up until after the 1st pass of irrb skeletal animation was completed, but have you given any thought to all things animable under a single animation system? In other words, an animation system that targets basic types (int, float, vector, matrix, scolor, rect, position, dimension, etc.). Having that would go along way for providing uniform animation support for nodes, skeletal, morph, texture, color, gui, and just about any other custom target under a single animation system.

No, I didn't consider that, but it does sound interesting but maybe too abstract. I'm not sure how such a thing would look either. Do you have an example?

[pc0de]

Do you plan to implement skinning and morph targets

Actually started out with morph targets and quickly realized that would require some mods to Irrlicht skinned mesh code. So the 1st pass will be skeletal only and will use the existing Irrlicht functionality already in place.

No, I didn't consider that, but it does sound interesting but maybe too abstract. I'm not sure how such a thing would look either. Do you have an example?

As a matter of fact I do . This began a while back when I started the irrb update to work with Blender 2.5x. In 2.5x "all things are animable", so I wondered how that could work in Irrlicht. That eventually lead to Chett Haase's "Timing Framework":

• Timing is everything
  Time again

Which then lead to a relatively quick port from java to c++:

• C++ Timing Framework

A feature that may not be so obvious at first glance is the ability to specify interpolators (linear, spline, ...) per keyframe. Here's an example that does both node (pos, rot, & scale) and SMaterial.emissive color animation: see "main.cpp".

Note that this port is a first pass as well, the idea of "property" targets are a little sketchy in that they assume the targets don't go away during animation. Probably better replaced with an "animatedObject" type that grabs a reference...

[3DModelerMan]

If we're making everything animable. Then I have two ideas:

1: It could be like some sort of IAnimationManager that you get at through the scene manager? You could add keyframe attributes to the scene nodes... Or you could have Irrlicht animation files (kinda like the scene files) that would store keyframes for scene nodes. It would have to use names or IDs to do that. At first it could just do simple transformations, like rotation, and scale, and position. But it could be extended later.

2: (This one's my favorite idea) It could be implemented as an animator. This way we could load animations on a per node basis. We don't need it to affect all the nodes.