Hardware skinning?

[Rikan]

Now there's something I've seen in Ogre what I'm clearly missing in Irrlicht...

The idea behind 'hardware skinning' is that the calculations to deform the skin mesh are actually done by the GPU instead of the CPU.

Normally the mesh is taken and deformed according to the blend weights given and the positions of the bones of the underlying skeleten, this is quite a consuming task for the CPU. After that the data is transferred to the GPU which needs to do the objectspace-to-viewport transformations.

Since both transformations - the mesh deforming and the viewport transformation - are simple matrix operations, why not rolling them up into one and having the GPU do that, since it's not that much additional work?

So, a vertex program is needed which is given the objectspace-to-viewport tranformation matrices for all the bones of the mesh and the vertex data containing the positions, maybe additional data like color and last but not least the blending weights (or a single blending weight if there is only one bone per vertex association).

The Ogre3D demo 'Skeletalanimation' shows it well enough that it may be a huge difference in the resulting frame rates. With me, it was for the better with hardware skinning, but your mileage may vary.

Limitations of the hardware skinning technique are imposed mainly by the number of constant registers the graphics hardware has to offer - a set of 100 registers would provide enough room for skeletons with no more than at around twelve bones.

Changes for the Irrlicht skeletal animators would include a function which tells the underlying animators whether to pass the deformed mesh to the GPU as usual or the original mesh together with the blending data (meaning bone index or indices per vertex and the blending weights if there is more than one bone attached to any vertex) and a function which either returns an array of transformation matrices for each bone or maybe fills the GPU constant registers by themselves with complete object-to-viewport transformation matrices and perhaps a function which returns a suitable vertex program (or sets the material with that vertex program) depending of the choice of material flags (transparency, texturing, whatever) given.

[hybrid]

Well said, we're looking forward for your first screenshots or patches

[Baal Cadar]

The 96 constant registers can hold more than twenty 4x3 matrices even.

[Rikan]

A bigger problem would be - how to pass the blending data along?

Vertices are stored in an array of S3DVertex structures which mimics the Direct3D vertex stream structure for the fixed vertex function pipeline.

Quite good for most meshes, speeds up passing the data along, but that leaves me with the question of where to fit the blending data - and how to tell the Direct3D driver (before you ask, I've already kind of figured it out with OpenGL) how to find them and fill the correct vertex attribute for the shader program?

I'd rather not invent yet another flavor of S3DVertex but it seems to be quite necessary to do so to allow for more complex vertex programs which need more - or simply other - data passed along instead of color, normal vector and textures. Or is it possible to conveniently extend the S3DVertex class to allow passing around any attributes to the shader programs? That would also spare the trouble of converting a mesh held in S3DVertex into another class just to make room for the blending data.

[Baal Cadar]

Ouch. Yeah that's a problem. I don't recommend to introduce yet another S3DVertex type. The switch(vertextype) is already a pita, no use to have all those changed. Imho it was a bad idea from the beginning.

Changing the whole vertex system to something more flexible is a good idea. A system that works well with hardware buffers and a system that allows for arbitrary layout of the vertex stream. Lightfeather, an Irrlicht derivate, has changed this to a more flexible format, but it still contains curiousities and is not easily extendable. As far as flexibility is concerned Ogre is unrivaled among the Open Source engines, but its system wouldn't fit Irrlicht's philosophy.

After a closer inspection of the animation code and how it is organised (mainly in CXAnimationPlayer) I'd say implementing hardware skinning can't be added easily. There was an extreme amount of changes necessary with a lot of design decisions to be made. And getting all those in the same way as Niko would like them to be is difficult, so the chance, that your patch will be applied to core irrlicht is small.

You could consult Niko and ask him how he'd do it, or if there are any changes coming to the vertex system.

[hybrid]

Or just don't count on core integration at all. In my opinion it is hopeless to wait for integration before publishing any changes. Just count the patches I have collected over the years. Only a handful has been integrated yet. It's much more work to do, but otherwise you won't ever proceed.
Maybe check for Spintz' changes cause he simplified the vertex structure (by makeing S3DVertex slightly larger). So you don'T have to decide everything on your own

[Guest]

Yes, had a look at it. Though extending the number of textures looks like a walk in the park compared to the changes which may be needed here.

Yet the base idea has merit - As far as I know, the vertex position must reside at the start of each array element, but that's not the point.

So - why not encapsulating everything from the second element onwards with a union and offer a simple array of vector4d<f32> (Yes, would need to be introduced but needn't offer that much functionality, just to hold the data in a structured manner) as an alternative to fill the vertex attributes?

The functions drawIndexedTriangle(...) and drawIndexedTrianleFan(...) of each driver may be extended with a parameter which instructs the driver to switch the the alternative stream reading scheme when set.

Perhaps the MeshBuffer which holds the vertex array may keep track of the type of vertices stored? AFAIK it is already done in regards to EVT_STANDARD, EVT_TANGENTS and EVT_2TCOORDS.

Well, it doesn't cut down the switch(type) thing... but at least one doesn't need to overload the necessary functions even more as it is already done with S3DVertex and S3DVertex2TCoords.

[mm765]

Lightfeather, an Irrlicht derivate, has changed this to a more flexible format, but it still contains curiousities and is not easily extendable.

could you elaborate on that please ? what are the curiosities and why is it not easily extendable ? id honestly like to hear your opinion (just so you dont think im offended or anything).

oh and i dont think that Lightfeather can still be called an irrlicht derivate - that would be misleading in my opinion. aside from the base-classes (and even those are changed a lot) there is really nothing left from irrlicht(nx++).

[Baal Cadar]

mm765, to be honest I just looked over the API docs when I made this comment. Now that I wanted to answer your question I realised, that I was under a misconception then. Sorry for that.

Actually it looks rather good. Though I don't understand, why blend weights are stored in a seperate kind of buffer, instead of a CVertexBuffer. The API is similiar enough.

As far as extensibility is concerned, a more generallised API might be easier to extend. Though I understand, that this means a trade-off between type safety and generalisation. Right now the buffer has a lot of status information fields, that are not used for every buffer. Even when the buffer only stores positions, then nevertheless there are the fields like colorOffset, normalOffset and so on.

Ogre for instance doesn't store any semantic info in its VertexBuffer. This is stored in a seperate structure. The vertex layout is defined in a VertexDeclaration which stores VertexElement structs, which define semantics and size and source of an element. (For instance this allows texture coordinates to be 1, 2 or 3dimensional)

Of course many approaches are valid and none is better per se. It is all a question of design goals and adjustment between multiple requirements.

[mm765]

hmm..i was under the impression i already posted a reply - weird.

mm765, to be honest I just looked over the API docs when I made this comment. Now that I wanted to answer your question I realised, that I was under a misconception then. Sorry for that.

no problem at all.

thanks for your comments, i really appreciate them (always looking for possible improvements).
i can see your point with the definition being "separate" from the data. i must say though that i am very satisfied with the current solution (havent had any problems yet).