Hardware skinning

[Funto]

Hi all,

I am a contributor to the game SuperTuxKart, which is based on Irrlicht, and I have done some attempts at implementing hardware skinning, as skinning has been identified as the biggest bottleneck in STK's performance.
I primarily took my inspiration from this thread: http://irrlicht.sourceforge.net/forum/v ... hp?t=42096 but heavily modified the code.

I tried to implement it while altering as little as possible the original code of Irrlicht, but unfortunately that was not really possible without sacrificing information on the vertex (color, normal).
This arised the need for either a flexible vertex format (think glVertexAttribPointer), or at least a vertex format adapted to hardware skinning.
This also showed a limitation as to the number of joints a given vertex can be associated to.

So I would like to know if there are plans to support such a flexible vertex format, hardware skinning, and in case I submit patches for those features, what are the chances it would be accepted in Irrlicht (I've seen many works in this forum that have not been integrated into the core of Irrlicht, so I am wondering).
More generally I wanted to get in touch with the Irrlicht community ^^.

Thanks for reading.

[hybrid]

The flexible vertex format will probably only come with Irrlicht 2.0, which will get those changes which have a stronger impact on backward compatibility and the overall engine design. This might take quite long from now, though. Adding yet another vertex format is not that much of an overhead, and might be possible with every major version of Irrlicht.
Patches to Irrlicht are all considered for integration. The chances simply depend on the quality and further work and maintainability issues related to it. As you might have seen, we have a patch tracker where such additions are collected. Many of the submitted patches are directly integrated. Many patches for STK have been integrated immediately, as e.g. Auria provides very high quality patches. If you provide more than bug fixes, it is definitely required to get into the discussion with the community. Since there are already several approaches for the flexible format and other issues, just take on these threads.

[Funto]

Ok, thanks for the quick answer ^^
Would it be ok if I add fixed vertex format(s) that support hardware skinning, while waiting for a flexible vertex format-based approach?
And if yes, I have 3 questions:
- should I add the 3 required variants (Standard, 2TCoord and Tangents), or would it be ok if I only add a "hardware skinning version" to the standard vertex format?
- would it be ok if I only provide an implementation for OpenGL? (I am using GNU/Linux)
- should a "default shader" be provided by the engine, that does standard vertex lighting + hardware skinning? I suppose yes...

I won't have much time to work on such a patch in the upcoming days, but I hope I will have more time later on, that's why I'm asking on the possibilities for contributing.

Thanks.

[hybrid]

Yes, only a basic vertex format would be ok. Having an implementation for only one driver would be one reason why a patch lies around for a long time. I don't see much time for me in the next time to add this support, and probably also none of the other dev team members. But usually having an implementation for only few platforms is a good start and eventually it gets completed by someone.
Not sure about the shader question. Hardware skinning will only help if it is someone usable with arbitrary materials. But since hw skinning only requires a vertex shader, it should be possible to use it also with fixed function pipeline?!

[Funto]

Hmm ok, so I will try to provide a DX9 implementation then...
Yes it only requires a vertex shader, but if you use vertex lighting (which the fixed function pipeline does by default IIRC, right?) then it has to be implemented in the vertex shader. Now, think of using vertex lighting with a number of light scene nodes that varies while the scene is rendered...:/

[fmx]

Since I have some experience with hardware skinning I might as well share some of my thoughts.
So hybrid, what you are suggesting for the short term (pre-2.0) is the introduction of a new fixed vertex format, alongside the the three already present, something like S3DVertexSkinned?

In most cases, there is very little need for more than 3 joint influences per vertex, and I personally feel that the overall size of the class should be as minimal as possible so Tangents might be overkill.

How about:

Code: Select all

vector3df Position
vector3df Normal
vector2df TCoords
SColor Color
f32 Weights[3] (or [4])
u16 Indices[3] (or [4])

Small and simple.
Ideally Indices should be bytes, but there is no guarantee the number of joints will be less than 255

Ofcourse, dont forget that file loaders will also need to be modified accordingly

[hybrid]

Yes, adding a new vertex format seems to be the least distrubing way of introducing this. I'd suggest 4 elements, since many file formats support this number.

[Mel]

I'd include also support for tangent space.

So, a skinned vertex could look like this:

Code: Select all

 
S3DSkinnedVertex{
vector3df position;
vector3df normal;
vector3df T;
vector3df B;
vector2df TCoords;
SColor Color;
u32 Indices;// 4 bones of influence out of 255 posible bones. 255 bones are a lot.
u32 Weights;//If posible. Or f32 weights[4]; if using unsigned chars for the bone influences had no sense
}
 

http://www.opengl.org/registry/specs/AR ... alette.txt

Open GL also has support for something like this, but i don't know how does it actually work. Not obstant, using u32 for the Indices still give a lot of room for skinning matrices, as i have read somewhere that the video card has around 16 or 32 slots for skinning matrices. The best implementation would be to split a mesh so each joint had the least bone influences (which translate into skinning matrices) used.

[fmx]

Your SkinnedVertex implementation is very heavy for general use.
Although I agree that having Tangents and/or Binormals would allow normal mapping, the amount of indices and weights required would bloat every vertex anyway.

255 bones is a lot, but sometimes limiting the bonecount is not possible.
Many users of irrlicht are not very efficient at even using polygonal models with less than 65535 polygon indices, I doubt some developers might be happy about having to stick within a tight bone-count limit

Perhaps irrlicht could have TWO new skinned vertex formats, to cater for both needs?

S3DSkinnedVertex:

Code: Select all

vector3df Position
vector3df Normal
vector2df TCoords
SColor Color
f32 Weights[4]
u16 Indices[4]

S3DSkinnedVertexTangents:

Code: Select all

vector3df Position
vector3df Normal
vector3df Tangent
vector3df Binormal   // optional, can be calculated by using Normal and Tangent alone
vector2df TCoords
SColor Color
f32 Weights[4]
u8 Indices[4]

The tangent one is obviously larger, but the smaller indices allow for them at a lower cost.
The standard skinned vertex would be the one used in most cases

I've implemented matrix palette skinning with GLES-1.x (same thing as what you just linked to) and I can assure you, it has virtually nothing to do with vertex structure, and everything to do with meshbuffer management.
It can be done, but isn't very relevant for the discussion, which was why I didn't mention it.

We shouldn't forget that this is only a temporary solution until flexible vertex formats are implemented in irrlicht-2.0, so the final decision should bias towards speed of implementation more than universal flexibility

[Mel]

The Tangent can't be calculated using the normal and the Bi- vector, because that vector is aligned to the U axis of the texture coordinates, as the Bi- is aligned to the V axis, both are necesary, and can't be calculated inside a shader.

Anyway, taking into consideration the nature of the N, T and B vectors, could it be posible to use half precision float vectors? 16 bits instead of 32 bits floats?

[shadowslair]

The Tangent can't be calculated using the normal and the Bi- vector, because that vector is aligned to the U axis of the texture coordinates, as the Bi- is aligned to the V axis, both are necesary, and can't be calculated inside a shader.

Of course it`s perfectly valid to calculate the binormal in the shader as a cross product of the tangent and normal or of the normal and tangent, depending on whether the normalmap is using flipped Y or not. Once knowing two of them, you can always calculate the other. As for me- I believe putting two new vertex formats- one without Tan and Bi, and one without them sounds reasonable. But, even in the short run it may turn a bit of pain, so one format containing all needed attributes, including tangent and bitangent can also be enough to tie our pants for a short time. Well, whatever you decide- even if you decide to not add it, I doubt someone will get hurt. I believe all the guys who needed hw skinning already have found their way to do it anyway. This is my opinion about the skinned vertex formats. As for the flexible vertex format- yeah, I vote "Yes" with my both hands, cannot think of an easy way for its implementation though.

Anyway, taking into consideration the nature of the N, T and B vectors, could it be posible to use half precision float vectors? 16 bits instead of 32 bits floats?

This will most probaly lead to visible artifacts, too lazy to test it right now. Anyway, we shouldn`t sacrifice visual quality for some almost invisible speed-up and gained memory, just my two cents.

[Funto]

I believe all the guys who needed hw skinning already have found their way to do it anyway.

I doubt that, at least that's not the case for STK, and that's the job of the engine anyway...

[Mel]

It wouldn't produce many noticeable artifacts, the good thing of the floating point numbers is that their accuracy grows as the number is closer to the range [0,1], and the ideal situation for those vectors is to be normalized.

Not obstant, the diference isn't the amount of memory used by a single vertex, but a whole mesh of thousands of them. Any memory saving within a graphics routines means a gain in performance. Compared to the precision loss, it could be an affordable change.

[hybrid]

I guess adding one vertex will make clear which steps need to be taken. Adding a second after that will be only a little copy/paste, so don't worry too much about it. Getting the first one working is the real thing.

[Funto]

Yup, and as I said, I'm a bit worried about the vertex shader thing (i.e. handling everything that is done in the FFP, which includes lighting with a possibly varying number of lights...).
I think that is where the real challenge is, but tell me if I'm wrong.