Hardware "friendly" skinned meshes

[Mel]

Seeing how Irrlicht is orienting towards the new hardware and AP's, i think it is time already to have a new hardware friendly skinned system, because it would become otherwise, the weakest spot of the engine.

I would like to point out some features so it was easier or at least, i think so, and check how posible would that be.

The current system uses the bones as the structures to store the meshbuffer ID, vertex ID and the weights, it reads the aprpriate meshbuffer, and builds one mesh for the rendering on the fly. the meshbuffer is then rendered as always using a normal material. While 100% flexible, this ends being slow on multiple ways, because:

• the mesh is built per frame
• it is transformed on the CPU
• and it is sent per frame to the video card.

The good thing of the current skinned meshbuffers is that the animation system can be kept almost unchanged. (or maybe created anew from scratch to optimized the current scene traverse issues that involve the skinned meshes, and the attached nodes)

The solution to the problems of the skinned meshes rely:

• on the ability to store the vertex influence on the mesh, and take it from the bones
• to be able to split the amount of bones that affect a single meshbuffer
• to be able to store statically the meshes on the video card
• and send to the meshbuffers only the necesary bone matrices.

To store the information on the meshes, it is necesary to have another vertex format that stores the bones that influence the vertex, and their weights besides the tangent space, and limit the amount of bone influences to 4, for space matters.

The point here is that the bone influences shouldn't be the ones from the original animated mesh, but from a reduced set which was associated to the meshbuffer that contains the vertex. Why? because the amount of bones would be limited otherwise, either using the fixed function palleted skinning, or a shader based approach, the amount of matrices that can be used here is limited. hence, there is need for a new type of hardware friendly meshbuffer in which there was a limited set of matrices that referenced the animations of the original mesh, and that these matrices were available for the meshbuffers to render using either a shader or a paletted skinning system.

Then, for the animation system, it would be necesary to store the bone transformation matrices after they were animated, and the rendering would use them.

This would solve everything:

• the mesh building process would be done on rendering time, and the CPU overhead for mesh creating/deleting would be avoided.
• It would help to move the whole vertex transformation process to the video card, gaining speed
• and finally, the mesh could be static, so there would be no need to send it multiple times, saving bandwidth.

The drawback is that the rendering would forcefully use a shader or use the fixed pipeline rendering, where either the material couldn't change, or their utility would be limited to fixed pipeline rendering drivers. and makes me wonder if is there a way to use a vertex shader and to be able to change the fragment shader at will... is it posible?

[Granyte]

In dx there is a such possibility to switch the ps shader at run time when building an effect pool how ever for the other drivers i don't know

[Mel]

In general, also, it would be a desirable feature that the shaders knew what meshbuffer and scene nodes are using them. Besides, in DX is posible to output the mesh to a buffer, and reuse it, so for instance, it would be posible to render a skinned mesh to a buffer like this, and use it later with any shader,

Speaking of, in DX, taking a peek to the DX10 skinning examples in the SDK, it says that shaders may have associated a thing called "shader resources", like a texture, or the constants table, but with random user data, that, unlike the constants table, can be very large, so this structure could also store the matrix transformations, is there something similar in GL? or the constants table in GL can have any size? would GL have something similar to outputting a skinning result to a buffer? I lack much knowledge of Open GL

[fmx]

i still use the old generic system of skinning gpu meshbuffers in shaders (managing mesh animations on CPU and uploading params to shader for each mesh that uses it), maybe not very efficient on new pc hardware but works well on mobile platforms at least

i have zero experience with DX10/11
is there any difference between DX11 and DX10 skinning examples/methods?

keep in mind irrlicht will need to support older and less powerful platforms too, so any new skinning system shouldn't be totally based on newest GL4/DX11 methods

[Granyte]

well we can still do it the old way but the power of dx10/11 is that the unified shader architecture preatty much removed the limit on what you could do on a gpu and how you were forced to do it.

and even the normal constant register in dx10/11 can hold 4x4096 registers. when i implemented instancing with dx11 i could upload 1000 matrix at once .

[Nadro]

In OpenGL we can use Uniform Buffers for skinning, instancing etc (a max buffer size is depend on GL_MAX_UNIFORM_BLOCK_SIZE value, but it should be 64KB or more) and Texture Buffers for skinned instancing etc (size is depend on GL_MAX_TEXTURE_BUFFER_SIZE, but it should be 128MB or more). For store a skinned vertices in OGL we can use Transform Feedback All these features are available in OGL 3.x Core. More advanced version of Transform Feedback is available in OGL 4.x Core and via extensions for a hardware which support only OGL3.

Compared to OGL2.x/DX9 from a C++ perspective for these cases (skinning, instancing etc) there isn't any special requirments for an Flexible Vertex Format and other Irrlicht shared parts via different video drivers. Only shaders and video drivers need modifications for support it.

[fmx]

Cool, I thought irrlicht might require different mesh management as well for different drivers

since we are discussing meshes
irrlicht could use its own skin/skeleton/animation file formats too

i haven't looked at .IRRMESH spec yet, but has a discussion been started on possible specs for import/export of Animated model data?
i'm guessing they would be called .IRRSKIN, .IRRSKEL and .IRRANIM

or maybe assimp can be integrated into irrlicht instead of having to write/maintain custom loaders
http://irrlicht.sourceforge.net/forum/v ... 19&t=47333

[Mel]

So, if uniform buffers are available on GL, then, there should be little diference between the DX implementation and the OpenGL counterpart.

You can load 1000 transform matrices inside a shader in DX11? compared to the room for 64 of the PS3.0 hardware it is a lot!. But using shader resources/uniform buffers then there is no need to load them as constants any more, and for instance, splitting a skinned mesh on newer hardware wouldn't be necesary. But on GL 2.0/DX9 would be other story.

A native skinned mesh format should be a must for Irrlicht already. It would remove the problems that the current importers provoke, (bad bones, bad animations...) also, it would help to normalize the (almost inexistent...) content creation pipeline for Irr, and in general, it would help the engine advance to the already current generation of hardware. The loaders still could be used to convert the meshes to the native format of Irr. And COLLADA has already support for bones and skinned meshes, so a loader that supported those would also be highly welcome.