Updating Tangents and Binormals for Skinned and Morphed Anim

[Vectrotek]

C++

When animating a mesh by "Morphing" or "Skeletal Animation" such as "*.md3", "*.x" and "*.b3d" using "Shaders" for rendering we can improve the final render if we "Cyclically Update" the "Tangents" and "Binormals"..
We presume that our meshes are, among others, textured with a "NORMAL MAP" used by the "Shader" (cg, hlsl, or glsl etc) in calculating diffuse and specular.
We also have one or more lights used by the shader.

This is one way we could go about it:

Code: Select all

 
 
 // Update TANGENTS & BINORMALS at every frame for a skinned animation..
   
 // We dont want to do this for static meshes like levels etc..
 // We also dont want to do it for Rotating, Scaled and translated meshes..(we can however, as a bonus, scale, rotate and translate these)
 // Only for animated skinned and morph based meshes..
 // This is loose code that works. If anyone can improve it for the engine itself that would be great..
 // You'll probably ID possible improvements immediately!
 
 // At every N'th Frame we loop through all the vertices..
 // 1. In the loop we Access the VERTEX of POINT A of the "INDEXED TRIANGLE"..
 // 2. We interrogate the "OTHER TWO" VERTICES (which thankfully do change at each frame) for their Positions, Normals, and UV Coords to 
 //    Genertate a "BRAND NEW" (animated) TANGENT and BINORMAL. (We may want to calculate the the "Binormal" in the SHADER to save time)
 // 3. We REWRITE the Tangent and Binormal for our SELECTED TRIANGLE POINT.
 // 4. We DO THE SAME for POINTS B and C..
 // 
 
 //  GENERATE "LIVING" TANGENTS & BINBORMALS 
 //  REMEMBER!
 //  WE NEED "LOOP THROUGH ALL MESHES" THAT ARE TYPE "Skinned" 
 //  WE NEED "LOOP THROUGH ALL ITS BUFFERS" 
 //  WE NEED "LOOP THROUGH ALL THOSE BUFFER VERTICES" 
 // Possible types of (animated) meshes.
 // Enumerator:
 // 1  EAMT_MD2            Quake 2 MD2 model file..
 // 2  EAMT_MD3            Quake 3 MD3 model file..
 // 10 EAMT_MDL_HALFLIFE   Halflife MDL model file..
 // Below is what an item type must be for it to qualify for Tangent Updates..
 // 11 EAMT_SKINNED        generic skinned mesh "*.x" "*.b3d" etc.. (see Morphed too!)
 //                                                                              
 // We want to change tangents for skinned meshes only so we must determine which ones are "Skinned"..
 // This may change if we add and remove meshes during runtime.. 
 
 // Added this for clarity..
 IMeshCache* TheSceneMeshCache; // Not to be assigned cyclically..
 irr::scene::IMeshBuffer* CurrMeshBuffPtr ;  // We want to assign this cyclically..
 video::S3DVertexTangents* CurrTanVertsPtr ;
 IAnimatedMesh* AcquiredAnimeshByIndex;  // This changes in the loop! 
 IMesh*  AcquiredImesh ; 
 SceneMeshCount = TheSceneManager ->getMeshCache()->getMeshCount();
 
 //                                                                              
 if (TanUpdStatus == true) // We "set" this bool in our App..
  {                
   //  SceneMeshCount
   for (u32 SMeshI = 0; SMeshI < SceneMeshCount; SMeshI++)  // There amy be other ways to iterate through all your items..
    {// start loop "SMESHI"     
     u16 TheMeshType = TheSceneMeshCache->getMeshByIndex(SMeshI)->getMeshType();
     if (TheMeshType == EAMT_SKINNED)   // EAMT_SKINNED (or morphed ?)..
      {// -start- Only "Skinned" meshes..
       AcquiredAnimeshByIndex = TheSceneMeshCache->getMeshByIndex(SMeshI); 
       AcquiredImesh = AcquiredAnimeshByIndex ;
       u32 TheMBufferCount = AcquiredImesh->getMeshBufferCount();
       for (u32 MBuffI = 0 ; MBuffI < TheMBufferCount ; MBuffI++)
        {// start Buffer Loop 
         CurrMeshBuffPtr = AcquiredImesh->getMeshBuffer(0);  // WE MUST ALSO LOOP BUFFERS..
         CurrTanVertsPtr  = (video::S3DVertexTangents*)CurrMeshBuffPtr->getVertices(); // Many Buffers for Many Meshes..
         u16* TheINDEXPtr =    TheSceneMeshCache->getMeshByIndex(SMeshI)->getMeshBuffer(0)->getIndices();
         u32 TheIndexCount = (u32)TheSceneMeshCache->getMeshByIndex(SMeshI)->getMeshBuffer(0)->getIndexCount();
         for (u32 IndexII = 0; IndexII < TheIndexCount; IndexII+=3) // Note "+=3" in For loop..
          {// start Faces..  Get all three of our triangle vertices..
            VertexTriA = CurrTanVertsPtr[TheINDEXPtr[IndexII]];
            VertexTriB = CurrTanVertsPtr[TheINDEXPtr[IndexII+1]];
            VertexTriC = CurrTanVertsPtr[TheINDEXPtr[IndexII+2]];
            // We have the animated normals as modelled, so we dont really want to mess with these 
            // because during the modelling phase we created "Sharp Edges" (important concept) 
            // which play a large role in the final rendering.. 
            // We could offcourse, when we optimise this, look at how Irrlicht generated these 
            // Tangents and Binormals internally (seemingly once, at model loading moment), but for now we'll do it ourselves..
 
            // Here we get the THREE POINTS XYZ Positions for the TRIANGLE..
            f32 TAX = VertexTriA.Pos.X;   f32 TAY = VertexTriA.Pos.Y; f32 TAZ = VertexTriA.Pos.Z; 
            f32 TBX = VertexTriB.Pos.X;   f32 TBY = VertexTriB.Pos.Y; f32 TBZ = VertexTriB.Pos.Z;
            f32 TCX = VertexTriC.Pos.X;   f32 TCY = VertexTriC.Pos.Y; f32 TCZ = VertexTriC.Pos.Z;
            // Here we get the UV Coordinates for each of the three Points.
            f32 TAU = VertexTriA.TCoords.X; f32 TAV = VertexTriA.TCoords.Y;
            f32 TBU = VertexTriB.TCoords.X; f32 TBV = VertexTriB.TCoords.Y;
            f32 TCU = VertexTriC.TCoords.X; f32 TCV = VertexTriC.TCoords.Y;
            // We introduce THREE new "Delta Vectors" which will eventually become "Triangle Edges"..
            // This is a special "recipe" using "Triangle Points" and "UV Coordinates"..
            f32 DV1X = TBX - TAX ; f32 DV1Y = TBU - TAU ; f32 DV1Z = TBV - TAV;
            f32 DV2X = TCX - TAX ; f32 DV2Y = TCU - TAU ; f32 DV2Z = TCV - TAV;
            f32 DV3X = TBY - TAY ; f32 DV3Y = TBU - TAU ; f32 DV3Z = TBV - TAV;
            f32 DV4X = TCY - TAY ; f32 DV4Y = TCU - TAU ; f32 DV4Z = TCV - TAV;
            f32 DV5X = TBZ - TAZ ; f32 DV5Y = TBU - TAU ; f32 DV5Z = TBV - TAV;
            f32 DV6X = TCZ - TAZ ; f32 DV6Y = TCU - TAU ; f32 DV6Z = TCV - TAV;
            // Now we introduce THREE "Cross Products". Cross Product A, Cross Product B and Cross Product C.
            // Dont sweat the math, but understand that these Cross Products are needed to get Tangents / Binormals..
            f32 CAX = (DV1Y * DV2Z) - (DV2Y * DV1Z); // "Subtraction" hence "Delta Vectors" above..
            f32 CAY = (DV1Z * DV2X) - (DV2Z * DV1X);
            f32 CAZ = (DV1X * DV2Y) - (DV2X * DV1Y);
            f32 CBX = (DV3Y * DV4Z) - (DV4Y * DV3Z);
            f32 CBY = (DV3Z * DV4X) - (DV4Z * DV3X);
            f32 CBZ = (DV3X * DV4Y) - (DV4X * DV3Y);
            f32 CCX = (DV5Y * DV6Z) - (DV6Y * DV5Z);
            f32 CCY = (DV5Z * DV6X) - (DV6Z * DV5X);
            f32 CCZ = (DV5X * DV6Y) - (DV6X * DV5Y);
            // Calculate our TANGENT..
            f32 TanX = (CAY / CAX);  f32 TanY = (CBY / CBX);  f32 TanZ = (CCY / CCX); 
            // ..and our BINORMAL..
            f32 BinX = (CAZ / CAX);  f32 BinY = (CBZ / CBX);  f32 BinZ = (CCZ / CCX); 
            // Normalise them.. (is this really necessary??) 
            // Apparently not! (I took TWO screengrabs of ON and OFF and they look the same..)
            // f32 TanXN = TanX; // / ((TanX * TanX)+(TanY * TanY)+(TanZ * TanZ));
            // f32 TanYN = TanY; // / ((TanX * TanX)+(TanY * TanY)+(TanZ * TanZ));
            // f32 TanZN = TanZ; // / ((TanX * TanX)+(TanY * TanY)+(TanZ * TanZ));
            // Now this "equals" aint needed.. (I'll keep it here just incase)    
            // Try doing the "Binormals" in the shader by Crossing the Normals with the Tangents..
            // The Irrlicht Coders would probably want this working for Direct X aswell before implementing it..
            // Normalisation apparently not needed here aswell..
            // f32 BinXN = BinX; // / ((BinX * BinX)+(BinY * BinY)+(BinZ * BinZ));
            // f32 BinYN = BinY; // / ((BinX * BinX)+(BinY * BinY)+(BinZ * BinZ));
            // f32 BinZN = BinZ; // / ((BinX * BinX)+(BinY * BinY)+(BinZ * BinZ));
            // Just incase our balls were facing the wrong way..
            // swap (TanXN , BinXN);  swap (TanYN , BinYN); swap (TanZN , BinZN);
            // TanXN *= 1.0; TanYN *= 1.0; TanZN *= 1.0;
            // BinXN *= 1.0; BinYN *= 1.0; BinZN *= 1.0;
            // Now we replace the Static Tangents/Binromals with our animated ones..
            CurrTanVertsPtr[TheINDEXPtr[IndexII]].Tangent.X = -TanX;  // Remember, if problems that these were +-TanXN (normalised ones)..  
            CurrTanVertsPtr[TheINDEXPtr[IndexII]].Tangent.Y = -TanY;
            CurrTanVertsPtr[TheINDEXPtr[IndexII]].Tangent.Z = -TanZ;
            CurrTanVertsPtr[TheINDEXPtr[IndexII+1]].Tangent.X = -TanX;
            CurrTanVertsPtr[TheINDEXPtr[IndexII+1]].Tangent.Y = -TanY;
            CurrTanVertsPtr[TheINDEXPtr[IndexII+1]].Tangent.Z = -TanZ;
            CurrTanVertsPtr[TheINDEXPtr[IndexII+2]].Tangent.X = -TanX;
            CurrTanVertsPtr[TheINDEXPtr[IndexII+2]].Tangent.Y = -TanY;
            CurrTanVertsPtr[TheINDEXPtr[IndexII+2]].Tangent.Z = -TanZ;
            // Could this be done in the shader??   I tried but no joy.. ..
            CurrTanVertsPtr[TheINDEXPtr[IndexII]].Binormal.X = BinX;
            CurrTanVertsPtr[TheINDEXPtr[IndexII]].Binormal.Y = BinY;
            CurrTanVertsPtr[TheINDEXPtr[IndexII]].Binormal.Z = BinZ;
            CurrTanVertsPtr[TheINDEXPtr[IndexII+1]].Binormal.X = BinX;
            CurrTanVertsPtr[TheINDEXPtr[IndexII+1]].Binormal.Y = BinY;
            CurrTanVertsPtr[TheINDEXPtr[IndexII+1]].Binormal.Z = BinZ;
            CurrTanVertsPtr[TheINDEXPtr[IndexII+2]].Binormal.X = BinX;
            CurrTanVertsPtr[TheINDEXPtr[IndexII+2]].Binormal.Y = BinY;
            CurrTanVertsPtr[TheINDEXPtr[IndexII+2]].Binormal.Z = BinZ;
            // Now we have "Living Tangents and Binormals" getting updated when we want them to..
            // It means that the interaction between light and surfaces will remain "Physically Correct"
            // during any change made to the geometry such as "Skinned Mesh Deformation"..
           }   // End Index Loop..     
         }  // End Buffer Loop 
       } // End loop only "Skinned" meshes..
     } // End loop "SMESHI"     
   }  // End conditional Tangents & Binormal Update..             
 // O.K.   
     
   
   

Get a working example and coplete VC 7 C++ projects here: http://s000.tinyupload.com/?file_id=837 ... 6460510590

[The_Glitch]

Thanks Irrlicht has been overdue for this.

[Vectrotek]

My pleasure! Keep up the good work you guys!

[The_Glitch]

One question can I take your cpp file above and add it to my project or do I have to update the Irrlicht sources to make use of your change?

[Vectrotek]

@ Glitch.. Yes, I understand why you ask the question. It is rather baffling when you try compile cool code and then discover that the person who posted the code
had changed his own Irlricht "*.dll" without including the DLL or instructions on how the LIB was changed. (I could be wrong on that, considering your "patches" etc which I know nothing of yet)

The "Tangent and Bi-Normal" updating snippet , I actually tried to code in such a way that it would compile with the originally shipped Irrlicht 1.8.1 or 1.8.2 "*.dll" and "*.lib" files!
It should work if your Items i.e. "Meshes", their types, their respective Buffers and their Vertices are accessible in the way that my posted code has it..
I think it is pretty standard, but some folks may have other fancy methods for handling their objects.

From the Irrlicht Engine's point of view, I think an optimized version the code would work just fine.
Note that Tangents and Binormals are already calculated in the engine itself, even if done ONLY ONCE with the "Convert to Tangents" functions..
The thing is, that I couldn't call these functions cyclically as they seemed to do other things too.

So.. The short answer is yes, it should work, because it accesses Meshes independently from how you would, in your App.
It should also work, irrespective of Shader Language, whether GLSL, CG, CGFX, FX or HLSL under OpenGL or Direct X.

Just be sure to add it to your App code in such a way that the declared "Variables" (at the start) is at the "right place" and the "Actual Loop" is at the "right place".
If you should "Add" more meshes to your scene during run-time, then the placement of the "Loop" is important.

I tried to make it as robust as possible by having the loop check all the MESHES for "type-skinned" (morph based "*md2" is possible too).
It may be possible that a "Skinned Mesh" is big enough to warrant "Multiple Buffers", so I also had the loop iterate through all the buffers based on "buffer count per mesh".
You'll see that it is a "for" loop, within a "for" loop, within a "for" loop, within a conditional "if".

I hope it works for Irrlicht Users App Code and eventually inside the Irrlicht Engine as well.
It definitely works in the project I had posted.

Keep me posted! Cheers!

[The_Glitch]

Okay thanks maybe this will help me fix my Tangent space shader issue on an animated mesh.

Thanks The_Glitch

[The_Glitch]

Vectrotek you application for this has far to many extra things, you should just take a simple Irrlicht demo that's provided with the SDK and add your code for tangents and binormals and only that to show properly how to use it with irrlicht, it would help me as well for converting for use with shader-pipeline without digging through all the extra code.

[Vectrotek]

Ill see what I can do!

[The_Glitch]

Vectrotek just add it to the default Irrlicht example it has the least bit of code for simplicity.
http://irrlicht.sourceforge.net/docu/example001.html

[Vectrotek]

Hi! The Glitch!
I did not merge the code with the example as you asked but I did post a FULL PROJECT
that handles both GLSL and HLSL..

It is a stream lined version of previous code with some cool features..

This is the best I could do so I hope it helps!

Here is the PROJECT..(I forgot to convert from *.x to *.b3d so the file is big.. sorry!)

http://s000.tinyupload.com/?file_id=457 ... 9221471855

Screenshot:

[The_Glitch]

I'll check into it bud thanks.
Did you get my PM about your fog shader?

[christianclavet]

Wow! Thanks Vectrotek! Just seen your example and you documented the shader! That's really great! I searched for something like this for a long time! Once they fully switch to shaders, I hope this (or and optimized version) or this will get in Irrlicht.

[mongoose7]

It's really not suitable. If you notice, there is only one (meaning four types, of course) texture per model. It doesn't integrate with the model loading system or the animation system. It would have been much better to have patched Irrlicht than to try and work outside it.

[The_Glitch]

Agree with Mongoose7

[christianclavet]

Irrlicht need something like this for a long time. There are issues each time we try to have an animated model with diffuse/spec/normal maps. (There even not a build in shader for this.) This is a good example of that it can be done.

I agree that Irrlicht would need a patch, but a lot of time, patches are not accepted (hendu provided patches for hardware instancing that have never been applied and now they are obsolete). If somebody want to have improved features, it's sometimes preferable to "work outside it", if you want to share it, and not give patches that will never be used.

So who will patch Irrlicht to have this "during the year" if the current devs are so busy?! At least this solution can be used NOW, and could serve as a reference to create a patch later when the devs have the time to do it. Or be used as a example in the distribution...

I'm really grateful that vectrotek showed me how to do it, than to wait ages to see this happening in Irrlicht core. Personally, I'm not an engine programmer at all, and I depend on theses contributions to do anything outside "the box".