Bump mapping for Animated meshes

[Vectrotek]

HI! This is an example to clarify my Tangent/Binormal Problem..

It is 6.5 Megs!! I really tried to get the data as small as possible.

If you run "000_TANGENT_BALL.exe" you will see that Skinned
Animation causes "final" GLSL normals to change in unexpected ways..

Try using keys "3", "6" and "7" to change the rendermode..
You can also Slow, Hasten or Pause the animation with the "I","O","P" Keys..
"[" Brings us to the First Frame i.e. T, or NULL POSE..

Here it is: (slight code change)
I deleted this. Check later posts.
Everything is there.

I now know that Updating Tangents and Binormals is the key, but how?
Please share your code if you crack it!
Im looking at how IRRLICHT updates the normals and trying
to get the same for Tangents / Binormals..
I am sure it wouldn't be too much of a "CPU TIME PENALTY" if
we have some kind of an OPTION to do this or not..
We could even have updating done only at every "Nth" frame (again, option would be nice)..

The Normal deviation only becomes really aparent at angles greater than approx 75.0 degrees..

Keep in mind that Static objects can be Translated,
Scaled, even Rotated and the shader will light
it correclty. (Thanks to You, the Forum Members ofcourse!)

Sorry about the large ZIP file, I did everything possible to keep it small..

If You'd like a HIRES Package let me know..

The models come from the Internet and are free for educational purposes
but not for Commercial material..

[Vectrotek]

Check the "Droplet Shader" at work in the "Nanosuit" eyes..
You might want to slow the animation down or pause it..

[mongoose7]

You can't update tangents in a shader because you have no information on adjacent vertices. But if you look at CSkinnedMesh.cpp:CSkinnedMesh::skinJoint() you will see that he original positions and normals are stored against the weight in each joint and the transformed values are stored in SkinningBuffers. You just need to store the tangents in these two places and transform them along with the positions. That is, SkinningBuffers will have to be defined in terms of the tangent vertices and not the standard vertices. The same goes for the weights.

[Vectrotek]

Aha! I see what you mean.. Thanks!

[Vectrotek]

Hi! The "Live" Updating of Tangents & Binormals have been solved!
(special thanks to "Mongoose7" for his advice)

I think this came out super cool and would like hear your comments.
Apart from the "Animated Tangent Problem" being solved there
are many more things you might enjoy, such as "Realtime Shader Parameter Control".
If you want GLSL Shaders in your App then this guarantees instant inspiration!
We've got Emmision Maps, R-G-Encoded Specularity Maps,
Droplet Masked Normal Maps, All sorts of Options and much more!
We've even got "PLAECF"! which makes "True Glossmaps" possile! (check it out)
We've also got a "Special Colour Syntaxed" version of the code you
can view in "Open Office".

-------- ---- --- -- -
There are "THREE" SEPARATE downloads you can CHOOSE from.
--- -- -
The "FIRST" Download (full VC 7 project) is quite big but has ALL the
Animated Characters "*.b3d" included. (16.2 Meg Zip File)
(sorry about the weight, but anything smaller
would simply not do the work I put into this any justice)
--- -- -
The "SECOND" Download (full VC 7 project) is much smaller but includes
ONE Animated Character "*.b3d". (a more reasonable 4.4 Meg Zip File)
You could always get the rest of the Characters
later in the THIRD Download if you wish.
--- -- -
The "THIRD" Download is "THE REST" of the Animated Characters.
It is also quite big (11.7 Meg Zip File)
If you see the first one you'd probably want
to see the rest!

I had included the ALL "*.exe" files (they are small) so running some of them without
having its related model in its directory will cause errors.
I have also kept the "Diretory structure" intact so you can just download them and
copy them over.. "001_Makron_II" is the one that is included in the smaller version.

If you have problems running the examples then please let me know.
I've included the latest Irrlicht 1.8.2 "*.dll".
Also, for compiling, the "*.lib" included is the latest one.


Id love to hear your opinions!
See how "Droplet Shading" works quite well for "Blood Effects"
in the "Hell Knight" model!


Enjoy!

DOWNLOADS:
=========================================
FIRST DOWN LOAD (big): http://s000.tinyupload.com/?file_id=901 ... 6605149362
=========================================
SECOND DOWN LOAD (small):http://s000.tinyupload.com/?file_id=837 ... 6460510590
=========================================
THIRD DOWN LOAD (big): http://s000.tinyupload.com/?file_id=075 ... 2256657122

[Vectrotek]

Here are some screenshots:

[Vectrotek]

Sorry for these screenshots being of such a demented nature. The thing is that these models are the only ones I could find with proper normal maps. Many times these maps get screw up in the baking process rendering a nice "looking" normal map useless. This happens when 3D Coat or other apps are used to transfer normal maps from one object to another where the UV Islands get "turned".. (normal maps then lose their direction) If anyone knows of good animated or still models with good normal maps let me know.
I hope that the shader stuff is useful. Ill try to post milder content next time! As for the mere mention of the formula used in gloss mapping.. What would you have done if you were onto something as ground breaking as this? I posted the whole project and all the resources.. Please don't let the PLAECF thing put you off (its just an example).. 99.999% of the stuff is yours to do what you like with. Its just my way of giving back to this forum for what I had gained from it. Read the bit on LSSOMs.. You'll definitely see this in the future and its yours to develop if you like! Also, Id really appreciate help with the Binary Encoded image matter..

[BlindSide]

Hi Vectrotek,

Thanks for your contribution, but I think it would benefit everybody if you posted it over at the code snippets forum as a new thread. The models look ok, standard stuff for first person shooters.

[The_Glitch]

Vectrotek could you do this for shader pipeline? So animated objects will be correct.

[Vectrotek]

I posted the "important bit" in "Code Snippets".. I hope that would simplify implementation in the "Pipeline" (which I'm not too familiar with yet).

[Vectrotek]

The last render with GLSL before we go CG..

Looks like shes trying to say something!

[Vectrotek]

Help! Does anyone know how to pass MORE THAN 4 IMAGES to a GLSL Shader?
I tried changing this in the API "IrrCompileConfig.h" file..

line 00117: #define _IRR_MATERIAL_MAX_TEXTURES_ 8 // was 4..

I've even tried to change the "Pixel Shader Profile".
I've tried everything but could NOT get the FIFTH Image (or anything above that) into the shader..

====================================================================

I'm placing GLSL development for for my Irrlicht Apps on hold for the following reasons:
1. I've done a lot of work in CG previously, and after seeing what can be done
with CG/HLSL in Irrlicht, Im think CG / HLSL is the way to go..
2. I could not get a proper Skycube to work in Irrlicht with GLSL..
(It's been done it in my other pure Win 32 and MFC OpenGL apps using "*.CG")
I may post some non Irrlicht related (yet usefull) material under "Anything 2D 3D Graphics"
A successfull implementation of the "Skycube" concept will be great for "Reflection" rendering.
Even if what exists currently in Irrlicht CG don't do skycubes right, There is a LOT that can be posted for a communial solution.
3. We could not feed more than FOUR images into the GLSL shader.. (grinding teeth)
4. Nvidia has good support for CG, HLSL and CGFX..("NVIDIA Composer" is still a valuable asset for developers)
5. Much of what I,ve learned with Irrlicht GLSL can be directly ported to CG..
6. I certainly hope upon hope that more than 4 images can be fed to CG shaders from Irrlicht!
if this is not the case then the Irrlicht Core Engine (textures) needs some serious looking at..

Here is my latest and last stuff done with Irrlicht "GLSL".
I'm posting it because most of it will be taken along on an Irrlicht "CG" journey.

Here is the latest stuff done with the GLSL shader:

1. The "Normal Map" and "Specularity & Gloss" is now encoded into a single "RGBA" image!
Packing the Normal Map into TWO 8-Bit channels (other two for Spec && Gloss)
became possible because we can "solve for" the Z value of the Normal Map in the
"Pythagorean Theorem" as follows: (in the Fragment Shader)

Step 1: NormalDist = sqrt(X^2 + Y^2 + Z^2) The original statement. We all know that the Normal Distance is ONE, thus..
Step 2: 1^2 = X^2 + Y^2 + Z^2 "Sqrt" went "over" and became a "Squared" ..
Step 3: 1 - X^2 = Y^2 + Z^2 1^2 is obviously 1 and X^2 went "over" becoming "negative" X^2.
The result was POSITIVE or NEGATIVE.. we chose Positive.. (lucky guess)
Step 4: 1 - X^2 - y^2 = Z^2 The Y^2 went "over" and became "negative" Y^2..
Step 5: sqrt(1 - X^2 - Y^2) = Z The "squared" for the "Z" went "over" and became "sqrt".. (again we had a lucky guess Positve)..
O.K.
The final CG / GLSL / HLSL / CGFX / FX conversion in the GPU Shader code:

// code..
// CALCULATE Z ON THE FLY USING A "SOLVED FOR Z" VERSION OF PYTHAGORAS'S THEOREM!!
UnCompressedNormal.z = sqrt(-(UnCompressedNormal.x * UnCompressedNormal.x) - (UnCompressedNormal.y * UnCompressedNormal.y) + 1.0);
// No need for the "Blue" i.e "Z" channel in the Normal Map!
// This saved a WHOLE LOT of space and time!

2. We used the ALPHA Channel in the "Diffuse" image to do two the following THREE things:
a. "Clipmapping" needs a "small" difference, somewhere between 40% and 60% intensity to work well.
Clipmapping does not work well with a flat "White or Black" contrasted Alpha Value (see the "Pixel Squares"),
so we "reserved" some "grey" between 40% and 60% in the Alpha Channel to maintain the "smooth cutoff" effect.
b. "Droplet Shading" A neat trick we want to hold on to, formerly activated through the normal
map Alpha Channel now became a problem as we used the Alpha Channel in the NEW "NSG" map for Gloss Mapping.
The "Droplet Mask" now lives in the "Higher 40%" of the Alpha Channel Intensity.
Like the Clipmap the Droplet Mask needs a degree of grey between white and black to avoid anoyingly visible borders..
Now, converting the Higher 40% of the Alpha Channel to a "0.0 to 1.0" value for the
"Droplet Mask" occurs in the Shader as follows:
// code..
if (MAPDiffuseRGBA.w > 0.6) // Referring to the Upper 40% of Alpha Intensity..
{DropletMask = (MAPDiffuseRGBA.w - 0.6) * 2.5; // Convert the Upper 40% into a 100% range for smooth bordering..
}

c. The "lower" 40% of the Diffuse Alpha Channel is now available for a "Reflection Factor" for which which we also need a "grey".
We could also use this domain or some of it for Transparency. (a lot can be saved by understanding that the full 256 levels of
grey is a waste for most grey based optical effects)
(obviously we need to know about "Mixing RGBA Channels" in "Photoshop", "Gimp" or other "Bitmap Editing Apps" for these things)
Where we "cut off" these domains for "what" in the Alpha Channel would really depend on the agreement between coders and artists.

3. I've added more lights in the shader resulting in a far better "Specular Show". (not from the App, as we will eventually do)

4. I tested a new painting method on an animated model from a famous game which worked quite well. (ask if you want to know how it was done)

Note: During Development I use "*.x" for animated models and "*.tga" for RGBA Images.
For uploading projects, to keep them small, I use "*.b3d" for animated models and "Colour Domain Reduced" "*.png" for RGBA Images.


1. Here is the COMPLETE PROJECT, DATA AND EXECUTABLE:
(being updated)


Im done with GLSL so if, in a few months, I do anything worthwhile in "CG" I'll post it..
The cool thing about "CG" is that any "problems solved" would be easily portable to "HLSL"!

Final note: The way matrices are passed from Irrlicht to the shaders of any language is the same.
The structure of the data may change so try some experimentation.
It is what your shader does with those matrices that is important.
(youll see a lot of "unnecessary stuff" sent to shaders in official examples, well...)
Some 3d models have their UV's mysteriously "Shifted" along an axis by 1.0 unit which is something you'll not be aware of
as most modelers automatically fix this in the actual view of the UV's (The actual shifted UV data remains unchanged however)
This "secret shift" plays havoc and results in some serious headaches when calculating Tangents. Goodluck!
Oh, by the way, "DO NOT TRY CALCULATING BINORMALS IN YOUR SHADER BY THINKING YOU CAN "CROSS" THE TANGENTS WITH NORMALS"..
It wont work! You can take it from me!

If the Irrlicht Authors could do something about the MAX 4 Images fed to the shader I'd be very glad!
The way that "Sampler Cubes" are sent to shaders is also something I'd really like to see addressed.

If you need help, when you post, try to send more information on what you use in terms of graphics apps and
shader languages. I feel that you can never put too much non-code plain english information in your posts.
Try give your "post reader" as much information as you can so he has a good understanding of your problem or your advice!
Dont be afraid to repeat things many times over many posts.

Enjoy! I hope my posts were usefull. Cheers!

[hendu]

If you changed the define and recompiled irrlicht, and still couldn't do so, then your GPU may not support more than four textures. Check with a GL limits viewer.

Use texture arrays instead.

[Vectrotek]

Hendu! Bless you a thousand times my man! No.. Bless you a million times!
There are two versions of "irrCompileConfig.h".. One in the Shipped API which most of us use, and one in the Engine Code.
I assumed, that if I change "_IRR_MATERIAL_MAX_TEXTURES_ 4" to "_IRR_MATERIAL_MAX_TEXTURES_ 8" in the API version, (I can't believe I actually thought that) then the Engine would operate differently, it doesn't..
So, I made the change in the "Engine Code" version and recompiled Irrlicht 1.8.2 which resulted in a NEW "irrlicht.dll" and "irrlicht.lib". (you know all of this, but I thought I'd mention it for people like me)
I then replaced the DLL in my system32 or App release directory and replaced the LIB in my VC IDE.
Finally I recompiled my App and, guess what! It worked, and I spilled boiling hot coffee all over myself!
I can breathe again!

Ill post the "irrlicht.dll" and "irrlicht.lib" files just in-case someone had the same problem but cant compile the engine code right now..

here: (DLL) http://s000.tinyupload.com/?file_id=009 ... 5513481354
and here: (LIB) http://s000.tinyupload.com/?file_id=901 ... 9008295008

The cherry on top is that now that I am sure about this image count, I can peacefully continue experimenting with CG/HLSL in Irrlicht!
O.K. Cg is out! HLSL it is.

Little note: I read that GLSL and most hardware can actually support 32 and more images? Why are there Core Engine constraints of 8 images/textures here?
It doesn't matter! I can fit a LOT into 8 x 32 bit images!
Anyhow, I'm sure this little revelation would mean something to a few coders out there!

Thanks again Hendu!

[Vectrotek]

I need to correct myself! The two "IrrCompileConfig.h" files I spoke of is ONE AND THE SAME FILE!
The "1.8.2 Engine Code" uses some of the API Shipped header files, already in your IDE directory from the API (again I assumed otherwise)..
Man! With this stuff you've got to keep your head about you!
When migrating from Irrlicht 1.8.1 to 1.8.2 and then on top of that recompiling 1.8.2 for 8 Shader images
you end up with quite a mess if you don't keep your eyes open! Add to this mess the fact that three versions of your
appcode ago is where you actually wanted the new functionality with the new DLL and LIB.
The Engine Code is not the API, but changes made in the API files will be included when you compile the "Engine Code".
(right?)