4 light normalmapping for animated meshes [C++/GLSL]

[mrcdrc]

You definitely need to align the tangents along u and v axes, see ShaderX^5. At least this is widely accepted to give good lighting results when generating precomputed tangents.

As tangent space is local to each face, and in order to get a correlation between all local texture space coordinate systems it's not 100% correct to just swap components of the vertex normal.

The error should become more pronounced on low resolution meshes with large poly's if i'm correct, am i?

But nontheless good work! One could implement the correct real time tangent space generation in your code with little effort i guess. Need to read that article in SX^5 in more detail...

mrc

[noreg]

One could implement the correct real time tangent space generation in your code with little effort i guess.

Bite, don't bark. If it's just a "little effort", ...

[omaremad]

Actually it would require you two send the data of the neighbouring vertcies as attributes. this would require either new vertex classes or do it in the irrlicht driver class before sending to the GPU.(ie: allot of wasted cpu time and ram, and more calls to the rendering api)

The problem is pumping more floats per vertex would lead to bloating of the pre t&l cache (so you get less vertex pushing speed) and also badly aligned vertex data would lead to cache misses and therefore slow down.

The optimum vertex data structure should be a multiple/factor of 32kb since thats size of each cahe row in the vertex cache.

I dont really see why people are complaing about "the poor accuracy of the tangent" if his method produces mathamateically correct tangents if the normals are correct (which they are).

[Luben]

But if the tangents are not aligned with the UV-coords, won't you get normal maps which's(Up/right in tex) are not aligned with up and right? Ehr, sortof. I'll draw a picture of what i mean tomorrow.

[Saturn]

Luben, yes sort of. The problem is, that light direction is transformed into tangent space, which is usually aligned so that the tangent is along U and binormal along V. The normals in the normal map are stored with this assumption. When it is not met, the light direction used for lighting is off. Mostly you don't notice it, as the effect itself (falloff, coloring, etc.) looks just fine, but the direction is wrong, subtly or notedly depending on how the mesh is unwrapped. This is most notable with long fine ridges on a mesh, since the light directions seems to change when the orientation of the unwrapped triangles on the uvmap changes.

omaremad, you probably mean 32 byte, not kilobyte. But anyway, this alignment is indeed a huge performance factor. I was really surprised when I first learned about it. The point I hope I could convey is that not any tangent space will do (there are infinite fitting ones, not putting into account limited float precision. ), but the tangent space that the normal map is made for.

[Luben]

Saturn, do you know where i can read about that alignment thing? It'd probably be interesting at least

[TheGameMaker]

I don´t think it is possible to align the tanents to the UV within the shaders...
You would need to have access to the other verts UV Coordinates

[Saturn]

TheGameMaker, no not with vertex shaders anyway. DirectX 10 Geometry Shader can do it though: http://msdn2.microsoft.com/en-us/librar ... ader_Stage

Luben, I have no web link at hand for this. The book ShaderX 4 has an article about how to create a high quality tangent space base.

[omaremad]

I ran a few visualisations through render moneky and it seems each method has its advantages.

The alignment is perfect using the dot product method if both the uv maps and the geometry coordinates increase in the same direction (no rotated vertices in the uv map, mirrored vertices need their own uv space)

however if your uvmap are mirrored or parts of your mesh are rotated in the uv map then your alignment is wrong.

The cpu calculated method produces artifacts especially around texture seams ( i call them artifacts because the cause sharp jaggies in the actual lighting). however no alignment issues are experienced.


Ill upload all the pics/render moneky files later but here is a comparision

[Saturn]

How did you create your tangents, omaremad? There is an abundance of different algorithms with different trade-offs. (Not that I know the details, but at least this much. )

And mirrored triangles are not a problem for CPU based algos, since the direction of the mapping can be determined. You can save the direction of the binormal in the tangent's w-component and evaluate it in the shader. 1 for forward mapped, -1 for backward mapped.

[TheGameMaker]

uhm.. DX.. I don´t like DX.. have you seen this:
http://www.ubuntuusers.de/ikhaya/488/??
No.. I definitiv don´t want to use Dx..

[Saturn]

Well, I don't intermingle politics with everything else. But this doesn't matter. Since you can as well use GL_EXT_geometry_shader4. If you have a SM4.0 card that is.

[Virion]

uhm.. DX.. I don´t like DX.. have you seen this:
http://www.ubuntuusers.de/ikhaya/488/??
No.. I definitiv don´t want to use Dx..

can i have an english version one?...

[Ico]

One of their sources linked in the article: http://money.cnn.com/magazines/fortune/ ... /index.htm

[mrcdrc]

One can align tangents in the pixel shader along uv axes using the built-in functions ddx, ddx (Cg) or dFdx, dFdy (GLSL). These functions alow for calculation of partial derivatives in terms of screen coordinates x and y.
This way you don't need to pass neighbouring verts/uv's...

I just tried that based on the mentioned article in Shader X5. The sad thing about it is that my card is not capable of doing the partial derivation in hardware, so the framerate is far below 1 fps...and testing was actually a pain and finally impossible for me.

Here comes the shader code i used (reduced the code to 1 light for simplicity)

Code: Select all

const stringc vertBumpShader =
"uniform vec3 fvLightPosition1;"
""
"uniform float fLightStrength1;"
""
"uniform mat4 matWorldInverse;"
""
"varying vec2 Texcoord;"
"varying vec3 ViewDirection;"
"varying vec3 LightDirection1;"
"varying vec4 LightDistMultiplier;"
"varying vec3 vertNormal;"
"varying vec3 vertPosition;"
""
"float getLengthSQR (vec3 vec)"
"{"
"return(vec.x*vec.x+vec.y*vec.y+vec.z*vec.z);"
"}"
""
"void main( void )"
"{"
""
"   mat4 LightTransform= gl_ModelViewMatrix;"
"   LightTransform=LightTransform*matWorldInverse;"
""
"   gl_Position = ftransform();"
"   Texcoord    = gl_MultiTexCoord0.xy;"
"   vertNormal  = vec3(gl_Normal);"
"   vertPosition = vec3(gl_Vertex);"
""
"   vec4 fvObjectPosition = gl_ModelViewMatrix * gl_Vertex;"
"   vec4 fvLightPos1 = LightTransform * vec4(fvLightPosition1,1.0);"
""
"   vec3 fvViewDirection  =  - fvObjectPosition.xyz;"
""
"   vec3 fvLightDirection1 = (fvLightPos1.xyz - fvObjectPosition.xyz);"
""
"   LightDistMultiplier[0]=1.0/(getLengthSQR (fvLightDirection1)/(fLightStrength1*10000.0));"
"   ViewDirection   = fvViewDirection;"
"   LightDirection1 = fvLightDirection1.xyz;"
"}";
// --------------------------------------------------------------------------

// OpenGL Fragment Program 1.1
const stringc fragBumpShader =
"uniform vec4 fvAmbient;"
"uniform vec4 fvLight1Color;"
""
"uniform float fSpecularPower;"
"uniform float fSpecularStrength;"
"uniform float fBumpStrength;"
""
"uniform sampler2D baseMap;"
"uniform sampler2D bumpMap;"
""
"varying vec2 Texcoord;"
"varying vec3 ViewDirection;"
"varying vec3 LightDirection1;"
"varying vec4 LightDistMultiplier;"
"varying vec3 vertNormal;"
"varying vec3 vertPosition;"
""
"/* Code taken from Shader X5: "
"   Normal Mapping without precomputed Tangents"
"*/"
""
"mat3 invert_3x3(mat3 M)"
"{"
"  float det = dot(M[0]*M[1], M[2]);"
"  /* get the transpose of M */"
"  mat3 T = mat3(vec3(M[0][0],M[1][0],M[2][0]),vec3(M[0][1],M[1][1],M[2][1]),vec3(M[0][2],M[1][2],M[2][2]));"
"  return mat3(T[1]*T[2], T[2]*T[0], T[0]*T[1])/det;"
"}"
""
"mat3 compute_tangent_frame(vec3 N, vec3 p, vec2 uv)"
"{"
"  /* get edge vectors of the pixel triangle */"
"  vec3 dp1  = dFdx(p);"
"  vec3 dp2  = dFdy(p);"
"  vec2 duv1 = dFdx(uv);"
"  vec2 duv2 = dFdy(uv);"
""
"  /* solve the linear system */"
"  mat3 M = mat3(dp1, dp2, dp1*dp2);"
"  mat3 inverseM = invert_3x3(M);"
"  vec3 T = inverseM*vec3(duv1.x, duv2.x, 0);"
"  vec3 B = inverseM*vec3(duv1.y, duv2.y, 0);"
""
"  /* construct tangent frame */"
"  float maxLength = max(length(T), length(B));"
"  return mat3(T/maxLength, B/maxLength, N);"
"}"
""
"void main( void )"
"{"
""
"   mat3 TBN = compute_tangent_frame(vertNormal, vertPosition, Texcoord);"
""
"   vec3  fvLightDirection1 = normalize( LightDirection1*TBN );"

"   vec3  fvNormal         = texture2D( bumpMap, Texcoord ).yxz;"
"   fvNormal.xy*=2.0;"
"   fvNormal.xy-=1.0;"

"   fvNormal=(vec3(0.0,0.0,1.0)-fvNormal)*fBumpStrength+fvNormal;"

"   fvNormal=normalize(fvNormal);"

"   float fNDotL1           = max(dot(fvNormal, fvLightDirection1),0.0)-0.1; "

"   vec3  fvReflection1     = normalize( ( ( 2.0 * fvNormal )  ) - fvLightDirection1 ); "

"   vec3  fvViewDirection  = normalize( ViewDirection );"

"   float fRDotV1          = max( 0.0, dot( fvReflection1, fvViewDirection ) );"

"   vec4  fvBaseColor      = texture2D( baseMap, Texcoord );"

"   vec4  fvTotalAmbient   = fvAmbient * fvBaseColor; "

"   vec4  fvTotalDiffuse   = fvLight1Color * fNDotL1* fvBaseColor*LightDistMultiplier[0]; "
"   vec4  fvTotalSpecular  = fNDotL1*fvLight1Color * ( pow( fRDotV1, fSpecularPower ) )*LightDistMultiplier[0];"

"   vec4 color=( fvTotalAmbient + fvTotalDiffuse+ (fvTotalSpecular*fSpecularStrength));"
"/*   if(color.r>1.0){color.gb+=color.r-1.0;}"
"   if(color.g>1.0){color.rb+=color.g-1.0;}"
"   if(color.b>1.0){color.rg+=color.b-1.0;}*/"
"   gl_FragColor = color;"

"}"
;

And for the expert calling me a "barker": Actually implementing it was "little effort" as it took me half an hour to type in the code from the book converting it from Cg to GLSL on the fly, which is not that hard.

But even the "smallest effort" can take ages to finish when you're lacking time man...


But i for one think that passing neighbouring verts/uv's to the vertex shader and construct the tangent frame there should work as well. One has to modify the engine therefor, as far as i thought about what has to be done...

mrc