This code preserves the artist's original normals when calculating tangent space. This way, normal-mapped models should look closer to the artist's intent. Smooth surfaces will stay smooth, and flat surfaces will stay flat.
Hey, cool. There had been some efforts in doing such an extension some month ago, but I think they never finished the stuff. I'll have a look at it soon. Is this under zlib-license in case we add it to the core library?
One question after looking at the code: You're just taking the average of the neighbouring tangents and binormals. Since the normal is left unchanged it won't fit the binormal then. I'd have thought that you need to calculate binormal and tangent based on the current normal (and leaving it unchanged).
You're right - If your normals aren't smooth they won't be perfectly orthogonal to the tangent planes. I don't know how much difference that makes. It seems to look good, but might look better if I add a postpass to rotate the tangent and bitangent into alignment with the pre-existing normal.
My newest code recomputes the normal vectors, so that won't be a problem, at least for this version.
It uses an angle-weighted average to compute a more realistic tangent space. The angle-weight method is used in recalculateNormals as well, so vertex-lighted materials are improved as well as normal-mapped ones.
Here's the new code. You can compare methods by toggling USE_ANGLE_WEIGHTED_NORMALS. Give it a spin!
I wasn't really sure about the old version, due to the concerns I posted. I think I lost track of this due to some other work, at least I don't know if I ever tried the new version. At least making it the default normals and tangent calculations might require lots of tests, making it just another manipulation method would be ok.
Also, last time I tried patched from ryanclark I had to port lots of stuff, because he did not develop against the latest SVN versions. This is no general problem, but can delay addition ot the library quite a lot.