Strange lighting with shader

You are an experienced programmer and have a problem with the engine, shaders, or advanced effects? Here you'll get answers.
No questions about C++ programming or topics which are answered in the tutorials!
Post Reply
The_Glitch
Competition winner
Posts: 523
Joined: Tue Jan 15, 2013 6:36 pm

Strange lighting with shader

Post by The_Glitch »

This is my tangent space shader which is working pretty well except I can't figure out why these harsh spots occur. I believe it may be due to my shader using the View Inverse and when I try to set that up in irrlicht maybe I've done something wrong.

Thanks The_Glitch

Image
Image

VertexShader


Code: Select all

 
class TangentSpaceNormalMapping_Shader2 : public video::IShaderConstantSetCallBack
{
public:
    TangentSpaceNormalMapping_Shader2() : FirstUpdate(true)
    {
    }
 
    virtual void OnSetConstants(video::IMaterialRendererServices* services,
        s32 userData)
    {
        video::IVideoDriver* driver = services->getVideoDriver();
 
        // get shader constants id.
 
        if (UseHighLevelShaders && FirstUpdate)
        {
 
 
            ViewMat = services->getVertexShaderConstantID("view_proj_matrix");
            InvViewMat = services->getVertexShaderConstantID("inv_view_matrix");
            LightAtt = services->getVertexShaderConstantID("Light1_Attenuation");
            Light = services->getVertexShaderConstantID("lightPos");
            cameraPosition = services->getVertexShaderConstantID("view_position");
            LightCol = services->getPixelShaderConstantID("Light1_Color");
            LightCol2 = services->getPixelShaderConstantID("Light2_Color");
 
 
        }
 
 
 
        core::matrix4 Proj, View, world;
        
        Proj = driver->getTransform(video::ETS_PROJECTION);
        View = driver->getTransform(video::ETS_VIEW);
        world = driver->getTransform(video::ETS_WORLD);
 
        core::matrix4 matWorldViewProj1 = Proj * View * world;
        services->setVertexShaderConstant(ViewMat, matWorldViewProj1.getTransposed().pointer(), 16);
        
        core::matrix4 matWorldViewProj2 = world;
        services->setVertexShaderConstant(InvViewMat, matWorldViewProj2.getTransposed().pointer(), 16);
        matWorldViewProj2.makeInverse();
 
 
        SLight light;
 
        light = driver->getDynamicLight(0);
 
        core::vector3df pos = device->getSceneManager()->
            getActiveCamera()->getAbsolutePosition();
 
        core::vector3df light_position = light.Position;
 
        
        f32 LA[4] = { 0.0f, 0.000250f, 0.0f ,1000.0f};
        services->setVertexShaderConstant(LightAtt, reinterpret_cast<f32*>(&LA), 4);
 
 
        f32 LP[4] = { light_position.X, light_position.Y, light_position.Z, 10.0f };
        services->setVertexShaderConstant(Light, reinterpret_cast<f32*>(&LP), 4);
 
 
        f32 C[4] = { pos.X, pos.Y, pos.Z, 10.0f };
        services->setVertexShaderConstant(cameraPosition, reinterpret_cast<f32*>(&C), 4);
 
 
        video::SColorf Col1(0.5922f, 0.6556f, 0.8421f, 1.0f);
        services->setPixelShaderConstant(LightCol, reinterpret_cast<f32*>(&Col1), 4);
 
        video::SColorf Col2(0.25f, 0.25f, 0.25f, 1.0f);
        services->setPixelShaderConstant(LightCol2, reinterpret_cast<f32*>(&Col1), 4);
 
 
    }
 
private:
 
 
    s32 ViewMat;
    s32 InvViewMat;
    s32 LightAtt;
    s32 Light;
    s32 cameraPosition;
    s32 LightCol;
    s32 LightCol2;
 
 
 
    bool FirstUpdate;
};
 
 
Last edited by The_Glitch on Tue Sep 15, 2015 12:09 am, edited 1 time in total.
Vectrotek
Competition winner
Posts: 1087
Joined: Sat May 02, 2015 5:05 pm

Re: Strange lighting with shader

Post by Vectrotek »

Hi The_Glitch! This is looking really cool!
You have made me decide to go back to CG or HLSL for Irrlicht!
(I've got the CG "Wrapper" so I'll try)
I'm going to go CG now.. Ill post my last GLSL stuff first.


Try rendering your normals as "Range Compressed" pixels..
and seeing if they look "Right", then tweak..
Do you have the light connected to the camera?
(a must for tests)

I don't know if you've got my previous post download but
in there you can see how..
It is GLSL but is fundamentally the same..
This is a good example of which matrices go where form IRR point of view. GLSL makes no diffs.


Ill keep you posted on my progress.
(by the way.. if you know how to feed MORE THAN FOUR IMAGES to a GLSL shader let me know pleeease)

Those "over-bright highlights" looks like something was not normalized ????
Also note the "Normal Map Standard" used i.e. Y-UP vs Y-DOWN..
Matrices.. (which ones to use where)
=====================

I've done some work in CG under my own win 32 Opengl Programs
so I cant give you any verbatim answers but I think the following should help..
Here is a piece of CG code that I designed in Nvidia Composer back in the day..

It uses all the elements you seem to use in your code and it rendered "physically correct".


I hope it helps! (first try delete the things you dont need)
I know youll make sense of it. (did the tangent update work?)

Code: Select all

 // ==== CONNECTOR DATA STRUCTURES ==== 
 // data from application vertex buffer..
 struct appdata 
  {float3 Position  : POSITION;
   float4 UV        : TEXCOORD0;
   float4 Normal    : NORMAL;
   float4 Tangent   : TANGENT0;
   float4 Binormal  : BINORMAL0;
  };
 // data passed from vertex shader to pixel shader..
 struct vertexOutput 
  {float4 HPosition     : POSITION;
   float2 UV            : TEXCOORD0;
   // The following values are passed in "World" coordinates since 
   // it tends to be the most flexible and easy for handling       
   // reflections, sky lighting, and other "global" effects.       
   float3 LightVec_001      : TEXCOORD1;
   float3 WorldNormal       : TEXCOORD2;
   float3 WorldTangent      : TEXCOORD3;
   float3 WorldBinormal     : TEXCOORD4;
   float3 WorldView         : TEXCOORD5;
   float3 LightVec_002      : TEXCOORD6;
   float3 LightVec_003      : TEXCOORD7;
 
   #ifdef USE_SHARED_SHADOW
     // This optional value expresses the current location in "light"
     //   coordinates for use with shadow mapping.
     float4 LProj       : LPROJ_COORD;
   #endif /* USE_SHARED_SHADOW */
  };
 
 // ===============  VERTEX PROGRAM  ================= 
 vertexOutput std_VS(appdata IN)
  {vertexOutput OUT = (vertexOutput)0;
   OUT.WorldNormal = mul(WorldITXf,IN.Normal).xyz;
   OUT.WorldTangent = mul(WorldITXf,IN.Tangent).xyz;
   OUT.WorldBinormal = mul(WorldITXf,IN.Binormal).xyz;
   float4 Po = float4(IN.Position.xyz,1);
   float3 Pw = mul(WorldXf,Po).xyz;
   OUT.LightVec_001 = (Lamp0Pos - Pw);
   OUT.LightVec_002 = (Lamp1Pos - Pw);
   OUT.LightVec_003 = (Lamp2Pos - Pw);
   #ifdef FLIP_TEXTURE_Y 
     OUT.UV = float2(IN.UV.x,(1.0-IN.UV.y));
   #else /* !FLIP_TEXTURE_Y */
     OUT.UV = IN.UV.xy;
   #endif /* !FLIP_TEXTURE_Y */
   #ifdef USE_SHARED_SHADOW
   float4 Pl = mul(ShadowViewProjXf,Pw);  // "P" in light coords
   float4x4 BiasXf = make_bias_mat(ShadBias);
   OUT.LProj = mul(BiasXf,Pl);      // bias to make texcoord
   #endif /* USE_SHARED_SHADOW */
   OUT.WorldView = normalize(float3(ViewIXf[0].w,ViewIXf[1].w,ViewIXf[2].w) - Pw);
   OUT.HPosition = mul(WvpXf,Po);
   return OUT;
  }
 // ===============  FRAGMENT SHADER  ================= 
 void phong_shading(vertexOutput IN,
                    float3 LightColor_01,
                    float3 LightColor_02,
                    float3 LightColor_03,
                    float3 Nn,
                    float3 Ln_01,
                    float3 Ln_02,
                    float3 Ln_03,
                    float3 Vn,
                    out float3 DiffuseContrib_01,
                    out float3 DiffuseContrib_02,
                    out float3 DiffuseContrib_03,
                    out float3 SpecularContrib_01,
                    out float3 SpecularContrib_02,
                    out float3 SpecularContrib_03
                   )
  {
   float3 Hn_01 = normalize(Vn + Ln_01);
   float3 Hn_02 = normalize(Vn + Ln_02);
   float3 Hn_03 = normalize(Vn + Ln_03);
 
   float4 litV_01 = lit(dot(Ln_01,Nn),dot(Hn_01,Nn),SpecExpon);
   float4 litV_02 = lit(dot(Ln_02,Nn),dot(Hn_02,Nn),SpecExpon);
   float4 litV_03 = lit(dot(Ln_03,Nn),dot(Hn_03,Nn),SpecExpon);
 
   DiffuseContrib_01 = litV_01.y * LightColor_01;
   DiffuseContrib_02 = litV_02.y * LightColor_02;
   DiffuseContrib_03 = litV_03.y * LightColor_03;
 
   float3 SpecularMapped = tex2D(SpecularSampler,IN.UV).rgb;
   SpecularContrib_01 = litV_01.y * litV_01.z * Ks * LightColor_01 * SpecularMapped;
   SpecularContrib_02 = litV_02.y * litV_02.z * Ks * LightColor_02 * SpecularMapped;
   SpecularContrib_03 = litV_03.y * litV_03.z * Ks * LightColor_03 * SpecularMapped;
  }
   
 
 float4 std_PS(vertexOutput IN) : COLOR 
  {
   float3 diffContrib_01;
   float3 diffContrib_02;
   float3 diffContrib_03;
 
   float3 specContrib_01;
   float3 specContrib_02;
   float3 specContrib_03;
 
 
   float3 Ln_01 = normalize(IN.LightVec_001);
   float3 Ln_02 = normalize(IN.LightVec_002);
   float3 Ln_03 = normalize(IN.LightVec_003);
   float3 Vn = normalize(IN.WorldView);
   float3 Nn = normalize(IN.WorldNormal);
   float3 Tn = normalize(IN.WorldTangent);
   float3 Bn = normalize(IN.WorldBinormal);
   float3 bump = Bump * (tex2D(NormalSampler,IN.UV).rgb - float3(0.5,0.5,0.5));
 
   float3 VertNormal = Vn;
 
 
   Nn = Nn + bump.x*Tn + bump.y*Bn;
   Nn = normalize(Nn);
   phong_shading(IN,Lamp0Color, Lamp1Color, Lamp2Color,
                       Nn,Ln_01, Ln_02, Ln_03,
                       Vn,diffContrib_01, diffContrib_02, diffContrib_03,
                          specContrib_01, specContrib_02, specContrib_03 );
   // == = == = == = == = == = == = == = == = == = == = == = == = == = == = == = == =
   float3 diffuseColor =  tex2D(ColorSampler,IN.UV).rgb;
 
  // diffContrib_01 *= DiffuseMultiplier;
  // diffContrib_02 *= DiffuseMultiplier;
  // diffContrib_03 *= DiffuseMultiplier;
 
    diffuseColor *= DiffuseMultiplier;
 
   #ifdef USE_SHARED_SHADOW
     //float shadowed = tex2Dproj(DepthShadSampler,IN.LProj).x;
     //float faded = 1.0-(ShadDens*(1.0-shadowed));
     //diffContrib_01 *= (faded);   // JACQUES..
     //specContrib_01 *= shadowed;
   #endif /* USE_SHARED_SHADOW */
 
   float3 result = specContrib_01+(diffuseColor*(diffContrib_01));
 
    result += specContrib_02+(diffuseColor*(diffContrib_02));     // CHECK THIS OUT!!
    result += specContrib_03+(diffuseColor*(diffContrib_03));     // CHECK THIS OUT!!
 
   float3 R = reflect(Vn,Nn);
   float3 reflColor = Kr * texCUBE(EnvSampler,R.xyz).rgb;
   float3 Reflop = (tex2D(ReflopSampler,IN.UV).xyz);
   reflColor.xyz *= Reflop.xyz;  // DO THIS AGAIN!!
 
 
   float3 FinalOutput;
 
    float3 Emmision = ( tex2D( EmmisionSampler,IN.UV ).rgb  ) ;
 
      result.xyz += Emmision.xyz;
 
 
   result += reflColor;
   result.x *= RedFilter;  result.y *= GreenFilter;  result.z *= BlueFilter;
   float3 NormalOut;
   NormalOut = ( tex2D( NormalSampler,IN.UV ).rgb  ) ;
 
 
   if (RenderMode == 1) {FinalOutput = result;}
   if (RenderMode == 2) {FinalOutput = NormalOut;}
   if (RenderMode == 3) {FinalOutput = specContrib_01 + specContrib_02 + specContrib_03;}
   if (RenderMode == 4) {FinalOutput = (Ln_01 / 2) + 0.5;}
   if (RenderMode == 5) {FinalOutput = (VertNormal / 2) + 0.5;}
   if (RenderMode == 6) {FinalOutput = (Tn / 2) + 0.5;}
   if (RenderMode == 7) {FinalOutput = (Bn / 2) + 0.5;}
   if (RenderMode == 8) {FinalOutput.xyz = bump.z;}
   if (RenderMode == 9) {FinalOutput = (-R / 2) + 0.5;}
   if (RenderMode == 10) {FinalOutput = result;}
 
 
    return float4(FinalOutput,1);
    
  } 
The_Glitch
Competition winner
Posts: 523
Joined: Tue Jan 15, 2013 6:36 pm

Re: Strange lighting with shader

Post by The_Glitch »

Well I've gotten most of the issue's fixed I'll post a video link. I just need to make sure my implementation of the ViewInverse is right in Irrlicht.

Code: Select all

 
 
               core::matrix4 Proj, View, world;
        
        Proj = driver->getTransform(video::ETS_PROJECTION);
        View = driver->getTransform(video::ETS_VIEW);
        world = driver->getTransform(video::ETS_WORLD);
 
        core::matrix4 matWorldViewProj1 = Proj * View * world;
        services->setVertexShaderConstant(ViewMat, matWorldViewProj1.getTransposed().pointer(), 16);
        
        core::matrix4 matWorldViewProj2 = View;
        matWorldViewProj2.makeInverse();
        services->setVertexShaderConstant(InvViewMat, matWorldViewProj2.pointer(), 16);
 
 
I set View matrix to make Inverse I assume this is how you do this in Irrlicht? Not to sure.
Also haven't used your tangent code snippet as there are things that need to be changed in it for it to work with shader pipeline.

Video
https://youtu.be/So9A5-rLc5U
Vectrotek
Competition winner
Posts: 1087
Joined: Sat May 02, 2015 5:05 pm

Re: Strange lighting with shader

Post by Vectrotek »

That looks great! (the video). One more post, and then I'm finished with GLSL. (the post will have some useful generic stuff)
I have to go "CG" in my own projects.
I'll let you know if I find anything useful. By the way, are you using CG or HLSL? (the GPU code looks the same)
(just asking) If you're using CG that would be cool cause I need help! I'm steering clear of HLSL. Is the video rendered by CG or HLSL?
The_Glitch
Competition winner
Posts: 523
Joined: Tue Jan 15, 2013 6:36 pm

Re: Strange lighting with shader

Post by The_Glitch »

I'm using Hlsl bud. It's similar to cg though. I'm still working out things with the shader it's not %100 yet.
Vectrotek
Competition winner
Posts: 1087
Joined: Sat May 02, 2015 5:05 pm

Re: Strange lighting with shader

Post by Vectrotek »

Thanks Glitch! I'm sure between HLSL and CG we'll sort it out! (check out my final GLSL post when it arrives) Who knows what the future holds! Cheers and keep up the good work!
jarlsburg
Posts: 3
Joined: Tue Oct 20, 2015 5:04 am

Re: Strange lighting with shader

Post by jarlsburg »

Vectrotek wrote:Thanks Glitch! I'm sure between HLSL and CG we'll sort it out! (check out my final GLSL post when it arrives) Who knows what the future holds! Cheers and keep up the good work!
Hi Vectrotek, just wanted to mention that CG is now deprecated and hasnt been updated for a while. This forced me to make the switch away from CG about a year ago, just wanted to give you a heads up incase this is going to cause trouble for you later.

See:

https://developer.nvidia.com/cg-toolkit
The Cg Toolkit is a legacy NVIDIA toolkit no longer under active development or support. Cg 3.1 is our last release and while we continue to make it available to developers, we do not recommend using it in new development projects because future hardware features may not be supported.
Post Reply