[Tutorial]Using HLSLshaders for bumpmapping + lighting (WIP)

[Abraxas)]

Hello irrlicht.

Recently I was having problems getting with getting detail mapping working on a mesh I made. I decided to stop being lazy and learn to program shaders myself because maybe I could do the texture mapping manually.

I realized there was a big gap of information between the tutorials and how to do some more advanced shading. So I figured we could pool our knowledge and write a nice shader FAQ to look at while coding shaders. I would really like to write a tutorial on using shaders for all our users.

First of all, some "lifesaver tips":

The compiler will (completely retartedly) delete variables from your code without letting you know. This leads to really stupid situations where you can't code things one step at a time. If anyone knows a way around this.....

If you CAN, do things a step at a time. Some errors are really hard to spot, especially when you're using a variable called normal : TEXCOORD and N:NORMAL. They can get really tricky.
----------------------


Personally, I have been following this:
http://digitalerr0r.wordpress.com/tutorials/

but the tuts have many errors in them (no clue how he managed to run them) and you need to understand the irr API to use them anyway.

I just finished 4+5 separately, and I figured I should document what I did for anyone else to figure out.

To get to this:


requires quite a bit.

What I did was run the standard diffuse/specular mapping logic, so you can see the light comes from the top and the bottom is mostly black.

Then I made two colors, one was the "texture color" and the other the "normal map difference". I add these two to give some bumpy looking geometry.

CAVEAT: I'm a complete beginner at shaders. More experienced programmers will look at the code and say "this could be better...more efficient..". So hopefully this will be a nice start for a tutorial!

STEP 1: Load the shader:

Code: Select all

void loadshaders()
        { //shaders
 
        io::path vsFileName; // filename for the vertex shader
        io::path psFileName; // filename for the pixel shader
 
            psFileName = "../media/Metallic.hlsl";
            vsFileName = psFileName; // both shaders are in the same file
 
        video::IGPUProgrammingServices* gpu = driver->getGPUProgrammingServices();
        shadermat1 = 0;
 
        TempCallBack* mc = new TempCallBack();
            
        shadermat1 = gpu->addHighLevelShaderMaterialFromFiles(
                vsFileName, "VertexShaderFunction", video::EVST_VS_3_0,
                psFileName, "PixelShaderFunction", video::EPST_PS_3_0,
                mc, video::EMT_SOLID);
 
        mc->drop();
    }

MAKE SURE TO PASS SHADERMAT1 (the custom shader material).
MAKE SURE THE "VertexShaderFunction" and "PixelShaderFunction" ARE THE NAMES OF THE "MAIN" FUNCTIONS IN YOUR SHADER FILE.

STEP 2: Set up the callback for shading:

Code: Select all

class TempCallBack : public video::IShaderConstantSetCallBack
{
public:
 
    virtual void OnSetConstants(video::IMaterialRendererServices* services,
            s32 userData)
    {
 
        // set clip matrix
 
        
        core::matrix4 Proj = driver->getTransform(video::ETS_PROJECTION);
        core::matrix4 View = driver->getTransform(video::ETS_VIEW);
        core::matrix4 world = driver->getTransform(video::ETS_WORLD);
 
        core::matrix4 matWorldViewProj = Proj*View*world;
 
 
            services->setVertexShaderConstant("World", world.pointer(), 16);
            services->setVertexShaderConstant("matWorldViewProj", matWorldViewProj.pointer(), 16);
            
        float dir[4] = {0,-1,0,0};
            services->setVertexShaderConstant("LightDirection", dir, 4);        
 
 
            vector3df p = camera->getAbsolutePosition();
 
        float pos[4] = {p.X,p.Y,p.Z,0};
            services->setVertexShaderConstant("EyePosition", pos, 4);
 
        f32 map = nmap;
            services->setPixelShaderConstant("usenmap", reinterpret_cast<f32*>(&map), 1);   //YOU CAN PRESS N and see the object without a normal mapping
 
    }
};
 

STEP 3: Set up the meshes:

Code: Select all

    IMesh* mesh = smgr->addSphereMesh("sphere",50);
        IMesh* tanmesh = smgr->getMeshManipulator()->createMeshWithTangents(mesh);
            
        ore[counter].node = smgr->addMeshSceneNode(tanmesh);
 
        ore[counter].node->setMaterialType((video::E_MATERIAL_TYPE)shadermat1);
        ore[counter].node->setMaterialTexture(0,driver->getTexture("../media/Brick.jpg")); //YOUR DIFFUSE (COLOR) TEXTURE
        ore[counter].node->setMaterialTexture(1,driver->getTexture("../media/BrickNormal.png")); //YOUR NORMAL MAPPING. (protip) Find a program online that makes normal maps for free.
 

STEP 4: Write a shader.

...j/k I'll show you my normal(bump)map shader:

Code: Select all

float4x4    matWorldViewProj;
float4x4    World;
float4      LightDirection;
float4      EyePosition;
float       usenmap;
 
struct OUT
{ 
        float4 Pos: POSITION; 
        float3 L:    TEXCOORD0; 
        float3 N:    TEXCOORD1; 
        float3 V:    TEXCOORD2;
        float3 T:   TEXCOORD3;
        float3 NORMAL: NORMAL;
        float3x3 WorldToTangentSpace : TEXCOORD4;
 
        float3 Binormal : BINORMAL0;
        float3 Tangent : TANGENT0;
 
 
};
 
texture2D ColorMap;
sampler2D ColorMapSampler
{
    Texture = <ColorMap>;   MinFilter = linear; MagFilter = linear; MipFilter = linear;
};
 
texture2D NormalMap;
sampler2D NormalMapSampler
{
    Texture = <NormalMap>;  MinFilter = linear; MagFilter = linear; MipFilter = linear;
};
 
 
 
 
OUT VertexShaderFunction( OUT I) 
{
    OUT Out = (OUT) 0; 
    Out.Pos = mul(I.Pos, matWorldViewProj); 
    Out.T = I.L;   
    Out.L = normalize(LightDirection); 
    Out.N = normalize(mul(I.NORMAL, World)); 
    
    float4 PosWorld = mul(I.Pos, World);
    Out.V = normalize(EyePosition - PosWorld);
 
 
    Out.WorldToTangentSpace[0] = mul(normalize(I.Tangent), World);
    Out.WorldToTangentSpace[1] = mul(normalize(I.Binormal), World);
    Out.WorldToTangentSpace[2] = mul(normalize(I.NORMAL), World);
 
    return Out; 
}
 
 
float4 PixelShaderFunction(OUT I) : COLOR
{       
    float4 color = tex2D(ColorMapSampler, I.T);    
    float3 normalMap = 2 * tex2D(NormalMapSampler, I.T) - 1;   
    float3 normalMap2 = normalize(mul(normalMap, I.WorldToTangentSpace));
    float4 normal;
    float4 normal2;
    if (usenmap == 1) normal = float4(normalMap2,1.0);
    if (usenmap == 0) normal2 = float4(normalMap,1);
 
    normal = float4(normalMap2,1.0);
    normal2 = float4(normalMap,1);
    
    float Diff = saturate(dot(-I.L, normal));
    float Diff2 = saturate(dot(-I.L, normal2));
 
    float Ai = 0.1f; 
    float4 Ac = float4(.1, .1, .1, .2);
    float Di = 1.5f; 
    float4 Dc = float4(1, 1, 1, .5); 
    float4 Sc = float4(1, 1, 1.0, 1.0);
 
 
        float3 R = saturate(reflect(I.L,normal));    
    float3 R2 = saturate(reflect(I.L,normal2));   
 
    float Specular = pow(dot(R, I.V), 5); // R.V^n
    float Specular2 = pow(dot(R2, I.V), 5); // R.V^n   
    
    float4 color1 = color * Di * Dc * Diff; //+ color * Sc * Specular; IF YOU WANT SPECULAR HIGHLIGHTS (shinyness!!)
    float4 color2 = color * Di * Dc * Diff2; //+ color * Sc * Specular2; IF YOU WANT SPECULAR HIGHLIGHTS (shinyness!!)  
    
        if (usenmap == 1) color2 = (0,0,0,0);
    
    return color1 + color2 ;// + Sc * Specular;
 
 
}

[Abraxas)]

Some improvements to the shader:

float Bumpiness = 5;

float4 color1 = color * Di * Dc * Diff + Sc * Specular;
float4 color2 = color * Di * Dc * Diff2 * Diff * Bumpiness;

Makes it so that unlit sides don't get bumped. Specular looks better without adding color material (IMO it looks more realistic) and a variable to set your "bump-factor".



It is overlit for sure, but it makes my point.

[Abraxas)]

float Bumpiness = 5;

float4 color1 = color * Di * Dc * Diff + Sc * Specular + color * Ai * Ac;
float4 color2 = color * Di * Dc * Diff2 * (Diff + (.01,.01,.01,.01)) * Bumpiness ;

Adding the ambient color in and making sure the unlit sides have a BIT of highlight to them, or else it looks weird.

EDIT:

I think I got it to be the most realistic-looking. I added a constant to the diffuse so that it wouldn't look flat when it was darker. Also, playing around with the colors makes a huge difference in how "real" it looks.

Code: Select all

    float Specular = pow(dot(R, I.V), 22); // R.V^n
    
    float Bumpiness = 2;
    
    float4 color1 = color * Di * Dc * Diff +  Sc * Specular * 0.5 + color * Ai * Ac;
    float4 color2 = color * Di * Dc * Diff2 * (Diff + (.05,.05,.05,.05)) * Bumpiness ;  

[christianclavet]

Hi, Thanks for posting this! I'm not using HLSL but its one of the few tutorial I've seen using Irrlicht used in shader creation. I would really nice if there would be more information about this. A book or on the wiki would be great! (Shaders creation tutorial using Irrlicht)

From what I see from the shader code, it seem that you use a global light (only take the direction of one light), would it be hard to add another light that is at a specific coordinate? (I searched a little on this to do my lighting shader and was not able to get the coordinates correctly, were always rendering the lighting from a object space coordinates)

Is there any place that have tutorial on multipass/draw pass, why it's used and how? Seen some comment about this on the forum (irrlicht draw passes). How is it acheived? I'm still a total noobs on using shaders with Irrlicht. What I would like to write when I understand more to this is to have a shader (or multipass) that render the diffuse map, normal map, specular map (only specular light), and lightmap with more than one light and be able in Irrlicht to set the surface with this shader and activate/deactivate the map I need to use on the model.

[Mel]

Whenever a shader detects a variable has become 0, it starts neurotically tracing back any occurrence where that variable is multiplying others so it can also delete the other variables, this optimizes the code, but for some matters, it also makes the shader execution fail, because it start to miss variables, and causes variable setting errors.

to overcome this drawback, i do a workaround, if i need to disable some part, i multiply that part by 0.00001, it is small enough for it not to be noticed by the output, but it isn't 0, so the shader can't disable those variables

By the way, using the OUT structure won't work in Irrlicht properly because Irr maps the B and T vectors to the TEXCOORD1 and TEXCOORD2, and not to the BINORMAL and TANGENT semantics. In irrlicht that needs to be changed,

[fmx]

^ i didn't notice that about the 0 thing before, thanks for pointing it out

@Christian
have a look at Blindside's XEffects framework
you dont have to use it in your project, but IMO it should help you understand how you can setup multiple render passes in your projects too

[hendu]

What a crappy compiler, if it craps on itself like that.

Come to the dark side, we have cookies

[christianclavet]

Ha? Thanks FMX I'll look into that, I need to learn more about shaders, put them on the side for too long. Thanks.

[mongoose7]

I don't think shaders try to detect 0 values. I don't even think you mean that. But I did find with Nvidia that, if a variable is undefined, everything depending on it will also be undefined.

I think it is a mistake to think that shaders are procedural. I know it looks that way, but that may be just to help the coders. They certainly don't behave as if they are procedural.

[Abraxas)]

mongoose if you try to multiply a variable by 0, it auto erases those variables from your code. Then when you try to run your program you get assaulted by "cannot find variable to set". It's a real pain.

[Granyte]

wow wich gpu and drivers are you guys running cause this clearly never happened to me

[Mel]

Never debugged a shader and needed to neutralize some variable? I don't know GL, but HLSL do reduce that code.

[Mikhail9]

Confirmed for HLSL. And not just things that get set to zero either: If the variables aren't referenced in the shader, they are stripped out entirely and you get the endless "cannot find variable to set" errors.

[Abraxas)]

Yeah, it's my first lifesaver tip, because it drove me absolutely nuts.

I'm thinking of posting more tutorials about doing postprocessing. I'm slowly learning how to do things

[Mel]

On rendermonkey you can debug the shaders better... at least, it isn't constantly reminding you to set the variables... Maybe they could be reported only once?