Shaders multi-pass, techniques support

[guruhenry]

Hello,

I'm trying to understand how to use shaders multi pass, techniques etc.. in Irrlicht.

It seems to be possible to define a material only through the predefined constants (i.e. EMT_SOLID.. ) and to load a pixel/vertex shaders without techniques.

Is it possible to define a material specifying a technique (with more than one pass... ), as I could do with Ogre?


Thanks in advance.

[slavik262]

"multi-pass" and "techniques" are just terms. In a single pass, you can only ever render a object (in any engine) with a single shader and material. To add additional passes into a technique, you can save that initial pass to a render target and re-render with a different material and shader, then combine the two in post-processing.

I haven't worked with OGRE, but I'm assuming it just does it for you. Irrlicht doesn't, but it's trivial to implement yourself.

[guruhenry]

Hi,

Ok, I'm new about programming shaders, but if I have understood correctly, starting from a .fx file:

- I can import the hlsl vertex and pixel shaders, discarding the techniques parts.
- I have to manually implement the techniques performing each passes setting the state values and drawing the target object and mixing the results with post-processing.

Is it correct?


Thanks.

[slavik262]

Even in .fx files with passes and techniques, DirectX doesn't implement these things automatically. Each pass is just a name for the shader and instructions about which Shader Model to compile it with, and techniques are just ways of containing multiple shaders in the same file and being able to fall back on, say, a simpler one if the standard one is too complex for the hardware its running on. They don't automatically do multiple passes; it's just a way to have different shading options.

Take a look at the shader example. It does a good job of showing you how to interface with shaders in your Irrlicht code.

And looking at the OGRE documentation, I don't think that it automatically blends multiple passes together for you either. I think (if I'm reading this right; correct me if I'm wrong) that the techniques and passes are what I described above - just different options for what shader to use.

Rendering can be repeated with many passes for more complex effects. Each pass is either a fixed-function pass (meaning it does not use a vertex or fragment program) or a programmable pass (meaning it does use either a vertex and fragment program, or both).
Programmable passes are complex to define, because they require custom programs and you have to set all constant inputs to the programs (like the position of lights, any base material colours you wish to use etc), but they do give you much total flexibility over the algorithms used to render your pass, and you can create some effects which are impossible with a fixed-function pass. On the other hand, you can define a fixed-function pass in very little time, and you can use a range of fixed-function effects like environment mapping very easily, plus your pass will be more likely to be compatible with older hardware. There are pros and cons to both, just remember that if you use a programmable pass to create some great effects, allow more time for definition and testing.

Now you can have one shader that renders multiple effects at once - you can use multiple render targets and draw to all of them at the same time (which is much more efficient than re-drawing over and over).

[3DModelerMan]

Check out IrrCG. It does CGFX files and multipass.

[slavik262]

Just keep in mind that rendering geometry over again will slow down your program.

[guruhenry]

Hi Slavik,

I'm still evaluating both Irrlicht and Ogre libraries.

What I can see is that the Ogre material definition is similar to .fx files, while in Irrlicht seems to not have the possibility to read the passes informations from a standardized file.

Using DirectX I read that I can load a .fx file and simply execute a for like this:

...
for (UINT i=0;i<numPasses;i++)
{
anEffect->BeginPass(i); // Set the pass
// render geometry e.g. DrawIndexedPrimitive
anEffect->EndPass();
}
...

Now, if I want to create an effect library, not embedded in my application source code, I'm thinking how could fully describe each effect, multi-pass included.

IrrCG could be an option, but I would like to use the main Irrlicht library without adding (..not so stable?..) extensions.

[serengeor]

I'm not Shader expert but i think that you could look at the irrCG code and than based on what you need write your own ('so much more stable?') effect library.

[slavik262]

You could do this extremely easily in Irrlicht (without Cg) with a custom scene node. Since Irrlicht uses scene nodes to represent game entities, you could create a scene node that has multiple materials. Take a look at tutorial 3. Notice the code inside the render function of the scene node. This is what actually draws what is contained in the scene node.

Code: Select all

driver->setMaterial(Material);
driver->setTransform(video::ETS_WORLD, AbsoluteTransformation);
driver->drawVertexPrimitiveList(&Vertices[0], 4, &indices[0], 4, video::EVT_STANDARD, scene::EPT_TRIANGLES, video::EIT_16BIT);

You could create a scene node that stores multiple materials. Each SMaterial can contain a shader as well as many parameters for that shader. You could combine these into a "technique," and each material would be a pass.

Code: Select all

driver->setTransform(video::ETS_WORLD, AbsoluteTransformation);
for(u32 c = 0; c < passes; ++c)
{
     driver->setMaterial(Materials[c]);
     driver->drawVertexPrimitiveList(/*args*/);
}

This would work identically to the DirectX code you posted,

Code: Select all

for (UINT i=0;i<numPasses;i++) 
{ 
anEffect->BeginPass(i); // Set the pass 
// render geometry e.g. DrawIndexedPrimitive 
anEffect->EndPass(); 
} 

The only difference is that you'd need to load in the materials yourself.


Feel free to ask any follow-up questions if I'm not making sense.

On a note about the comparison of OGRE and Irrlicht:

Don't let anybody tell you that one engine is better than the other. It's a preference. Both are capable of doing the exact same things (with the exception that Irrlicht doesn't have DirectX 10 supprt yet, but that's not an issue unless you're planning on using geometry shaders and targeting very high-end computers). The quality of your game depends much, much more on the quality of your code and art assets (models, textrues, etc.) than on whether you use Irrlicht or OGRE.

The tradeoff is that while OGRE has more features "built-in," this also adds a lot to the complexity of the engine, and in my opinion makes it difficult to use. I've considered switching to OGRE multiple times, but every time I do a single look at the OGRE documentation scares me back to Irrlicht. Irrlicht offers a simple and well-laid out code base that OGRE just can't match for me.

P.S. I apologize for what I said earlier about using post-processing - it's an overcomplicated solution that I thought of while I was still waking up.

[guruhenry]

First of all, I apologize for the sentence "not so stable?" related to IrrCG extension, or other extensions...

About defining a custom scene node, It seems a good way. I'll try it as soon as possible.


Thanks for your informations.

[Bate]

I've considered switching to OGRE multiple times, but every time I do a single look at the OGRE documentation scares me back to Irrlicht.

So damn true

[haffax]

And looking at the OGRE documentation, I don't think that it automatically blends multiple passes together for you either. I think (if I'm reading this right; correct me if I'm wrong) that the techniques and passes are what I described above - just different options for what shader to use.

Actually Ogre blends passes for you, according to the options you set for the pass attributes scene_blend/scene_blend_op.
Techniques are different options for how to render a material. A technique can contain multiple passes, all passes of the chosen technique are blended.

Code: Select all

for (UINT i=0;i<numPasses;i++) 
{ 
anEffect->BeginPass(i); // Set the pass 
// render geometry e.g. DrawIndexedPrimitive 
anEffect->EndPass(); 
} 

This is not how Ogre does it, and actually way too expensive. Due to the many state changes and due to not being able to early z-out the fragments.

Ogre first renders all first passes of all the meshes, then all second passes and so on. The advantage is that state changes can be minimized. Imagine you have 20 multipass objects all with the same two passes. With your method you'd set pass1 then render mesh1 with it, then change to pass2 render mesh1, change to pass1 render mesh2, change to pass2 render mesh2 and so on. Changing render states is expensive, especially when shaders and textures are involved. Better to render all first passes, then all second passes.

Also imagine you have a very complex shader, that is expensive to execute. When you have a cheap first pass (e.g just the ambient) then the second pass is omitted for some of the meshes, because a wall is between the camera and your mesh. The depth check, which all modern GPUs can do before rendering, may render rendering the second pass unneeded.

[slavik262]

I realize the cost of state changes and the importance of batching, but I don't see how your implied method of rendering all the first passes, then the second passes, etc. uses any less state changes. You still need to set the material once per mesh, per pass, unless you optimize it so that setMaterial isn't called if two nodes share the same material (which Irrlicht doesn't do in the first place).

Also, if something fails a z-test, why would you render any passes? Z-tests can also get quite tricky with transparent materials.

[haffax]

I realize the cost of state changes and the importance of batching, but I don't see how your implied method of rendering all the first passes, then the second passes, etc. uses any less state changes. You still need to set the material once per mesh, per pass, unless you optimize it so that setMaterial isn't called if two nodes share the same material (which Irrlicht doesn't do in the first place).

I don't know how Irrlicht handles this, but Ogre orders all rendering operations according to the pass (as in material pass) used, so the hypothetical 20 meshes are rendered using the same material (but of course shader parameters declared as auto parameters like world transforms etc are reset for each render operation) A simple and computational inexpensive heuristic is used to have similiar passes next to each so that render state change from one pass to the next is reduced.

Also, if something fails a z-test, why would you render any passes? Z-tests can also get quite tricky with transparent materials.

Just like in Irrlicht, Ogre also handles transparent passes differently. But in a typical scene these are few.
Ogre itself never does a z-test here, but the GPU does. The second pass is still prepared and the geometry drawn, but the GPU rejects the pixel before the shader sees them. And todays GPUs are very good at doing this. Because the potential performance gain is huge, GPU vendors developed clever techniques for this. The last actual number I have on this is that a Radeon 9800Pro is able to reject 160 pixels per clock cycle. Today's cards are probably even better. 160 pixels that otherwise had held several shader units busy for many clock cycles.

[slavik262]

I don't know how Irrlicht handles this, but Ogre orders all rendering operations according to the pass (as in material pass) used, so the hypothetical 20 meshes are rendered using the same material (but of course shader parameters declared as auto parameters like world transforms etc are reset for each render operation) A simple and computational inexpensive heuristic is used to have similiar passes next to each so that render state change from one pass to the next is reduced.

Unfortunately, Irrlicht doesn't do this. Unless you implement it differently yourself, Irrlicht calls setMaterial for every scene node.

EDIT: After a brief swim in the source code, however, it seems that Irrlicht does some batching automatically. Despite making calls to setMaterial for every node, the driver implementation of setMaterial is smart enough to not make SetTexture calls if the tuxtures haven't changed.