Stencilshadows&&smoothing

[TheGameMaker]

Hey, I´m currently messing around with shaders... so I came over a question:
The Stencil shadows are handeld as scencenodes, rigth?? So.. There is a Render call... (a assume, this rendercall is called after everything else, right??) so.. wouldn´t it be possible to render the shadows into one rendertarget, the other one into a secon one, and combine them via Pixelsahder?? this would give us the possibility of smooth stencil shadows!
correct me if I´m wrong, but i cannot see any thing wrong about this...
greets TGM
(sry, if i have forgotten something, or just didn´t read the manula good enough..)

[omaremad]

I did this but you have to edit the dll to capture the stencil shaddows at the right stage. ill pm you the code if you want.

[TheGameMaker]

yeah.. it was pretty obvious that you have to change irrlicht a little bit.
But it works^^ yeah.. damned i thought it must work.. but i wasn´t realy shure.
The code would be great... is it opensource, or don´t you want it to be released. I think there are a lot of people that are very interested in smoot shadows...

*edit* I thought you would change the onPreRenderMethod of the shadowNode, and call the render function manualy... but how have you done it??

[omaremad]

Its really ugly it was just a quick prototype with no decent interface.

Any shadows had to be black while anything recieving shadows had to be white (you have to create copies of the nodes) (you can use grey's to give partial shadows). You give shadow casting objects and shadoow recieving objects both an id of 999999 and the a statement in the scenemanager finds them, makes the visible renders them to RTT with shadows and then makes the invisible.

a better method is to use lists in the scene mangaer insatead of the 999999 id hack.

First we create in the public section of the scenemanager class a texture and set it up as an RTT

then we modify the drawall call like so

Code: Select all

 

void CSceneManager::drawAll(bool shaddow) 
        { 
            if (!Driver) 
                return; 

            // calculate camera pos. 
            camTransPos.set(0,0,0); 
            if (ActiveCamera) 
                camTransPos = ActiveCamera->getAbsolutePosition(); 

            // let all nodes register themselfes 
            OnPreRender(); 

            //render lights and cameras 


            u32 i; // new ISO for scoping problem in some compilers 
            if(shaddow) 
            { 
                Driver->setRenderTarget(shaddowtex, true,true, video::SColor(0,0,0,0)); 



                // render default objects 

                CurrentRendertime = ESNRP_SOLID; 

                SolidNodeList.sort(); // sort by textures 

                for (i=0; i<SolidNodeList.size(); ++i) 
                { 
                    if(SolidNodeList[i].node->getID ()==9999) 
                    { 

                        SolidNodeList[i].node->render(); 

                    } 

                } 


                CurrentRendertime = ESNRP_SHADOW; 

                for ( i=0; i<ShadowNodeList.size(); ++i) 
                    ShadowNodeList[i]->render(); 

                if (!ShadowNodeList.empty()) 
                    Driver->drawStencilShadow(true,ShadowColor, ShadowColor, 
                                              ShadowColor, ShadowColor); 


                ShadowNodeList.clear(); 
                Driver->setRenderTarget(0); 
                // render transparent objects. 

            } 
            if(!shaddow) 
            { 
                CurrentRendertime = ESNRP_LIGHT_AND_CAMERA; 

                Driver->deleteAllDynamicLights(); 



                for (i=0; i<LightAndCameraList.size(); ++i) 
                    LightAndCameraList[i]->render(); 

                LightAndCameraList.clear(); 

                // render skyboxes 

                CurrentRendertime = ESNRP_SKY_BOX; 

                for (i=0; i<SkyBoxList.size(); ++i) 
                    SkyBoxList[i]->render(); 

                SkyBoxList.clear(); 


                // render default objects 

                CurrentRendertime = ESNRP_SOLID; 

                SolidNodeList.sort(); // sort by textures 

                for (i=0; i<SolidNodeList.size(); ++i) 
                { 
                    if(SolidNodeList[i].node->getID ()!=9999) 
                    { 
                        SolidNodeList[i].node->render(); 
                    } 
                } 
                SolidNodeList.clear(); 



                CurrentRendertime = ESNRP_TRANSPARENT; 

                TransparentNodeList.sort(); // sort by distance from camera 

                for (i=0; i<TransparentNodeList.size(); ++i) 
                    TransparentNodeList[i].node->render(); 

                TransparentNodeList.clear(); 
            } 
            SolidNodeList.clear(); 
            // do animations and other stuff. 
            OnPostRender(os::Timer::getTime()); 

            clearDeletionList(); 

            CurrentRendertime = ESNRP_COUNT; 

        } 

you can then grab the pointer for the sahdow texture and apply it to your favourite postprocessor.

Its really bad code as you can see .

[TheGameMaker]

hmm wouldn´t it be much faster if you just use a rendertarget with white background, and the just render the shadows into it???

or even just like:

Code: Select all

void CSceneManager::drawAll()
{
	if (!Driver)
		return;

	// calculate camera pos.
	camTransPos.set(0,0,0);
	if (ActiveCamera)
		camTransPos = ActiveCamera->getAbsolutePosition();

	// let all nodes register themselfes
	OnPreRender();

	//render lights and cameras

	CurrentRendertime = ESNRP_LIGHT_AND_CAMERA;

	Driver->deleteAllDynamicLights();
	
	u32 i; // new ISO for scoping problem in some compilers

	for (i=0; i<LightAndCameraList.size(); ++i)
		LightAndCameraList[i]->render();

	LightAndCameraList.clear();

	// render skyboxes

	CurrentRendertime = ESNRP_SKY_BOX;

	for (i=0; i<SkyBoxList.size(); ++i)
		SkyBoxList[i]->render();

	SkyBoxList.clear();


	// render default objects

	CurrentRendertime = ESNRP_SOLID;

	SolidNodeList.sort(); // sort by textures

	for (i=0; i<SolidNodeList.size(); ++i)
		SolidNodeList[i].node->render();

	SolidNodeList.clear();

	// render shadows
[b]
	CurrentRendertime = ESNRP_SHADOW;
        Driver->setRenderTarget(shaddowtex, true,true, video::SColor(255,255,255,255));
	for (i=0; i<ShadowNodeList.size(); ++i)
		ShadowNodeList[i]->render();

	if (!ShadowNodeList.empty())
		Driver->drawStencilShadow(true,ShadowColor, ShadowColor,
			ShadowColor, ShadowColor);

	ShadowNodeList.clear();
        Driver->setRenderTarget(0); [/b]
     
	// render transparent objects.

	CurrentRendertime = ESNRP_TRANSPARENT;

	TransparentNodeList.sort(); // sort by distance from camera

	for (i=0; i<TransparentNodeList.size(); ++i)
		TransparentNodeList[i].node->render();

	TransparentNodeList.clear();

	// do animations and other stuff.
	OnPostRender(os::Timer::getTime());

	clearDeletionList();

	CurrentRendertime = ESNRP_COUNT;
}

(I dunno if this work, but i think it would..)
then just blur the rt, and multiply it with the rendered orginal...
but this code will also hav problems with transparent objekts, since they won´t hide the shadow.. but shadows will be always in front of them. So maybe you even would have to render the transparent objekts into a rt... +depthbuffer, and then mixing it.. but this won´t even work if there are more transparent objekts overlapping each other.. [/b]

[omaremad]

hmm wouldn´t it be much faster if you just use a rendertarget with white background, and the just render the shadows into it???

you have to have the scenes polygons so you can show the contents of the stencil buffer and hence use the stencil in the stencil buffer to paint the shadow over the scene.

If you want other nodes to block the shadow just add them as white in the shadow pass.For transparent objects just render as white with transparent areas where the hole are.

[TheGameMaker]

yeah... so i have to call the drawAll() twice,
like this:

Code: Select all

allNodes->setMaterialToWhite();//pseudocode^^
smgr->drawAll(true);
allNodes->setMaterialToStandart();
smgr->drawAll();
PostProccesFilter->Render();

to hell thats slow..
Won´t there be any way to just render the scene, and save the stencilbuffer seperat?? (sry... i just want to know...)
tgm

[omaremad]

Most real games dont do soft shadows(projected shadow maps) on the actual level, most just project their textures on a simplified mesh ussually also used for collsion detection.

Using the same leveldata is a huge waste since that level already needs textures of its own and already has a complex shader.

In this case a white simplified level would require a very low transform cost, since the level is untextured its fill rate cost will be next to nothing.

[TheGameMaker]

ok.. that sounds logical... hmm... well... but it means that i have to have two meshes in ram.. not that good in big levels...
ok.. thx... maybe i´ll gonna test this some time(currently working on some mor simple things^^ like bloom, fake refraction etc...)

[omaremad]

ok.. that sounds logical... hmm... well... but it means that i have to have two meshes in ram.. not that good in big levels... Laughing Laughing

yeah its still faster than running collsion detection on a complex mesh, the extra boundries can be used for other stuff like invisible walls to prevent players falling off the level, and when you use a physics engine (not sure about irrlicht) you are making a copy of the complex mesh anyway... A list of a few thousand vertices is nothing compared to the slow down of complex meshes.

[TheGameMaker]

yeah its still faster than running collsion detection on a complex mesh, the extra boundries can be used for other stuff like invisible walls to prevent players falling off the level, and when you use a physics engine (not sure about irrlicht) you are making a copy of the complex mesh anyway... A list of a few thousand vertices is nothing compared to the slow down of complex meshes.

yeah.. totaly right.. makes the mesh being safed 3 times^^ yeah.. the levelmesh won´t be that a problem.. a singel 1024x1024 Tex should take more ram than a normal level without textures..