(HLSL) Cel (Toon) Shading

[nickle]

I went through the EffectHandler, seems the screenResolution is already sent to the shader

Code: Select all

sPP.addShaderDefine("SCREENX", core::stringc(ScreenRTTSize.Width));
	sPP.addShaderDefine("SCREENY", core::stringc(ScreenRTTSize.Height));

My bad...

However, I am wondering how outlines can be extracted in the postprocessing way. I think sobel or other edge detection is not actually outlines of the models. The dot(N,E) < threshold cannot be done in Screen Space since only depth information is provided. Do you think there is a way to achieve Sketchup style outlines of the model using postprocessing?[/code]

[Cloudef]

Can you convert this to GLSL? Would be nice if irrMaple had some 3d cell shaded effects

[BlindSide]

I went through the EffectHandler, seems the screenResolution is already sent to the shader

Code: Select all

sPP.addShaderDefine("SCREENX", core::stringc(ScreenRTTSize.Width));
	sPP.addShaderDefine("SCREENY", core::stringc(ScreenRTTSize.Height));

My bad...

However, I am wondering how outlines can be extracted in the postprocessing way. I think sobel or other edge detection is not actually outlines of the models. The dot(N,E) < threshold cannot be done in Screen Space since only depth information is provided. Do you think there is a way to achieve Sketchup style outlines of the model using postprocessing?[/code]

Did you see this topic?

http://irrlicht.sourceforge.net/phpBB2/ ... hp?t=32740

[nickle]

Oh... I didn't see it. So it is. Thanks

[B@z]

hi
continuing this post

i think i got it.

i did this before:

Code: Select all

		for (u32 i = 0; i < body[part]->getMaterialCount(); i++)
		{
			body[part]->getMaterial(i).setTexture(1, driver->getTexture("data\\effects\\cel.png"));
			body[part]->getMaterial(i).TextureLayer[1].BilinearFilter = false;
			body[part]->setMaterialFlag(EMF_LIGHTING, false);
			if (celMat != -1)
				body[part]->getMaterial(i).MaterialType = (E_MATERIAL_TYPE)celMat;
		}

to make sure that every meshbuffer gets the toon shade. it worked, but not when celMat's materialtype is EMT_TRANSPARENT_ALPHA_CHANNEL
so, thats when i made celMat without transparency, and celMatAlpha with alpha channel transparency

Code: Select all

		for (u32 i = 0; i < body[part]->getMaterialCount(); i++)
		{
			body[part]->getMaterial(i).setTexture(1, driver->getTexture("data\\effects\\cel.png"));
			body[part]->getMaterial(i).TextureLayer[1].BilinearFilter = false;
			body[part]->setMaterialFlag(EMF_LIGHTING, false);
			if (celMat != -1)
				body[part]->getMaterial(i).MaterialType = (E_MATERIAL_TYPE)celMat;
		}
		if (celMatAlpha != -1)
			body[part]->getMaterial(0).MaterialType = (E_MATERIAL_TYPE)celMatAlpha;

only the first meshbuffer gets the alpha channel transparency, and the others are solid.
it works for me, even if the first meshbuffer doesnt have transparent texture, but the others do.
maybe the problem was that all of them had transparency?

[Avidan84]

Hey Irrlicht gurus,
Im not sure how active this thread is but i figured this would be an appropriate place to start, i am running this toon shader and followed the example and when i run it from the IDE it looks absolutely fine, but when i run it from the executable in debug or release i am getting some wierd look artifacts, i was wondering if anyone was able to tell me what it could be, or point me in the right direction.

this is my code for it

Code: Select all

	Castor::IrrlichtSetup *Engine = Castor::IrrlichtSetup::Instance();

	//Load Model
	_mesh = (IAnimatedMesh*) mesh;

	//Load animated scene node
	_meshNode = (IAnimatedMeshSceneNode*) Engine->GetSceneManager()->addAnimatedMeshSceneNode(_mesh);

	//Load First Texture
	_meshNode->getMaterial(0).setTexture(0, Engine->GetVideoDriver()->getTexture(textureFile.c_str()));

	//Load Second Texture
	_meshNode->getMaterial(0).setTexture(1, Engine->GetVideoDriver()->getTexture(textureFileTwo.c_str()));

	_meshNode->getMaterial(0).TextureLayer[1].BilinearFilter = false;
	_meshNode->getMaterial(0).TextureLayer[2].BilinearFilter = false;

	_gpu = Engine->GetVideoDriver()->getGPUProgrammingServices();

	//Load the cel shading file
	s32 cellShade = _gpu->addHighLevelShaderMaterialFromFiles(
	"cel.hlsl", "vertexMain", EVST_VS_2_0,
	"cel.hlsl", "pixelMain", EPST_PS_2_0, this);

	_meshNode->getMaterial(0).MaterialType = (E_MATERIAL_TYPE)cellShade;
	lightPosition = vector3df(1000, 2500, 0);

And the resulting image when running from .exe
The models are exported from 3dsmax 2009 using Panda X exporter to a .X format, then loaded in.

[JuicyLobster]

Odd, getting a celshading.exe has encountered a problem and needs to close type of error at startup. I tried compiling the program myself and fiddling around with a few things, even commenting out the shader parts because my video card is only shader 2.0 (which I assume would be fine though?). Anyone else may know why? I am using irrlicht 1.5. And its not the IDE because im running it from the actual directory.

EDIT: oops, forgot to replace irrlicht.dll with my version, LOL. Anyway, thanks, got the cel shading i needed