(HLSL) Cel (Toon) Shading

B@z · Post by **B@z** » Mon Mar 02, 2009 8:00 pm

well, i made that work in rendermonkey, just some step to get it work in irrlicht, but doesn't look so good with complex objects...

BlindSide · Post by **BlindSide** » Mon Mar 02, 2009 9:01 pm

Because of the recent interest in this, I have decided to improve it:

Shader and Demo can be downloaded from: http://irrlichtirc.g0dsoft.com/BlindSide/celshading.zip

Katsumi · Post by **Katsumi** » Mon Mar 02, 2009 10:11 pm

Impressive! how rotate this donut in exe-file? its not better to rotate the camera around the donut? i can only move the donut.

BlindSide · Post by **BlindSide** » Mon Mar 02, 2009 10:18 pm

I don't understand, what do you mean? Do you have trouble rotating the donut?

If I add this to the demo code:

Code: Select all

ISceneNodeAnimator* anim = smgr->createRotationAnimator(vector3df(0.5f, 0.0f, 0.0f));
torus->addAnimator(anim);
anim->drop();

The donut rotates fine.

sudi · Post by **sudi** » Tue Mar 03, 2009 10:05 am

wow impressive effect...of course i directly tried it and canged the mesh. then i wondered why it looks wrong but after checking the shadersource i got it...it only works with deteiled meshes or at least with detailed normals.
Bc when using a normal cube u get the following.

BlindSide · Post by **BlindSide** » Tue Mar 03, 2009 7:44 pm

Haha I only tried on the donut.

I did a quick touch up on it and added outlining and specular directly into the shader, if I was going the full route I would have made the outlining a post processing effect based on edge detection (Although this way may be more preferable to some as the performance is better and in general you get thicker lines, unless the object is far away).

You can disable the outline effect from the shader and do it the other way (With front face culling and wireframe) if you want to use it on a cube for example.

B@z · Post by **B@z** » Wed Mar 04, 2009 11:37 am

yeah its good but i have the same problem

it looks cool in her arms, but not at his clothes and so on ><
outline should be everywhere the same width..
should i try that front face culling and wireframe thingy?

BlindSide · Post by **BlindSide** » Wed Mar 04, 2009 8:44 pm

Yeah, outline should be easy to disable in the shader, just comment out all the lines with "fresnal" in 'em. You can do the outline as a post processing effect if you use XEffects and render to depth map then add a post processing effect that compares the depth of different pixels and makes them dark if theres a reasonable difference, quite similar to simple ssao shader, check out jinquan's thread for an example of this type of shader.

bicunisa · Post by **bicunisa** » Wed Mar 04, 2009 9:04 pm

Hi BlindSide,

Is it Direct3d only? I tried running it on Linux with OpenGL and the program just shows a black torus. I wonder what am I missing...

I get these messages in the console:

Code: Select all

Irrlicht Engine version 1.5
Linux 2.6.27-12-generic #1 SMP Thu Feb 5 09:26:35 UTC 2009 i686
Creating X window...
Visual chosen: : 39
Using renderer: OpenGL 2.1.2
GeForce Go 7600/PCI/SSE2: NVIDIA Corporation
OpenGL driver version is 1.2 or better.
GLSL version: 1.3
read ChunkNODE: Torus
read ChunkMESH
read ChunkVRTS
read ChunkTRIS
Loaded mesh: torus.b3d
Loaded texture: /home/lobot/development/NetBeansProjects/cellshad/lialique.bmp
Loaded texture: /home/lobot/development/NetBeansProjects/cellshad/cel.png
GLSL shader program failed to link
Vertex info
-----------
0(1) : warning C7506: OpenGL does not define the global type float4x4
0(2) : warning C7506: OpenGL does not define the global type float3
0(7) : warning C7506: OpenGL does not define the global type float4
0(8) : warning C7506: OpenGL does not define the global type float2
0(32) : warning C7554: OpenGL requires samplers to be explicitly declared as uniform
0(33) : warning C7554: OpenGL requires samplers to be explicitly declared as uniform
(0) : error C3001: no program defined

Fragment info
-------------
0(1) : warning C7506: OpenGL does not define the global type float4x4
0(2) : warning C7506: OpenGL does not define the global type float3
0(7) : warning C7506: OpenGL does not define the global type float4
0(8) : warning C7506: OpenGL does not define the global type float2
0(32) : warning C7554: OpenGL requires samplers to be explicitly declared as uniform
0(33) : warning C7554: OpenGL requires samplers to be explicitly declared as uniform
(0) : error C3001: no program defined

BlindSide · Post by **BlindSide** » Thu Mar 05, 2009 3:42 am

Yes this is HLSL shader for Direct3D9, I can translate into OpenGL for you if you really need it, I think there is another cel shader by sio2 that is glsl if you want that.

B@z · Post by **B@z** » Thu Mar 05, 2009 3:56 am

yeah this is a hlsl file, so its dx only..
opengl goes with glsl

Blindside: nah i dun understand a thing -.-
downloaded something about xeffects and ssao, but dunno what to do.. xD

Nadro · Post by **Nadro** » Thu Mar 05, 2009 2:39 pm

You can use it HLSL (Cg) shader via IrrCg if You need it for Direct3D9 and OpenGL without translating HLSL code to GLSL

BlindSide · Post by **BlindSide** » Thu Mar 05, 2009 11:58 pm

Ah yes good point, CG == HLSL in most cases.

oldskoolPunk · Post by **oldskoolPunk** » Mon Mar 09, 2009 6:53 pm

This is weird. Why cant you just get the light position in the shader? Also, why do you keep passing all these different matrices from Irrlicht ? Are they not available to you in HLSL as they are in GLSL ?

sudi · Post by **sudi** » Mon Mar 09, 2009 9:07 pm

they are avaible but noone uses them....