(HLSL) Cel (Toon) Shading

BlindSide · Post by **BlindSide** » Tue Mar 10, 2009 1:53 am

I'm pretty sure they are only available in GLSL...

oldskoolPunk · Post by **oldskoolPunk** » Tue Mar 10, 2009 3:39 pm

ahh! Now I understand. Good work by the way.

BlindSide · Post by **BlindSide** » Wed Mar 11, 2009 12:21 am

Yes but I am using the same outlining algo that you tried and threw away

That's why I suggested using a post process for the outlines instead. Did you end up using something similar to that yourself?

oldskoolPunk · Post by **oldskoolPunk** » Wed Mar 11, 2009 7:30 am

Still reading.

Learning convolution is convoluted !

Lol not really. It seems image processing is the easiest form of shader programming so far. Dont know why I have put it off for so long. One shader can do all kinds of filters by just changing the matrix.

B@z · Post by **B@z** » Fri Mar 13, 2009 5:03 pm

Hi
I have a little problem...
I want to use transparent textures in my model with this shader, and it works, but just partially.

As you can see, it doesn't just make the model at that point transparent, but the other poligons behind it too.
in my model, there is the skin group, and in top of that there is the hotpants group.
probably its because it calculates the texture of that place (the toon), and gives the alpha value (if i set always the alpha to 255, it isn't transparent, but i can see the full model, so probably its because this), so that place become transparent..

can you help me?

BlindSide · Post by **BlindSide** » Sat Mar 14, 2009 5:14 am

in my model, there is the skin group, and in top of that there is the hotpants group.

Ok first make sure you are using EMT_TRANSPARENT_ALPHA_CHANNEL (Without the _REF) as your base material.

Next you will have to change the last line of the shader to

Code: Select all

return(float4(saturate(lightColor * texCol + specular4).xyz, texCol.a));

and it should work correctly.

About the things behind it being invisible, etc, you may have to try enabling/disabling ZWriteEnable on the mesh. If it is too much trouble in the end I recommend just drawing the hotpants as a seperate object.

B@z · Post by **B@z** » Sat Mar 14, 2009 10:33 am

thanks for the reply.
i changed that line, and nothing happened. xD
same us before... what does it do? xD

enabling zwrite did nothing (probably it was enabled before), disabling it made my model quite ugly.
so no good

BlindSide · Post by **BlindSide** » Sat Mar 14, 2009 11:23 am

Ok it seems you will have to draw the hotpants separately, just load them as a separate scene node.

B@z · Post by **B@z** » Sat Mar 14, 2009 11:31 am

thats what i can't do.
it will be an rpg game, so all of the clothes has to be the same format.

BlindSide · Post by **BlindSide** » Sun Mar 15, 2009 2:21 am

It's a transparency problem B@z. You usually need to draw as seperate objects to get around this.

If you still insist. Try using EMT_TRANSPARENT_ALPHA_CHANNEL_REF as a last shot. Also maybe playing around with the 1 Texture Blend material set as the base will get you something.

Cheers

shadowking97 · Post by **shadowking97** » Sun Mar 15, 2009 4:12 am

Just split the pants into a different mesh. Have it copy the position, rotation, and scale of the girl, play the same animation, and if you have everything right, The vertexes of both meshes will line up so it looks like one. You can do this with all your clothes, and after youve loaded all the clothes files, you can change the settings on the ones you need to.

B@z · Post by **B@z** » Sun Mar 15, 2009 3:37 pm

i know it, but it wont work.
coz there will be clothes what puts texture in the body model (for example tatoos and so on), so i HAVE to put it into the same model..
ohh and no, i dont want to save the clothes, and the clothes body in different models...
with this, i have around 60 model, and this is the begining.. if i put it seperated, it will be 120 xD no thanks

EMT_TRANSPARENT_ALPHA_CHANNEL_REF didnt do anything to me... maybe im using it wrong? xD

grumpymonkey · Post by **grumpymonkey** » Mon Mar 23, 2009 9:10 pm

I used to do this a long time ago when i used gamemaker, but not using this method.
what i did was this:

first, i got my regular model, and then loaded it with the texture, then I loaded another model with all the faces backwards, and i drew it ontop of the original one. I turned on back face culling(so that only the front of the faces are visible), and set the second models texture to just be completely black.

this method works, but i dont recomend it for complicated meshes as there will be double the amount of polygons as a regular model.

here is a crappy diagram of what i mean:
GREEN = VISIBLE
RED = INVISIBLE
O = CAMERA

O--> || ||

the player can only see the parts of the second mesh that are behind the oribinal mesh, so regardless of the angle it will always be outlined correctly

now, about the actual SHADING, i have no clue how to do that since gamemaker's 3D had really crappy shadows, and im still new to irrlicht(started a week ago)

B@z · Post by **B@z** » Mon Mar 23, 2009 10:21 pm

yep its for outline, but as you said it doesnt work with complex models...
so wont work with players xD
and it wont also work with concave models. so out of question xD

nickle · Post by **nickle** » Wed Apr 15, 2009 6:40 am

XEffects and render to depth map then add a post processing effect that compares the depth of different pixels and makes them dark if theres a reasonable difference, quite similar to simple ssao shader

by different pixels, do you mean the neighbouring 8 pixels depth? If so, I also checked out the screenQuad shader, I think the postprocessing shader for edge detection should have a input of the current screenRes in order to get the neighbouring 8 pixels texcoords. If not, could you show us how this could be done with more details? An edge detection postprocessing in a general purpose (not only cel shading) is also very useful. I am not so sure how to get this done without modifying the current screenQuad shader. [/quote]

Irrlicht Engine

(HLSL) Cel (Toon) Shading

outline extraction, edge detection