Making a "cartoon" shader work?(ASM ps.1.1)

[eye776]

Hi. I was experimenting with various (really old) ps1.1 ASM shaders I found on this website http://www.talula.demon.co.uk/postproce ... ssing.html.

This is not a standard "toon shader", the effect you obtain is similar to GIMP/Pthotoshop's "Cartoon" filter (it is applied on a rendertarget, or a texture).

While I got a few to work, I can't seem to make heads or tails of this one:

Code: Select all

ps.1.1

def c0, 0.3, 0.59, 0.11, 0

tex t0                     // sample the source image
tex t1                     // sample offset by (-1, -1)
tex t2                     // sample offset by (1, 1)

dp3 r0.rgba, t1, c0        // greyscale sample #1 in r0.a
dp3 r1.rgba, t2, c0        // greyscale sample #2 in r1.a

sub_x4 r0.a, r0, r1        // diagonal edge detect difference
mul_x4_sat r0.a, r0, r0    // square edge difference to get absolute value

mul r0.rgb, r0, 1-r0.a     // output color * edge detect
+ mov r0.a, t0.a           // output alpha

Any pointers or help to get this working?

I'm trying to use it for a preprocessing class (process textures when loading with various effects).

PS: I get a white output, but no errors regarding the shader itself.

[slavik262]

Is there a reason you're using ASM instead of a higher level shader language like HLSL, GLSL, or Cg? They make debugging a lot simpler. Are you just targeting old platforms? You needn't worry about performance - the higher level shader compilers do a really good job of compiling the shaders into efficient code. Plus, they can be aware of the graphics card hardware they're compiling for, which can help them optimize it.

[Bate]

Also, you could develop it in a high level language and compile it once when you're done.

[eye776]

I just realized that "t1" and "t2" were not set in the material.
Also, does irrlicht even support 3 textures on a material?

@slavik262:
It's not that I'm really "targeting" anything right now... I just don't want another aborted project on my HDD.
The shader was found on an old website, it appeared to do what I wanted it to without costing an arm and a leg in terms of processing power.

Anyway, will try setting the textures properly and seeing what happens.

[slavik262]

Irrlicht supports up to 4 textures by default and up to 8 if you recompile it. Calls to SetMaterial automatically place the textures in the texture registers S0 through S7.