Mesh-based Toon Shader

[Lonesome Ducky]

I was originally planning on putting this in code snippets, but I plan on it getting quite a lot bigger, so I decided to put it here.

This code is used to create a cartoon outline effect for any unanimated mesh (animated mesh support coming up soon). It works by moving the vertices out by their normals, and flipping the faces.



Depending on the mesh, it may take a while to combine duplicate vertices (I'm not terribly good at optimization). Without it though, it ends up looking like this:

Animated mesh support coming up as soon as I get the hang of the ISkinnedMesh interface. Anyway, here's the code:

Code: Select all

struct indexRedirect {
	int oldIndex;
	int newIndex;
};
struct vertCombine {
	irr::core::vector3df totalPos;
	irr::core::vector3df totalNormal;
	int count;
};

// Adds a toon outline to a mesh. Works by copying the mesh, and expanding by the normals. It then flips the faces.
irr::scene::IMesh* createToonOutlineMesh(irr::scene::IMesh* mesh, float size, irr::video::SColor color) {

	irr::scene::SMesh* toonMesh = new irr::scene::SMesh();

	// Copy the mesh buffers
	for (int i = 0; i < mesh->getMeshBufferCount(); i++) {
		irr::scene::SMeshBuffer* sourceBuffer = (irr::scene::SMeshBuffer*)mesh->getMeshBuffer(i);
		irr::scene::SMeshBuffer* destBuffer = new irr::scene::SMeshBuffer();

		irr::video::S3DVertex* copyVertices = new irr::video::S3DVertex[sourceBuffer->getVertexCount()];
		for (int j = 0; j < sourceBuffer->getVertexCount(); j++) {
			copyVertices[j] = sourceBuffer->Vertices[j];
		}
		irr::u16* copyIndices = new irr::u16[sourceBuffer->getIndexCount()];
		for (int j = 0; j < sourceBuffer->getIndexCount(); j++) {
			copyIndices[j] = sourceBuffer->Indices[j];
		}

		// Find duplicate vertices, then add an indexRedirect so indices can be updated and the vertices can later be collapsed
		irr::video::S3DVertex* sourceVertices = (irr::video::S3DVertex*)sourceBuffer->getVertices();
		vertCombine* combiners = new vertCombine[sourceBuffer->getVertexCount()];
		irr::core::array<indexRedirect> redirects;
		bool *check = new bool[sourceBuffer->getVertexCount()];
		for (int j = 0; j < sourceBuffer->getVertexCount(); j++) {
			check[j] = true;
			combiners[j].totalPos = sourceVertices[j].Pos;
			combiners[j].totalNormal = sourceVertices[j].Normal;
			combiners[j].count = 1;
		}
		for (int j = 0; j < sourceBuffer->getVertexCount(); j++) {
			if (check[j]) {
				// Check for vertices that are very close
				for (int k = j; k < sourceBuffer->getVertexCount(); k++) {
					if (check[k]) {
						if (sourceVertices[j].Pos.getDistanceFromSQ(sourceVertices[k].Pos) <= irr::core::ROUNDING_ERROR_f32*irr::core::ROUNDING_ERROR_f32) {
							check[k] = false;
							indexRedirect redir;
							redir.newIndex = j;
							redir.oldIndex = k;
							redirects.push_back(redir);
						}
					}
				}
			}
		}
		delete [] check;

		// Combine vertices
		for (int j = 0; j < redirects.size(); j++) {
			combiners[redirects[j].newIndex].totalPos += sourceVertices[redirects[j].oldIndex].Pos;
			combiners[redirects[j].newIndex].totalNormal += sourceVertices[redirects[j].oldIndex].Normal;
			combiners[redirects[j].newIndex].count++;
		}
		for (int j = 0; j < redirects.size(); j++) {
			copyVertices[redirects[j].newIndex].Pos = combiners[redirects[j].newIndex].totalPos / combiners[redirects[j].newIndex].count;
			copyVertices[redirects[j].newIndex].Normal = combiners[redirects[j].newIndex].totalNormal / combiners[redirects[j].newIndex].count;
			copyVertices[redirects[j].newIndex].Normal.normalize();
		}
		// After combining the vertices, fix indices so they no longer reference dead vertices
		for (int j = 0; j < redirects.size(); j++) {
			for (int k = 0; k < sourceBuffer->getIndexCount(); k++) {
				if (copyIndices[k] == redirects[j].oldIndex) {
					copyIndices[k] = redirects[j].newIndex;
				}
			}
		}
		delete [] combiners;

		// Now expand the mesh, moving the vertices along their normals
		for (int j = 0; j < sourceBuffer->getVertexCount(); j++) {
			copyVertices[j].Color = color;
			copyVertices[j].Pos += copyVertices[j].Normal*size;
		}

		destBuffer->append(copyVertices,sourceBuffer->getVertexCount(),copyIndices,sourceBuffer->getIndexCount());
		destBuffer->recalculateBoundingBox();

		// Now set the material
		destBuffer->Material = destBuffer->Material;
		destBuffer->Material.Lighting = false;
		destBuffer->Material.BackfaceCulling = false;
		destBuffer->Material.FrontfaceCulling = true;
		destBuffer->Material.DiffuseColor = color;
		destBuffer->Material.ColorMaterial = irr::video::ECM_DIFFUSE_AND_AMBIENT;
		destBuffer->Material.TextureLayer[0].Texture = 0;
		toonMesh->addMeshBuffer(destBuffer);

	}

	toonMesh->recalculateBoundingBox();
	return (irr::scene::IMesh*)toonMesh;
}

[polylux]

That looks quite promising!

Didn't go through your code, but there's also this quite classy approach to blow up backfaces a bit, render them first (-> cull front faces) and then render the front faces as usual. That way you shouldn't have to do optimizations as in fig. 2.

Cheers,
p.

[Mel]

Looks great!

[hybrid]

We have a scale-by-normals and a mesh-flip function in the mesh manipulator. Do you do anything else here?

[Lonesome Ducky]

Haha, I wasn't aware of that in the mesh manipulator. I do combine redundant vertices that have different texcoords though, which I believe the meshmanipulator doesn't do. I know it combines duplicates, but only if they don't have different normals, colors, or uv's. I'll change the code to use the mesh manipulator, and later add skinned meshes.

[hybrid]

Yeah, the mesh combiner is indeed somewhat limited so far. But you could at least reduce the initialisation then to call copy/scale/invert on the mesh with the mesh manipulator, and then combine manually on your own. Would also help us to see if those functions are working properly and have all the parameters which are needed. And also promotes the use of the manipulator some more

[Lonesome Ducky]

Updated to use the mesh manipulator:

Code: Select all

#ifndef TOONOUTLINE_H
#define TOONOUTLINE_H
#include <irrlicht.h>

struct indexRedirect {
	int oldIndex;
	int newIndex;
};
struct vertCombine {
	irr::core::vector3df totalPos;
	irr::core::vector3df totalNormal;
	int count;
};

// Adds a toon outline to a mesh. Works by copying the mesh, and expanding by the normals. It then flips the faces.
irr::scene::IMesh* createToonOutlineMesh(irr::scene::ISceneManager* smgr, irr::scene::IMesh* mesh, float size, irr::video::SColor color) {

	// Copy the mesh buffers
	irr::scene::SMesh* toonMesh = smgr->getMeshManipulator()->createMeshCopy(mesh);


	for (int i = 0; i < toonMesh->getMeshBufferCount(); i++) {

		// Find duplicate vertices, then add an indexRedirect so indices can be updated and the vertices can later be collapsed
		irr::video::S3DVertex* sourceVertices = (irr::video::S3DVertex*)toonMesh->getMeshBuffer(i)->getVertices();
		irr::u16* sourceIndices = toonMesh->getMeshBuffer(i)->getIndices();
		vertCombine* combiners = new vertCombine[toonMesh->getMeshBuffer(i)->getVertexCount()];
		irr::core::array<indexRedirect> redirects;
		bool *check = new bool[toonMesh->getMeshBuffer(i)->getVertexCount()];
		for (int j = 0; j < toonMesh->getMeshBuffer(i)->getVertexCount(); j++) {
			check[j] = true;
			combiners[j].totalPos = sourceVertices[j].Pos;
			combiners[j].totalNormal = sourceVertices[j].Normal;
			combiners[j].count = 1;
		}
		for (int j = 0; j < toonMesh->getMeshBuffer(i)->getVertexCount(); j++) {
			if (check[j]) {
				// Check for vertices that are very close
				for (int k = j; k < toonMesh->getMeshBuffer(i)->getVertexCount(); k++) {
					if (check[k]) {
						if (sourceVertices[j].Pos.getDistanceFromSQ(sourceVertices[k].Pos) <= irr::core::ROUNDING_ERROR_f32*irr::core::ROUNDING_ERROR_f32) {
							check[k] = false;
							indexRedirect redir;
							redir.newIndex = j;
							redir.oldIndex = k;
							redirects.push_back(redir);
						}
					}
				}
			}
		}
		delete [] check;

		// Combine vertices
		for (int j = 0; j < redirects.size(); j++) {
			combiners[redirects[j].newIndex].totalPos += sourceVertices[redirects[j].oldIndex].Pos;
			combiners[redirects[j].newIndex].totalNormal += sourceVertices[redirects[j].oldIndex].Normal;
			combiners[redirects[j].newIndex].count++;
		}
		for (int j = 0; j < redirects.size(); j++) {
			sourceVertices[redirects[j].newIndex].Pos = combiners[redirects[j].newIndex].totalPos / combiners[redirects[j].newIndex].count;
			sourceVertices[redirects[j].newIndex].Normal = combiners[redirects[j].newIndex].totalNormal / combiners[redirects[j].newIndex].count;
			sourceVertices[redirects[j].newIndex].Normal.normalize();
		}
		// After combining the vertices, fix indices so they no longer reference dead vertices
		for (int j = 0; j < redirects.size(); j++) {
			for (int k = 0; k < toonMesh->getMeshBuffer(i)->getIndexCount(); k++) {
				if (sourceIndices[k] == redirects[j].oldIndex) {
					sourceIndices[k] = redirects[j].newIndex;
				}
			}
		}
		delete [] combiners;

		// Now expand the mesh, moving the vertices along their normals
		smgr->getMeshManipulator()->setVertexColors(toonMesh,color);
		for (int j = 0; j < toonMesh->getMeshBuffer(i)->getVertexCount(); j++) {
			sourceVertices[j].Color = color;
			sourceVertices[j].Pos += sourceVertices[j].Normal*size;
		}

		toonMesh->getMeshBuffer(i)->recalculateBoundingBox();

		// Now set the material
		toonMesh->getMeshBuffer(i)->getMaterial().Lighting = false;
		toonMesh->getMeshBuffer(i)->getMaterial().BackfaceCulling = false;
		toonMesh->getMeshBuffer(i)->getMaterial().FrontfaceCulling = true;
		toonMesh->getMeshBuffer(i)->getMaterial().DiffuseColor = color;
		toonMesh->getMeshBuffer(i)->getMaterial().ColorMaterial = irr::video::ECM_DIFFUSE_AND_AMBIENT;
		toonMesh->getMeshBuffer(i)->getMaterial().TextureLayer[0].Texture = 0;

	}

	toonMesh->recalculateBoundingBox();
	return (irr::scene::IMesh*)toonMesh;
}

#endif

I couldn't find the scale by normals btw, hybrid.