Issues with applied recalculated normals being again changed

[Yarcies]

I have a skinned/animated model with hugely messed up normals (most likely some bug in the exporter or importer) after importing it into Irrlicht.

Now I have two major issues I ran into when attempting to fix this:

I wrote a little function that walks through all meshbuffers of a given animated mesh, inspects all vertices in there and recalculates all normals completely from scratch. That works fine and gives acceptable results if I use it on my scenenode->getMesh()'s mesh.

The first problem I ran into is, instead of creating a new animated mesh scene node directly from scenemanger->getMesh()'s mesh, I did this:

Code: Select all

	if (manager->getMeshCache()->isMeshLoaded(meshpath) == false) {
		//mesh is new, fix normals for it:
		mesh = manager->getMesh(meshpath);
		if (mesh && strendswith(meshpath, ".b3d")) {
			//fix the normals!
			recalculatesmoothnormals((SAnimatedMesh*)mesh);
		}
	}else{
		//mesh is already present and fixed:
		mesh = manager->getMesh(meshpath);
	}

The idea was that I don't do it on *afterwards* on every single scene node, but once on the gobal mesh copy in the scene manager's mesh cache.

Sadly this didn't work. When I created any scene nodes from that mesh, they would have totally ruined normals again.

Update: Fixed this now by using setDirty() for the meshbuffer which I previously didn't by mistake - it doesn't fix the second issue though:

The second problem I ran into is, as soon as I activate updateNormalsWhenAnimating(true) on the scenenode->getMesh() *after* fixing the normals successfully, this ruins them again - if I leave this at false, they work fine (but aren't adjusted for animations). I expected them to be based on my recalculated, fixed normals - instead they appear to be taken from the original broken ones for some reason? (Please note this still happens despite me setting the meshbuffer dirty after the manipulations!)

This problem is pretty visible and not just a performance issue, rather trashing all model's normals when animated since I can either have them all borked or not adjusted during animation (which results in them also being borked for complex animations).

Does anyone have an idea what I could do about those two problems (especially the latter one)?

[Yarcies]

It is some bunch of code and might also not be the most clean way to do it, but here the function recalculatesmoothnormals(). Please note it visibly perfectly gives the intended result, just without using updateNormalsWhenAnimating(false) it will revert to the OLD normals I had before I run that function (despite I clearly called it). Why?

Code: Select all

void meshbuf_setvnormal(IMeshBuffer* meshb, int vertexindex, vector3df normal) {
	//set the vertex normal
	void* vertexbuffer = meshb->getVertices();
	if (meshb->getVertexType() == EVT_2TCOORDS) {
		((S3DVertex2TCoords*)vertexbuffer)[vertexindex].Normal = normal;return;
	}
	if (meshb->getVertexType() == EVT_TANGENTS) {
		((S3DVertexTangents*)vertexbuffer)[vertexindex].Normal = normal;return;
	}
	((S3DVertex*)vertexbuffer)[vertexindex].Normal = normal;
}
vector3df meshbuf_getvnormal(IMeshBuffer* meshb, int vertexindex) {
	//get the vertex normal
	void* vertexbuffer = meshb->getVertices();
	if (meshb->getVertexType() == EVT_2TCOORDS) {
		return ((S3DVertex2TCoords*)vertexbuffer)[vertexindex].Normal;
	}
	if (meshb->getVertexType() == EVT_TANGENTS) {
		return ((S3DVertexTangents*)vertexbuffer)[vertexindex].Normal;
	}
	return ((S3DVertex*)vertexbuffer)[vertexindex].Normal;
}

vector3df obtainposfromvertex(IMeshBuffer* meshb, int vertexindex) {
	//get a vertex position
	void* vertexbuffer = meshb->getVertices();
	if (meshb->getVertexType() == EVT_2TCOORDS) {
		return ((S3DVertex2TCoords*)vertexbuffer)[vertexindex].Pos;
	}
	if (meshb->getVertexType() == EVT_TANGENTS) {
		return ((S3DVertexTangents*)vertexbuffer)[vertexindex].Pos;
	}
	return ((S3DVertex*)vertexbuffer)[vertexindex].Pos;
}

triangle3df meshbuf_addtrianglenormals(IMeshBuffer* meshb, int triangleid) {
	//will add the triangle normals of a given polygon to its vertices' normals
	void* vertexbuffer = meshb->getVertices();
	u16* indexbuffer = meshb->getIndices();
	vector3df pos1,pos2,pos3;
	pos1 = obtainposfromvertex(meshb, indexbuffer[(triangleid*3)+0]);
	pos2 = obtainposfromvertex(meshb, indexbuffer[(triangleid*3)+1]);
	pos3 = obtainposfromvertex(meshb, indexbuffer[(triangleid*3)+2]);
	vector3df n = triangle3df(pos1,pos2,pos3).getNormal();
	int i = triangleid*3;
	while (i < triangleid*3+3) {
		meshbuf_setvnormal(meshb, indexbuffer[i], meshbuf_getvnormal(meshb, indexbuffer[i])+n);
		i++;
	}
	return triangle3df(pos1,pos2,pos3);
}


void recalculatesmoothnormals(SAnimatedMesh* animmesh) {
	int i = animmesh->getMeshBufferCount();
	int r = 0;
	while (r < i) {
		//first step: zero out all vertex normals
		int vcount = animmesh->getMeshBuffer(r)->getVertexCount();
		int k = 0;
		while (k < vcount) {
			meshbuf_setvnormal(animmesh->getMeshBuffer(r), k, vector3df(0,0,0));
			k++;
		}
		//second step: go through all polygons and add up their face normals on all vertices
		int polycount = animmesh->getMeshBuffer(r)->getIndexCount()/3;
		k = 0;
		while (k < polycount) {
			//obtain triangle
			meshbuf_addtrianglenormals(animmesh->getMeshBuffer(r), k);
			k++;
		}
		r++;
	}
	animmesh->setDirty(EBT_VERTEX_AND_INDEX);
}

I mean, is this even supposed to work? Modifying a mesh buffer's normals manually, then doing updateNormalsWhenAnimating(true) afterwards?

I expected updateNormalsWhenAnimating(true) to use the normals of the mesh and twist them around when it is animated. Or alternatively, to sanely recalculate them from scratch (but it doesn't seem to do that, since that is basically what I do manually and it results in different results). But it seems to use normals I had in the mesh before changing them, so I am unsure about where it even takes them from (I changed them... so wtf?).

Any hints appreciated!

[Gorbstein]

At a guess, it sounds like not all of the keyframes are getting their normals uploaded properly - just the first one. When you're telling Irrlicht to relcalculate the normals during animation it'll be interpolating between your fixed normals and the bad ones.

Does it help if you call setDirty on each individual meshbuffer rather than on the SAnimatedMesh?

Ie in your normal recalculation loop of recalculatesmoothnormals:

Code: Select all

      k = 0; 
      while (k < polycount) { 
         //obtain triangle 
         meshbuf_addtrianglenormals(animmesh->getMeshBuffer(r), k); 

          //FLAG INDIVIDUAL MESHBUFFER AS RECALCULATED
         animesh->getMeshBuffer(r)->setDirty(EBT_VERTEX_AND_INDEX);

         k++; 
      }

[Yarcies]

I tried what you suggested, sadly it didn't help... I am still stuck with the old normals as soon as update normals when animating is active.

And when I set update normals when animating to off, I got the nice recalculated normals of me again... just not updated on animation

Also for the animation, actually I'm just setting that CONTROL mode and messing around with the bone scene nodes myself instead of triggering/playing some of the animations that might be buried in the 3d model. So I'm making use of the bone rigging, but not of any of the animations.

[Gorbstein]

Hmm.. ok. Not completely familiar with the bone/animation system in Irr so not sure where it's getting its normal data from. What if you try using a Mesh Manipulator to recalculate the normals?

Code: Select all

virtual void 	recalculateNormals (IMeshBuffer *buffer, bool smooth=false, bool angleWeighted=false) const =0
 	Recalculates all normals of the mesh buffer. 

If that doesn't work, I'd try to eliminate the reason for the initial borked normals; try another file format? A different tool? What are you using to model/export atm?

Any chance of a screenshot, or a code snippet where you're manipulating the bones?

D

[Yarcies]

What are you using to model/export atm?

Blender -> Gandaldf's .b3d export -> .b3d -> Irrlicht import -> IAnimatedMesh

try another file format? A different tool?

I think with Blender and Irrlicht and without any additional tools, .b3d and Gandaldf's .b3d exporter is the only sane solution if you don't want to write tools from scratch or use some non-blender stuff to convert things around.

What if you try using a Mesh Manipulator to recalculate the normals?

I think while this is a good idea, I am sure it won't work for me: the model has, similarly to .3ds, double and triple vertices added for each single face again, so if you do a simple smooth normals calculation, you will get hard edges due to the doubled and tripled vertices. While I can work around this with my own code which does a second pass and searches all doubled and tripled vertices and averages their normals which fixes it, I guess the Mesh Manipulator won't do this if alone for performance reasons.

Therefore I think own code to do the recalculation is kinda required, but I could try to strip the whole thing down to some sort of demo case with source that demonstrates the problem (or the bug?).

Any chance of a screenshot, or a code snippet where you're manipulating the bones?

I think this is more related to the way I setup the model or mesh actually... I am not doing much animation wise yet and I basically observe the normals in the unchanged position as originally in the first keyframe.

I'll go put together a test case thing now with as few code as possible, since if noone is having another idea (thanks a lot for your ideas so far!) I guess that's the only thing that can advance this weird problem of mine