Problems picking scaled scene nodes / getPickedNodeBB

[Rv]

Hi folks,

I've been through the threads about this problem, and seen some of Vitek's posts make it into the source base, but as some have noticed, with scaled objects the problem persist. As previously noted, the problem with scaled objects is that the scale is applied to the ray as it is transformed to object space, skewing the results.

My fix is to remove the scale from the matrix before it's applied to the ray, but save the scale and apply it to the bounding box before checking against that.

This brings to light another problem, in that Irrlicht's matrix getScale() does not work for rotated objects. Let's fix that first:

Code: Select all

	inline vector3df matrix4::getScale() const
	{
		vector3df vScale;
		vScale.X = (f32) vector3df(M[0],M[1],M[2]).getLength();
		vScale.Y = (f32) vector3df(M[4],M[5],M[6]).getLength();
		vScale.Z = (f32) vector3df(M[8],M[9],M[10]).getLength();
		return vScale;
	}

If you don't want to change getScale(), you could always make a getTrueScale() and use that instead (I did to begin with), but I think it's fairly safe just to replace the original.

Now we can patch up getPickedNodeBB:

Code: Select all

      if (current->isVisible() &&
          (bNoDebugObjects ? !current->isDebugObject() : true) &&
          (bits==0 || (bits != 0 && (current->getID() & bits))))
      {
		 // get world to object space transform
		 core::matrix4 mat;
		 core::matrix4 orient = current->getAbsoluteTransformation();

		 //. Remove scale
		 core::vector3df vScale = orient.getScale();
		 orient.Normalise();

		 if (!orient.getInverse(mat))
			continue;

		 // transform vector from world space to object space
		 core::line3df line(ray);
		 mat.transformVect(line.start);
		 mat.transformVect(line.end);

		 core::aabbox3df box = current->getBoundingBox();

		 //. Apply scale to bounds
		 box.MinEdge *= vScale;
		 box.MaxEdge *= vScale;

		 // do intersection test in object space
		 if (box.intersectsWithLine(line))
		 {
			...

For us, this works much better. The sort order is still an issue, but that is by design rather than a bug. Ideally, the picker would return an ordered list of the nodes found to allow the caller to choose the most appropriate, but that's beyond the scope of this quick fix.

Hope this is of some use to others, since this problem has been driving me nuts!

Cheers,

[omar shaaban]

ya man the helped a lot good job