How to get the mouse position relative to the camera.

[SanekFPS]

I am using a split screen and I have two cameras displayed. I'm drawing a picture on one of the cameras. It is necessary that an event occurs when the cursor hovers over the image. But the position of the picture and the position of the mouse do not match. Moreover, when I move the image, the x coordinate difference is calculated to become larger.

[CuteAlien]

Are you using ISceneCollisionManager::getScreenCoordinatesFrom3DPosition? There's 3 things to note about that one.
1.) If your viewport isn't identical to the full render-target (by default the screen) - as in your case - you have to pass true as last parameter so it gives you results relative to view-port instead of screen.
2.) Second one is part of an open TODO, but has some workaround. It ignores the left-top coordinate of the view-port when it's used. So result of getScreenCoordinatesFrom3DPosition should add the value of videoDriver->getViewPort().UpperLeftCorner.
3.) If there is more than one view-port in your code be aware that the function always uses the active one in your videoDriver.

If you are not using this directly please describe in a bit more detail which Irrlicht functions you are working with. Maybe post the code.

[SanekFPS]

No, I don't use this function. Thanks for the help

[SanekFPS]

I tried using this function
core::position2d<s32> asd = smgr->getSceneCollisionManager()->getScreenCoordinatesFrom3DPosition(
core::vector3df(receiver.GetMouseState().Position.X,receiver.GetMouseState().Position.Y,0),camera[0],true);
it turns out somehow strange. The report from the upper left corner does not start from zero and the y position changes even when I move the mouse left to right

[CuteAlien]

It's from 3D (!) position, so don't pass in screen positions. I thought you used that to draw a picture at a certain 3d position maybe. So forget about that one I realize now you are maybe working all in 2D and it's about gui-images probably.

Please describe again what exactly you are trying to do. How do you draw an image? Aka - is this an IGUIImage from Irrlicht GUI or do you draw something with video driver or do you use 3d billboards? Where do you want to draw that image exactly (some 2d position, some 3d position?). How do you check for an event? Which functions are you using? So how do you know those positions are different? Are you using mouse-events or are you working the the ICursorControl class?

Sorry, always hard for me to understand what's going on when I don't see the full code as there are so many options to do things and I don't know what it is you are doing. Easiest for me is if you can make a quick code example (modify one of the Irrlicht examples or so). Then I can simply run it and see what's going on.

My guess is it's something about the viewport. For example when working with ICursorControl class you might want to restrict cursor to one camera. But I really can't tell without more info.

[SanekFPS]

Sorry. I should have shown the code first. Here:
#include <iostream>
#include <irrlicht.h>
#include <windows.h>

using namespace std;

using namespace irr;

using namespace core;
using namespace scene;
using namespace video;
using namespace io;
using namespace gui;
#pragma comment(lib, "Irrlicht.lib")
#pragma comment(linker, "/subsystem:windows /ENTRY:mainCRTStartup")
ICameraSceneNode *camera[2]={0,0};


class MyEventReceiver : public IEventReceiver
{
public:
struct SMouseState
{
core::position2di Position;
bool LeftButtonDown;
SMouseState() : LeftButtonDown(false) { }
} MouseState;
// This is the one method that we have to implement
virtual bool OnEvent(const SEvent& event)
{
// Remember whether each key is down or up
if (event.EventType == irr::EET_KEY_INPUT_EVENT)
KeyIsDown[event.KeyInput.Key] = event.KeyInput.PressedDown;
if (event.EventType == irr::EET_MOUSE_INPUT_EVENT)
{
switch(event.MouseInput.Event)
{
case EMIE_LMOUSE_PRESSED_DOWN:
MouseState.LeftButtonDown = true;
break;

case EMIE_LMOUSE_LEFT_UP:
MouseState.LeftButtonDown = false;
break;

case EMIE_MOUSE_MOVED:
MouseState.Position.X = event.MouseInput.X;
MouseState.Position.Y = event.MouseInput.Y;
break;

default:
// We won't use the wheel
break;
}
}



return false;
}
const SMouseState & GetMouseState(void) const
{
return MouseState;
}

// This is used to check whether a key is being held down
virtual bool IsKeyDown(EKEY_CODE keyCode) const
{
return KeyIsDown[keyCode];
}

MyEventReceiver()
{
for (u32 i=0; i<KEY_KEY_CODES_COUNT; ++i)
KeyIsDown = false;
}

private:
// We use this array to store the current state of each key
bool KeyIsDown[KEY_KEY_CODES_COUNT];
};

void led(IVideoDriver* driver,video::ITexture* images,int x, int y, bool log)
{
if(log)
driver->draw2DImage(images, core::position2d<s32>(x,y),
core::rect<s32>(96,0,143,47), 0,video::SColor(255,255,255,255), true);
else
driver->draw2DImage(images, core::position2d<s32>(x,y),
core::rect<s32>(48,0,95,47), 0,video::SColor(255,255,255,255), true);


}

bool tubler(IVideoDriver* driver,video::ITexture* images,int x, int y, bool log, core::position2di mouse,bool clock)
{
if(log)
driver->draw2DImage(images, core::position2d<s32>(x,y),
core::rect<s32>(0,48,31,95), 0,video::SColor(255,255,255,255), true);
else
driver->draw2DImage(images, core::position2d<s32>(x,y),
core::rect<s32>(0,0,31,47), 0,video::SColor(255,255,255,255), true);

if((mouse.X>x)&&(mouse.Y>y)&&clock)
{
return true;
}
else
return false;


}

void indic3(IVideoDriver* driver,video::ITexture* images,core::position2di pos, int cheslo)
{
int a=cheslo/100;
int b=(cheslo%100)/10;
int c=cheslo%10;
driver->draw2DImage(images, pos,core::rect<s32>(16+32*a,144,47+32*a,207), 0,video::SColor(255,255,255,255), true);
driver->draw2DImage(images, core::position2d<s32>(pos.X+32,pos.Y),core::rect<s32>(16+32*b,144,47+32*b,207), 0,video::SColor(255,255,255,255), true);
driver->draw2DImage(images, core::position2d<s32>(pos.X+64,pos.Y),core::rect<s32>(16+32*c,144,47+32*c,207), 0,video::SColor(255,255,255,255), true);
}
void indic2(IVideoDriver* driver,video::ITexture* images,core::position2di pos, int cheslo)
{
int a=cheslo/10;
int b=cheslo%10;

driver->draw2DImage(images, pos,core::rect<s32>(16+32*a,144,47+32*a,207), 0,video::SColor(255,255,255,255), true);
driver->draw2DImage(images, core::position2d<s32>(pos.X+32,pos.Y),core::rect<s32>(16+32*b,144,47+32*b,207), 0,video::SColor(255,255,255,255), true);
}
void draw2DImageR(irr::video::IVideoDriver *driver, irr::video::ITexture* texture, irr::core::rect<irr::s32> sourceRect,
irr::core::position2d<irr::s32> position,
irr::core::position2d<irr::s32> rotationPoint, irr::f32 rotation,
irr::core::vector2df scale, bool useAlphaChannel, irr::video::SColor color) {

// Store and clear the projection matrix
irr::core::matrix4 oldProjMat = driver->getTransform(irr::video::ETS_PROJECTION);
driver->setTransform(irr::video::ETS_PROJECTION,irr::core::matrix4());

// Store and clear the view matrix
irr::core::matrix4 oldViewMat = driver->getTransform(irr::video::ETS_VIEW);
driver->setTransform(irr::video::ETS_VIEW,irr::core::matrix4());

// Store and clear the world matrix
irr::core::matrix4 oldWorldMat = driver->getTransform(irr::video::ETS_WORLD);
driver->setTransform(irr::video::ETS_WORLD,irr::core::matrix4());

// Find horizontal and vertical axes after rotation
irr::f32 c = cos(-rotation*irr::core::DEGTORAD);
irr::f32 s = sin(-rotation*irr::core::DEGTORAD);
irr::core::vector2df horizontalAxis(c,s);
irr::core::vector2df verticalAxis(s,-c);

// First, we'll find the offset of the center and then where the center would be after rotation
irr::core::vector2df centerOffset(position.X+sourceRect.getWidth()/2.0f*scale.X-rotationPoint.X,position.Y+sourceRect.getHeight()/2.0f*scale.Y-rotationPoint.Y);
irr::core::vector2df center = centerOffset.X*horizontalAxis - centerOffset.Y*verticalAxis;
center.X += rotationPoint.X;
center.Y += rotationPoint.Y;

// Now find the corners based off the center
irr::core::vector2df cornerOffset(sourceRect.getWidth()*scale.X/2.0f,sourceRect.getHeight()*scale.Y/2.0f);
verticalAxis *= cornerOffset.Y;
horizontalAxis *= cornerOffset.X;
irr::core::vector2df corner[4];
corner[0] = center + verticalAxis - horizontalAxis;
corner[1] = center + verticalAxis + horizontalAxis;
corner[2] = center - verticalAxis - horizontalAxis;
corner[3] = center - verticalAxis + horizontalAxis;

// Find the uv coordinates of the sourceRect
irr::core::vector2df textureSize(texture->getSize().Width, texture->getSize().Height);
irr::core::vector2df uvCorner[4];
uvCorner[0] = irr::core::vector2df(sourceRect.UpperLeftCorner.X,sourceRect.UpperLeftCorner.Y);
uvCorner[1] = irr::core::vector2df(sourceRect.LowerRightCorner.X,sourceRect.UpperLeftCorner.Y);
uvCorner[2] = irr::core::vector2df(sourceRect.UpperLeftCorner.X,sourceRect.LowerRightCorner.Y);
uvCorner[3] = irr::core::vector2df(sourceRect.LowerRightCorner.X,sourceRect.LowerRightCorner.Y);
for (irr::s32 i = 0; i < 4; i++)
uvCorner /= textureSize;

// Vertices for the image
irr::video::S3DVertex vertices[4];
irr::u16 indices[6] = { 0, 1, 2, 3 ,2 ,1 };

// Convert pixels to world coordinates
irr::core::vector2df screenSize(driver->getViewPort().getWidth(), driver->getViewPort().getHeight());
for (irr::s32 i = 0; i < 4; i++) {
vertices.Pos = irr::core::vector3df(((corner.X/screenSize.X)-0.5f)*2.0f,((corner.Y/screenSize.Y)-0.5f)*-2.0f,1);
vertices.TCoords = uvCorner;
vertices.Color = color;
}

// Create the material
// IMPORTANT: For irrlicht 1.8 and above you MUST ADD THIS LINE:
//material.BlendOperation = irr::video::EBO_ADD;
irr::video::SMaterial material;
material.Lighting = false;
material.ZWriteEnable = false;
material.ZBuffer = false;
material.BackfaceCulling = false;
material.TextureLayer[0].Texture = texture;
material.MaterialTypeParam = irr::video::pack_textureBlendFunc(irr::video::EBF_SRC_ALPHA, irr::video::EBF_ONE_MINUS_SRC_ALPHA, irr::video::EMFN_MODULATE_1X, irr::video::EAS_TEXTURE | irr::video::EAS_VERTEX_COLOR);
material.BlendOperation = irr::video::EBO_ADD;
if (useAlphaChannel)
material.MaterialType = irr::video::EMT_ONETEXTURE_BLEND;
else
material.MaterialType = irr::video::EMT_SOLID;

driver->setMaterial(material);
driver->drawIndexedTriangleList(&vertices[0],4,&indices[0],2);

// Restore projection, world, and view matrices
driver->setTransform(irr::video::ETS_PROJECTION,oldProjMat);
driver->setTransform(irr::video::ETS_VIEW,oldViewMat);
driver->setTransform(irr::video::ETS_WORLD,oldWorldMat);
}



int main(int argc, char** argv)
{
int screen_width = GetSystemMetrics(SM_CXSCREEN);
int screen_height = GetSystemMetrics(SM_CYSCREEN);

MyEventReceiver receiver;

IrrlichtDevice *device =createDevice(EDT_OPENGL, dimension2du(screen_width, screen_height), 16,false, false, false, &receiver);
device->setResizable(true);
IVideoDriver* driver = device->getVideoDriver();
ISceneManager* smgr = device->getSceneManager();
IGUIEnvironment* guienv = device->getGUIEnvironment();
//загрузка антены
IAnimatedMesh* ant = smgr->getMesh("./models/ant1.obj");
IAnimatedMeshSceneNode* antNode = smgr->addAnimatedMeshSceneNode(ant );
antNode->setMaterialFlag(EMF_LIGHTING, false);
antNode->setPosition(vector3df(0, 8, 0));
//загрузка основания
IAnimatedMesh* osn = smgr->getMesh("./models/osnovanie.obj");
IAnimatedMeshSceneNode* osnNode = smgr->addAnimatedMeshSceneNode(osn );
osnNode->setMaterialFlag(EMF_LIGHTING, false);
osnNode->setPosition(vector3df(0, 0, 0));
//загрузка неба
IAnimatedMesh* sky = smgr->getMesh("./models/skybox.obj");
IAnimatedMeshSceneNode* skyNode = smgr->addAnimatedMeshSceneNode(sky );
skyNode->setMaterialFlag(EMF_LIGHTING, false);
skyNode->setPosition(vector3df(0, 0, 0));
//загрузка текстуры элементов
video::ITexture* images = driver->getTexture("./models/pribor.png");
driver->makeColorKeyTexture(images, core::position2d<s32>(0,0));
/*
координаты
тумблер:0 0 31 47
тумблер выкл 0 48 31 95
лампочка выключена: 48 0 95 47
лампочяка вкл 96 0 143 47
метр 152 16 559 223
рукоятка 208 0 335 127
индикатор 22 140 64 212
*/


camera[0] = smgr->addCameraSceneNode(0, vector3df(5,40,40), vector3df(0,0,0));
//Top
camera[1] = smgr->addCameraSceneNode(0, vector3df(5,18,28), vector3df(0,0,0));


int lastFPS = -1;

u32 then = device->getTimer()->getTime();

const f32 MOVEMENT_SPEED = 5.f;
bool old=false;
while(device->run())
{

u32 now = device->getTimer()->getTime();
f32 frameDeltaTime = (f32)(now - then) / 1000.f; // Time in seconds
then = now;

core::vector3df camPosition = antNode->getPosition();
core::vector3df camRorarion = antNode->getRotation();

if(receiver.IsKeyDown(irr::KEY_UP))
camRorarion.Y += MOVEMENT_SPEED * frameDeltaTime;

else if(receiver.IsKeyDown(irr::KEY_DOWN))

camRorarion.Y -= MOVEMENT_SPEED * frameDeltaTime;

antNode->setPosition(camPosition);
antNode->setRotation(camRorarion);
driver->setViewPort(rect<s32>(0,0,screen_width*2/3,screen_width*2/3));
driver->beginScene(true, true, SColor(0,208,208,208));
//камера 2 3д модель
smgr->setActiveCamera(camera[1]);
//Set viewpoint to the second quarter (right top)
driver->setViewPort(rect<s32>(screen_width*2/3,0,screen_width,screen_height*2/3));
smgr->drawAll();

//камера1 интерфейс
smgr->setActiveCamera(camera[0]);
//Set viewpoint to the first quarter (left top)
driver->setViewPort(rect<s32>(0,0,screen_width*2/3,screen_height));
//Draw scene


bool a;
led(driver,images,50,50,false);
a=tubler(driver,images,500,500,a,receiver.GetMouseState().Position,receiver.GetMouseState().LeftButtonDown);
core::position2d<s32> asd = smgr->getSceneCollisionManager()->getScreenCoordinatesFrom3DPosition(
core::vector3df(receiver.GetMouseState().Position.X,receiver.GetMouseState().Position.Y,0),camera[0],true);
if(receiver.GetMouseState().LeftButtonDown&&!old)
{
cout<<"Y:"<<asd.X<<endl;
cout<<"X:"<<asd.Y<<endl;
old=true;
}
if(!receiver.GetMouseState().LeftButtonDown&&old)old=false;

indic3(driver,images,core::position2d<s32>(300,300),123);

//driver->draw2DImage(images, core::position2d<s32>(300,100),core::rect<s32>( 208,0,335,127), 0,video::SColor(255,255,255,255), true);
draw2DImageR(driver,images,core::rect<s32>(208,0,335,127),core::position2d<s32>(300,100),core::vector2d<s32>(325,125),30,core::vector2df(1,1),
false,
SColor(255,255,255,255));




driver->endScene();
}

device->drop();

return 0;
}

[SanekFPS]

I'm drawing a picture. I need to check that the mouse is in the picture. Checking the mouse coordinates follows the example from the documentation. The pore x coordinate does not match the coordinates of the image. And the further away from 0 0 the greater the difference is obtained

[CuteAlien]

Thank you. I'll look at it in the evening. Btw, you can use a code tag in the forum to keep code formatted.
Edit: Unrelated to bug, but check your KeyIsDown intitialization - that loop not doing what you think.
Edit2: Similar bug in the loop with vertices, corner annd uvCorner after "// Convert pixels to world coordinates" - that shouldn't compile.
Edit3: Using getScreenCoordinatesFrom3DPosition like that makes no sense. Just remove that again and forget I ever mentioned it (you pass a 2d screen coordinate as parameter where it expects a 3d world coordinate).

Edit4: OK... this is not a quick read. You transform your images before drawing. What I can tell is that receiver.GetMouseState().Position is exactly returning the screen position of your mouse. So I'd guess the problem is that your own draw2DImageR doesn't end up where you expect it? So what about using a simple draw2DImage for a start - does that behave as you'd expect? Or maybe you have to scale it by the viewport size and add lef-top coordinate of viewport? Sorry, I'm still a bit struggling to figure out _what_ your exact problem is...

[SanekFPS]

Code: Select all

a=tubler(driver,images,500,500,a,receiver.GetMouseState().Position,receiver.GetMouseState().LeftButtonDown);

I have described the function. She draws a button with my image and checks if the mouse coordinates fall within the limits of the button coordinates. If it matches and the mouse is pressed, then true is returned, otherwise false. When I put the button and tried to press true, it returned not in the place where the button is drawn, but twice to the right.

[SanekFPS]

draw2DImageR I need to rotate a couple of objects in place

[CuteAlien]

So the draw2DImageR stuff is completely unrelated and doesn't have anything to do with your bug? What about indic3 - is that needed in any way?
The only function of interest is tubler?
But ... you only pass fixed values in there. x and y are always 500 and log is always false. All it does is return true when mouse-coordinates are both > 500 - but it doesn't _do_ anything with that result afterwards (assigns it to a, but you overwrite that one in the next loop before it's used for anything).

[SanekFPS]

draw2dimage is not related to the problem. indic3 is also not relevant. The problem is in tubler. I put the table at 500 on X. But on the screen, in order for table to return true, the mouse must be slightly to the right of the image itself.

[SanekFPS]

https://yadi.sk/d/ZJIHU_Cxb5KJ5w

[CuteAlien]

If you check the image on resizing you'll notice it is the image that moves. The mouse is always returning the correct coordinates.

And the reason your image is more to the left is - because your viewport is smaller than the full screen. So everything you usually see full-screen get's scaled down to the viewport sizes. If you don't want that for 2d images then reset your viewport first to have again the full screen-size.

If you want your mouse-coordinates not be real mouse-coordinates (those are always for the screen), then you have to scale them by your viewport-sizes.

I rewrote your example a bit so you can see the difference:

Code: Select all

#include <iostream>
#include <irrlicht.h>
#include <windows.h>

using namespace std;

using namespace irr;

using namespace core;
using namespace scene;
using namespace video;
using namespace gui;
#pragma comment(lib, "Irrlicht.lib")

ICameraSceneNode *camera[2]={0,0};


class MyEventReceiver : public IEventReceiver
{
public:
	struct SMouseState
	{
		core::position2di Position;
		bool LeftButtonDown;
		SMouseState() : LeftButtonDown(false) { }
	} MouseState;
	// This is the one method that we have to implement
	virtual bool OnEvent(const SEvent& event)
	{
		// Remember whether each key is down or up
		if (event.EventType == irr::EET_KEY_INPUT_EVENT)
			KeyIsDown[event.KeyInput.Key] = event.KeyInput.PressedDown;
		if (event.EventType == irr::EET_MOUSE_INPUT_EVENT)
		{
			switch(event.MouseInput.Event)
			{
			case EMIE_LMOUSE_PRESSED_DOWN:
				MouseState.LeftButtonDown = true;
				break;

			case EMIE_LMOUSE_LEFT_UP:
				MouseState.LeftButtonDown = false;
				break;

			case EMIE_MOUSE_MOVED:
				MouseState.Position.X = event.MouseInput.X;
				MouseState.Position.Y = event.MouseInput.Y;
				break;

			default:
				// We won't use the wheel
				break;
			}
		}

		return false;
	}
	const SMouseState & GetMouseState(void) const
	{
		return MouseState;
	}

	// This is used to check whether a key is being held down
	virtual bool IsKeyDown(EKEY_CODE keyCode) const
	{
		return KeyIsDown[keyCode];
	}

	MyEventReceiver()
	{
		for (u32 i=0; i<KEY_KEY_CODES_COUNT; ++i)
			KeyIsDown[i] = false;
	}

private:
	// We use this array to store the current state of each key
	bool KeyIsDown[KEY_KEY_CODES_COUNT];
};

int main(int argc, char** argv)
{
	int screen_width = GetSystemMetrics(SM_CXSCREEN);
	int screen_height = GetSystemMetrics(SM_CYSCREEN);

	MyEventReceiver receiver;

	IrrlichtDevice *device = createDevice(EDT_OPENGL, dimension2du(screen_width, screen_height), 16,false, false, false, &receiver);
	device->setResizable(true);
	IVideoDriver* driver = device->getVideoDriver();
	ISceneManager* smgr = device->getSceneManager();
	IGUIEnvironment* guienv = device->getGUIEnvironment();

	video::ITexture* images = driver->getTexture("./models/pribor.png");
	guienv->addImage(images, vector2di(500, 100));

	camera[0] = smgr->addCameraSceneNode(0, vector3df(5,40,40), vector3df(0,0,0));
	camera[1] = smgr->addCameraSceneNode(0, vector3df(5,18,28), vector3df(0,0,0));


	bool old=false;
	while(device->run())
	{
		driver->setViewPort(rect<s32>(0,0,screen_width*2/3,screen_width*2/3));
		driver->beginScene(true, true, SColor(0,208,208,208));
		//
		smgr->setActiveCamera(camera[1]);
		//Set viewpoint to the second quarter (right top)
		driver->setViewPort(rect<s32>(screen_width*2/3,0,screen_width,screen_height*2/3));
		smgr->drawAll();

		//
		smgr->setActiveCamera(camera[0]);
		//Set viewpoint to the first quarter (left top)
		driver->setViewPort(rect<s32>(0,0,screen_width*2/3,screen_height));
		//Draw scene

		// Images with partial viewport
		driver->draw2DImage(images, core::position2d<s32>(500, 500));

		if(receiver.GetMouseState().LeftButtonDown&&!old)
		{
			cout<<"X:"<<receiver.GetMouseState().Position.X<<endl;
			cout<<"Y:"<<receiver.GetMouseState().Position.Y<<endl;
			old=true;
		}
		if(!receiver.GetMouseState().LeftButtonDown&&old)old=false;

		guienv->drawAll();

		// Images with full viewport
		driver->setViewPort(rect<s32>(0,0,screen_width,screen_width));
		driver->draw2DImage(images, core::position2d<s32>(500, 500));
		guienv->drawAll();

		driver->endScene();
	}

	device->drop();

	return 0;
}

Btw, when posting code, please always kick out everything which has nothing to do with your question/bug. Takes you like an extra minute as you know which parts are necessary and then it's easier for others to read.

[SanekFPS]

understood. thanks for the help.