Viewports and Raycasts

[Bate]

Hey everybody,
I'm currently trying to figure out how to do a raycast for collision detection inside a viewport that is rendered on top of another viewport.


So how could I make the usual cam-to-cursor ray...

Code: Select all

  line3df ray = smgr1->getSceneCollisionManager()->getRayFromScreenCoordinates(device->getCursorControl()->getPosition(), smgr1->getActiveCamera());
  vector3df collPoint;
  triangle3df collTriangle;
  SelectedNode = smgr1->getSceneCollisionManager()->getSceneNodeAndCollisionPointFromRay(ray, collPoint, collTriangle, 0, 0);

...hit a node inside the 2nd smgr/viewport if the cursor hovers its rect? Or is there a totally different approach I should use?

Code: Select all

  // Render loop
  CameraUpdate(cam1);

  driver->beginScene(true, true, SColor(0, 0,0,0));
    smgr1->drawAll();
    guienv->drawAll();

    driver->setViewPort(recti(100, 100, 100 + 512, 100 + 512));
      smgr2->drawAll();
    driver->setViewPort(recti(0, 0, 1024, 768));
  driver->endScene();

Thanks in advance.

[Lonesome Ducky]

I think you'll have to do ray collision testing in the part where the camera is facing where you want. So if it's in the smaller viewport you want to test, do the ray testing before you set the camera to the bigger viewport.

[Bate]

Hm, but where would I perform the cast exactly? I want it to happen onclick so that's why I do it in my eventreceiver. Also, I don't change the camera at all since each smgr has its on camera (and the camera of smgr2 is fixed).

Anyhow, I simply tried to cast using the 2nd cam/smgr and the cursor position. This basically works but the collision is wrong, so there are hits when there shouldn't be and vice versa.

Code: Select all

recti smallviewport = recti(0,0,512,512);

[...]

// on click
if ( smallviewport.isPointInside(device->getCursorControl()->getPosition()) )
{
    line3df ray2 = smgr2->getSceneCollisionManager()->getRayFromScreenCoordinates(device->getCursorControl()->getPosition(), smgr2->getActiveCamera());
    vector3df collPoint2;
    triangle3df collTriangle2;
    SelectedNode = smgr2->getSceneCollisionManager()->getSceneNodeAndCollisionPointFromRay(ray2, collPoint2, collTriangle2, 0, 0, true);

    if (SelectedNode) mySound->play(SFX_CLICK_1);
}

Obviously, you cannot use the cursor position directly since it is the position used for camera1 and the full screen only. I assume that it should be possible to calculate the relative cursor position for camera2 so that I could use that calculated position to perform the cast on camera2 just like for camera1 (if the cursor is inside the viewport rect).

However, since not only the sizes of the viewports are different but also the aspect ratios of the cameras I have no clue how to calculate that.
camera1 : 4.0f/3.0f , recti(0,0,1024,768)
camera2 : 1.0f , recti(0,0,512,512) or in general any rect

Feels like there must be a way but hell -- I don't get it. Trial & error didn't help much either.

I could really need some help here, please

[Lonesome Ducky]

Aspect ration is width/height. Anyway, to find the right mouse position, just subtract the rectangles upper left corner position from the cursor position. That will give you the mouse position relative to that viewport.

[Bate]

Thank you, it's not work though (I've already tested several ideas).

Here's where I get a hit with

Code: Select all

recti viewrect = recti(0,0,512,512);
// this doesn't do anything in my example, same as using the mouse coords directly
vector2di test = device->getCursorControl()->getPosition() - viewrect.UpperLeftCorner;

line3df ray2 = smgr2->getSceneCollisionManager()->getRayFromScreenCoordinates(test, smgr2->getActiveCamera());

blue : 1024x768 viewport (camera1 aspect ratio 4/3)
red : 512x512 viewport (camera2 aspect ratio 1.0)
green : cube mesh, centered, face orthogonal to camera2
black : hits inside this rect

that's why I don't get it seems to be a little more complicated.

[Lonesome Ducky]

Hmm, it looks like it's still doing the ray testing in the larger viewport, perhaps you can show us all of the code giving you a problem so one of us can better help?

[Bate]

Ah alright, i've got it

Code: Select all

vector2di test = device->getCursorControl()->getPosition();
test.X = test.X * 2;
test.Y = static_cast<s32>(test.Y * 1.5f);

there's a little inaccuracy because of the integer cast. so it's better to use the same aspect ratio of 4/3 for the 2nd viewport.

Thanks for your efforts Ducky.

[hybrid]

Ok, time to dig this up. I was just working on some other bugs in Irrlicht, which are related to view ports. As it showed up, the bug was not in viewport, but in the interpretation of the collision results.
I think that we need to define whether the collision manager results are respecting the viewport, or are based on the full screen. I think the results should be correct when being used with the currently applied viewport. I.e., if I do a collision check, and render to that position, the result should be corectly positioned inside the viewport. This seems to be the case here, but I guess we need some discussion about that. Any opinions?

[slavik262]

I'd say that it should be relative to the viewport, seeing as that's the "window into the world" you're drawing anyways.