How to avoid clipping the view port rect?

[Andrey01]

I'm making some API for Minetest engine allowing to render and handle meshes in the screen space of the player camera. When some new mesh is added, it creates a separate scene manager for it (smgr->createNewSceneManager), adds to there the own camera and the mesh node itself. When rendering all meshes, it sets the screen-sized view port for each of them sequentially and moves its core::rect by the mesh position vector started from the screen center. However, since the view port size is equal to the screen one, after moving and setting it (driver->setViewPort()), the mesh is rendered distorted. I tracked how the originally set view port size is changed just getting it from getViewPort() directly after setting and it appears with another size. I guess because it goes beyong the screen, does it get clipped automatically by Irrlicht? Then e.g. how could I render the mesh locating around the screen border (that's the case where only the certain part of the mesh that is within the screen rect should be rendered)?

For example, I have an one mesh (steel ingot) with the original view port (highlighted by red color):


How it looks like after moving its view port rect taking into consideration that "clipping" effect (shift position vector: -1.0, 0.0):

[CuteAlien]

Maybe you are missing some aspect ratio changes? Your first image has 15/8 your second has something like 2 times 7.5/8 (with the images you posted here, I guess real ones are likely not exactly the same).

The camera always renders the full height of your given field of view (ICameraSceneNode::setFOV). The width the camera renders is then calculated from the aspect ratio (ICameraSceneNode::setAspectRatio). So as you noticed - whatever your viewport is, the parts you see in it are the same as long as you don't change those 2 values in the camera. Just scaled to your viewport. If you don't want that scaling you usually set your aspect ratio to the real aspect ratio of your viewport. Which then means some parts of the image width will be clipped.

edit: That's if I understand what you are doing here... not quite sure I did on reading this second time. Now I get the feeling that maybe you talk about camera shifting instead. Rendering an off-center part (like taking a larger render and then using one of it's corners. If that's the case - check ICameraSceneNode::setLensShift (maybe not yet in Minecraft, but easy to port).

[Andrey01]

Maybe you are missing some aspect ratio changes? Your first image has 15/8 your second has something like 2 times 7.5/8 (with the images you posted here, I guess real ones are likely not exactly the same).

The camera always renders the full height of your given field of view (ICameraSceneNode::setFOV). The width the camera renders is then calculated from the aspect ratio (ICameraSceneNode::setAspectRatio). So as you noticed - whatever your viewport is, the parts you see in it are the same as long as you don't change those 2 values in the camera. Just scaled to your viewport. If you don't want that scaling you usually set your aspect ratio to the real aspect ratio of your viewport. Which then means some parts of the image width will be clipped.

I did to update the aspect ratio every time when it was necessary to move the view port rect. Although the proportions stayed the same, but it caused changing the mesh scale (it became either smaller or bigger than the original one). It doesn't solve the problem which I described above. Also, if I moved it by (-1, 0) on the screen, it didn't appear on the screen border, but near of it.

edit: That's if I understand what you are doing here... not quite sure I did on reading this second time. Now I get the feeling that maybe you talk about camera shifting instead. Rendering an off-center part (like taking a larger render and then using one of it's corners. If that's the case - check ICameraSceneNode::setLensShift (maybe not yet in Minecraft, but easy to port).

No, the camera position (it's always to 0, 0, 0) doesn't change. The camera itself is added to the separate scene manager of each mesh object. What I am doing here is moving the view port rect through the screen depending on the mesh location whose size is always equal to the screen one. Why I do it at all: I could contain all meshes in the one manager, however I want to save the perspective constant because the meshes renders are actually just images (they are located on the screen, not in the view or world space).

[CuteAlien]

Sorry, either I don't get what you do or what you expect.

If you move an object left-right in 3D space with perspective camera then objects do change (if you are in a tunnel you can see both side-walls at once because of that). That's what perspective does. You can't move things to the side and have them stay the same size as rendered in the center. That's unless you work with orthographic views (which I think you don't).

And you either have a scaling or clipping going on when you change the viewport size. Those are the only 2 options. Well, 3 options as you can also decide to do the clipping vertically or horizontally.

Note that CCameraSceneNode does set it's original aspect ratio once automatically when it's created new. Maybe that gets in your way?

Maybe you have to write a quick example code so I understand what you are doing here, because I really don't get it.

Ow... or maybe I do get it ... you try to have a viewport which is partially "outside" the screen? That might get clipped. You can't render outside the screen when you render to the screen, not an Irrlicht thing (double-buffers needs memory etc, there is no screen memory for non-screen areas). So the viewport you set won't be the viewport you get then I guess. If you need that you need render-target textures.
Or with a combination of the things mentioned above - aspect ratio and the new lens shifting (you would have to shift the lens to render off-center and set the aspect ratio to the clipped area). Thought I do have trouble imagining a use-case for that (if it's about buffering stuff you likely need render-target-textures anyway, but I really don't get how this could work with real perspective).

[Andrey01]

As I already said before, the screen mesh objects doesn't move around their scenes as well as their cameras. Only their view port rectangles shift happens if the coords of the object were changed. Ok, I show my code snippets and explain what they do.

I have a HUD scene class which exists per a mesh object:

Code: Select all

class HUDScene : virtual public IReferenceCounted
{
public:
	HUDScene(scene::ISceneManager *mgr, Client *client,
		const HudElement *elem=nullptr, const v3f &mesh_pos=v3f(),
		const v3f &mesh_rot=v3f(), bool lighting=false, bool is_hand=false);

	~HUDScene();

	----------------------------
	
	void step(f32 dtime);
private:
	Client *m_client;
	scene::ISceneManager *m_smgr;
	scene::ICameraSceneNode *m_cam;
	MeshHUDSceneNode *m_hud_node;

	// Constant rectangle. Actually is (1920, 1008)
	core::recti m_original_rect;
	// The rect which is continuously updated
	core::recti m_rect;
	// Position on the screen in the range: [-1.0..1.0]
	v2f m_scene_pos{0.0f, 0.0f};

	const HudElement *m_hud_elem;

	// Last set 'scale.X' hud def field
	v3f m_last_relative_scale{1.0f};
	// Mainly WIELD_SCALE_FACTOR/WIELD_SCALE_FACTOR_EXTRUDED
	// calculated by getOrCreateMesh()
	v3f m_last_constant_scale{1.0f};
};

It actually implements functions of the container and management over the MeshHUDSceneNode (mesh object) and ordinary ICameraSceneNode, that also saves an own scene manager in which those previous objects are added. That class in its each step() call, which is continuously gets called outside, retrieves a new screen mesh position ranging from 0.0f - 1.0f (it gets from the Lua mods scripts), rotation in degrees and relative scale.

Code: Select all

void HUDScene::step(f32 dtime)
{
	-------------------------
	// Updates both end points of the rect depending on new m_screen_pos
	updateRect();

	// Updates the rotation of the mesh object
	m_hud_node->setRotation(m_hud_elem->rotation);

	// Updates the scale
	updateScale();

	-------------------------
}

How updateRect() works:

Code: Select all

void HUDScene::updateRect()
{
	// Translates m_scene_pos to [0..1] range
	v2f uv_pos = m_scene_pos / 2.0f + 0.5f;

	// Compares it with the external one which gets from Lua side.
	// If they are equal, it is unnecessary to update it
	if (uv_pos == m_hud_elem->pos)
		return;

	// Translates the new position to [-1..-1] and saves it
	m_scene_pos = (m_hud_elem->pos - 0.5f) * 2.0f;

	// Calculates the shift vector in the pixel coords starting from the screen center
	v2s32 new_pos(s32(m_scene_pos.X*m_original_rect.getWidth()/2.0f),
		s32(m_scene_pos.Y*m_original_rect.getHeight()/2.0f));

	// And adds it with the constant screen rectangle. Thus we gets the shifted rectangle relatively the screen center.
	m_rect = m_original_rect + new_pos;
}

The following function is called in the general class that saves and manages all mesh objects.

Code: Select all

void Hud::drawMeshes()
{
	driver->clearBuffers(video::ECBF_DEPTH);

	core::recti oldViewPort = driver->getViewPort();

	// Loop through all mesh objects
	for (const auto &p : m_hud_scenes) {
		// Gets a current view port rectangle and set it
		driver->setViewPort(p.second->getRect());
		// Note: "new_rect" != p.second->getRect() !!! That actually gets clipped if 
		core::recti new_rect = driver->getViewPort();
		
		// Draws the mesh object and its camera for the given mesh
		p.second->getSceneManager()->drawAll();
	}

	driver->setViewPort(oldViewPort);
}

As I noted in the comment above "new_rect" line, the rect which is set is not equal to the original rect. It gets clipped by Irrlicht itself because its borders get beyond the screen borders. And since the displayed viewport rectangle has distorted proportions, therefore the mesh gets distorted also. That is certainly not what I expect. I want the viewport size being unchanged after setting allowing it appear partially beyond the screen and correspondingly the mesh itself also.

To clarify finally what result I want to have, I will show once again how it looks like now after moving at (-1, 0) (in the device coords):


and how it actually should look (edited in Gimp):


Hope, I explained very in detail and comprehensively my issue

[CuteAlien]

Isn't that what I wrote after the "Ow... or maybe I do get it ..."?

I'm not quite sure what would happen. I searched a bit around the web and found answers from "gpu will just clip it, no problem" to "this is badly defined in OpenGL" to "might depend on the graphic card what happens". If you want to try disabling the clipping in Irrlicht try the following:

In COpenGLDriver::setViewPort remove the 2 lines with rendert (aka the clipping to rendertarget borders).

But I have no idea if this will work in general. Maybe it does, maybe only for rendering to screen but not render-targets (or vice versa), or maybe it just works and that clipping there is useless. It was in there since first svn version, so I have no commit message or any further info about those lines and also could only write my own experiments and then get a ton of people to try the results. Or maybe you can find more info on the web? To me it sounds a bit risky as you basically tell the driver to render outside of the screen-memory and then hope it clips correct, but without changing aspect ratio. (edit: I checked glViewport documentation and it mentions no restrictions ... so maybe it's indeed fine and Irrlicht should get rid of those lines).

The other 2 solutions are:
a) Use render target textures and draw them (they get clipped correct).
b) Use the mixture of changing aspect ratio (to the clipped area) and then use lens-shifting in the camera to get an off-center rendering (you have to port the lens-shifting code first to Minetest as this was added after you split, but it's pretty trivial).

[Andrey01]

Isn't that what I wrote after the "Ow... or maybe I do get it ..."?

I'm not quite sure what would happen. I searched a bit around the web and found answers from "gpu will just clip it, no problem" to "this is badly defined in OpenGL" to "might depend on the graphic card what happens". If you want to try disabling the clipping in Irrlicht try the following:

In COpenGLDriver::setViewPort remove the 2 lines with rendert (aka the clipping to rendertarget borders).

But I have no idea if this will work in general. Maybe it does, maybe only for rendering to screen but not render-targets (or vice versa), or maybe it just works and that clipping there is useless. It was in there since first svn version, so I have no commit message or any further info about those lines and also could only write my own experiments and then get a ton of people to try the results. Or maybe you can find more info on the web? To me it sounds a bit risky as you basically tell the driver to render outside of the screen-memory and then hope it clips correct, but without changing aspect ratio. (edit: I checked glViewport documentation and it mentions no restrictions ... so maybe it's indeed fine and Irrlicht should get rid of those lines).

The other 2 solutions are:
a) Use render target textures and draw them (they get clipped correct).
b) Use the mixture of changing aspect ratio (to the clipped area) and then use lens-shifting in the camera to get an off-center rendering (you have to port the lens-shifting code first to Minetest as this was added after you split, but it's pretty trivial).

Thanks for the help! I removed the clipping to the screen rect in COpenGLDriver::setViewPort() and that worked as I wanted:


However, that is the Irrlicht changes, not Minetest engine ones. So I will need to make another PR to the IrrlichtMt repo before pusblishing one to the engine repo.

[CuteAlien]

Yeah, the more I thought about it the more certain I was this should work. It's only a transformation and unrelated to the clipping. But still wondering why it's in there then, maybe to make harder to mess up when working with rtt's? So people don't have to care about it most of the time and it still works?
If I find some energy/time I'll experiment a bit with it to see if we can remove it in Irrlicht in general or maybe add another flag about clipping to the function.

[CuteAlien]

I've patched Irrlicht in [r6636] for this by adding a parameter to setViewPort. Thought it only works with OpenGL (and NullDriver which does nothing...), all other drivers will not handle it correctly (D3D 9 can handle it with larger coordinates, but not with negative ones, software drivers will crash).
For a driver independent solution you will have to use one of the other 2 options.
Not tested yet with ogl-es drivers, but will add it there as well.

[Andrey01]

Thanks for the patch.

I have a small question unrelating to the topic (I just don't want to create a separate thread for it): is there some support of GLSL > 1.2 version currently in Irrlicht? I would like to try to use in and out specifiers in the vertex and fragment shaders as I need in passing a multiple of vertex attributes (a few colors per a vertex in my case such way without using long ago outdated gl_Color that represents the only color). I didn't find in the official docs any API allowing to set attribute locations, e.g. in IMaterialRendererServices (that allows only to set uniforms so far through setVertex/PixelShaderConstant().

[CuteAlien]

Take what I say with grain of salt as this is not an area of expertise for me as I always got away with what Irrlicht offered so far. I think GLSL versions are not the problem for this as you can require any GLSL version you want in the shader (with something like "#version 430 compatibility" in the first line of the shader). The problem you have with Irrlicht is that the vertex format is very restricted and there are really only 3 different types: S3DVertex, S3DVertex2TCoords and S3DVertexTangents. With the last one being the largest amount of data you can send per vertex in Irrlicht. The data in those structs there can be re-used in different ways and that's some way to get a bit around this limitation (like passing colors in the Tangent/Binarmal which can be calculated on GPU instead). But sadly there is no way in Irrlicht to define your own flexible vertex format. Those 3 structs are all you got when it comes to per vertex data.