Display 2D image as 3D surface

[dadreamer]

Hi, everyone!

My task is not so closely associated with the game development, it's likely image processing. I'm very new to Irrlicht and 3D manipulations so maybe the solution to my problem is very simple. Well, I have grayscale image from 3rd party application. This image is in form of 8-bit 2D array where each element represents pixel intensity in range 0...255. It may look like this:



I need to convert this 2D image to 3D image, it looks like this:



That is, in final image z-coordinate corresponds to pixel intensity. Further I need to display the result in window, which would have an ability to rotate an image (by mouse click or keyboard).

I explored some Irrlicht examples but they seem to be slightly complex for my goal. I was able to display simple line and point in window but I don't know how better to translate 8-bit array to 3D image. I think, I need to create array of vertexes and then display it with drawVertexPrimitiveList. Is it right way? Wouldn't it hang up (work very slowly) when I create ~16 000 vertexes? Initial picture's resolution may be larger, for instance 700*700 = 490 000 points. Which way is the most optimal?..

Thanks.

[hybrid]

Before some good ways can be proposed, I guess a little more details about the 2d array would help. Is every pixel really to be converted to a distinct vertex? In your example, it looks more like a 1D array, as each column has only one value across the whole column. In case you have multiple pixel values per column, is the value linear or can it change arbitrarily?
drawVertexPrimitiveList is only necessary when rendering special geometry types, such as lines or reduced triangle strips. Otherwise, use of the SMesh/SMeshBuffer structure is of certain advantages. Most important, you can enable hw buffers that way, making rendering even with huge (static) structures quite fast. Example 23 (or near) is showing the use of user-defined SMesh structures.
Depending on your requirements, also the use of shaders could be interesting, as you have some kind of displacement map here. The vertex shader can offset vertices based on the texture values, which directly maps to your scenario. However, you still need the proper number of vertices in the beginning. The latter could be handled by geometry shaders, though, so maybe you can do all this with a simple quad rendering and highly dedicated shaders.
We simply need some more info here - also about the update frequency of the mesh / the texture and whether you manually edit vertex positions (in code or with the mouse) or if everything could be done on the GPU.

[dadreamer]

In fact, I have given first image as an example - such scans are coming from my camera. I collect them and unite into resulting array, which of course can contain different pixel values across one column. When I'm scanning an object, there may be some defects (scratches etc.) and it looks like dark spots on the scan. So, it is desirable to me that 3D image matches 2D image as close as possible. I can accept small degradation or distortion of 3D though. My second picture is a screenshot from 3rd vendor's application. It's compiled and there are no sources for it. I just know it uses opengl32.dll and it builds 3D pictures without any delays.
I will take a look at example #23 and SMesh/SMeshBuffer structures. For the time, I know a little about rendering and shaders. Could you suggest me how to enable/activate them? Answering to last two questions... There's no need to change something in 3D picture, just display it, rotate by mouse/arrow keys and maybe zoom. We can change initial picture a little if it helps for further processings. Update time may be near 300-400 ms, so that is not very conspicuous. I just don't want to have glitches and delays in output process.

[smso]

You may display the 2D image as terrain in irrlicht, like:

Code: Select all

    // add terrain scene node
    scene::ITerrainSceneNode* terrain = smgr->addTerrainSceneNode("HEIGHT_MAP.png"); // your grey color image here:
    terrain->setScale(core::vector3df(50.0f, 10.0f, 50.0f));
    terrain->setMaterialFlag(video::EMF_LIGHTING, false);
    
    terrain->setMaterialTexture(0, driver->getTexture("TEXTURE.jpg"));
    terrain->setMaterialTexture(1, driver->getTexture("DETAILMAP_TEXTURE.jpg"));
        
    terrain->setMaterialType(video::EMT_DETAIL_MAP);
    terrain->scaleTexture(1.0f, 20.0f);
 

Regards,
smso

[markmze]

or else, i think this could help you : http://irrlicht.sourceforge.net/forum/v ... ap#p280769
It's a "normal map", that you can apply to a plane

[dadreamer]

Thank you all for the answers! Currently I'm playing around with example #23 and SMeshBuffers. I'm not native C++ programmer so things are moving very slowly. Have tested terrain variant, it works but has some limitations. I'll consider it as last variant.

markmze
Did you mean functions set(Vertex/Pixel)ShaderConstant? I can't understand where I should place my array and what to do with it to get everything working...

It seems I need a wrapper for Irrlicht because it's very complex

[markmze]

By the way, forget about what I said. Juste use it as a heightMap as smso said, pretty easy. In fact, i'm wondering why you guys call normal map "height maps". Nevermind, It's explained here : http://irrlicht.sourceforge.net/forum/v ... hp?t=43377
And it's already in one of the tutorials, check at your media folder, there is some different heightmap. Probably for the terrain tutorial

If you don't need LOD you can use the terrain mesh.

Code: Select all

 smgr->getGeometryCreator()->createTerrainMesh(video::IImage* texture,
            video::IImage* heightmap,
            const core::dimension2d<f32>& stretchSize,
            f32 maxHeight, video::IVideoDriver* driver,
            const core::dimension2d<u32>& defaultVertexBlockSize,
            bool debugBorders=false) const =0; 

Pass in the heightmap as second parameter (first texture is diffuse map). This will automagically create the meshes from your heightmap. Supports arbitrary image sizes etc.

[hybrid]

Yes, in case you want to test the terrain thingy, you should definitly use the terrain mesh, not the terrain scene node. The latter is optimized for huge terrains, which don't need to be exact in their representation. Terrain mesh is also able to provide arbitrarily large heightmaps.