openCV & CvIrrCamTexture - distorted image

[EvilBob]

I've been playing with openCV 2.0 and Irrlicht 1.6, using Visual C++ 2008 Express. I have cut & pasted CvIrrCamTexture.h and CvIrrCamTexture.cpp from this thread by FreakNigh. I had to make the following changes to compile;

In CvIrrCamTexture.cpp, lines 22-26

Code: Select all

//make the texture
if(use_rgba_texture)
    this->irr_texture = this->driver->addTexture(core::dimension2d<s32>(cv_img->width, cv_img->height), unique_tex_name, video::ECF_A8R8G8B8);
else
    this->irr_texture = this->driver->addTexture(core::dimension2d<s32>(cv_img->width, cv_img->height), unique_tex_name, video::ECF_R8G8B8);

In my project, I've changed core::dimension2d<s32> to core::dimension2d<u32>, which is the same change I've been making to the IrrlichtDevice *device = createDevice(...) call in other projects.

When I run this, I get an object with a recognisable webcam image, but it is distorted/repeated. My wife says that it's the best I've ever looked on a webcam but that's besides the point



I'm trying to work out where in the process this is going wrong, and what elements I would need to study further and redress. I'm slightly out of my depth with this aspect of things and any pointers in the right direction would be gratefully received.

I am using the following code - again taken the previously noted thread, but without the extra meshes;

Code: Select all

int main(){

    IrrlichtDevice *device =  createDevice(video::EDT_OPENGL, dimension2d<u32>(640, 480), 16, false, false, false, 0);
    IVideoDriver* driver = device->getVideoDriver();
    ISceneManager* smgr = device->getSceneManager();
    IGUIEnvironment* guienv = device->getGUIEnvironment();


    ICameraSceneNode* camera = smgr->addCameraSceneNodeFPS();
    camera->setPosition(vector3df(60,10,0));
    camera->setTarget(vector3df(0,0,0)); 

    CvIrrCamTexture* cv_text_mgr = new CvIrrCamTexture(driver,0,0); 

    IBillboardSceneNode* our_bill_node = smgr->addBillboardSceneNode(NULL, dimension2d<f32>(25.0f, 25.0f),vector3df(0,0,0));
    our_bill_node->setMaterialFlag(video::EMF_LIGHTING, false); 
    our_bill_node->setMaterialTexture( 0, cv_text_mgr->getTexture() );

    while(device->run())
    {
        //update webcam texture
        cv_text_mgr->UpdateTexture();
      
        driver->beginScene(true, true, SColor(255,100,101,140));

        smgr->drawAll();

        driver->endScene();
    }
   
    device->drop(); 

    return 0;

}

[jawdy]

Yet another revival of an old post, for me... but I'm suffering the same problem.

On my main dev' machine, everything works perfectly, but when I transfer the release executable my laptop, I get the same problem as EvilBob.

Something else I noticed is that on my working machine, I've set the texture to draw as 2D in the bottom right hand corner, relative to the camera's image size.
If I switch over to my laptop, the image is distorted and it's as though the resolution of this distorted image is somehow larger than the source camera input.

Does anyone have any idea? Is Nigh about, could you potentially shed some light?

Cheers!

[bitplane]

Looks to me like the width is wrong, causing pixels to wrap around. I haven't looked at the project but I'd imagine that the webcam image isn't a power of two but your destination image is, and you don't pad it, or something similar.

[jawdy]

Thanks for the reply, BitPlane

The source image from the camera is 320 x 240 and the destination texture is the same resolution.
I thought it was the power of 2 problem - I've had that before, haha - but it doesn't seem to be the case...

I've just tried it on another "gaming" PC with a proper graphics card and it works flawlessly, same as my Dev PC. Only the little laptop, with it's embedded graphics chipset has any problems.
So, I honestly think it's not an Irrlicht/OpenCV problem, but hardware or at the very least driver/software.

Not very much of a useful answer in actually finding the problem, but when it works on two machines out of three (for me at least) I'd lean towards the lack of power in my laptop.

[bitplane]

Maybe your low power PC is using a different pixel format then? If the source and dest images use different bits per pixel, you'll see something like that.

An image and some code would save a lot of guesswork

[bartmanb]

Hi everybody,

I'm actually working on a project using irrlicht and OpenCV and 'im confronted to the same kind of problem. The rendered picture is distorted like EvilBob shows it in its capture.

Does anybody have a real issue to capture webcam frames using opencv and rendering with irrlicht ???

[bartmanb]

hi everybody,

I just want to add that in my personal computer, the picture rendered from the webcam is just fine... so i think that the problem provides grom the graphic card. Does anybody solve this problem ?

[polylux]

To me, this seems to be exactly the issue bitplane described.
Normally you get an image like this if you set up the params incorrectly, most likely width, height, and very well known: the depth (8 to 24 bits per pixel).
Check this, when your cam works normally with other programs it is veeery likely you'll find the problem in your code

[bartmanb]

Ok i'm looking for that but it's not normal that the program will work on my personal computer and not in another computer... that's the problem that i don't understand...

[hybrid]

Some machines can only provide a 16bit depth. So even if you use the same app on the two machines, you can get different results. You just need to check all the values you rely on.

[bartmanb]

I have tested many configurations (16 or 32bits, ECF_A1R5G5B5 or ECF_R5G6B5) but the result still the same...

Here is the code if somebody wan to help me :

Code: Select all

irr::video::ITexture* create_ITexture_from_CvCapture(CvCapture* capture, irr::video::IVideoDriver* driver)
{
   char unique_tex_name[50];

   //for irrlicht
   sprintf(unique_tex_name, "cv_image:%d", capture);

   //grab an image
   IplImage* img = 0;
   if(!cvGrabFrame(capture))
   {
      printf("Could not grab a frame\n");
      return 0;
   }
   img=cvRetrieveFrame(capture);

   //get these variables about the video
   int frameH    = (int) cvGetCaptureProperty(capture, CV_CAP_PROP_FRAME_HEIGHT);
   int frameW    = (int) cvGetCaptureProperty(capture, CV_CAP_PROP_FRAME_WIDTH);
   //printf("height : %d, width : %d ",frameH, frameW );
   //int fps       = (int) cvGetCaptureProperty(capture, CV_CAP_PROP_FPS);
   int fps=0;

   //make our texture
   irr::video::ITexture* m_poTileTexture = driver->addTexture(irr::core::dimension2d<irr::s32>(frameW, frameH), unique_tex_name, irr::video::ECF_A1R5G5B5);

   // read the pixels directly into the texture
   irr::u8* pixels = (irr::u8*)(m_poTileTexture->lock());
   irr::u8* ardata = (irr::u8*)img->imageData;
   int max_pixels = frameW * frameH;

   for(int i=0;i<max_pixels;i++)
   {
      *pixels = *ardata;
      pixels++; ardata++;
      *pixels = *ardata;
      pixels++; ardata++;
      *pixels = *ardata;
      pixels++; ardata++;

      pixels++;
   }

   m_poTileTexture->unlock();

   return m_poTileTexture;
}


int main( int argc, char **argv )
{
	// creation du device
	irr::IrrlichtDevice *device = irr::createDevice(    
        irr::video::EDT_OPENGL,
        irr::core::dimension2d<irr::u32>(320,240),
        32,false,true,false,0);
 
    // creation du driver
	irr::video::IVideoDriver* driver = device->getVideoDriver();	
    // creation du scene manager
	irr::scene::ISceneManager *sceneManager = device->getSceneManager ();							
 
	irr::scene::IBillboardSceneNode* webcam = sceneManager->addBillboardSceneNode();
	float distance=100; //distance de l'image à la camera
	webcam->setSize(irr::core::dimension2d<irr::f32>(distance*5.85,distance*4.35));
    webcam->setPosition(irr::core::vector3df(0,0,distance*3));
    webcam->setMaterialFlag(irr::video::EMF_LIGHTING,false);
	//irr::scene::ITriangleSelector* selector = sceneManager->createTriangleSelectorFromBoundingBox(webcam);
	irr::scene::ICameraSceneNode *camera = sceneManager->addCameraSceneNode();

	// GESTION OPENCV
	CvMemStorage *storage = 0;
    CvCapture* capture = 0;
    IplImage* frame = 0;
    //storage = cvCreateMemStorage(0);
    //capture = cvCaptureFromCAM(0);
	//irr::video::ITexture* CvImage = create_ITexture_from_CvCapture(capture, driver);
	//webcam->setMaterialTexture(0,CvImage);

	irr::video::ITexture* CvImage;
    int frames =0;
	while (device->run ())                         // boucle infinie
    {
		storage = cvCreateMemStorage(0);
		capture = cvCaptureFromCAM(0);
		CvImage = create_ITexture_from_CvCapture(capture, driver);
		webcam->setMaterialTexture(0,CvImage);
		
		driver->beginScene (true, true,irr::video::SColor (255,255,255,255)); // on demarre la scene
		
        sceneManager->drawAll ();                  // on calule le rendu
        driver->endScene ();                       // on affiche le rendu
    }
	cvReleaseCapture( &capture );
    device->drop ();                               // liberation de la memoire
}

[hybrid]

As you can see, you try to copy 3 of 4 bytes to 3 of 4 bytes. However, your texture format only has two bytes. This cannot work. You should check which format your camera image really has, and create your texture accordingly. Next, check the texture format (because it's not guaranteed to get this format) and copy the pixels (not bytes!) properly.

[bartmanb]

As you can see, you try to copy 3 of 4 bytes to 3 of 4 bytes. However, your texture format only has two bytes. This cannot work. You should check which format your camera image really has, and create your texture accordingly. Next, check the texture format (because it's not guaranteed to get this format) and copy the pixels (not bytes!) properly.

Ok thanks, but how can i check the format of the camera's image ? and also the texture format ?

[hybrid]

For the things in Irrlicht you can use the getColorFormat methods for images or textures. For OpenCV you'd have to check their API.