Eye toy project

[YoJi]

Hello
I'm making a eye toy project ( dragon who is "driver" with your arms ) but i have a FPS Problem , to can making operation with my webcam i making some frame ... I pick up the frame in the loop of the main ... a webcam have a max FPS of 30 but 30 FPS to run 3D animation is too slow ...

Code: Select all

void Jeu(IGUIEnvironment* guienv, IVideoDriver* driver , ISceneManager* smgr,IrrlichtDevice* device,CvCapture* capture)
{
    /** Un piti cadre autour de l'image de Webcam **/ 
    guienv->addImage(driver->getTexture("cadre.png"),core::position2d<s32>(341,20));

    /** Chargement Dragon **/
    IAnimatedMeshSceneNode* dragon = DraGon.AffichageDragon(smgr, driver);
    
    /** Gestion Caméra 3D **/
    ICameraSceneNode* camera = Irrlichtdessin.GestionCamera(smgr, driver, dragon);

    /** Affichage Terrain **/
    Irrlichtdessin.AffichageSkybox(smgr, driver);
    ITriangleSelector* selector = Irrlichtdessin.AffichageTerrain(smgr, driver , camera);
	Irrlichtdessin.GestionCollisionTerrain(smgr, driver, camera, selector);

    /** Enleve le curseur de la souris **/  	
    device->getCursorControl()->setVisible(0);


 

    /** Initialisation variables utiles dans la boucle **/
    Imagebool=-1;   
    int lastFPS = -1;
    ITexture * TextureWebcam;
    IplImage * frame;
    
    
    while(device->run())
    if (device->isWindowActive())
    {
        
        /** Scene Draw **/
    	driver->beginScene(true, true, SColor(0,0,0,0)); 
        

        smgr->drawAll();
    	guienv->drawAll();
    	
        /** Grab Webcam image and do motion operation **/        
        
       Webcam(guienv,driver,smgr,device,capture,TextureWebcam,frame);
        
        
        driver->endScene();
        
    	/** Pour récuperer le FPS **/
    	int fps = driver->getFPS();
    	if (lastFPS != fps)        // Pour eviter d'afficher toujours le meme FPS si celui ci reste inchangé
		{
			stringw str = L"Irrlicht Engine - BiGMaC's Adventure ["; str += driver->getName();str += "] FPS:";str += fps;
            device->setWindowCaption(str.c_str());
			lastFPS = fps;
		}
    }

  
}

my fps with webcam : 6
without : 600

i don't have 100% CPU process and don't have all my Ram use

Webcam decrease my fps by 100 !!!
May be someone have a solution to my problem ?

[vitek]

This is closely related to your original post http://irrlicht.sourceforge.net/phpBB2/ ... hp?t=11937.

A big chunk of your time is going to be spent copying pixel data from the webcam image to your texture. If there are ways to do less pixel copying, that is a good thing. Here are two ways to improve it...

If you make the webcam image smaller. Fewer pixels to copy means less time spent copying them. As your image grows, the time spent copying pixels will increase exponentially. i.e. a 512x512 image takes 4x as long to copy as a 256x256 [there are 4 times as many pixels].

If your camera only updates at 30hz, then it is useless to regenerate the webcam image every render. Seperate your the image generation functionality from the image rendering. Render the image every frame, but update it at some constant frame rate...

It might also be a good idea to avoid hitting the heap [allocating memory] every time you want to read data from the webcam. You should allocate the texture once and then lock/update/unlock it. This would also avoid the leak that I believe you may be experiencing because you call driver->addTexture repeatedly with the same texture name.

If the leak I mentioned is actually a leak, then fixing it will improve speed also.

Code: Select all

  ITexture* webcam = driver->addTexture(core::dimension2d<s32>(256, 256), "WebCam", video::ECF_A8R8G8B8);

  const u32 camera_fps = 30;
  const u32 ms_at_camera_fps = 1000 / camera_fps;

  u32 then = device->getTimer()->getRealTime();
  while(device->run()) 
  {
    if (device->isWindowActive()) 
    { 
        
        /** Scene Draw **/ 
       driver->beginScene(true, true, SColor(0,0,0,0)); 
       smgr->drawAll(); 
        
        /** Grab Webcam image and do motion operation **/
        driver->draw2DImage(webcam, position2d<s32>(x, y));

       // Draw GUI last so it renders over the 2d webcam
       guienv->drawAll();

       // only update the webcam texuture every 30hz.
       u32 now = device->getTimer()->getRealTime();
       if (ms_at_camera_fps < (now - then))
       {
         then = now;

         void* lock = webcam->lock();
         if (lock)
         {
           // update the webcam textures pixel data

           webcam->unlock();
         }
       }

[YoJi]

Hi Vitek

how do you want to update the texture ? , i update with driver->addtexture();

[vitek]

The pointer returned by webcam->lock() in my example above is a pointer to the texture pixel data. You need to ask the texture for the format and then write the data to the texture as is appropriate for the pixel format.

[sdi2000]

update ur texture between these lines from vitek

Code: Select all

   void* lock = webcam->lock();
         if (lock)
         {
           // update the webcam textures pixel data
           //HERE!!
           memcpy(lock, yourImageBuffer, sizeof youriamgebuffer);

           webcam->unlock();
         } 

be sure that you image size has the right size. irrlicht resize all textures to an optimal size.
use the function in irrlicht or ask the old texture size and then resize the image. if you dont resize ur image u get an access violation if ur grabed picture is bigger than the irrlicht texture size!

the irrlicht function to determine the new size

Code: Select all

int getTextureSizeFromImageSize(int _size)
{
	int ts = 0x01;

	while(ts < _size)
		ts <<= 1;

	if (ts > _size && ts > 64)
		ts >>= 1;

	return ts;
}

call them for x and y and resize your image with the results.

then u can copy ur image with memcopy. if u want a fast copy routine with sse2 support download the virtual dub project and look into the source.

tip. dont call the webcam and the irrlicht engine in the same thread. create a new tread for the cam and write in a buffer like linux v4l.
update ur image only if the buffer is filled.
Use codevis to grab images from a webcam. codevis has callbacks and simple examples. i watch tv with the codevis lib and irrlicht with. up to 350fps

[YoJi]

I use Opencv to grab a frame ...
My scene is lagging when it's grab a frame that better but ... :'(

I can have 400 fps if i put a very little camera_fps but sometimes the scene is lagging

[sdi2000]

oups u need ur lib... first read then answer oli... my error it´s okay u want to make a game with a cam or some interactions with the 3d world...

[hybrid]

As your image grows, the time spent copying pixels will increase exponentially. i.e. a 512x512 image takes 4x as long to copy as a 256x256 [there are 4 times as many pixels].

No, actually this is not exponential, but quadratic (or in general polynomial). Anyway, its much work.

[vitek]

It is exponential in the sense that it involves exponents not natural logarithms. But yes, it is expensive.

[YoJi]

I want thanks you for your help

Thank you very much

[sdi2000]

I can have 400 fps if i put a very little camera_fps but sometimes the scene is lagging

try a new webcam driver. i had the same problem, but after an driver update the frames goes up. (the solution in my case was... the company build a driver with directx memory accessing and not using the winapi calls)

some other speed features in irrlicht windowmode are: let irrlicht only render at onidle and onpaint mode (testet under windows) because some other process need also cpu time. an sleep is not to advise, let the system handle it.
if irrlicht runs in onidle and onpaint mode the speed goes up.
in full screen mode u can render without onidle and onpaint, because the system call for fullscreen disable partly the rendering time for any window in background.
but if u want this u must fix the engine... this is a big drawback =)
fix... recompile ... and so on...
hm anyone another solution? vitek? hybrid? nico?

[Guest]

Code: Select all

   void* lock = webcam->lock();
         if (lock)
         {
           // update the webcam textures pixel data
           //HERE!!
           memcpy(lock, yourImageBuffer, sizeof youriamgebuffer);

           webcam->unlock();
         } 

[/quote]


memcpy(lock, yourImageBuffer, sizeof youriamgebuffer);
???? i don't understand what type is yourImageBuffer ...
so i don't knwo how to update the webcam texture
what can i do with the void* return by webcam->lock; ??

[YoJi]

ops i forgot to log in ^^ the previous post was mine

[sdi2000]

the void* is the pixel buffer of current used image by the texture.
u can copy your pixels in this buffer.
be sure that the buffers has the same format...

with lock you can access through the buffer and with unlock the engine copy the pixels into the texture

void *t = texture->lock();

memcpy(t, imagePixelBuffer, sizeofImagePixelBuffer);

texture->unlock();

[vitek]

The memcpy will only work if the two images have the same pitch [bytes per row] and size. i.e. both images would need to be the same format and the exact same size. Usually you have to just write loops that reads pixels from the source format to the destination.

Your code should look something like the makeColorKeyTexture code in CNullDriver.cpp. If you wanted to support scaling the texture, there is code for that in CImage.cpp.

Travis