Irrlicht Beginner 2D. Live video tweaking.

[irrlicht_ist_toll]

Hi all I've used several 3D engines in the past and have some experience with 3d meshes and all.
But I'm just starting with Irrlicht and win32, getting away from .NET

For now I'm using irrlicht to show some 2D stuff on a texture on a viewport as part of my
visual recognition project.



This is really fun, I'm just getting the live video frame data from a webcam into a buffer, and after doing any desired modification to the data, I lock and update the texture.

Code: Select all

 ptr= (unsigned int*)(texture->lock(ETLM_READ_WRITE,0)); 
 
   for(i=0;i<size;i++){ ptr[i] = source[i]; }   //copy
   //for(i=0;i<size;i++){ ptr[i] = 0xFFFF0000;} //red-fill-test
 
 texture->unlock();//

One of the best parts is that the program has worked consistently on different computers I've tested on, without having to install anything.

[chronologicaldot]

Cool beans
What did you use to decode the video?

[irrlicht_ist_toll]

There is this whole convoluted formula to go from YUY2 to RGB. See below.
YUY2 goes like y0, u , y1, v,
so Y luma is per-pixel while uv is shared by two pixels.
The conversion is pretty convoluted but it's performing surprisingly fast.
I guess one could also use a GPU shader on the application, but i haven't tried that since it doesn't really benefit me here, as i need the RGB data in application memory to manipulate it.

Code: Select all

 
LRESULT CALLBACK Camera_onFrameCall(HWND hWnd, LPVIDEOHDR lpVHdr){
 
 //Copy rgb version to application buffer
 //This convertion is pretty long but still performed really fast.
 //still need to optimize this 
 
 unsigned int s= lpVHdr->dwBufferLength;
 
 if(camera.user_frame_array_rgb != NULL){//if user set pointer
   int rgb,r,g,b,p;
   int y0,u,y1,v, C, C2, D, E;
   //
   p=0;
   for(i=0;i<s;i+=2){
     y0 = lpVHdr->lpData[p];
      u = lpVHdr->lpData[p+1];
     y1 = lpVHdr->lpData[p+2];
      v = lpVHdr->lpData[p+3];
     p +=4;//yuyu2 is 16bpp
     if(p>=s){break;}
     //
     C  = y0 - 16;  
     C2 = y1 - 16;  
     D  = u  - 128;
     E  = v  - 128;
     //
     //Pixel 1
         r = (298*C+409*E+128)/256;
     if(r<0){r=0;}if(r>255){r=255;}
     g = (298*C-100*D-208*E+128)/256;
     if(g<0){g=0;}if(g>255){g=255;}
     b = (298*C+516*D+128)/256;
     if(b<0){b=0;}if(b>255){b=255;}
         camera.user_frame_array_rgb[i]=(r<<16)|(g<<8)|(b);  //rgb
     //
     //Pixel 2
         r = (298*C2+409*E+128)/256;
     if(r<0){r=0;}if(r>255){r=255;}
         g = (298*C2-100*D-208*E+128)/256;
     if(g<0){g=0;}if(g>255){g=255;}
         b = (298*C2+516*D+128)/256;
     if(b<0){b=0;}if(b>255){b=255;}
     camera.user_frame_array_rgb[i+1]=(r<<16)|(g<<8)|(b);  //rgb
     //
     //camera.user_frame_array_rgb[i]=0xFFFF0000;//red-fill-test
   }//
 }//
}

[chronologicaldot]

Messy indeed. Good job at making it work.