Page 1 of 1
Irrlicht Beginner 2D. Live video tweaking.
Posted: Tue Oct 13, 2015 2:05 am
by irrlicht_ist_toll
Hi all I've used several 3D engines in the past and have some experience with 3d meshes and all.
But I'm just starting with Irrlicht and win32, getting away from .NET
For now I'm using irrlicht to show some 2D stuff on a texture on a viewport as part of my
visual recognition project.
This is really fun, I'm just getting the live video frame data from a webcam into a buffer, and after doing any desired modification to the data, I lock and update the texture.
Code: Select all
ptr= (unsigned int*)(texture->lock(ETLM_READ_WRITE,0));
for(i=0;i<size;i++){ ptr[i] = source[i]; } //copy
//for(i=0;i<size;i++){ ptr[i] = 0xFFFF0000;} //red-fill-test
texture->unlock();//
One of the best parts is that the program has worked consistently on different computers I've tested on, without having to install anything.
Re: Irrlicht Beginner 2D. Live video tweaking.
Posted: Sat Oct 17, 2015 2:14 am
by chronologicaldot
Cool beans
What did you use to decode the video?
Re: Irrlicht Beginner 2D. Live video tweaking.
Posted: Mon Oct 19, 2015 3:43 am
by irrlicht_ist_toll
There is this whole convoluted formula to go from YUY2 to RGB. See below.
YUY2 goes like y0, u , y1, v,
so Y luma is per-pixel while uv is shared by two pixels.
The conversion is pretty convoluted but it's performing surprisingly fast.
I guess one could also use a GPU shader on the application, but i haven't tried that since it doesn't really benefit me here, as i need the RGB data in application memory to manipulate it.
Code: Select all
LRESULT CALLBACK Camera_onFrameCall(HWND hWnd, LPVIDEOHDR lpVHdr){
//Copy rgb version to application buffer
//This convertion is pretty long but still performed really fast.
//still need to optimize this
unsigned int s= lpVHdr->dwBufferLength;
if(camera.user_frame_array_rgb != NULL){//if user set pointer
int rgb,r,g,b,p;
int y0,u,y1,v, C, C2, D, E;
//
p=0;
for(i=0;i<s;i+=2){
y0 = lpVHdr->lpData[p];
u = lpVHdr->lpData[p+1];
y1 = lpVHdr->lpData[p+2];
v = lpVHdr->lpData[p+3];
p +=4;//yuyu2 is 16bpp
if(p>=s){break;}
//
C = y0 - 16;
C2 = y1 - 16;
D = u - 128;
E = v - 128;
//
//Pixel 1
r = (298*C+409*E+128)/256;
if(r<0){r=0;}if(r>255){r=255;}
g = (298*C-100*D-208*E+128)/256;
if(g<0){g=0;}if(g>255){g=255;}
b = (298*C+516*D+128)/256;
if(b<0){b=0;}if(b>255){b=255;}
camera.user_frame_array_rgb[i]=(r<<16)|(g<<8)|(b); //rgb
//
//Pixel 2
r = (298*C2+409*E+128)/256;
if(r<0){r=0;}if(r>255){r=255;}
g = (298*C2-100*D-208*E+128)/256;
if(g<0){g=0;}if(g>255){g=255;}
b = (298*C2+516*D+128)/256;
if(b<0){b=0;}if(b>255){b=255;}
camera.user_frame_array_rgb[i+1]=(r<<16)|(g<<8)|(b); //rgb
//
//camera.user_frame_array_rgb[i]=0xFFFF0000;//red-fill-test
}//
}//
}
Re: Irrlicht Beginner 2D. Live video tweaking.
Posted: Tue Oct 20, 2015 2:26 am
by chronologicaldot
Messy indeed. Good job at making it work.