android port and qualcomm AR

ronaldylee · Post by **ronaldylee** » Mon Mar 21, 2011 3:29 am

Hi,

I am playing around with Qualcomm AR lib and trying to make it work with Irrlicht android port. I play with the Sydney sample and with some hacking around I can kind of make the 3D model move relative to an image (Qualcomm calls it Trackable). But I couldn't get the texture working on the model. It seems like it uses the image fed by the camera as the texture, make it looks like an invisible model (but still able to see the shape and running animation). OR it just fails to apply the texture at all, dunno. Another thing I found out is that in the COGLESDriver.cpp's "setTexture" method, if I force the "stage" to be 1 when it was passed in, then it will be able to show the model with correct texture (kind of), and it moves relatively, but there is no camera feeds in the background.. only a black bg color. I am so new to OpenGL (know nothing about it when I start playing with AR and this engine, was hoping to get it work by brute force) and this 3D engine so if anyone can help me out would be greatly appreciated!

thanks.
Ronald

hybrid · Post by **hybrid** » Mon Mar 21, 2011 7:51 am

The texture of the sydney model is a non-power-of-two sized texture. This can cause problems on many systems. Try to resize the image file to 256x256 and check again.
Don't know how you get the video image onto a texture by chance, maybe there's more going wrong then just a little texture glitch. Layer one is not used for rendering the sydney model, so a texture stored there should be completely invisible (note: the texture is not visible, the model would be completely white or black if no texture is properly set).
Maybe post some code and screenshots so we can better understand what the problem is.

ronaldylee · Post by **ronaldylee** » Mon Mar 21, 2011 8:33 am

thanks for quick response. I wil try to resize the .bmp image, but it was running fine on my Nexus One without Qualcomm AR stuff, just the original Android port sample.

Is there a way for me to zip up my whole project directory and post it to the forum or email to u? My test project is mainly to merge the Qualcomm AR sample project code to Irrhlict android port code together. In Irrhlict code I comment out code that handle the modelview/project logic and let Qualcomm AR handles that (it generates a project matrix), so it solves the "moving the model relative to the trackable image" problem (total hack and I guess there's better way to do it). I am totally stuck at the texture issue probably because i pretty much don't know enough the 3D engine and openGL in general.

Also, when I move my phone, the "texture" on the model will change also. It almost looks like the model is distorting light coming out from the background fed by the camera.

hybrid · Post by **hybrid** » Mon Mar 21, 2011 9:25 am

You can upload larger sources to a free hoster. But it's probably not that useful, as most people won't have ways to let this run properly. Better show some screenshots and the main parts of the code as citation directly in the forum.

ronaldylee · Post by **ronaldylee** » Tue Mar 22, 2011 2:51 am

Here is the main "renderFrame" call:

Java_com_xxx_renderFrame(JNIEnv *, jobject)
{
// Clear color and depth buffer
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

// Render video background:
QCAR::State state = QCAR::Renderer::getInstance().begin();

// Did we find any trackables this frame?
int numActTrackables = state.getNumActiveTrackables();
for(int tIdx = 0; tIdx < numActTrackables; tIdx++)
{
// Get the trackable:
const QCAR::Trackable* trackable = state.getActiveTrackable(tIdx);
QCAR::Matrix44F modelViewMatrix =
QCAR::Tool::convertPose2GLMatrix(trackable->getPose());

// Load projection matrix:
glMatrixMode(GL_PROJECTION);
glLoadMatrixf(projectionMatrix.data);

// Load model view matrix:
glMatrixMode(GL_MODELVIEW);
glLoadMatrixf(modelViewMatrix.data);
glTranslatef(0.f, 0.f, kObjectScale);
glScalef(kObjectScale, kObjectScale, kObjectScale);

nativeDrawIteration();
}

QCAR::Renderer::getInstance().end();
}

//=======
File COGLESDriver.cpp, in the COGLES1Driver::setTransform function, i commented out in the switch statement for cases ETS_VIEW,ETS_WORLD, ETS_PROJECTION, so it uses the matrix created by Qualcomm above (projectionMatrix.data).

//=======
in app-android.cpp, function "void nativeDrawIterationSydney()" I commented out calls to the "driver" which seems to drawing stuff that covering the background camera feed:
//driver->beginScene(true, true, SColor(255,255,0,0));
....
//driver->endScene();

//========
in app-android.cpp, function "initSydney" I just use a different camera setup which i think is ok:
//smgr->addCameraSceneNode(0, vector3df(0,10,-40), vector3df(0,5,0));
camera = smgr->addCameraSceneNodeFPS();
camera->setPosition(core::vector3df(-50,50,-150));

//=========
COGLESDriver.cpp, for the call "bool COGLES1Driver::setTexture(u32 stage, const video::ITexture* texture)". if I put "stage = 1" it shows the model with correct texture but no background. The model is still kind of strange looking when i rotate it to look at its back (see thru the face).

I think thats all the major code change I have made to try to make it work. I will try to get some screen shots also later on so you see what I mean when I say the model is transparent like glass which reflecting the background camera feed. thx!!

ronaldylee · Post by **ronaldylee** » Tue Mar 22, 2011 5:06 am

Here is a screen shot showing the transparent model in motion:

http://72.249.23.79/photo.JPG

ronaldylee · Post by **ronaldylee** » Wed Mar 30, 2011 6:01 am

Still no luck with this integration. Texture still using the camera feed. What needs to be clear for texture in order for Irrlicht to work?

hybrid · Post by **hybrid** » Wed Mar 30, 2011 8:12 am

MAybe the texture update changes the active texture currently set? When dealing with native gl commands, you have to be very careful with not messing around with the assumed render states in Irrlicht.

ronaldylee · Post by **ronaldylee** » Wed Mar 30, 2011 4:23 pm

what i did is pretty much wrap the irrlicht android code with simple qualcomm ar code:

qualcomm AR code render camera;

irrlicht render frame code that render 3d model;

qualcomm AR code clean up;

So I found that if removing the qualcomm AR code for rendering camera, the 3D model texture works. So I just don't know what Qualcomm camera code changes the openGL states that causes irrlicht not able to render the texture correctly. What OpenGL commands needs to call to reset everything?? It seems that those calls that try to initialize/cleanup opengl in irrlicht is not enough(?)

kine · Post by **kine** » Fri May 27, 2011 12:13 pm

It is due to the fact that QualcommAR and irrlicht share the same ogl context, and so there is a conflict in texture. I guess they are both the first assigned,and as QualcommAR refresh everytime, it takes the lead.