android camera SurfaceTexture

[kornwaretm]

i create a camera capture on java side using SurfaceTexture so i dont need to display anything on screen. i got stuck on transfering SurfaceTexture to ITexture. i kinda lost here no clues at all. can i read gles context from java ? or somehow create rendertarget in irrlicht and use it on java?

thanks in advance

[kornwaretm]

so my progress, i read that i have to render the camera image via offscreen GLES, and i manage to create a secondary GLES context from java android. the creation, shader compile, texture creation are successful, but somehow when the surfaceTexture gets the OnFrameAvailableListener called, android's GLES context always get disposed, while irrlicht's keep running.

i call GLES20.glGetString(GLES20.GL_RENDERER); GLES20.glGetString(GLES20.GL_VENDOR)); GLES20.glGetString(GLES20.GL_VERSION)); after the creation and in the OnFrameAvailableListener, i clearly see the frist one is a valid gles andreno 3.0 and stuff, second time they always undefined.

desperado... still no clue how to fix this. if anyone had any idea, i'd love to hear

[kornwaretm]

i manage to do continues drawing the camera capture, and copy the pixels from android's gles to irrlicht's ITexture. still facing problem. somehow camera capture always gives black screen. here is my code so far.
frame available listener

Code: Select all

 
cameraTexture = new SurfaceTexture(textureId);
cameraTexture.setOnFrameAvailableListener(
    new SurfaceTexture.OnFrameAvailableListener()
    {
        @Override
        public void onFrameAvailable(SurfaceTexture surfaceTexture)
        {
            try
            {
                // activate our offscreen context n surface
                EGL14.eglMakeCurrent(display, surface, surface, context);
                
                // update
                qrActivity.cameraTexture.updateTexImage();
                
                // draw
                cam.drawTexture();
                
                // frame is drawn, read all pixel to bitmap
                GLES20.glReadPixels(0, 0, mWidth, mHeight, GLES20.GL_RGBA, GLES20.GL_UNSIGNED_BYTE, mPixelBuf);
                
                // tell c++ frame ready
                markFrameReady();
                
            }
            catch(Exception e)
            {
                printExceptionStackTrace(e);
            }
            catch(Error e)
            {
                printErrorStackTrace(e);
            }
        }
    }
);
 

gles drawing code

Code: Select all

 
GLES20.glClearColor(1,1,1,1);
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT);
GLES20.glDisable(GLES20.GL_CULL_FACE);
GLES20.glViewport(0, 0, 512, 512);
 
// activate texture
GLES20.glUseProgram(program);
GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, getTextureId());
GLES20.glUniform1i(textureID, 0);
 
//vertex position
GLES20.glEnableVertexAttribArray(positionID);
GLES20.glVertexAttribPointer(
        positionID,         // position of the atributte
        2,                  // 2 values for each vertex
        GLES20.GL_FLOAT,    // data type
        false,              // normalize
        2 * BYTES_PER_FLOAT,// stride
        VERTEX_BUFFER);     // the vertex buffer
 
// coordinate
GLES20.glEnableVertexAttribArray(texcoordID);
GLES20.glVertexAttribPointer(
            texcoordID,         // position of the atributte
            2,                  // data size per vertex
            GLES20.GL_FLOAT,    // data type
            false,              // normalize
            2 * BYTES_PER_FLOAT,// stride
            UVCOORD_BUFFER);    // the uv coordinate buffer
 
// draw
GLES20.glDrawArrays(
    GLES20.GL_TRIANGLE_STRIP,   // drawing type
    0,                          // vertex offset
    4);                         // vertex count
 
// reset render state
GLES20.glDisableVertexAttribArray(positionID);
GLES20.glDisableVertexAttribArray(texcoordID);
GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, 0);
GLES20.glUseProgram(0);
 

[kornwaretm]

find the problem finally. it was stupid, "uniform sampler2D tex", what i need is "uniform samplerExternalOES tex"

Code: Select all

 
    private static final String PIXEL_SHADER =
        "#extension GL_OES_EGL_image_external : require\n" +
            "varying vec2 coor;\n" +
            "uniform samplerExternalOES tex;\n" +
            "void main() {\n" +
                "vec4 color = texture2D(tex, coor);\n" +
                "gl_FragColor = color;\n" +
            "}\n";
 

[CuteAlien]

Phew, thanks for sharing the solution - I also didn't know about this one. Pretty cool if it works! I didn't even expect it to be possible with NDK (everything seems hard/impossible with NDK on Android...).