OpenCV + OpenGL:复制gl纹理,使用OpenCV修改并渲染

问题描述 投票:0回答:1

[我正在编写一个Android应用程序,该应用程序使用CameraX将相机中的图像下载到SurfaceTexture,然后将纹理绑定到OpenGL的纹理,以在GLSurfaceView上执行进一步的渲染。我想使用OpenCV对onDrawFrame中的帧“注入”一些修改。

我曾考虑过将GL像素缓冲区复制到cv Mat,然后对其进行转换,然后使用glTexSubImage2D将其复制回纹理。问题是,我的代码不起作用,但是在打印cv::error(): OpenCV(4.2.0) Error: Assertion failed (!_src.empty()) in cvtColor之后立即抛出device/generic/goldfish-opengl/system/GLESv2_enc/GL2Encoder.cpp:s_glReadPixels:4413 GL error 0x500,因此我怀疑将像素复制到缓冲区的过程有问题。

有人可以告诉我,我在哪里弄乱了缓冲区?另外,如果有人有想法,如果有更好,更友好的方法可以做到这一点,您能指出吗?

我的代码:

@Override
public void onDrawFrame(GL10 gl) {
    GLES20.glClear( GLES20.GL_COLOR_BUFFER_BIT );

    surfaceTexture.updateTexImage();

    GLES20.glUseProgram(hProgram);

    int ph = GLES20.glGetAttribLocation(hProgram, "vPosition");
    int tch = GLES20.glGetAttribLocation ( hProgram, "vTexCoord" );
    int th = GLES20.glGetUniformLocation ( hProgram, "sTexture" );

    GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
    GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, hTex[0]);
    GLES20.glUniform1i(th, 0);

    GLES20.glVertexAttribPointer(ph, 2, GLES20.GL_FLOAT, false, 4*2, pVertex);
    GLES20.glVertexAttribPointer(tch, 2, GLES20.GL_FLOAT, false, 4*2, pTexCoord );
    GLES20.glEnableVertexAttribArray(ph);
    GLES20.glEnableVertexAttribArray(tch);

    Mat img = new Mat(this.bufferHeight, this.bufferWidth, CV_8UC3);

    GLES20.glPixelStorei(GL_PACK_ALIGNMENT, (img.step1() & 3) == 0 ? 1 : 4);

    GLES20.glPixelStorei(GL_PACK_ROW_LENGTH, (int)img.step1()/(int)img.elemSize());

    byte rarr[] = new byte[this.bufferWidth * this.bufferHeight];
    ByteBuffer buff = ByteBuffer.wrap(rarr);

    GLES20.glReadPixels(0, 0, img.cols(), img.rows(), GL_BGRA, GL_UNSIGNED_BYTE, buff);

    img.get(0, 0, buff.array());

        Imgproc.cvtColor(img, img, Imgproc.COLOR_BGR2GRAY);

    img.put(0, 0, buff.array());

    GLES20.glTexSubImage2D(hTex[0], 0, 0, 0, img.width(), img.height(),GL_BGRA,GL_UNSIGNED_BYTE, buff);

    GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, 0, 4);
    GLES20.glFlush();
}

崩溃日志:

2020-02-01 23:05:48.690 15840-15912/com.myapp.app I/ReactNativeJS: Running "appApp" with {"rootTag":1}
2020-02-01 23:05:48.756 15840-15840/com.myapp.app D/appView: Camera permission granted
2020-02-01 23:05:48.879 15840-15929/com.myapp.app D/eglCodecCommon: setVertexArrayObject: set vao to 0 (0) 0 0
2020-02-01 23:05:48.879 15840-15929/com.myapp.app D/EGL_emulation: eglCreateContext: 0xd751b800: maj 2 min 0 rcv 2
2020-02-01 23:05:48.924 15840-15929/com.myapp.app D/EGL_emulation: eglMakeCurrent: 0xd751b800: ver 2 0 (tinfo 0xb4c801a0)
2020-02-01 23:05:48.926 15840-15929/com.myapp.app E/EGL_emulation: eglQueryContext 32c0  EGL_BAD_ATTRIBUTE
2020-02-01 23:05:48.926 15840-15929/com.myapp.app E/EGL_emulation: tid 15929: eglQueryContext(1902): error 0x3004 (EGL_BAD_ATTRIBUTE)
2020-02-01 23:05:48.926 15840-15929/com.myapp.app D/appView: appRenderer fired appRendererInitialized callback
2020-02-01 23:05:48.929 15840-15929/com.myapp.app W/appap: Core platform API violation: Ljava/nio/Buffer;->position:I from Landroid/opengl/GLES20; using JNI
2020-02-01 23:05:48.930 15840-15929/com.myapp.app W/appap: Core platform API violation: Ljava/nio/Buffer;->limit:I from Landroid/opengl/GLES20; using JNI
2020-02-01 23:05:48.930 15840-15929/com.myapp.app W/appap: Core platform API violation: Ljava/nio/Buffer;->_elementSizeShift:I from Landroid/opengl/GLES20; using JNI
2020-02-01 23:05:48.930 15840-15929/com.myapp.app W/appap: Core platform API violation: Ljava/nio/Buffer;->address:J from Landroid/opengl/GLES20; using JNI
2020-02-01 23:05:48.941 15840-15929/com.myapp.app E/emuglGLESv2_enc: device/generic/goldfish-opengl/system/GLESv2_enc/GL2Encoder.cpp:s_glPixelStorei:536 GL error 0x500
2020-02-01 23:05:48.942 15840-15929/com.myapp.app W/appap: Accessing hidden method Ljava/nio/NIOAccess;->getBaseArray(Ljava/nio/Buffer;)Ljava/lang/Object; (greylist, JNI, allowed)
2020-02-01 23:05:48.942 15840-15929/com.myapp.app W/appap: Accessing hidden method Ljava/nio/NIOAccess;->getBaseArrayOffset(Ljava/nio/Buffer;)I (greylist, JNI, allowed)
2020-02-01 23:05:48.942 15840-15929/com.myapp.app E/emuglGLESv2_enc: device/generic/goldfish-opengl/system/GLESv2_enc/GL2Encoder.cpp:s_glReadPixels:4413 GL error 0x500
2020-02-01 23:05:48.976 15840-15929/com.myapp.app E/cv::error(): OpenCV(4.2.0) Error: Assertion failed (!_src.empty()) in cvtColor, file /build/master_pack-android/opencv/modules/imgproc/src/color.cpp, line 182
2020-02-01 23:05:48.977 15840-15929/com.myapp.app E/org.opencv.imgproc: imgproc::cvtColor_11() caught cv::Exception: OpenCV(4.2.0) /build/master_pack-android/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'
2020-02-01 23:05:48.980 15840-15929/com.myapp.app E/AndroidRuntime: FATAL EXCEPTION: GLThread 721
Process: com.myapp.app, PID: 15840
CvException [org.opencv.core.CvException: cv::Exception: OpenCV(4.2.0) /build/master_pack-android/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'
]
    at org.opencv.imgproc.Imgproc.cvtColor_1(Native Method)
    at org.opencv.imgproc.Imgproc.cvtColor(Imgproc.java:5051)
    at com.myapp.app.app.ui.appRenderer.onDrawFrame(AppRenderer.java:209)
    at android.opengl.GLSurfaceView$GLThread.guardedRun(GLSurfaceView.java:1573)
    at android.opengl.GLSurfaceView$GLThread.run(GLSurfaceView.java:1272)
java android opencv opengl android-camerax
1个回答
0
投票

最后通过固定缓冲区的分辨率(大小)并将GL_BGRA中的GL_RGBA更改为GLES20.glReadPixels,使该工作正常进行,根据文档,这是与GL_UNSIGNED_BYTE唯一允许的组合。现在,垫子已正确填充。

© www.soinside.com 2019 - 2024. All rights reserved.