Android 具有来自 JNI 代码的输入缓冲区的 MediaCodec

Question

我使用标准形式的 MediaCodec，例如：

public void run() {
    MediaExtractor extractor = new MediaExtractor();
    try {
        extractor.setDataSource("/sdcard/video-only.mpg");
    } catch (Exception e1) {
    }
    MediaFormat format = extractor.getTrackFormat(0);
    extractor.selectTrack(0);
    MediaCodec decoder = MediaCodec.createDecoderByType("video/avc");
    decoder.configure(format, mSurface, null, 0);
    decoder.start();
    ByteBuffer[] inputBuffers = decoder.getInputBuffers();
    ByteBuffer[] outputBuffers = decoder.getOutputBuffers();
    BufferInfo info = new BufferInfo();
    boolean isEOS = false;
    long startMs = System.currentTimeMillis();
    while (!Thread.interrupted()) {
        if (!isEOS) {
            int inIndex = decoder.dequeueInputBuffer(10000);
            if (inIndex >= 0) {
                ByteBuffer buffer = inputBuffers[inIndex];
                int sampleSize = extractor.readSampleData(buffer, 0);
                if (sampleSize < 0) {
                    decoder.queueInputBuffer(inIndex, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM);
                    isEOS = true;
                } else {
                    decoder.queueInputBuffer(inIndex, 0, sampleSize, extractor.getSampleTime(), 0);
                    extractor.advance();
                }
            }
        }
        int outIndex = decoder.dequeueOutputBuffer(info, 10000);
        switch (outIndex) {
        case MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED:
            outputBuffers = decoder.getOutputBuffers();
            break;
        case MediaCodec.INFO_OUTPUT_FORMAT_CHANGED:
            break;
        case MediaCodec.INFO_TRY_AGAIN_LATER:
            break;
        default:
            ByteBuffer buffer = outputBuffers[outIndex];
            while (info.presentationTimeUs / 1000 > System.currentTimeMillis() - startMs) {
                try {
                    Thread.sleep(10);
                } catch (Exception e) {
                    break;
                }
            }
            decoder.releaseOutputBuffer(outIndex, true);
            break;
        }
        if ((info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0)
            break;
    }
    decoder.stop();
    decoder.release();
    extractor.release();
}

我现在从 JNI 中的单独线程运行获取输入缓冲区。在此线程中，我收到回调并获得（已解析的）h264 数据。我打算将 java 代码修改为以下内容（不再提取器）：

public void run() {
    MediaFormat format = MediaFormat format = MediaFormat.createVideoFormat("video/avc", 640, 480);
    MediaCodec decoder = MediaCodec.createDecoderByType("video/avc");
    decoder.configure(format, mSurface, null, 0);
    decoder.start();
    ByteBuffer[] inputBuffers = decoder.getInputBuffers();
    ByteBuffer[] outputBuffers = decoder.getOutputBuffers();
    BufferInfo info = new BufferInfo();
    boolean isEOS = false;
    long startMs = System.currentTimeMillis();
    while (!Thread.interrupted()) {
        if (!isEOS) {
            int inIndex = decoder.dequeueInputBuffer(10000);
            if (inIndex >= 0) {
                ByteBuffer buffer = inputBuffers[inIndex];
                int sampleSize = ??????? // How to memcpy from JNI
                if (sampleSize < 0) {
                    decoder.queueInputBuffer(inIndex, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM);
                    isEOS = true;
                }
            }
        }
        int outIndex = decoder.dequeueOutputBuffer(info, 10000);
        switch (outIndex) {
        case MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED:
            outputBuffers = decoder.getOutputBuffers();
            break;
        case MediaCodec.INFO_OUTPUT_FORMAT_CHANGED:
            break;
        case MediaCodec.INFO_TRY_AGAIN_LATER:
            break;
        default:
            ByteBuffer buffer = outputBuffers[outIndex];
            while (info.presentationTimeUs / 1000 > System.currentTimeMillis() - startMs) {
                try {
                    Thread.sleep(10);
                } catch (Exception e) {
                    break;
                }
            }
            decoder.releaseOutputBuffer(outIndex, true);
            break;
        }
        if ((info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0)
            break;
    }
    decoder.stop();
    decoder.release();
}

在 JNI 代码中，我收到以下回调：

static void callback_with_buffer (uint8 *buffer, uint size) {
}

我应该如何将 JNI 缓冲区 memcpy 到 Java inputBuffers[inIndex]，尤其是在两个线程都没有真正同步的情况下？

Answer 1

在 JNI 中：

    JNIEnv *env = get_jni_env();
    jbyteArray jBuf = (*env)->NewByteArray(env, map.size);
    (*env)->SetByteArrayRegion(env, jBuf, 0, map.size, (jbyte*) map.data);
    (*env)->CallVoidMethod(env, data->app, set_buffer_method_id, jBuf);
    if ((*env)->ExceptionCheck(env)) {
        GST_ERROR("Failed to call Java method");
        (*env)->ExceptionClear(env);
    }
    (*env)->DeleteLocalRef(env, jBuf);

在Java中：

public LinkedList<byte[]> mData = new LinkedList<byte[]>();
private void setBuffer(byte[] buf) {
    mData.add(buf);
}

在 MediaCodec 解码器线程中：

                ByteBuffer buffer = inputBuffers[inIndex];
                int sampleSize = 0;
                buffer.clear();
                while (!((IVESampleActivity) mContext).mData.isEmpty()) {
                    byte[] buf = ((IVESampleActivity) mContext).mData.removeFirst();
                    buffer.put(buf);
                    sampleSize += buf.length;
                }

我想知道是否可以避免一些重复...

Android 具有来自 JNI 代码的输入缓冲区的 MediaCodec

Android MediaCodec with input buffer from JNI code

android

android-ndk

java-native-interface

android-mediacodec