为什么 MediaConfig 在简单解码设置时抛出 UnsupportedOperationException？

Question

我正在尝试使用 Android 上的 MediaConfig 解码一个简单的 H.264 视频流。它只是在第一个数据包上抛出异常，对于我来说，我无法理解为什么。

这是最少的代码。它所做的只是获取编解码器、启动解码器、获取输入缓冲区、填充并提交。但是在调试设备上，我得到一个 UnsupportedOperationException（也在下面）。

这让我很生气，在模拟器上花了一天时间才意识到它根本没有进行任何视频解码，这无济于事。

代码（除去问题之外的所有内容）：

package com.AndroidH264VideoTest;

import android.app.Activity;
import android.os.Bundle;

import android.media.MediaCodec;
import android.media.MediaFormat;

import java.nio.ByteBuffer;

public class AndroidH264VideoTest extends Activity
{
    private static final int SHORT_TIMEOUT = 1;  // Provide some timeout to getting a buffer

    private MediaCodec m_decoder = null;

    private ByteBuffer[] m_decoderInputBuffers;

    @Override
    public void onCreate(Bundle savedInstanceState)
    {
        super.onCreate(savedInstanceState);

        String MIME_TYPE = "video/avc";    // This is the type for H.264

        int TEST_WIDTH  = 8;  // Our test image is 8x8 pixels
        int TEST_HEIGHT = 8;

        m_decoder = MediaCodec.createDecoderByType(MIME_TYPE);

        MediaFormat decoderFormat = MediaFormat.createVideoFormat(MIME_TYPE, TEST_WIDTH, TEST_HEIGHT);

        m_decoder.configure(decoderFormat, null, null, 0);  // bytebuffer input/output
        m_decoder.start();

        m_decoderInputBuffers  = m_decoder.getInputBuffers();
    }

    @Override
    public void onResume()
    {
        byte SPS_PACKET[] = 
        {
            (byte)0x00, (byte)0x00, (byte)0x00, (byte)0x01,      // NAL header identifier
            (byte)0x67,                                          // NAL info - (byte)0x67 = 0 11 00111 = type 7 = SPS (1st packet) 
            (byte)0x42,                                          // SPS profile_idc = (byte)0x42/66 - Baseline. Should be decodable by anything
            (byte)0xC0,                                          // Constrained - should be even more decodable.
            (byte)0x0A,                                          // SPS level_idc = (byte)0x0a/10  
            (byte)0xDA, (byte)0x7E, (byte)0x59, (byte)0x66, (byte)0xA0, (byte)0xC0, (byte)0x20,
            (byte)0xC8, (byte)0x00, (byte)0x00, (byte)0x03, (byte)0x00, (byte)0x08, (byte)0x00, (byte)0x00,
            (byte)0x03, (byte)0x03, (byte)0xc4, (byte)0x78, (byte)0x91, (byte)0x35, 
        };

        super.onResume();  // Always call the superclass method first

        // Take ownership of buffer
        int inputBufferId = m_decoder.dequeueInputBuffer(SHORT_TIMEOUT);
        if (inputBufferId < 0)
        {
            // No buffer available error
            return;
        }

        // Copy data to buffer
        m_decoderInputBuffers[inputBufferId] = ByteBuffer.wrap(SPS_PACKET);

        // Submit buffer for processing
        m_decoder.queueInputBuffer(inputBufferId, 0, SPS_PACKET.length, 0, MediaCodec.BUFFER_FLAG_SYNC_FRAME);
    }
}

所以，那应该是无害的。不会做任何事情，因为我只传递了一个 NAL 数据包，但它不应该抱怨。

但是，在我的 Marshmallow 测试设备上发生了这种情况：

8-21 16:21:45.295 21871 21871 V ActivityThread: Performing resume of ActivityRecord{fa15f01 token=android.os.BinderProxy@e6df1a6 {com.AndroidH264VideoTest.AndroidH264VideoTest/com.AndroidH264VideoTest.AndroidH264VideoTest}}

08-21 16:21:45.296 21871 21871 D AndroidRuntime: Shutting down VM

--------- beginning of crash

08-21 16:21:45.297 21871 21871 E AndroidRuntime: FATAL EXCEPTION: main

08-21 16:21:45.297 21871 21871 E AndroidRuntime: Process: com.AndroidH264VideoTest.AndroidH264VideoTest, PID: 21871

08-21 16:21:45.297 21871 21871 E AndroidRuntime: java.lang.RuntimeException: Unable to resume activity {com.AndroidH264VideoTest.AndroidH264VideoTest/com.AndroidH264VideoTest.AndroidH264VideoTest}: java.lang.UnsupportedOperationException

08-21 16:21:45.297 21871 21871 E AndroidRuntime:    at android.app.ActivityThread.performResumeActivity(ActivityThread.java:3325)

08-21 16:21:45.297 21871 21871 E AndroidRuntime:    at android.app.ActivityThread.handleResumeActivity(ActivityThread.java:3356)

08-21 16:21:45.297 21871 21871 E AndroidRuntime:    at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2670)

08-21 16:21:45.297 21871 21871 E AndroidRuntime:    at android.app.ActivityThread.-wrap11(ActivityThread.java)

08-21 16:21:45.297 21871 21871 E AndroidRuntime:    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1499)

08-21 16:21:45.297 21871 21871 E AndroidRuntime:    at android.os.Handler.dispatchMessage(Handler.java:111)

08-21 16:21:45.297 21871 21871 E AndroidRuntime:    at android.os.Looper.loop(Looper.java:207)

08-21 16:21:45.297 21871 21871 E AndroidRuntime:    at android.app.ActivityThread.main(ActivityThread.java:5765)

08-21 16:21:45.297 21871 21871 E AndroidRuntime:    at java.lang.reflect.Method.invoke(Native Method)

08-21 16:21:45.297 21871 21871 E AndroidRuntime:    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:789)

08-21 16:21:45.297 21871 21871 E AndroidRuntime:    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:679)

08-21 16:21:45.297 21871 21871 E AndroidRuntime: Caused by: java.lang.UnsupportedOperationException

08-21 16:21:45.297 21871 21871 E AndroidRuntime:    at java.nio.ByteBuffer.setAccessible(ByteBuffer.java:636)

08-21 16:21:45.297 21871 21871 E AndroidRuntime:    at android.media.MediaCodec.invalidateByteBuffer(MediaCodec.java:2646)

08-21 16:21:45.297 21871 21871 E AndroidRuntime:    at android.media.MediaCodec.queueInputBuffer(MediaCodec.java:2174)

08-21 16:21:45.297 21871 21871 E AndroidRuntime:    at com.AndroidH264VideoTest.AndroidH264VideoTest.onResume(AndroidH264VideoTest.java:68)

08-21 16:21:45.297 21871 21871 E AndroidRuntime:    at android.app.Instrumentation.callActivityOnResume(Instrumentation.java:1268)

08-21 16:21:45.297 21871 21871 E AndroidRuntime:    at android.app.Activity.performResume(Activity.java:6392)

08-21 16:21:45.297 21871 21871 E AndroidRuntime:    at android.app.ActivityThread.performResumeActivity(ActivityThread.java:3310)
08-21 16:21:45.297 21871 21871 E AndroidRuntime:    ... 10 more

那么，为什么 queueInputBuffer() 中的操作不受支持？

更新：奇怪的是在不同的设备上（Android 4.4.4 KitKat，S4 mini）我收到了不同的错误消息，但它仍然爆炸：

E/AndroidRuntime(25080): FATAL EXCEPTION: main

E/AndroidRuntime(25080): Process: com.AndroidH264VideoTest.AndroidH264VideoTest, PID: 25080

E/AndroidRuntime(25080): java.lang.RuntimeException: Unable to start activity ComponentInfo{com.AndroidH264VideoTest.AndroidH264VideoTest/com.AndroidH264VideoTest.AndroidH264VideoTest}: java.lang.IllegalStateException

E/AndroidRuntime(25080):    at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2548)

E/AndroidRuntime(25080):    at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2607)

E/AndroidRuntime(25080):    at android.app.ActivityThread.access0(ActivityThread.java:174)

E/AndroidRuntime(25080):    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1325)

E/AndroidRuntime(25080):    at android.os.Handler.dispatchMessage(Handler.java:102)

E/AndroidRuntime(25080):    at android.os.Looper.loop(Looper.java:146)

E/AndroidRuntime(25080):    at android.app.ActivityThread.main(ActivityThread.java:5756)

E/AndroidRuntime(25080):    at java.lang.reflect.Method.invokeNative(Native Method)

E/AndroidRuntime(25080):    at java.lang.reflect.Method.invoke(Method.java:515)

E/AndroidRuntime(25080):    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1291)

E/AndroidRuntime(25080):    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1107)

E/AndroidRuntime(25080):    at dalvik.system.NativeStart.main(Native Method)

E/AndroidRuntime(25080): Caused by: java.lang.IllegalStateException

E/AndroidRuntime(25080):    at android.media.MediaCodec.native_configure(Native Method)

E/AndroidRuntime(25080):    at android.media.MediaCodec.configure(MediaCodec.java:262)

E/AndroidRuntime(25080):    at com.AndroidH264VideoTest.AndroidH264VideoTest.onCreate(AndroidH264VideoTest.java:33)

E/AndroidRuntime(25080):    at android.app.Activity.performCreate(Activity.java:5619)

E/AndroidRuntime(25080):    at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1093)

E/AndroidRuntime(25080):    at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2512)

E/AndroidRuntime(25080):    ... 11 more

Answer 1

我会说你的问题出在这里：

    // Copy data to buffer
    m_decoderInputBuffers[inputBufferId] = ByteBuffer.wrap(SPS_PACKET);

这只会用对另一个 ByteBuffer 的引用替换本地数组中的缓冲区 - 但解码器本身仍会尝试读取 m_decoder.getInputBuffers()[inputBufferId]。因此，您需要将输入字节复制到解码器提供的 ByteBuffer 中。

Answer 2

4.4.4 上的问题是试图解码 8x8 图像，这对于它的小大脑来说太小了。如果我将 8x8 替换为 320x240、160x160 甚至 67x71，只要它是 64x64 或更大，它就会很高兴。

我得看看这个任意大小限制是否也适用于 Marshmallow 设备。

为什么 MediaConfig 在简单解码设置时抛出 UnsupportedOperationException？

Why is MediaConfig throwing UnsupportedOperationException on simple decode setup?

video

android

h.264

android-mediacodec