Android: MediaCodec+MediaMuxer 编码的音频 MP4 无法播放

Question

我有以下函数，它使用 Android 的 MediaCode 和 MediaMuxer 类将 WAV (PCM) 文件编码为 AAC 编码的 MP4 文件。这只是音频。该函数成功运行并输出一个被识别为 AAC 编码的 reasonable .mp4。但它无法在 Android、网络或 iOS 玩家上播放，并且会导致 Audacity 崩溃。有什么我想念的吗？代码如下所示。

public void encode(final String from, final String to, final Callback callback) {
    new Thread(new Runnable() {
        @Override
        public void run() {
            try {
                extractor.setDataSource(from);
                int numTracks = extractor.getTrackCount();
                for (int i = 0; i < numTracks; ++i) {
                    MediaFormat format = extractor.getTrackFormat(i);
                    String mime = format.getString(MediaFormat.KEY_MIME);
                    Log.d(TAG, "Track " + i + " mime-type: " + mime);
                    if (true) {
                        extractor.selectTrack(i);
                    }
                }

                MediaCodec codec = MediaCodec.createEncoderByType("audio/mp4a-latm");
                MediaFormat format = new MediaFormat();
                format.setString(MediaFormat.KEY_MIME, "audio/mp4a-latm");
                format.setInteger(MediaFormat.KEY_BIT_RATE, 128 * 1024);
                format.setInteger(MediaFormat.KEY_CHANNEL_COUNT, 2);
                format.setInteger(MediaFormat.KEY_SAMPLE_RATE, 44100);
                format.setInteger(MediaFormat.KEY_AAC_PROFILE, MediaCodecInfo.CodecProfileLevel.AACObjectLC);
                codec.configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
                final MediaMuxer muxer = new MediaMuxer(to, MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
                final ByteBuffer byteBuffer = ByteBuffer.allocate(65536);
                codec.setCallback(new MediaCodec.Callback() {
                    @Override
                    public void onInputBufferAvailable(MediaCodec codec, int bufferIndex) {
                        ByteBuffer inputBuffer = codec.getInputBuffer(bufferIndex);
                        if (isEndOfStream) {
                            return;
                        }
                        int sampleCapacity = inputBuffer.capacity() / 8;
                        if (numAvailable == 0) {
                            numAvailable = extractor.readSampleData(byteBuffer, 0);
                            if (numAvailable <= 0) {
                                codec.queueInputBuffer(bufferIndex, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM);
                                isEndOfStream = true;
                                return;
                            }
                            long presentationTimeUs = extractor.getSampleTime();
                            extractor.advance();
                        }
                        if (numAvailable < sampleCapacity) {
                            codec.queueInputBuffer(bufferIndex, 0, numAvailable * 8, 0, 0);
                            numAvailable = 0;
                        } else {
                            codec.queueInputBuffer(bufferIndex, 0, sampleCapacity * 8, 0, 0);
                            numAvailable -= sampleCapacity;
                        }
                    }

                    @Override
                    public void onOutputBufferAvailable(MediaCodec codec, int index, MediaCodec.BufferInfo info) {
                        ByteBuffer outputBuffer = codec.getOutputBuffer(index);
                        muxer.writeSampleData(audioTrackIndex,outputBuffer,info);
                        codec.releaseOutputBuffer(index, true);
                        if ((info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {
                            Log.d(TAG, "end of encoding!");
                            codec.stop();
                            codec.release();
                            extractor.release();
                            extractor = null;
                            muxer.stop();
                            muxer.release();
                            callback.run(true);
                        }
                    }

                    @Override
                    public void onError(MediaCodec codec, MediaCodec.CodecException e) {
                        Log.e(TAG, "codec error", e);

                    }

                    @Override
                    public void onOutputFormatChanged(MediaCodec codec, MediaFormat format) {
                        audioTrackIndex = muxer.addTrack(format);
                        muxer.start();

                    }
                });
                codec.start();
            } catch (IOException e) {
                Log.e(TAG,"Unable to encode",e);
                callback.run(false);
            }

        }
    }).run();

Answer 1

您似乎将 AAC 编码为不太流行的 LATM 格式。也许这就是玩家不愿玩的原因。尝试使用其他媒体类型，audio/mp4 或 audio/3gpp.

参见AAC container formats。

Answer 2

您需要：

正确添加时间戳信息，因为媒体复用器需要使用它来标记数据包时间信息。
添加逻辑以将数据缓冲区从提取器数据缓冲区 (PCM) 复制到 mediacodec 输入缓冲区，仅引用缓冲区索引将仅编码一个没有初始的随机数据缓冲区。
添加代码以将输入源属性（例如通道和采样率）应用到媒体编解码器。不确定您是否打算使用不同的通道和采样率进行编码！

示例代码如下：

MediaExtractor extractor = null;
int numAvailable = 0;
boolean isEndOfStream = false;
int audioTrackIndex = 0;
long totalen = 0;
int channels = 0;
int sampleRate = 0;
public void encode(final String from, final String to) {
    new Thread(new Runnable() {
        @Override
        public void run() {
            try {
                extractor = new MediaExtractor();
                extractor.setDataSource(from);
                int numTracks = extractor.getTrackCount();
                for (int i = 0; i < numTracks; ++i) {
                    MediaFormat format = extractor.getTrackFormat(i);
                    String mime = format.getString(MediaFormat.KEY_MIME);
                    Log.d(TAG, "Track " + i + " mime-type: " + mime);
                    if (true) {
                        extractor.selectTrack(i);
                        channels = extractor.getTrackFormat(i).getInteger(MediaFormat.KEY_CHANNEL_COUNT);
                        sampleRate = extractor.getTrackFormat(i).getInteger(MediaFormat.KEY_SAMPLE_RATE);
                        Log.e(TAG,"sampleRate:" + sampleRate + " channels:" + channels);
                    }
                }
                String mimeType = "audio/mp4a-latm";
                MediaCodec codec = MediaCodec.createEncoderByType(mimeType);
                MediaFormat format = new MediaFormat();
                format.setString(MediaFormat.KEY_MIME, mimeType);
                format.setInteger(MediaFormat.KEY_BIT_RATE, 128 * 1024);
                format.setInteger(MediaFormat.KEY_CHANNEL_COUNT, channels);
                format.setInteger(MediaFormat.KEY_SAMPLE_RATE, sampleRate);
                format.setInteger(MediaFormat.KEY_AAC_PROFILE, MediaCodecInfo.CodecProfileLevel.AACObjectLC);
                codec.configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
                final MediaMuxer muxer = new MediaMuxer(to, MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
                final ByteBuffer byteBuffer = ByteBuffer.allocate(65536);
                codec.setCallback(new MediaCodec.Callback() {
                    @Override
                    public void onInputBufferAvailable(MediaCodec codec, int bufferIndex) {
                        ByteBuffer inputBuffer = codec.getInputBuffer(bufferIndex);
                        inputBuffer.clear();
                        if (isEndOfStream) {
                            return;
                        }
                        int sampleCapacity = inputBuffer.capacity();
                        if (numAvailable == 0) {
                            numAvailable = extractor.readSampleData(byteBuffer, 0);
                            if (numAvailable <= 0) {
                                codec.queueInputBuffer(bufferIndex, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM);
                                isEndOfStream = true;
                                return;
                            }
                            extractor.advance();
                        }
                        long timestampUs = 1000000l * totalen / (2 * channels * sampleRate);
                        if (numAvailable < sampleCapacity) {
                            byte[] byteArray = new byte[numAvailable];
                            byteBuffer.get(byteArray);
                            inputBuffer.put(byteArray, 0, (int)numAvailable);
                            totalen += numAvailable;
                            codec.queueInputBuffer(bufferIndex, 0, numAvailable, timestampUs, 0);
                            numAvailable = 0;
                        } else {
                            byte[] byteArray = new byte[sampleCapacity];
                            byteBuffer.get(byteArray);
                            inputBuffer.put(byteArray, 0, (int)sampleCapacity);
                            totalen += sampleCapacity;
                            codec.queueInputBuffer(bufferIndex, 0, sampleCapacity, timestampUs, 0);
                            numAvailable -= sampleCapacity;
                        }
                    }
                    @Override
                    public void onOutputBufferAvailable(MediaCodec codec, int index, MediaCodec.BufferInfo info) {
                        ByteBuffer outputBuffer = codec.getOutputBuffer(index);
                        muxer.writeSampleData(audioTrackIndex,outputBuffer,info);
                        codec.releaseOutputBuffer(index, true);
                        if ((info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {
                            Log.d(TAG, "end of encoding!");
                            codec.stop();
                            codec.release();
                            extractor.release();
                            extractor = null;
                            muxer.stop();
                            muxer.release();
                            //callback.run(true);
                        }
                    }
                    @Override
                    public void onError(MediaCodec codec, MediaCodec.CodecException e) {
                        Log.e(TAG, "codec error", e);

                    }
                    @Override
                    public void onOutputFormatChanged(MediaCodec codec, MediaFormat format) {
                        audioTrackIndex = muxer.addTrack(format);
                        muxer.start();
                    }
                });
                codec.start();
            } catch (IOException e) {
                Log.e(TAG,"Unable to encode",e);
                //callback.run(false);
            }
        }
    }).run();
}

顺便说一句，为什么要用缓冲区长度除以8？回调 class 是什么？请分享导入模块！我几乎无法通过回调参数传递构建所以将其注释掉！

Android: MediaCodec+MediaMuxer 编码的音频 MP4 无法播放

Android: MediaCodec+MediaMuxer encoded audio MP4 won't play

android

audio

mp4

aac