MediaMuxer 将 H264 流写入 mpg 文件

Question

MediaMuxer 已经让我抓狂了两天两夜:-(

情况：我通过 UDP 接收到 H264 编码的 1280x720 视频流。 h264 流包含 NALU 1 - 切片和 NALU 5 - 关键帧（5 总是在 NALU 7 - SPS 和 NALU 8 - PPS 之前）。该流似乎稳定 30fps，每秒至少有一个 NALU 5 关键帧。比特率可变但小于 4Mbps。 MediaCodec 成功解码流并将其呈现在表面视图中，以便该部分运行良好。

但现在我需要将 H.264 保存到本地 mpg 文件中。我使用我拥有的所有 MediaFormat 信息设置了一个 MediaMuxer，并将流中的示例数据提供给它。每个样本包含一个帧（NALU 1 或 5），发送到 MediaMuxer 的第一个数据是关键帧（NALU 5）。呈现时间是根据帧数和帧率计算的。所有涉及的方法都是从同一个线程调用的。

但从未创建 mpg 文件。正如您在下面的输出中看到的，ByteBuffer 中的数据确实以 NALU headers 开头，后跟不同大小的数据。 MediaMuxer 在计算帧数时似乎在数据中 "see" 帧。那么这里有什么问题呢？

最小 API 是 21，我已经用三星 Galaxy S4 运行 stock Android 5 和一些设备运行 Lineageos Oreo 和 Nougat 进行了测试。

这是设置 MediaMuxer 的代码：

void setupMuxer(File f) throws IOException {
    if (DEBUG) Log.d(TAG, "Setup Muxer: " + f.getAbsolutePath() +" can write: " + f.canWrite());
    MediaFormat format = MediaFormat.createVideoFormat(MediaFormat.MIMETYPE_VIDEO_AVC, decoderWidth, decoderHeight);
    format.setInteger(MediaFormat.KEY_BIT_RATE, 4000000);
    format.setInteger(MediaFormat.KEY_FRAME_RATE, 29);
    format.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 1);
    format.setByteBuffer("csd-0", ByteBuffer.wrap(sps)); // sps and pps have been retrieved from the stream's NAL 7/8
    format.setByteBuffer("csd-1", ByteBuffer.wrap(pps));
    format.setInteger(MediaFormat.KEY_MAX_INPUT_SIZE, 1920 * 1080);

    muxer = new MediaMuxer(f.getPath(), MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
    videoTrack = muxer.addTrack(format);
    muxer.start();
}

为每个（完整的）NALU 1 和 NALU 5 调用此方法：

 void muxFrame(ByteBuffer buf, int frame) {
    MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();
    bufferInfo.offset = buf.arrayOffset();
    bufferInfo.size = buf.position() - bufferInfo.offset;
    bufferInfo.flags = (buf.get(4) & 0x1f) == 5 ? MediaCodec.BUFFER_FLAG_KEY_FRAME : 0;
    bufferInfo.presentationTimeUs = computePresentationTime(frame);


    if (DEBUG)
        Log.d(TAG, "muxFrame frame: " + frame + " size: " + bufferInfo.size + " NAL: " + (buf.get(4) & 0x1f) + " Flags: " + bufferInfo.flags + " PTS: " + bufferInfo.presentationTimeUs + " content: " + BitByteUtil.toByteString(buf.array(), buf.arrayOffset(), 8));

    try {
        muxer.writeSampleData(videoTrack, buf, bufferInfo);
    } catch (Exception e) {
        Log.w(TAG, "muxer failed", e);
    } finally {
    }
}

private static long computePresentationTime(int frameIndex) {
    return 42 + frameIndex * 1000000 / FRAME_RATE;
}

如果 MediaMuxer 在消耗 100 个 NALU 后停止，这是我的输出。

05.651 D/VideoDecoderView: Setup Muxer: /storage/emulated/0/Pictures/test.mpg can write: true
05.656 I/MPEG4Writer: limits: 4294967295/0 bytes/us, bit rate: -1 bps and the estimated moov size 3317 bytes
06.263 D/VideoDecoderView: muxFrame frame: 2 size: 7257 NAL: 5 Flags: 1 PTS: 66708 content: 0:000 1:000 2:000 3:001 4:101 5:184 6:000 7:015 
06.264 I/MPEG4Writer: setStartTimestampUs: 66708
06.264 I/MPEG4Writer: Earliest track starting time: 66708
06.308 D/VideoDecoderView: muxFrame frame: 3 size: 8998 NAL: 1 Flags: 0 PTS: 100042 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:034 7:020 
06.342 D/VideoDecoderView: muxFrame frame: 4 size: 13664 NAL: 1 Flags: 0 PTS: 133375 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:066 7:020 
06.375 D/VideoDecoderView: muxFrame frame: 5 size: 13674 NAL: 1 Flags: 0 PTS: 166708 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:098 7:020 
06.409 D/VideoDecoderView: muxFrame frame: 6 size: 13772 NAL: 1 Flags: 0 PTS: 200042 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:130 7:020 
06.483 D/VideoDecoderView: muxFrame frame: 7 size: 13707 NAL: 1 Flags: 0 PTS: 233375 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:162 7:020 
06.520 D/VideoDecoderView: muxFrame frame: 8 size: 13778 NAL: 1 Flags: 0 PTS: 266708 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:194 7:020 
06.555 D/VideoDecoderView: muxFrame frame: 9 size: 13743 NAL: 1 Flags: 0 PTS: 300042 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:226 7:020 
06.575 D/VideoDecoderView: muxFrame frame: 10 size: 7338 NAL: 5 Flags: 1 PTS: 333375 content: 0:000 1:000 2:000 3:001 4:101 5:184 6:000 7:015 
06.593 D/VideoDecoderView: muxFrame frame: 11 size: 9059 NAL: 1 Flags: 0 PTS: 366708 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:034 7:020 
06.618 D/VideoDecoderView: muxFrame frame: 12 size: 13587 NAL: 1 Flags: 0 PTS: 400042 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:066 7:020 
06.644 D/VideoDecoderView: muxFrame frame: 13 size: 13650 NAL: 1 Flags: 0 PTS: 433375 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:098 7:020 
06.671 D/VideoDecoderView: muxFrame frame: 14 size: 13797 NAL: 1 Flags: 0 PTS: 466708 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:130 7:020 
.... [snip]
09.620 D/VideoDecoderView: muxFrame frame: 97 size: 7212 NAL: 5 Flags: 1 PTS: 3233375 content: 0:000 1:000 2:000 3:001 4:101 5:184 6:000 7:015 
09.661 D/VideoDecoderView: muxFrame frame: 98 size: 8814 NAL: 1 Flags: 0 PTS: 3266708 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:034 7:020 
09.692 D/VideoDecoderView: muxFrame frame: 99 size: 13566 NAL: 1 Flags: 0 PTS: 3300042 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:066 7:020 
09.737 D/VideoDecoderView: muxFrame frame: 100 size: 13733 NAL: 1 Flags: 0 PTS: 3333375 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:098 7:020 
09.771 D/VideoDecoderView: muxFrame frame: 101 size: 13771 NAL: 1 Flags: 0 PTS: 3366708 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:130 7:020 
09.775 D/MPEG4Writer: Video track stopping. Stop source
09.775 I/MPEG4Writer: Received total/0-length     (100/1) buffers and encoded 100 frames. - Video
09.775 D/MPEG4Writer: Video track source stopping
09.775 D/MPEG4Writer: Video track source stopped
09.775 D/MPEG4Writer: Video track stopped. Stop source
09.775 D/MPEG4Writer: Stopping writer thread
09.776 D/MPEG4Writer: 0 chunks are written in the last batch
09.779 D/MPEG4Writer: Writer thread stopped
09.780 I/MPEG4Writer: Ajust the moov start time from 66708 us -> 66708 us
09.780 D/MPEG4Writer: Video track stopping. Stop source

Answer 1

@greeble31：你是对的。第一个日志条目明确指出 "Pictures" 而不是 "Videos"。我花了几个小时来研究这个问题，却没有注意到我的首选项键中有一个简单的剪切和粘贴错误。多蠢啊！！？！

自言自语：连续两天两夜编码不是英雄，而是愚蠢。

MediaMuxer 将 H264 流写入 mpg 文件

MediaMuxer writing H264 stream to mpg file

android

h.264

android-mediacodec

mediamuxer