使用显示方向 SEI 旋转 h264 视频
Rotate a h264 video using the Display Orientation SEI
我正在尝试将 h264 视频逆时针旋转 90 度。显示方向补充增强信息 (SEI) 的语法如下：
我第一次尝试编码如下：
val prefix = byteArrayOf(0, 0, 0, 1)
val nalHeader = byteArrayOf(6) // 0 forbidden_zero_bit = 0, nal_ref_idc = 0, nal_unit_type = 6
val display = byteArrayOf(47 /* Display orientation type*/, 3 /*payload size*/)
val displayOrientationCancelFlag = "0" // u(1); Rotation information follows
val horFlip = "0" // hor_flip; u(1); Do not flip horizontally
val verFlip = "0" // ver_flip; u(1); Do not flip vertically
val anticlockwiseRotation = "0100000000000000" // u(16); value / 2^16 -> 90 degrees
val displayOrientationRepetitionPeriod = "010" // ue(v); Persistent till next video sequence
val displayOrientationExtensionFlag = "0" // u(1); No other value is permitted by the spec atm
val byteAlignment = "1"
以上是我生成SEI的kotlin代码。这是一个静态变量，所以我选择了人类可读的版本。
删除附件 B 起始码，十六进制 SEI 为
06 2f 03 08 00 09
然而，ffmpeg 抱怨说我的有效载荷大小被指定为 3 个字节（24 位），但它在读取 NAL 停止位之前只读取了 23 位。为了解决这个问题，我用一个零位填充了我的 SEI 有效载荷，以允许 ffmpeg 成功解析它，并再次添加了停止位和更多字节对齐位：
val byteAlignment = "010000000"
这是十六进制的
06 2f 03 08 00 08 80
当我在我的 h264 rbsp 中的第一个 IDR NAL 单元之前添加它时，ffmpeg 将接受它以转换为我要求的任何格式，jpeg 或 mp4。但是输出没有旋转。用 ffplay 播放它也不会旋转它。我很清楚自己做错了什么。 ffprobe 产生以下输出：
[NULL @ 0x7fca03808a00] Opening 'sei2.264' for reading
[file @ 0x7fca01c289c0] Setting default whitelist 'file,crypto'
Probing h264 score:51 size:1502
Probing mp3 score:1 size:1502
[h264 @ 0x7fca03808a00] Format h264 probed with size=2048 and score=51
[h264 @ 0x7fca03808a00] Before avformat_find_stream_info() pos: 0 bytes read:1502 seeks:0 nb_streams:1
[AVBSFContext @ 0x7fca02808680] nal_unit_type: 7(SPS), nal_ref_idc: 3
[AVBSFContext @ 0x7fca02808680] nal_unit_type: 8(PPS), nal_ref_idc: 3
[AVBSFContext @ 0x7fca02808680] nal_unit_type: 6(SEI), nal_ref_idc: 0
[AVBSFContext @ 0x7fca02808680] nal_unit_type: 5(IDR), nal_ref_idc: 3
[h264 @ 0x7fca0381d400] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0x7fca0381d400] nal_unit_type: 8(PPS), nal_ref_idc: 3
[h264 @ 0x7fca0381d400] nal_unit_type: 6(SEI), nal_ref_idc: 0
[h264 @ 0x7fca0381d400] nal_unit_type: 5(IDR), nal_ref_idc: 3
[h264 @ 0x7fca0381d400] Format yuv420p chosen by get_format().
[h264 @ 0x7fca0381d400] Reinit context to 176x144, pix_fmt: yuv420p
[h264 @ 0x7fca03808a00] decoding for stream 0 failed
[h264 @ 0x7fca03808a00] stream 0: start_time: -7686143364045.646 duration: -7686143364045.646
[h264 @ 0x7fca03808a00] format: start_time: -9223372036854.775 duration: -9223372036854.775 bitrate=0 kb/s
[h264 @ 0x7fca03808a00] After avformat_find_stream_info() pos: 1502 bytes read:1502 seeks:0 frames:1
Input #0, h264, from 'sei2.264':
Duration: N/A, bitrate: N/A
Stream #0:0, 1, 1/1200000: Video: h264 (Constrained Baseline), 1 reference frame, yuv420p(progressive, left), 176x144, 0/1, 25 tbr, 1200k tbn, 50 tbc
[h264 @ 0x7fca0380dc00] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0x7fca0380dc00] nal_unit_type: 8(PPS), nal_ref_idc: 3
[AVIOContext @ 0x7fca02801b00] Statistics: 1502 bytes read, 0 seeks
我的包含 SPS、PPS、SEI 和 IDR 的完整 RBSP 如下：
unsigned char rbsp[1502] = {
// Offset 0x00000000 to 0x00001501
0x00, 0x00, 0x00, 0x01, 0x67, 0x42, 0xc0, 0x29, 0x8d, 0x68, 0x2c, 0x4e,
0x80, 0x78, 0x44, 0x23, 0x50, 0x00, 0x00, 0x00, 0x01, 0x68, 0xce, 0x01,
0xa8, 0x35, 0xc8, 0x00, 0x00, 0x00, 0x01, 0x06, 0x2f, 0x03, 0x08, 0x00,
0x08, 0x80, 0x00, 0x00, 0x00, 0x01, 0x65, 0xb8, 0x00, 0x04, 0x05, 0x9f,
0xff, 0xff, 0x04, 0x51, 0x40, 0x00, 0x41, 0x63, 0xc7, 0x31, 0xcf, 0xff,
0xff, 0x27, 0xff, 0xfe, 0x4f, 0xff, 0xfc, 0x9f, 0xff, 0xf9, 0x3f, 0xff,
0xf2, 0x7f, 0xff, 0xe4, 0xff, 0xff, 0xc9, 0xff, 0xff, 0x93, 0xff, 0xff,
0x27, 0xff, 0xfe, 0x4f, 0xff, 0xfc, 0x9f, 0xff, 0xf9, 0x3f, 0xff, 0xf2,
0x7f, 0xff, 0xe4, 0xff, 0xfc, 0x43, 0x05, 0xd0, 0x03, 0x2b, 0x2d, 0x11,
0x03, 0x16, 0x84, 0xb4, 0x5e, 0xc2, 0x00, 0x03, 0xa0, 0x20, 0xc7, 0x42,
0xb6, 0xb8, 0x00, 0x57, 0x66, 0x66, 0x09, 0xdc, 0xc1, 0x92, 0x50, 0x86,
0x38, 0x04, 0x8f, 0x6d, 0x83, 0xff, 0xff, 0x41, 0x5e, 0xef, 0xbb, 0xff,
0xff, 0xd0, 0x57, 0xbb, 0xee, 0xff, 0xff, 0xf4, 0x15, 0xee, 0xfb, 0xbf,
0xff, 0xfd, 0x05, 0x7b, 0xbe, 0xef, 0xff, 0xff, 0x41, 0x5e, 0xef, 0xbb,
0xff, 0xff, 0xd0, 0x57, 0xbb, 0xc0, 0x25, 0x55, 0x9a, 0xdf, 0x9f, 0x22,
0x63, 0xff, 0xff, 0x27, 0xff, 0xfe, 0x4f, 0xff, 0xfc, 0x9f, 0xff, 0xf9,
0x3f, 0xff, 0xec, 0x86, 0xfc, 0x7d, 0x2d, 0x2f, 0xf8, 0x07, 0xfa, 0xf8,
0x00, 0x37, 0x66, 0x35, 0x11, 0x29, 0x61, 0x8b, 0x30, 0x53, 0x41, 0x3c,
0xb3, 0xee, 0x59, 0xa7, 0xb7, 0xc2, 0x00, 0x03, 0x00, 0x07, 0x05, 0x86,
0x02, 0x10, 0x88, 0x38, 0x00, 0x53, 0x68, 0x93, 0x1b, 0xf4, 0x22, 0x3b,
0x90, 0x00, 0x08, 0x30, 0xd9, 0x64, 0x00, 0x02, 0x0c, 0x36, 0xe6, 0x10,
0xef, 0xa0, 0xf6, 0x52, 0xc3, 0xd9, 0x5c, 0xf8, 0x1f, 0xff, 0xb0, 0xdf,
0x80, 0x06, 0xc6, 0xaa, 0x46, 0x19, 0xd8, 0x55, 0x96, 0x5f, 0xfb, 0x38,
0xc2, 0xf8, 0x40, 0x00, 0x64, 0x0a, 0x0a, 0x30, 0x11, 0x20, 0xd4, 0x00,
0x8c, 0x99, 0xb0, 0x6c, 0xd9, 0x4a, 0x10, 0xde, 0x0c, 0x80, 0x00, 0x83,
0x8c, 0xb9, 0x88, 0xed, 0x77, 0x3c, 0x85, 0x1d, 0xcd, 0x2f, 0xff, 0xc7,
0x9c, 0x3b, 0xc0, 0xe8, 0x22, 0xb2, 0xbe, 0x04, 0xdf, 0xdf, 0xfe, 0x42,
0x80, 0x0b, 0x83, 0x88, 0xa5, 0x81, 0xc0, 0x02, 0x05, 0x08, 0x96, 0x00,
0x43, 0x64, 0x44, 0x09, 0xdf, 0xa1, 0x03, 0x88, 0xbe, 0x0d, 0x9c, 0xb1,
0x23, 0x3a, 0xe7, 0xff, 0xfd, 0x93, 0xff, 0xff, 0x27, 0xe2, 0x1f, 0xf8,
0x2c, 0x80, 0xf2, 0x10, 0xf2, 0xdf, 0x08, 0x00, 0x44, 0x00, 0x18, 0x0e,
0x81, 0x20, 0x02, 0x24, 0x0e, 0x00, 0x10, 0x20, 0x44, 0xbf, 0x06, 0xa7,
0x2f, 0xfc, 0xbf, 0x0c, 0x01, 0x02, 0xf8, 0xc4, 0xcf, 0x07, 0xc0, 0xe0,
0x00, 0x80, 0x08, 0x0f, 0x65, 0x84, 0x04, 0x0a, 0x8a, 0x03, 0x83, 0x44,
0xc0, 0x02, 0x92, 0x22, 0x20, 0x6c, 0xd9, 0x08, 0x1c, 0xfe, 0x03, 0x80,
0x8c, 0x5c, 0xb2, 0x00, 0x46, 0x2e, 0x5e, 0x04, 0x6c, 0xbb, 0x80, 0x71,
0x08, 0xf2, 0xc8, 0x10, 0x8f, 0x2c, 0x5a, 0xf2, 0x74, 0x0d, 0xb7, 0xf8,
0xd3, 0xed, 0x6d, 0x7f, 0xfc, 0x38, 0x03, 0x0e, 0xf8, 0x00, 0xee, 0x12,
0x89, 0xef, 0x22, 0x09, 0x97, 0xd3, 0xdf, 0x30, 0xc1, 0x48, 0x01, 0x80,
0x06, 0x9b, 0x80, 0x05, 0x64, 0xcd, 0x98, 0x9b, 0x29, 0x48, 0x74, 0x22,
0x32, 0x4b, 0xe0, 0x0f, 0x06, 0xd8, 0x61, 0x77, 0xb8, 0x70, 0x11, 0x8b,
0x97, 0xff, 0xc1, 0x30, 0x06, 0x1d, 0xf0, 0x00, 0xdc, 0xa1, 0x16, 0x88,
0xd5, 0xcc, 0x49, 0x17, 0xd9, 0xe8, 0x36, 0x61, 0x05, 0x55, 0x80, 0xe0,
0x0c, 0x37, 0x00, 0x21, 0xb4, 0x4a, 0x1b, 0xf4, 0x25, 0x8a, 0x40, 0x08,
0xc5, 0xcb, 0xe0, 0x70, 0x64, 0xa2, 0x13, 0xbd, 0xc3, 0x88, 0x47, 0x97,
0xf9, 0xf8, 0x07, 0xb0, 0x5f, 0x8e, 0xcc, 0xe0, 0xf8, 0x00, 0xac, 0x2f,
0x8e, 0xe6, 0x11, 0x2f, 0x51, 0xea, 0xf0, 0x40, 0xbb, 0x30, 0x4b, 0x4b,
0x0e, 0x02, 0x31, 0x72, 0xfc, 0x38, 0x84, 0x79, 0x7f, 0xe1, 0xff, 0xb0,
0x59, 0xea, 0xcf, 0xdc, 0x66, 0x12, 0xc3, 0xd5, 0xf2, 0xbd, 0x3c, 0x76,
0x02, 0x47, 0xae, 0x3d, 0xff, 0xf0, 0x24, 0x5c, 0xb3, 0xd2, 0x26, 0x2b,
0x9f, 0xf1, 0xfe, 0xc1, 0x6c, 0xf4, 0x1e, 0xe1, 0x00, 0x00, 0x80, 0x30,
0x03, 0x07, 0x97, 0x08, 0x4c, 0x85, 0x81, 0xc0, 0x02, 0x04, 0x08, 0x97,
0xe0, 0xd4, 0xe5, 0xff, 0xff, 0xf6, 0x16, 0xdb, 0xee, 0xfd, 0x1f, 0x6b,
0x6b, 0x6b, 0x6b, 0x6b, 0xc3, 0xff, 0xf6, 0x08, 0xbe, 0x0b, 0x44, 0xaf,
0xf9, 0xff, 0xb0, 0x59, 0xe0, 0x0d, 0x5f, 0x6a, 0x38, 0xde, 0x82, 0x01,
0x20, 0xc4, 0x80, 0x78, 0x00, 0x56, 0x46, 0x66, 0x84, 0xd9, 0x4a, 0xac,
0xd2, 0x00, 0x02, 0x05, 0x08, 0xb9, 0x04, 0xdf, 0xf6, 0x95, 0x9d, 0xce,
0x01, 0x82, 0xfc, 0x34, 0x3b, 0xee, 0x47, 0xc0, 0x4f, 0xaf, 0xad, 0x6f,
0x5e, 0x10, 0x17, 0x1d, 0x82, 0xc6, 0x35, 0xa0, 0x01, 0x4d, 0x91, 0x13,
0x1b, 0xf4, 0x26, 0x57, 0xc8, 0x00, 0x04, 0x08, 0x13, 0x71, 0xc6, 0x81,
0x23, 0x5b, 0xd6, 0x51, 0x2b, 0x9f, 0xff, 0xf6, 0x1f, 0xef, 0x3b, 0xcf,
0x53, 0xd7, 0x5d, 0x75, 0xd7, 0x4f, 0xff, 0xff, 0xc1, 0x07, 0x01, 0xc0,
0x21, 0xce, 0x2e, 0x5b, 0xff, 0xff, 0xb0, 0x43, 0xe0, 0x03, 0x6d, 0x10,
0xe1, 0x0a, 0x27, 0xa4, 0xdd, 0xa1, 0xce, 0x03, 0xda, 0x72, 0xdf, 0xfe,
0x29, 0x6a, 0xa5, 0xea, 0xa5, 0x80, 0x06, 0x0b, 0x68, 0x8a, 0x66, 0x62,
0xdf, 0xfc, 0x7e, 0xc1, 0x07, 0x01, 0xe6, 0x3a, 0x88, 0x7f, 0xd0, 0xe8,
0x00, 0xfc, 0x02, 0x67, 0xb1, 0xe0, 0x73, 0xd2, 0x2e, 0x03, 0xf8, 0x00,
0xc4, 0x00, 0x6f, 0x9b, 0x15, 0x6b, 0x2d, 0xcf, 0x07, 0xc0, 0x02, 0x09,
0x6a, 0x32, 0xd8, 0xba, 0x3f, 0x0c, 0x28, 0x20, 0x28, 0x20, 0x20, 0x42,
0x00, 0x83, 0xe0, 0x70, 0x23, 0x15, 0x2c, 0x01, 0x03, 0x2a, 0x63, 0xcc,
0x42, 0x99, 0x02, 0x19, 0xee, 0x01, 0xc8, 0x46, 0x96, 0x00, 0x47, 0x6c,
0xd8, 0x04, 0xaa, 0x41, 0xbc, 0x41, 0x05, 0x32, 0x00, 0x08, 0x20, 0xab,
0x9f, 0xff, 0xf4, 0x0a, 0xb4, 0xab, 0x8d, 0x3e, 0x09, 0x25, 0xff, 0xff,
0xe8, 0x9f, 0xff, 0xf4, 0x4f, 0xff, 0xfa, 0x27, 0xff, 0xfd, 0x07, 0x30,
0x04, 0x2d, 0x20, 0x47, 0x38, 0x95, 0xab, 0xfc, 0x00, 0x6b, 0x19, 0x05,
0xce, 0x53, 0x10, 0xa6, 0xed, 0x08, 0x12, 0xc0, 0x90, 0x00, 0x26, 0x1a,
0x00, 0x21, 0xfb, 0x90, 0xfc, 0x84, 0x24, 0x10, 0xcd, 0x72, 0x00, 0x14,
0xd2, 0x35, 0x91, 0x86, 0x37, 0x5e, 0x9b, 0x20, 0x00, 0x41, 0x05, 0xdc,
0xff, 0xe0, 0x18, 0x57, 0xe0, 0x02, 0x98, 0xf7, 0x33, 0x98, 0x55, 0x3d,
0x47, 0xab, 0xdf, 0xe8, 0x30, 0x01, 0xab, 0x58, 0x48, 0x53, 0xb1, 0x8a,
0x7a, 0xbd, 0xd5, 0x8f, 0x87, 0xe0, 0x01, 0x0c, 0xa4, 0x12, 0xad, 0xcc,
0x29, 0x1a, 0xd0, 0x83, 0xd9, 0xc2, 0x9a, 0xf4, 0xd0, 0x38, 0x08, 0xa4,
0xcb, 0xf2, 0xfc, 0xb0, 0xe0, 0x9c, 0x79, 0x7f, 0xff, 0xfe, 0x10, 0x61,
0xef, 0xc0, 0x01, 0x93, 0x46, 0x83, 0x76, 0xea, 0x41, 0x8d, 0xf7, 0x4b,
0x14, 0x1c, 0x00, 0x04, 0x10, 0x20, 0x18, 0x00, 0x08, 0x14, 0xf0, 0x91,
0xcd, 0xb8, 0x08, 0xd4, 0xb7, 0xcf, 0x7f, 0xff, 0xd0, 0x43, 0xdf, 0xd0,
0x81, 0x83, 0xca, 0xc0, 0x05, 0x34, 0x63, 0x27, 0x15, 0x4d, 0x52, 0x0d,
0xda, 0x5e, 0x30, 0x5e, 0x05, 0x2a, 0x0d, 0xbb, 0x90, 0x22, 0x03, 0x7a,
0x68, 0x1d, 0x21, 0xdc, 0xd4, 0xc8, 0x91, 0x4d, 0xf3, 0xcf, 0xff, 0xfa,
0x05, 0x9c, 0x53, 0xfa, 0x45, 0xff, 0xe3, 0x4f, 0xff, 0xfe, 0x82, 0xdb,
0x7d, 0xdf, 0x1c, 0x00, 0x0d, 0xc7, 0x00, 0x01, 0x01, 0x5f, 0xff, 0xfd,
0x06, 0xae, 0xf9, 0x7d, 0x31, 0xf1, 0x08, 0xe2, 0x00, 0x00, 0x80, 0x30,
0x03, 0xea, 0x2e, 0x30, 0x92, 0x6b, 0xc8, 0x70, 0x98, 0x6b, 0x90, 0xe1,
0x30, 0xd7, 0x3f, 0xf0, 0xc0, 0x34, 0x0b, 0x60, 0x00, 0x99, 0x90, 0xc8,
0xe1, 0x98, 0xdc, 0x41, 0x23, 0xcc, 0x71, 0xf0, 0x40, 0xc6, 0x17, 0xc2,
0x80, 0x07, 0xc0, 0x81, 0x40, 0x00, 0xf8, 0x09, 0xc0, 0x27, 0xb1, 0x37,
0x72, 0x02, 0xa7, 0xd5, 0xe2, 0xc0, 0x80, 0xa6, 0x78, 0x04, 0xe9, 0x89,
0x95, 0xc8, 0x0e, 0xbf, 0x57, 0x8b, 0x00, 0x80, 0x53, 0x3f, 0xf5, 0xf0,
0xd8, 0x76, 0x00, 0x09, 0xe4, 0x38, 0xcc, 0x7e, 0x90, 0x66, 0x90, 0xb5,
0x86, 0x08, 0x50, 0x00, 0x6c, 0x99, 0xb4, 0x44, 0x8c, 0x6a, 0x89, 0xc4,
0xf8, 0x27, 0xf8, 0x41, 0x1f, 0xdc, 0x10, 0x47, 0xf7, 0x60, 0x04, 0x9d,
0x0f, 0x31, 0x5c, 0x9e, 0xaf, 0x0e, 0x09, 0x89, 0x81, 0x0d, 0x34, 0xdb,
0x56, 0x00, 0x48, 0xd4, 0x34, 0x62, 0xb9, 0xbd, 0x5e, 0x1c, 0x05, 0xd9,
0x81, 0x0d, 0x34, 0xdb, 0x57, 0xfc, 0x03, 0x0e, 0x81, 0x54, 0x20, 0x04,
0x86, 0x28, 0x40, 0x09, 0x0c, 0x5f, 0xc0, 0x15, 0x24, 0x63, 0x05, 0x38,
0xc3, 0x6d, 0x5f, 0x58, 0x02, 0xa1, 0x67, 0x70, 0x53, 0x0c, 0x36, 0xd5,
0xff, 0xf0, 0x0d, 0x87, 0x70, 0x00, 0x4c, 0xc8, 0x64, 0x70, 0xcc, 0x6e,
0x20, 0x91, 0xe6, 0x38, 0xfc, 0x61, 0x70, 0x40, 0xfc, 0x20, 0x08, 0x75,
0x94, 0x10, 0x04, 0x15, 0x66, 0x61, 0x35, 0xc3, 0xec, 0x47, 0x2f, 0xab,
0xc5, 0x81, 0x01, 0x4c, 0xe1, 0x32, 0x70, 0xd5, 0x88, 0xe7, 0xf5, 0x78,
0xb0, 0x08, 0x05, 0x33, 0xff, 0x05, 0xfd, 0x82, 0xff, 0xaf, 0x80, 0x02,
0x79, 0x0e, 0x33, 0x1f, 0xa4, 0x19, 0xa4, 0x2d, 0x70, 0x42, 0x08, 0x01,
0x3c, 0x06, 0x08, 0x01, 0x38, 0x25, 0x43, 0x82, 0x61, 0x6e, 0x00, 0x12,
0xf0, 0x9b, 0x39, 0x41, 0x51, 0xea, 0xf1, 0x00, 0x98, 0x5b, 0x90, 0xe4,
0x64, 0xb8, 0x00, 0x4a, 0xd0, 0x99, 0x1c, 0xa0, 0x55, 0x7a, 0xbc, 0x41,
0x19, 0x2e, 0x7f, 0xff, 0xd0, 0x63, 0x2e, 0xa6, 0x3c, 0x00, 0x19, 0xb4,
0x49, 0x8d, 0xfa, 0x11, 0x1d, 0xd2, 0x33, 0xe8, 0xef, 0xff, 0xfe, 0xfc,
0x07, 0x00, 0x10, 0x41, 0xaa, 0x4b, 0x7e, 0x00, 0x0d, 0xd3, 0x26, 0xc4,
0xf3, 0x15, 0x1d, 0x13, 0x9f, 0x04, 0x2c, 0x0e, 0x00, 0xc1, 0x02, 0x40,
0x07, 0x81, 0xb4, 0x70, 0x75, 0x43, 0xf0, 0x3c, 0xc5, 0x70, 0x8f, 0xff,
0xff, 0xb0, 0xc7, 0x80, 0x03, 0x36, 0x89, 0x31, 0xbf, 0xc4, 0x47, 0x7d,
0x04, 0xfb, 0xf8, 0x07, 0xff, 0xb0, 0x45, 0xcb, 0x9e, 0xc2, 0x05, 0xd9,
0x82, 0x00, 0x11, 0x58, 0xaf, 0xff, 0xfb, 0x08, 0xdf, 0xff, 0xff, 0x61,
0x1b, 0x7f, 0xff, 0xec, 0x15, 0x76, 0xff, 0xff, 0xdd, 0xff, 0xff, 0xec,
0x15, 0x44, 0x7b, 0xff, 0xcd, 0x7f, 0xff, 0xec, 0x23, 0x77, 0xff, 0xfe,
0xc8, 0x9f
}
payloadSize只是sei_message()的大小，停止位不计入这个大小。所以SEI的最后一定有一个0x80，
我正在尝试将 h264 视频逆时针旋转 90 度。显示方向补充增强信息 (SEI) 的语法如下：
我第一次尝试编码如下：
val prefix = byteArrayOf(0, 0, 0, 1)
val nalHeader = byteArrayOf(6) // 0 forbidden_zero_bit = 0, nal_ref_idc = 0, nal_unit_type = 6
val display = byteArrayOf(47 /* Display orientation type*/, 3 /*payload size*/)
val displayOrientationCancelFlag = "0" // u(1); Rotation information follows
val horFlip = "0" // hor_flip; u(1); Do not flip horizontally
val verFlip = "0" // ver_flip; u(1); Do not flip vertically
val anticlockwiseRotation = "0100000000000000" // u(16); value / 2^16 -> 90 degrees
val displayOrientationRepetitionPeriod = "010" // ue(v); Persistent till next video sequence
val displayOrientationExtensionFlag = "0" // u(1); No other value is permitted by the spec atm
val byteAlignment = "1"
以上是我生成SEI的kotlin代码。这是一个静态变量，所以我选择了人类可读的版本。
删除附件 B 起始码，十六进制 SEI 为
06 2f 03 08 00 09
ffmpeg 抱怨说我的有效载荷大小被指定为 3 个字节（24 位），但它在读取 NAL 停止位之前只读取了 23 位。为了解决这个问题，我用一个零位填充了我的 SEI 有效载荷，以允许 ffmpeg 成功解析它，并再次添加了停止位和更多字节对齐位：
val byteAlignment = "010000000"
这是十六进制的
06 2f 03 08 00 08 80
当我在我的 h264 rbsp 中的第一个 IDR NAL 单元之前添加它时，ffmpeg 将接受它以转换为我要求的任何格式，jpeg 或 mp4。但是输出没有旋转。用 ffplay 播放它也不会旋转它。我很清楚自己做错了什么。 ffprobe 产生以下输出：
[NULL @ 0x7fca03808a00] Opening 'sei2.264' for reading
[file @ 0x7fca01c289c0] Setting default whitelist 'file,crypto'
Probing h264 score:51 size:1502
Probing mp3 score:1 size:1502
[h264 @ 0x7fca03808a00] Format h264 probed with size=2048 and score=51
[h264 @ 0x7fca03808a00] Before avformat_find_stream_info() pos: 0 bytes read:1502 seeks:0 nb_streams:1
[AVBSFContext @ 0x7fca02808680] nal_unit_type: 7(SPS), nal_ref_idc: 3
[AVBSFContext @ 0x7fca02808680] nal_unit_type: 8(PPS), nal_ref_idc: 3
[AVBSFContext @ 0x7fca02808680] nal_unit_type: 6(SEI), nal_ref_idc: 0
[AVBSFContext @ 0x7fca02808680] nal_unit_type: 5(IDR), nal_ref_idc: 3
[h264 @ 0x7fca0381d400] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0x7fca0381d400] nal_unit_type: 8(PPS), nal_ref_idc: 3
[h264 @ 0x7fca0381d400] nal_unit_type: 6(SEI), nal_ref_idc: 0
[h264 @ 0x7fca0381d400] nal_unit_type: 5(IDR), nal_ref_idc: 3
[h264 @ 0x7fca0381d400] Format yuv420p chosen by get_format().
[h264 @ 0x7fca0381d400] Reinit context to 176x144, pix_fmt: yuv420p
[h264 @ 0x7fca03808a00] decoding for stream 0 failed
[h264 @ 0x7fca03808a00] stream 0: start_time: -7686143364045.646 duration: -7686143364045.646
[h264 @ 0x7fca03808a00] format: start_time: -9223372036854.775 duration: -9223372036854.775 bitrate=0 kb/s
[h264 @ 0x7fca03808a00] After avformat_find_stream_info() pos: 1502 bytes read:1502 seeks:0 frames:1
Input #0, h264, from 'sei2.264':
Duration: N/A, bitrate: N/A
Stream #0:0, 1, 1/1200000: Video: h264 (Constrained Baseline), 1 reference frame, yuv420p(progressive, left), 176x144, 0/1, 25 tbr, 1200k tbn, 50 tbc
[h264 @ 0x7fca0380dc00] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0x7fca0380dc00] nal_unit_type: 8(PPS), nal_ref_idc: 3
[AVIOContext @ 0x7fca02801b00] Statistics: 1502 bytes read, 0 seeks
我的包含 SPS、PPS、SEI 和 IDR 的完整 RBSP 如下：
unsigned char rbsp[1502] = {
// Offset 0x00000000 to 0x00001501
0x00, 0x00, 0x00, 0x01, 0x67, 0x42, 0xc0, 0x29, 0x8d, 0x68, 0x2c, 0x4e,
0x80, 0x78, 0x44, 0x23, 0x50, 0x00, 0x00, 0x00, 0x01, 0x68, 0xce, 0x01,
0xa8, 0x35, 0xc8, 0x00, 0x00, 0x00, 0x01, 0x06, 0x2f, 0x03, 0x08, 0x00,
0x08, 0x80, 0x00, 0x00, 0x00, 0x01, 0x65, 0xb8, 0x00, 0x04, 0x05, 0x9f,
0xff, 0xff, 0x04, 0x51, 0x40, 0x00, 0x41, 0x63, 0xc7, 0x31, 0xcf, 0xff,
0xff, 0x27, 0xff, 0xfe, 0x4f, 0xff, 0xfc, 0x9f, 0xff, 0xf9, 0x3f, 0xff,
0xf2, 0x7f, 0xff, 0xe4, 0xff, 0xff, 0xc9, 0xff, 0xff, 0x93, 0xff, 0xff,
0x27, 0xff, 0xfe, 0x4f, 0xff, 0xfc, 0x9f, 0xff, 0xf9, 0x3f, 0xff, 0xf2,
0x7f, 0xff, 0xe4, 0xff, 0xfc, 0x43, 0x05, 0xd0, 0x03, 0x2b, 0x2d, 0x11,
0x03, 0x16, 0x84, 0xb4, 0x5e, 0xc2, 0x00, 0x03, 0xa0, 0x20, 0xc7, 0x42,
0xb6, 0xb8, 0x00, 0x57, 0x66, 0x66, 0x09, 0xdc, 0xc1, 0x92, 0x50, 0x86,
0x38, 0x04, 0x8f, 0x6d, 0x83, 0xff, 0xff, 0x41, 0x5e, 0xef, 0xbb, 0xff,
0xff, 0xd0, 0x57, 0xbb, 0xee, 0xff, 0xff, 0xf4, 0x15, 0xee, 0xfb, 0xbf,
0xff, 0xfd, 0x05, 0x7b, 0xbe, 0xef, 0xff, 0xff, 0x41, 0x5e, 0xef, 0xbb,
0xff, 0xff, 0xd0, 0x57, 0xbb, 0xc0, 0x25, 0x55, 0x9a, 0xdf, 0x9f, 0x22,
0x63, 0xff, 0xff, 0x27, 0xff, 0xfe, 0x4f, 0xff, 0xfc, 0x9f, 0xff, 0xf9,
0x3f, 0xff, 0xec, 0x86, 0xfc, 0x7d, 0x2d, 0x2f, 0xf8, 0x07, 0xfa, 0xf8,
0x00, 0x37, 0x66, 0x35, 0x11, 0x29, 0x61, 0x8b, 0x30, 0x53, 0x41, 0x3c,
0xb3, 0xee, 0x59, 0xa7, 0xb7, 0xc2, 0x00, 0x03, 0x00, 0x07, 0x05, 0x86,
0x02, 0x10, 0x88, 0x38, 0x00, 0x53, 0x68, 0x93, 0x1b, 0xf4, 0x22, 0x3b,
0x90, 0x00, 0x08, 0x30, 0xd9, 0x64, 0x00, 0x02, 0x0c, 0x36, 0xe6, 0x10,
0xef, 0xa0, 0xf6, 0x52, 0xc3, 0xd9, 0x5c, 0xf8, 0x1f, 0xff, 0xb0, 0xdf,
0x80, 0x06, 0xc6, 0xaa, 0x46, 0x19, 0xd8, 0x55, 0x96, 0x5f, 0xfb, 0x38,
0xc2, 0xf8, 0x40, 0x00, 0x64, 0x0a, 0x0a, 0x30, 0x11, 0x20, 0xd4, 0x00,
0x8c, 0x99, 0xb0, 0x6c, 0xd9, 0x4a, 0x10, 0xde, 0x0c, 0x80, 0x00, 0x83,
0x8c, 0xb9, 0x88, 0xed, 0x77, 0x3c, 0x85, 0x1d, 0xcd, 0x2f, 0xff, 0xc7,
0x9c, 0x3b, 0xc0, 0xe8, 0x22, 0xb2, 0xbe, 0x04, 0xdf, 0xdf, 0xfe, 0x42,
0x80, 0x0b, 0x83, 0x88, 0xa5, 0x81, 0xc0, 0x02, 0x05, 0x08, 0x96, 0x00,
0x43, 0x64, 0x44, 0x09, 0xdf, 0xa1, 0x03, 0x88, 0xbe, 0x0d, 0x9c, 0xb1,
0x23, 0x3a, 0xe7, 0xff, 0xfd, 0x93, 0xff, 0xff, 0x27, 0xe2, 0x1f, 0xf8,
0x2c, 0x80, 0xf2, 0x10, 0xf2, 0xdf, 0x08, 0x00, 0x44, 0x00, 0x18, 0x0e,
0x81, 0x20, 0x02, 0x24, 0x0e, 0x00, 0x10, 0x20, 0x44, 0xbf, 0x06, 0xa7,
0x2f, 0xfc, 0xbf, 0x0c, 0x01, 0x02, 0xf8, 0xc4, 0xcf, 0x07, 0xc0, 0xe0,
0x00, 0x80, 0x08, 0x0f, 0x65, 0x84, 0x04, 0x0a, 0x8a, 0x03, 0x83, 0x44,
0xc0, 0x02, 0x92, 0x22, 0x20, 0x6c, 0xd9, 0x08, 0x1c, 0xfe, 0x03, 0x80,
0x8c, 0x5c, 0xb2, 0x00, 0x46, 0x2e, 0x5e, 0x04, 0x6c, 0xbb, 0x80, 0x71,
0x08, 0xf2, 0xc8, 0x10, 0x8f, 0x2c, 0x5a, 0xf2, 0x74, 0x0d, 0xb7, 0xf8,
0xd3, 0xed, 0x6d, 0x7f, 0xfc, 0x38, 0x03, 0x0e, 0xf8, 0x00, 0xee, 0x12,
0x89, 0xef, 0x22, 0x09, 0x97, 0xd3, 0xdf, 0x30, 0xc1, 0x48, 0x01, 0x80,
0x06, 0x9b, 0x80, 0x05, 0x64, 0xcd, 0x98, 0x9b, 0x29, 0x48, 0x74, 0x22,
0x32, 0x4b, 0xe0, 0x0f, 0x06, 0xd8, 0x61, 0x77, 0xb8, 0x70, 0x11, 0x8b,
0x97, 0xff, 0xc1, 0x30, 0x06, 0x1d, 0xf0, 0x00, 0xdc, 0xa1, 0x16, 0x88,
0xd5, 0xcc, 0x49, 0x17, 0xd9, 0xe8, 0x36, 0x61, 0x05, 0x55, 0x80, 0xe0,
0x0c, 0x37, 0x00, 0x21, 0xb4, 0x4a, 0x1b, 0xf4, 0x25, 0x8a, 0x40, 0x08,
0xc5, 0xcb, 0xe0, 0x70, 0x64, 0xa2, 0x13, 0xbd, 0xc3, 0x88, 0x47, 0x97,
0xf9, 0xf8, 0x07, 0xb0, 0x5f, 0x8e, 0xcc, 0xe0, 0xf8, 0x00, 0xac, 0x2f,
0x8e, 0xe6, 0x11, 0x2f, 0x51, 0xea, 0xf0, 0x40, 0xbb, 0x30, 0x4b, 0x4b,
0x0e, 0x02, 0x31, 0x72, 0xfc, 0x38, 0x84, 0x79, 0x7f, 0xe1, 0xff, 0xb0,
0x59, 0xea, 0xcf, 0xdc, 0x66, 0x12, 0xc3, 0xd5, 0xf2, 0xbd, 0x3c, 0x76,
0x02, 0x47, 0xae, 0x3d, 0xff, 0xf0, 0x24, 0x5c, 0xb3, 0xd2, 0x26, 0x2b,
0x9f, 0xf1, 0xfe, 0xc1, 0x6c, 0xf4, 0x1e, 0xe1, 0x00, 0x00, 0x80, 0x30,
0x03, 0x07, 0x97, 0x08, 0x4c, 0x85, 0x81, 0xc0, 0x02, 0x04, 0x08, 0x97,
0xe0, 0xd4, 0xe5, 0xff, 0xff, 0xf6, 0x16, 0xdb, 0xee, 0xfd, 0x1f, 0x6b,
0x6b, 0x6b, 0x6b, 0x6b, 0xc3, 0xff, 0xf6, 0x08, 0xbe, 0x0b, 0x44, 0xaf,
0xf9, 0xff, 0xb0, 0x59, 0xe0, 0x0d, 0x5f, 0x6a, 0x38, 0xde, 0x82, 0x01,
0x20, 0xc4, 0x80, 0x78, 0x00, 0x56, 0x46, 0x66, 0x84, 0xd9, 0x4a, 0xac,
0xd2, 0x00, 0x02, 0x05, 0x08, 0xb9, 0x04, 0xdf, 0xf6, 0x95, 0x9d, 0xce,
0x01, 0x82, 0xfc, 0x34, 0x3b, 0xee, 0x47, 0xc0, 0x4f, 0xaf, 0xad, 0x6f,
0x5e, 0x10, 0x17, 0x1d, 0x82, 0xc6, 0x35, 0xa0, 0x01, 0x4d, 0x91, 0x13,
0x1b, 0xf4, 0x26, 0x57, 0xc8, 0x00, 0x04, 0x08, 0x13, 0x71, 0xc6, 0x81,
0x23, 0x5b, 0xd6, 0x51, 0x2b, 0x9f, 0xff, 0xf6, 0x1f, 0xef, 0x3b, 0xcf,
0x53, 0xd7, 0x5d, 0x75, 0xd7, 0x4f, 0xff, 0xff, 0xc1, 0x07, 0x01, 0xc0,
0x21, 0xce, 0x2e, 0x5b, 0xff, 0xff, 0xb0, 0x43, 0xe0, 0x03, 0x6d, 0x10,
0xe1, 0x0a, 0x27, 0xa4, 0xdd, 0xa1, 0xce, 0x03, 0xda, 0x72, 0xdf, 0xfe,
0x29, 0x6a, 0xa5, 0xea, 0xa5, 0x80, 0x06, 0x0b, 0x68, 0x8a, 0x66, 0x62,
0xdf, 0xfc, 0x7e, 0xc1, 0x07, 0x01, 0xe6, 0x3a, 0x88, 0x7f, 0xd0, 0xe8,
0x00, 0xfc, 0x02, 0x67, 0xb1, 0xe0, 0x73, 0xd2, 0x2e, 0x03, 0xf8, 0x00,
0xc4, 0x00, 0x6f, 0x9b, 0x15, 0x6b, 0x2d, 0xcf, 0x07, 0xc0, 0x02, 0x09,
0x6a, 0x32, 0xd8, 0xba, 0x3f, 0x0c, 0x28, 0x20, 0x28, 0x20, 0x20, 0x42,
0x00, 0x83, 0xe0, 0x70, 0x23, 0x15, 0x2c, 0x01, 0x03, 0x2a, 0x63, 0xcc,
0x42, 0x99, 0x02, 0x19, 0xee, 0x01, 0xc8, 0x46, 0x96, 0x00, 0x47, 0x6c,
0xd8, 0x04, 0xaa, 0x41, 0xbc, 0x41, 0x05, 0x32, 0x00, 0x08, 0x20, 0xab,
0x9f, 0xff, 0xf4, 0x0a, 0xb4, 0xab, 0x8d, 0x3e, 0x09, 0x25, 0xff, 0xff,
0xe8, 0x9f, 0xff, 0xf4, 0x4f, 0xff, 0xfa, 0x27, 0xff, 0xfd, 0x07, 0x30,
0x04, 0x2d, 0x20, 0x47, 0x38, 0x95, 0xab, 0xfc, 0x00, 0x6b, 0x19, 0x05,
0xce, 0x53, 0x10, 0xa6, 0xed, 0x08, 0x12, 0xc0, 0x90, 0x00, 0x26, 0x1a,
0x00, 0x21, 0xfb, 0x90, 0xfc, 0x84, 0x24, 0x10, 0xcd, 0x72, 0x00, 0x14,
0xd2, 0x35, 0x91, 0x86, 0x37, 0x5e, 0x9b, 0x20, 0x00, 0x41, 0x05, 0xdc,
0xff, 0xe0, 0x18, 0x57, 0xe0, 0x02, 0x98, 0xf7, 0x33, 0x98, 0x55, 0x3d,
0x47, 0xab, 0xdf, 0xe8, 0x30, 0x01, 0xab, 0x58, 0x48, 0x53, 0xb1, 0x8a,
0x7a, 0xbd, 0xd5, 0x8f, 0x87, 0xe0, 0x01, 0x0c, 0xa4, 0x12, 0xad, 0xcc,
0x29, 0x1a, 0xd0, 0x83, 0xd9, 0xc2, 0x9a, 0xf4, 0xd0, 0x38, 0x08, 0xa4,
0xcb, 0xf2, 0xfc, 0xb0, 0xe0, 0x9c, 0x79, 0x7f, 0xff, 0xfe, 0x10, 0x61,
0xef, 0xc0, 0x01, 0x93, 0x46, 0x83, 0x76, 0xea, 0x41, 0x8d, 0xf7, 0x4b,
0x14, 0x1c, 0x00, 0x04, 0x10, 0x20, 0x18, 0x00, 0x08, 0x14, 0xf0, 0x91,
0xcd, 0xb8, 0x08, 0xd4, 0xb7, 0xcf, 0x7f, 0xff, 0xd0, 0x43, 0xdf, 0xd0,
0x81, 0x83, 0xca, 0xc0, 0x05, 0x34, 0x63, 0x27, 0x15, 0x4d, 0x52, 0x0d,
0xda, 0x5e, 0x30, 0x5e, 0x05, 0x2a, 0x0d, 0xbb, 0x90, 0x22, 0x03, 0x7a,
0x68, 0x1d, 0x21, 0xdc, 0xd4, 0xc8, 0x91, 0x4d, 0xf3, 0xcf, 0xff, 0xfa,
0x05, 0x9c, 0x53, 0xfa, 0x45, 0xff, 0xe3, 0x4f, 0xff, 0xfe, 0x82, 0xdb,
0x7d, 0xdf, 0x1c, 0x00, 0x0d, 0xc7, 0x00, 0x01, 0x01, 0x5f, 0xff, 0xfd,
0x06, 0xae, 0xf9, 0x7d, 0x31, 0xf1, 0x08, 0xe2, 0x00, 0x00, 0x80, 0x30,
0x03, 0xea, 0x2e, 0x30, 0x92, 0x6b, 0xc8, 0x70, 0x98, 0x6b, 0x90, 0xe1,
0x30, 0xd7, 0x3f, 0xf0, 0xc0, 0x34, 0x0b, 0x60, 0x00, 0x99, 0x90, 0xc8,
0xe1, 0x98, 0xdc, 0x41, 0x23, 0xcc, 0x71, 0xf0, 0x40, 0xc6, 0x17, 0xc2,
0x80, 0x07, 0xc0, 0x81, 0x40, 0x00, 0xf8, 0x09, 0xc0, 0x27, 0xb1, 0x37,
0x72, 0x02, 0xa7, 0xd5, 0xe2, 0xc0, 0x80, 0xa6, 0x78, 0x04, 0xe9, 0x89,
0x95, 0xc8, 0x0e, 0xbf, 0x57, 0x8b, 0x00, 0x80, 0x53, 0x3f, 0xf5, 0xf0,
0xd8, 0x76, 0x00, 0x09, 0xe4, 0x38, 0xcc, 0x7e, 0x90, 0x66, 0x90, 0xb5,
0x86, 0x08, 0x50, 0x00, 0x6c, 0x99, 0xb4, 0x44, 0x8c, 0x6a, 0x89, 0xc4,
0xf8, 0x27, 0xf8, 0x41, 0x1f, 0xdc, 0x10, 0x47, 0xf7, 0x60, 0x04, 0x9d,
0x0f, 0x31, 0x5c, 0x9e, 0xaf, 0x0e, 0x09, 0x89, 0x81, 0x0d, 0x34, 0xdb,
0x56, 0x00, 0x48, 0xd4, 0x34, 0x62, 0xb9, 0xbd, 0x5e, 0x1c, 0x05, 0xd9,
0x81, 0x0d, 0x34, 0xdb, 0x57, 0xfc, 0x03, 0x0e, 0x81, 0x54, 0x20, 0x04,
0x86, 0x28, 0x40, 0x09, 0x0c, 0x5f, 0xc0, 0x15, 0x24, 0x63, 0x05, 0x38,
0xc3, 0x6d, 0x5f, 0x58, 0x02, 0xa1, 0x67, 0x70, 0x53, 0x0c, 0x36, 0xd5,
0xff, 0xf0, 0x0d, 0x87, 0x70, 0x00, 0x4c, 0xc8, 0x64, 0x70, 0xcc, 0x6e,
0x20, 0x91, 0xe6, 0x38, 0xfc, 0x61, 0x70, 0x40, 0xfc, 0x20, 0x08, 0x75,
0x94, 0x10, 0x04, 0x15, 0x66, 0x61, 0x35, 0xc3, 0xec, 0x47, 0x2f, 0xab,
0xc5, 0x81, 0x01, 0x4c, 0xe1, 0x32, 0x70, 0xd5, 0x88, 0xe7, 0xf5, 0x78,
0xb0, 0x08, 0x05, 0x33, 0xff, 0x05, 0xfd, 0x82, 0xff, 0xaf, 0x80, 0x02,
0x79, 0x0e, 0x33, 0x1f, 0xa4, 0x19, 0xa4, 0x2d, 0x70, 0x42, 0x08, 0x01,
0x3c, 0x06, 0x08, 0x01, 0x38, 0x25, 0x43, 0x82, 0x61, 0x6e, 0x00, 0x12,
0xf0, 0x9b, 0x39, 0x41, 0x51, 0xea, 0xf1, 0x00, 0x98, 0x5b, 0x90, 0xe4,
0x64, 0xb8, 0x00, 0x4a, 0xd0, 0x99, 0x1c, 0xa0, 0x55, 0x7a, 0xbc, 0x41,
0x19, 0x2e, 0x7f, 0xff, 0xd0, 0x63, 0x2e, 0xa6, 0x3c, 0x00, 0x19, 0xb4,
0x49, 0x8d, 0xfa, 0x11, 0x1d, 0xd2, 0x33, 0xe8, 0xef, 0xff, 0xfe, 0xfc,
0x07, 0x00, 0x10, 0x41, 0xaa, 0x4b, 0x7e, 0x00, 0x0d, 0xd3, 0x26, 0xc4,
0xf3, 0x15, 0x1d, 0x13, 0x9f, 0x04, 0x2c, 0x0e, 0x00, 0xc1, 0x02, 0x40,
0x07, 0x81, 0xb4, 0x70, 0x75, 0x43, 0xf0, 0x3c, 0xc5, 0x70, 0x8f, 0xff,
0xff, 0xb0, 0xc7, 0x80, 0x03, 0x36, 0x89, 0x31, 0xbf, 0xc4, 0x47, 0x7d,
0x04, 0xfb, 0xf8, 0x07, 0xff, 0xb0, 0x45, 0xcb, 0x9e, 0xc2, 0x05, 0xd9,
0x82, 0x00, 0x11, 0x58, 0xaf, 0xff, 0xfb, 0x08, 0xdf, 0xff, 0xff, 0x61,
0x1b, 0x7f, 0xff, 0xec, 0x15, 0x76, 0xff, 0xff, 0xdd, 0xff, 0xff, 0xec,
0x15, 0x44, 0x7b, 0xff, 0xcd, 0x7f, 0xff, 0xec, 0x23, 0x77, 0xff, 0xfe,
0xc8, 0x9f
}
payloadSize只是sei_message()的大小，停止位不计入这个大小。所以SEI的最后一定有一个0x80，