需要帮助配置 ffmpeg 以使用 android ndk 解码原始 AAC
need help configuring ffmpeg to decode raw AAC with android ndk
我有一个 android 应用程序可以从外部设备获取原始 AAC 字节，我想解码该数据，但我似乎无法让解码器工作，但 ffmpeg 似乎可以工作适用于解码包含相同音频数据的 mp4 文件（使用 isoviewer 验证）。最近我能够在 android 上获得这个 ffmpeg 库来解码来自同一外部设备的视频帧，但音频似乎无法工作。
这是具有相同数据的文件的 ffmpeg 输出：
$ ffmpeg -i Video_2000-01-01_0411.mp4
ffmpeg version 2.6.1 Copyright (c) 2000-2015 the FFmpeg developers
built with Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
configuration: --prefix=/usr/local/Cellar/ffmpeg/2.6.1 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-libx264 --enable-libmp3lame --enable-libvo-aacenc --enable-libxvid --enable-vda
libavutil 54. 20.100 / 54. 20.100
libavcodec 56. 26.100 / 56. 26.100
libavformat 56. 25.101 / 56. 25.101
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 11.102 / 5. 11.102
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'AXON_Flex_Video_2000-01-01_0411.mp4':
Metadata:
major_brand : mp42
minor_version : 1
compatible_brands: isom3gp43gp5
Duration: 00:00:15.73, start: 0.000000, bitrate: 1134 kb/s
Stream #0:0(eng): Audio: aac (LC) (mp4a / 0x6134706D), 8000 Hz, mono, fltp, 40 kb/s (default)
Metadata:
handler_name : soun
Stream #0:1(eng): Video: mpeg4 (Simple Profile) (mp4v / 0x7634706D), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 1087 kb/s, 29.32 fps, 26.58 tbr, 90k tbn, 1k tbc (default)
Metadata:
handler_name : vide
这是我用于设置和解码音频的 ndk 代码：
jint ffmpeg_init(JNIEnv * env, jobject this) {
audioCodec = avcodec_find_decoder(AV_CODEC_ID_AAC);
if (!audioCodec) {
LOGE("audio codec %d not found", AV_CODEC_ID_AAC);
return -1;
}
audioContext = avcodec_alloc_context3(audioCodec);
if (!audioContext) {
LOGE("Could not allocate codec context");
return -1;
}
int openRet = avcodec_open2(audioContext, audioCodec, NULL);
if (openRet < 0) {
LOGE("Could not open codec, error:%d", openRet);
return -1;
}
audioContext->sample_rate = 8000;
audioContext->channel_layout = AV_CH_LAYOUT_MONO;
audioContext->profile = FF_PROFILE_AAC_LOW;
audioContext->bit_rate = 48 * 1024;
audioContext->sample_fmt = AV_SAMPLE_FMT_FLTP;
// unsigned char extradata[] = {0x15, 0x88};
// audioContext->extradata = extradata;
// audioContext->extradata_size = sizeof(extradata);
audioFrame = av_frame_alloc();
if (!audioFrame) {
LOGE("Could not create audio frame");
return -1;
}
}
jint ffmpeg_decodeAudio(JNIEnv *env, jobject this, jbyteArray aacData, jbyteArray output, int offset, int len) {
LOGI("ffmpeg_decodeAudio()");
char errbuf[128];
AVPacket avpkt = {0};
av_init_packet(&avpkt);
LOGI("av_init_packet()");
int error, got_frame;
uint8_t* buffer = (uint8_t *) (*env)->GetByteArrayElements(env, aacData,0);
uint8_t* copy = av_malloc(len);
memcpy(copy, &buffer[offset], len);
av_packet_from_data(&avpkt, copy, len);
if ((error = avcodec_decode_audio4(audioContext, audioFrame, &got_frame, &avpkt)) < 0) {
ffmpeg_log_error(error);
av_free_packet(&avpkt);
return error;
}
if (got_frame) {
LOGE("Copying audioFrame->extended_data to output jbytearray, linesize[0]:%d", audioFrame->linesize[0]);
(*env)->SetByteArrayRegion(env, output, 0, audioFrame->linesize[0], *audioFrame->extended_data);
}
return 0;
}
如您所见，我有一个 init 函数可以打开解码器并创建上下文，这些都工作正常，没有错误。但是，当我调用 avcodec_decode_audio4 时出现错误：
FFMPEG 错误：-1094995529，处理输入时发现无效数据
我尝试了各种 AVCodecContext 属性的组合。我不确定我需要为解码器设置哪个来完成它的工作，但从在线阅读来看，我应该只需要设置频道布局和 sample_rate（我自己试过）。我还尝试将 extradata/extradata_size 参数设置为应与视频设置匹配的参数：http://wiki.multimedia.cx/index.php?title=MPEG-4_Audio
但运气不好。
因为我们从中获取数据包的设备发送的 aac 数据在开始时没有声音（但是是有效数据包），所以我尝试只发送这些数据，因为它们肯定应该正确解码。
以下是静音的初始音频数据包示例：
010c9eb43f21f90fc87e46fff10a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5dffe214b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4bbd1c429696969696969696969696969696969696969696969696969696969696969696969696969696969696969696969696969696969696969696969696969696969696969696969696969696969696969696969696978
请注意，上面显示的数据只是我放入 AVPacket 中的数据的十六进制编码，它是从外部设备发送到 android 应用程序的。我的应用程序没有直接访问文件的权限，所以我需要解码原始 frames/samples 文件。当我在 isoviewer 中查看音轨数据时，我可以看到音轨的第一个样本与我从包含该文件的设备获得的数据相同（因此，外部设备只是向我发送样本的原始数据）。我相信这些数据可以从从文件的 mdat 框读取 stsz（样本大小）框，从 stco（块偏移）框开始。
此外，isoviewer 显示 esds 框具有以下内容：
ESDescriptor{esId=0, streamDependenceFlag=0, URLFlag=0, oCRstreamFlag=0, streamPriority=0, URLLength=0, URLString='null', remoteODFlag=0, dependsOnEsId=0, oCREsId=0, decoderConfigDescriptor=DecoderConfigDescriptor{objectTypeIndication=64, streamType=5, upStream=0, bufferSizeDB=513, maxBitRate=32000, avgBitRate=32000, decoderSpecificInfo=null, audioSpecificInfo=AudioSpecificConfig{configBytes=1588, audioObjectType=2 (AAC LC), samplingFrequencyIndex=11 (8000), samplingFrequency=0, channelConfiguration=1, syncExtensionType=0, frameLengthFlag=0, dependsOnCoreCoder=0, coreCoderDelay=0, extensionFlag=0, layerNr=0, numOfSubFrame=0, layer_length=0, aacSectionDataResilienceFlag=false, aacScalefactorDataResilienceFlag=false, aacSpectralDataResilienceFlag=false, extensionFlag3=0}, configDescriptorDeadBytes=, profileLevelIndicationDescriptors=[[]]}, slConfigDescriptor=SLConfigDescriptor{predefined=2}}
二进制文件是这样的：
00 00 00 30 65 73 64 73 00 00 00 00 03 80 80 80
1f 00 00 00 04 80 80 80 14 40 15 00 02 01 00 00
7d 00 00 00 7d 00 05 80 80 80 02 15 88 06 01 02
我发现了上面代码的主要问题。当您调用 avcodec_open2 时解码器被初始化。因此，我应该在打开之前先设置上下文字段：
jint ffmpeg_init(JNIEnv * env, jobject this) {
//....
audioContext = avcodec_alloc_context3(audioCodec);
audioContext->sample_rate = 8000;
audioContext->channel_layout = AV_CH_LAYOUT_MONO;
audioContext->channels = 1;
int openRet = avcodec_open2(audioContext, audioCodec, NULL);
if (openRet < 0) {
LOGE("Could not open codec, error:%d", openRet);
return -1;
}
解码器正在无误地解码音频。
我有一个 android 应用程序可以从外部设备获取原始 AAC 字节，我想解码该数据，但我似乎无法让解码器工作，但 ffmpeg 似乎可以工作适用于解码包含相同音频数据的 mp4 文件（使用 isoviewer 验证）。最近我能够在 android 上获得这个 ffmpeg 库来解码来自同一外部设备的视频帧，但音频似乎无法工作。
这是具有相同数据的文件的 ffmpeg 输出：
$ ffmpeg -i Video_2000-01-01_0411.mp4
ffmpeg version 2.6.1 Copyright (c) 2000-2015 the FFmpeg developers
built with Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
configuration: --prefix=/usr/local/Cellar/ffmpeg/2.6.1 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-libx264 --enable-libmp3lame --enable-libvo-aacenc --enable-libxvid --enable-vda
libavutil 54. 20.100 / 54. 20.100
libavcodec 56. 26.100 / 56. 26.100
libavformat 56. 25.101 / 56. 25.101
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 11.102 / 5. 11.102
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'AXON_Flex_Video_2000-01-01_0411.mp4':
Metadata:
major_brand : mp42
minor_version : 1
compatible_brands: isom3gp43gp5
Duration: 00:00:15.73, start: 0.000000, bitrate: 1134 kb/s
Stream #0:0(eng): Audio: aac (LC) (mp4a / 0x6134706D), 8000 Hz, mono, fltp, 40 kb/s (default)
Metadata:
handler_name : soun
Stream #0:1(eng): Video: mpeg4 (Simple Profile) (mp4v / 0x7634706D), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 1087 kb/s, 29.32 fps, 26.58 tbr, 90k tbn, 1k tbc (default)
Metadata:
handler_name : vide
这是我用于设置和解码音频的 ndk 代码：
jint ffmpeg_init(JNIEnv * env, jobject this) {
audioCodec = avcodec_find_decoder(AV_CODEC_ID_AAC);
if (!audioCodec) {
LOGE("audio codec %d not found", AV_CODEC_ID_AAC);
return -1;
}
audioContext = avcodec_alloc_context3(audioCodec);
if (!audioContext) {
LOGE("Could not allocate codec context");
return -1;
}
int openRet = avcodec_open2(audioContext, audioCodec, NULL);
if (openRet < 0) {
LOGE("Could not open codec, error:%d", openRet);
return -1;
}
audioContext->sample_rate = 8000;
audioContext->channel_layout = AV_CH_LAYOUT_MONO;
audioContext->profile = FF_PROFILE_AAC_LOW;
audioContext->bit_rate = 48 * 1024;
audioContext->sample_fmt = AV_SAMPLE_FMT_FLTP;
// unsigned char extradata[] = {0x15, 0x88};
// audioContext->extradata = extradata;
// audioContext->extradata_size = sizeof(extradata);
audioFrame = av_frame_alloc();
if (!audioFrame) {
LOGE("Could not create audio frame");
return -1;
}
}
jint ffmpeg_decodeAudio(JNIEnv *env, jobject this, jbyteArray aacData, jbyteArray output, int offset, int len) {
LOGI("ffmpeg_decodeAudio()");
char errbuf[128];
AVPacket avpkt = {0};
av_init_packet(&avpkt);
LOGI("av_init_packet()");
int error, got_frame;
uint8_t* buffer = (uint8_t *) (*env)->GetByteArrayElements(env, aacData,0);
uint8_t* copy = av_malloc(len);
memcpy(copy, &buffer[offset], len);
av_packet_from_data(&avpkt, copy, len);
if ((error = avcodec_decode_audio4(audioContext, audioFrame, &got_frame, &avpkt)) < 0) {
ffmpeg_log_error(error);
av_free_packet(&avpkt);
return error;
}
if (got_frame) {
LOGE("Copying audioFrame->extended_data to output jbytearray, linesize[0]:%d", audioFrame->linesize[0]);
(*env)->SetByteArrayRegion(env, output, 0, audioFrame->linesize[0], *audioFrame->extended_data);
}
return 0;
}
如您所见，我有一个 init 函数可以打开解码器并创建上下文，这些都工作正常，没有错误。但是，当我调用 avcodec_decode_audio4 时出现错误：
FFMPEG 错误：-1094995529，处理输入时发现无效数据
我尝试了各种 AVCodecContext 属性的组合。我不确定我需要为解码器设置哪个来完成它的工作，但从在线阅读来看，我应该只需要设置频道布局和 sample_rate（我自己试过）。我还尝试将 extradata/extradata_size 参数设置为应与视频设置匹配的参数：http://wiki.multimedia.cx/index.php?title=MPEG-4_Audio 但运气不好。
因为我们从中获取数据包的设备发送的 aac 数据在开始时没有声音（但是是有效数据包），所以我尝试只发送这些数据，因为它们肯定应该正确解码。
以下是静音的初始音频数据包示例：
010c9eb43f21f90fc87e46fff10a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5a5dffe214b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4b4bbd1c429696969696969696969696969696969696969696969696969696969696969696969696969696969696969696969696969696969696969696969696969696969696969696969696969696969696969696969696978
请注意，上面显示的数据只是我放入 AVPacket 中的数据的十六进制编码，它是从外部设备发送到 android 应用程序的。我的应用程序没有直接访问文件的权限，所以我需要解码原始 frames/samples 文件。当我在 isoviewer 中查看音轨数据时，我可以看到音轨的第一个样本与我从包含该文件的设备获得的数据相同（因此，外部设备只是向我发送样本的原始数据）。我相信这些数据可以从从文件的 mdat 框读取 stsz（样本大小）框，从 stco（块偏移）框开始。
此外，isoviewer 显示 esds 框具有以下内容：
ESDescriptor{esId=0, streamDependenceFlag=0, URLFlag=0, oCRstreamFlag=0, streamPriority=0, URLLength=0, URLString='null', remoteODFlag=0, dependsOnEsId=0, oCREsId=0, decoderConfigDescriptor=DecoderConfigDescriptor{objectTypeIndication=64, streamType=5, upStream=0, bufferSizeDB=513, maxBitRate=32000, avgBitRate=32000, decoderSpecificInfo=null, audioSpecificInfo=AudioSpecificConfig{configBytes=1588, audioObjectType=2 (AAC LC), samplingFrequencyIndex=11 (8000), samplingFrequency=0, channelConfiguration=1, syncExtensionType=0, frameLengthFlag=0, dependsOnCoreCoder=0, coreCoderDelay=0, extensionFlag=0, layerNr=0, numOfSubFrame=0, layer_length=0, aacSectionDataResilienceFlag=false, aacScalefactorDataResilienceFlag=false, aacSpectralDataResilienceFlag=false, extensionFlag3=0}, configDescriptorDeadBytes=, profileLevelIndicationDescriptors=[[]]}, slConfigDescriptor=SLConfigDescriptor{predefined=2}}
二进制文件是这样的：
00 00 00 30 65 73 64 73 00 00 00 00 03 80 80 80
1f 00 00 00 04 80 80 80 14 40 15 00 02 01 00 00
7d 00 00 00 7d 00 05 80 80 80 02 15 88 06 01 02
我发现了上面代码的主要问题。当您调用 avcodec_open2 时解码器被初始化。因此，我应该在打开之前先设置上下文字段：
jint ffmpeg_init(JNIEnv * env, jobject this) {
//....
audioContext = avcodec_alloc_context3(audioCodec);
audioContext->sample_rate = 8000;
audioContext->channel_layout = AV_CH_LAYOUT_MONO;
audioContext->channels = 1;
int openRet = avcodec_open2(audioContext, audioCodec, NULL);
if (openRet < 0) {
LOGE("Could not open codec, error:%d", openRet);
return -1;
}
解码器正在无误地解码音频。