如何使用 CoreAudio 的 AudioConverter 实时编码 AAC？

Question

我能找到的所有使用 AudioConverterRef 的示例代码都侧重于我预先拥有所有数据的用例（例如转换磁盘上的文件）。他们通常调用 AudioConverterFillComplexBuffer 将要转换的 PCM 作为 inInputDataProcUserData 并将其填写在回调中。（这真的应该如何使用吗？那为什么它需要回调？）对于我的用例，我正在尝试从麦克风流式传输 aac 音频，所以我没有文件，并且我的 PCM 缓冲区正在实时填写。

因为我没有预先获得所有数据，所以我尝试在输入数据输出后在回调中执行 *ioNumberDataPackets = 0，但这只会让 AudioConverter 处于死机状态需要 AudioConverterReset()ted，但我没有从中得到任何数据。

我在网上看到的一种方法是 return 如果我存储的数据太小，回调会出错，一旦我有更多数据再试一次，但看起来是这样的浪费资源，我什至无法尝试。

我真的需要做 "retry until my input buffer is big enough"，还是有更好的方法？

Answer 1

AudioConverterFillComplexBuffer实际上并不意味着"fill the encoder with my input buffer that I have here"。意思是"fill this output buffer here with encoded data from the encoder"。从这个角度来看，回调突然变得有意义了——它用于获取源数据以满足 "fill this output buffer for me" 请求。也许这对其他人来说是显而易见的，但我花了很长的时间来理解这一点（从我看到的所有 AudioConverter 示例代码中，人们都在通过 inInputDataProcUserData 发送输入数据的地方徘徊，我猜我不是唯一一个）。

AudioConverterFillComplexBuffer 调用处于阻塞状态，需要您从回调中同步向其传送数据。如果您实时编码，则需要在您自己设置的单独线程上调用 FillComplexBuffer。在回调中，您可以检查可用的输入数据，如果不可用，则需要阻塞信号量。使用 NSCondition，编码器线程将看起来像这样：

- (void)startEncoder
{
    OSStatus creationStatus = AudioConverterNew(&_fromFormat, &_toFormat, &_converter);

    _running = YES;
    _condition = [[NSCondition alloc] init];
    [self performSelectorInBackground:@selector(_encoderThread) withObject:nil];
}

- (void)_encoderThread
{
    while(_running) {
        // Make quarter-second buffers.
        size_t bufferSize = (_outputBitrate/8) * 0.25;
        NSMutableData *outAudioBuffer = [NSMutableData dataWithLength:bufferSize];
        AudioBufferList outAudioBufferList;
        outAudioBufferList.mNumberBuffers = 1;
        outAudioBufferList.mBuffers[0].mNumberChannels = _toFormat.mChannelsPerFrame;
        outAudioBufferList.mBuffers[0].mDataByteSize = (UInt32)bufferSize;
        outAudioBufferList.mBuffers[0].mData = [outAudioBuffer mutableBytes];

        UInt32 ioOutputDataPacketSize = 1;

        _currentPresentationTime = kCMTimeInvalid; // you need to fill this in during FillComplexBuffer
        const OSStatus conversionResult = AudioConverterFillComplexBuffer(_converter, FillBufferTrampoline, (__bridge void*)self, &ioOutputDataPacketSize, &outAudioBufferList, NULL);

        // here I convert the AudioBufferList into a CMSampleBuffer, which I've omitted for brevity.
        // Ping me if you need it.
        [self.delegate encoder:self encodedSampleBuffer:outSampleBuffer];
    }
}

回调可能如下所示：（请注意，我通常使用此蹦床立即转发到我的实例上的一个方法（通过在 inUserData 中转发我的实例；为简洁起见省略此步骤）） :

static OSStatus FillBufferTrampoline(AudioConverterRef               inAudioConverter,
                                        UInt32*                         ioNumberDataPackets,
                                        AudioBufferList*                ioData,
                                        AudioStreamPacketDescription**  outDataPacketDescription,
                                        void*                           inUserData)
{
    [_condition lock];

    UInt32 countOfPacketsWritten = 0;

    while (true) {
        // If the condition fires and we have shut down the encoder, just pretend like we have written 0 bytes and are done.
        if(!_running) break;

        // Out of input data? Wait on the condition.
        if(_inputBuffer.length == 0) {
            [_condition wait];
            continue;
        }

        // We have data! Fill ioData from your _inputBuffer here.
        // Also save the input buffer's start presentationTime here.

        // Exit out of the loop, since we're done waiting for data
        break;
    }

    [_condition unlock];

        // 2. Set ioNumberDataPackets to the amount of data remaining


    // if running is false, this will be 0, indicating EndOfStream
    *ioNumberDataPackets = countOfPacketsWritten;

    return noErr;
}

为了完整起见，下面是您将如何向此编码器提供数据，以及如何正确关闭它：

- (void)appendSampleBuffer:(CMSampleBufferRef)sampleBuffer
{
    [_condition lock];
    // Convert sampleBuffer and put it into _inputBuffer here
    [_condition broadcast];
    [_condition unlock];
}

- (void)stopEncoding
{
    [_condition lock];
    _running = NO;
    [_condition broadcast];
    [_condition unlock];
}

Answer 2

为了将来参考，有一种方法更简单。

CoreAudio header 的状态：

If the callback returns an error, it must return zero packets of data. AudioConverterFillComplexBuffer will stop producing output and return whatever output has already been produced to its caller, along with the error code. This mechanism can be used when an input proc has temporarily run out of data, but has not yet reached end of stream.

那么，就这样做吧。而不是 returning noErr with *ioNumberDataPackets = 0, return any error (just make one, I used -1), 已经转换的数据将被 returned, 而音频转换器保持活动状态，不需要重置。

如何使用 CoreAudio 的 AudioConverter 实时编码 AAC？

How do I use CoreAudio's AudioConverter to encode AAC in real-time?

audio

aac

core-audio

audio-converter

ios