Superpowered

Question

我正在尝试将用户的声音与音乐混合并将其保存到文件中。

我创建了 2 个解码器 - 1 个用于语音，1 个用于音乐，并将它们放入 Mixer 的输入中。我解码每一帧并使用 FILE/createWAV/fwrite.

将其保存到文件中

当我的歌曲是 .wav 并且具有与录制语音 (48000/1024) 相同的 sampleRate 和 samplesPerFrame 时一切正常。

然而，当我想使用具有不同参数 (44100/1152) 的 .mp3 文件时，最终文件不正确 - 它被拉伸或有一些噼啪声。我认为这是因为我们为每个解码器获得了不同的 sampledDecoded，当它被放入 Mixer 或保存到文件时 - 这些样本之间的差异丢失了。

据我所知，当我们执行 voiceDecoder->decode(buffer, &samplesDecoded) 时，它会将 samplePosition 移动 samplesDecoded。

我试图做的是使用两个解码器的最小值。然而，根据上面的句子，每次循环迭代歌曲都会丢失 (1152 - 1024 = 128) 128 个样本，所以我也试图寻找 songDecoder 与 voiceDecoder 相同：songDecoder->seek(voiceDecoder->samplePosition, true) 但它导致文件完全不正确。

总结：当每个解码器具有不同的采样率和采样率时，我应该如何处理 mixer/offlineProcessing 2 个解码器？

代码：

void AudioProcessor::startProcessing() {
    SuperpoweredStereoMixer *mixer = new SuperpoweredStereoMixer();
    float *mixerInputs_[] = {0,0,0,0};
    float *mixerOutputs_[] = {0,0};
    float inputLevels_[]= {0.5f, 0.5f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f};
    float outputLevels_[] = { 1.0f, 1.0f };

    SuperpoweredDecoder *voiceDecoder = new SuperpoweredDecoder();
    SuperpoweredDecoder *songDecoder = new SuperpoweredDecoder();

    if (voiceDecoder->open(voiceInputPath, false) || songDecoder->open(songInputPath, false, songOffset, songLength)) {
        delete voiceDecoder;
        delete songDecoder;
        delete mixer;
        callJavaVoidMethodWithBoolParam(jvm, jObject, processingFinishedMethodId, false);
        return;
    };

    FILE *fd = createWAV(outputPath, songDecoder->samplerate, 2);
    if (!fd) {
        delete voiceDecoder;
        delete songDecoder;
        delete mixer;
        callJavaVoidMethodWithBoolParam(jvm, jObject, processingFinishedMethodId, false);
        return;
    };

    // Create a buffer for the 16-bit integer samples coming from the decoder.
    short int *voiceIntBuffer = (short int *)malloc(voiceDecoder->samplesPerFrame * 4 * sizeof(short int) + 32768);
    short int *songIntBuffer = (short int *)malloc(songDecoder->samplesPerFrame * 4 * sizeof(short int) + 32768);
    short int *outputIntBuffer = (short int *)malloc(voiceDecoder->samplesPerFrame * 4 * sizeof(short int) + 32768);

    // Create a buffer for the 32-bit floating point samples required by the effect.
    float *voiceFloatBuffer = (float *)malloc(voiceDecoder->samplesPerFrame * 4 * sizeof(float) + 32768);
    float *songFloatBuffer = (float *)malloc(songDecoder->samplesPerFrame * 4 * sizeof(float) + 32768);
    float *outputFloatBuffer = (float *)malloc(voiceDecoder->samplesPerFrame * 4 * sizeof(float) + 32768);

    bool isError = false;

    // Processing.
    while (true) {
        if (isCanceled) {
            isError = true;
            break;
        }

        // Decode one frame. samplesDecoded will be overwritten with the actual decoded number of samples.
        unsigned int voiceSamplesDecoded = voiceDecoder->samplesPerFrame;
        if (voiceDecoder->decode(voiceIntBuffer, &voiceSamplesDecoded) == SUPERPOWEREDDECODER_ERROR) {
            break;
        }
        if (voiceSamplesDecoded < 1) {
            break;
        }

        //
        // Decode one frame. samplesDecoded will be overwritten with the actual decoded number of samples.
        unsigned int songSamplesDecoded = songDecoder->samplesPerFrame;
        if (songDecoder->decode(songIntBuffer, &songSamplesDecoded) == SUPERPOWEREDDECODER_ERROR) {
            break;
        }
        if (songSamplesDecoded < 1) {
            break;
        }

        unsigned int samplesDecoded = static_cast<unsigned int>(fmin(voiceSamplesDecoded, songSamplesDecoded));

        // Convert the decoded PCM samples from 16-bit integer to 32-bit floating point.
        SuperpoweredShortIntToFloat(voiceIntBuffer, voiceFloatBuffer, samplesDecoded);
        SuperpoweredShortIntToFloat(songIntBuffer, songFloatBuffer, samplesDecoded);

        //setup mixer inputs
        mixerInputs_[0] = voiceFloatBuffer;
        mixerInputs_[1] = songFloatBuffer;
        mixerInputs_[2] = NULL;
        mixerInputs_[3] = NULL;

        // setup mixer outputs, might have two separate outputs (L/R) if second not null
        mixerOutputs_[0] = outputFloatBuffer;
        mixerOutputs_[1] = NULL;

        mixer->process(mixerInputs_, mixerOutputs_, inputLevels_, outputLevels_, NULL, NULL, samplesDecoded);

        // Convert the PCM samples from 32-bit floating point to 16-bit integer.
        SuperpoweredFloatToShortInt(outputFloatBuffer, outputIntBuffer, samplesDecoded);

        // Write the audio to disk.
        fwrite(outputIntBuffer, 1, samplesDecoded * 4, fd);

        // songDecoder->seek(voiceDecoder->samplePosition, true);
    }

    // Cleanup.
    closeWAV(fd);
    delete voiceDecoder;
    delete songDecoder;
    delete mixer;
    free(voiceIntBuffer);
    free(voiceFloatBuffer);
    free(songIntBuffer);
    free(songFloatBuffer);
    free(outputFloatBuffer);
    free(outputIntBuffer);
}

提前致谢！

Answer 1

您需要使用 SuperpoweredResampler class 来匹配采样率。您还需要为两个输入设置一些循环缓冲区，因为在许多情况下可用的样本数量不匹配。

Answer 2

好的，我设法让它工作了。我按照@Gabor 的建议做了，但没有完全奏效。我缺少的是频道 - 我必须将它包含在我的 buffer/shift 操作中，现在没问题了！

Superpowered - 导出到带有混音器和解码器问题的文件（不同的 sampleRates 和 samplesPerFrame）

Superpowered - export to file with Mixer and Decoder problem (different sampleRates and samplesPerFrame)

android

c++

audio