InputStream 混音(MODE_STREAM)
InputStream audio mixing (MODE_STREAM)
我正在 Android...
制作鼓音序器
我正在写入 MODE_STREAM
中的 AudioTrack
,这样我就可以实现与所有 InputStreams
的同步音频播放(可通过 'active' InputStreams 列表获得, activeStreams
在下面的代码中)
音频始终为:PCM (WAV),16 位立体声 44100 Hz。
显然,我无法在 UI 线程上实时合成音频,所以我使用 AsyncTask
来排队所有音频缓冲。
我的缓冲回放可以正常工作,但是当谈到合并两个(或更多)InputStream
的缓冲区时,互联网似乎在讨论下一步该怎么做。 "Convert the byte[] to short[]!"、"No, do the bit mixing on-the-fly!"、"But if you don't use shorts the byte Endianness is ignored!"、"It gets ignored anyway!" - 我什至不知道了。
如何混合两个或多个 InputStreams 的缓冲区?我不明白为什么我当前的实现失败
我试过 4 种不同的 Whosebug 解决方案来将 byte[] 转换为 short[],这样我就可以将样本加在一起,但是转换总是立即崩溃 Java 并出现一些神秘的错误消息我无法理解。所以现在我放弃了。这是我的代码实现 one such Whosebug solution...
protected Long doInBackground(Object ... Object) {
int bytesWritten = 0;
InputStream inputStream;
int si = 0, i = 0;
//The combined buffers. The 'composition'
short[] cBuffer = new short[Synth.AUDIO_BUFFER_SIZE];
//The 'current buffer', the segment of inputStream audio.
byte[] bBuffer = new byte[Synth.AUDIO_BUFFER_SIZE];
//The 'current buffer', converted to short?
short[] sBuffer = new short[Synth.AUDIO_BUFFER_SIZE];
int curStreamNum;
int numStreams = activeStreams.size();
short mix;
//Start with an empty 'composition'
cBuffer = new short[Synth.AUDIO_BUFFER_SIZE];
boolean bufferEmpty = false;
try {
while(true) { // keep going forever, until stopped or paused.
for(curStreamNum = 0;curStreamNum < numStreams;curStreamNum++){
inputStream = activeStreams.get(curStreamNum);
i = inputStream.read(bBuffer);
bufferEmpty = i<=-1;
if(bufferEmpty){
//Input stream buffer was empty. It's out of audio. Close and remove the stream.
inputStream.close();
activeStreams.remove(curStreamNum);
curStreamNum--; numStreams--; continue; // hard continue.
}else{
//Take the now-read buffer, and convert to shorts.
ByteBuffer.wrap(bBuffer).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer().get(sBuffer);
//Take the short buffer, merge into composition buffer.
//TODO: Optimize by making the 'first layer' of the composition the first buffer, on its own.
for(si=0;si<Synth.AUDIO_BUFFER_SIZE;si++){
mix = (short) (sBuffer[si] + cBuffer[si]);
//This part is probably completely wrong too. I'm not up to here yet to evaluate whats needed...
if(mix >= 32767){
mix = 32767;
}else if (mix <= -32768){
mix = -32768;
}
cBuffer[si] = mix;
}
}
}
track.write(sBuffer, 0, i);
//It's always full; full buffer of silence, or of composited audio.
totalBytesWritten += Synth.AUDIO_BUFFER_SIZE;
//.. queueNewInputStreams ..
publishProgress(totalBytesWritten);
if (isCancelled()) break;
}
} catch (IOException e) {e.printStackTrace();}
return Long.valueOf(totalBytesWritten);
}
我目前在这条线上收到 BufferUnderflowException
:ByteBuffer.wrap(bBuffer).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer().get(sBuffer);
。
怎么可能出现缓冲区欠载?我只是将 byte[] 转换为 short[].
请帮忙!
我已经发布了我的整个函数,希望这个更完整的代码示例和相当适应性强的用法可以帮助其他人。
(P.S。byte[] 到 short[] 的转换之后是一些脆弱的硬剪辑,我什至还没有调试,但也将不胜感激)
您的解决方案似乎不错,我看到两个问题和一个潜在问题:
- 短数组的长度:必须是字节数组的一半,否则会下溢
- 短裤的总和必须是短裤的平均值而不仅仅是总和,否则你只会得到噪音
- (潜在问题)您通过 InputStream 读取的数组长度不能完全免费,因为您必须为每个 InputStream 求和 2 个字节(那么它必须是一个偶数数组)并且您应该注意单声道与单声道。立体声音频文件(如果立体声,左声道有 2 个字节,右声道有 2 个字节交错)
在这里你可以找到一个片段,我会用它来求和两个 WAV 数组(16 位,单声道)
Random random = new Random();
int bufferLength = 20;
byte[] is1 = new byte[bufferLength];
byte[] is2 = new byte[bufferLength];
byte[] average = new byte[bufferLength];
random.nextBytes(is1);
random.nextBytes(is2);
short[] shorts1 = new short[bufferLength/2];
ByteBuffer.wrap(is1).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer().get(shorts1);
short[] shorts2 = new short[bufferLength/2];
ByteBuffer.wrap(is2).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer().get(shorts2);
short[] result = new short[bufferLength/2];
for (int i=0; i<result.length; i++) {
result[i] = (short) ((shorts1[i] + shorts2[i])/2);
}
ByteBuffer.wrap(average).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer().put(result);
对于 32 位立体声,解决方案可能是
Random random = new Random();
int bufferLength = 8 * 50;
byte[] is1 = new byte[bufferLength];
byte[] is2 = new byte[bufferLength];
byte[] average = new byte[bufferLength];
random.nextBytes(is1);
random.nextBytes(is2);
System.out.println(bytesToHex(is1));
System.out.println(bytesToHex(is2));
int[] ints1 = new int[bufferLength/4];
ByteBuffer.wrap(is1).order(ByteOrder.LITTLE_ENDIAN).asIntBuffer().get(ints1);
int[] ints2 = new int[bufferLength/4];
ByteBuffer.wrap(is2).order(ByteOrder.LITTLE_ENDIAN).asIntBuffer().get(ints2);
int[] result = new int[bufferLength/4];
for (int i=0; i<result.length; i++) {
result[i] = ((ints1[i] + ints2[i])/2);
}
ByteBuffer.wrap(average).order(ByteOrder.LITTLE_ENDIAN).asIntBuffer().put(result);
我正在 Android...
制作鼓音序器我正在写入 MODE_STREAM
中的 AudioTrack
,这样我就可以实现与所有 InputStreams
的同步音频播放(可通过 'active' InputStreams 列表获得, activeStreams
在下面的代码中)
音频始终为:PCM (WAV),16 位立体声 44100 Hz。
显然,我无法在 UI 线程上实时合成音频,所以我使用 AsyncTask
来排队所有音频缓冲。
我的缓冲回放可以正常工作,但是当谈到合并两个(或更多)InputStream
的缓冲区时,互联网似乎在讨论下一步该怎么做。 "Convert the byte[] to short[]!"、"No, do the bit mixing on-the-fly!"、"But if you don't use shorts the byte Endianness is ignored!"、"It gets ignored anyway!" - 我什至不知道了。
如何混合两个或多个 InputStreams 的缓冲区?我不明白为什么我当前的实现失败
我试过 4 种不同的 Whosebug 解决方案来将 byte[] 转换为 short[],这样我就可以将样本加在一起,但是转换总是立即崩溃 Java 并出现一些神秘的错误消息我无法理解。所以现在我放弃了。这是我的代码实现 one such Whosebug solution...
protected Long doInBackground(Object ... Object) {
int bytesWritten = 0;
InputStream inputStream;
int si = 0, i = 0;
//The combined buffers. The 'composition'
short[] cBuffer = new short[Synth.AUDIO_BUFFER_SIZE];
//The 'current buffer', the segment of inputStream audio.
byte[] bBuffer = new byte[Synth.AUDIO_BUFFER_SIZE];
//The 'current buffer', converted to short?
short[] sBuffer = new short[Synth.AUDIO_BUFFER_SIZE];
int curStreamNum;
int numStreams = activeStreams.size();
short mix;
//Start with an empty 'composition'
cBuffer = new short[Synth.AUDIO_BUFFER_SIZE];
boolean bufferEmpty = false;
try {
while(true) { // keep going forever, until stopped or paused.
for(curStreamNum = 0;curStreamNum < numStreams;curStreamNum++){
inputStream = activeStreams.get(curStreamNum);
i = inputStream.read(bBuffer);
bufferEmpty = i<=-1;
if(bufferEmpty){
//Input stream buffer was empty. It's out of audio. Close and remove the stream.
inputStream.close();
activeStreams.remove(curStreamNum);
curStreamNum--; numStreams--; continue; // hard continue.
}else{
//Take the now-read buffer, and convert to shorts.
ByteBuffer.wrap(bBuffer).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer().get(sBuffer);
//Take the short buffer, merge into composition buffer.
//TODO: Optimize by making the 'first layer' of the composition the first buffer, on its own.
for(si=0;si<Synth.AUDIO_BUFFER_SIZE;si++){
mix = (short) (sBuffer[si] + cBuffer[si]);
//This part is probably completely wrong too. I'm not up to here yet to evaluate whats needed...
if(mix >= 32767){
mix = 32767;
}else if (mix <= -32768){
mix = -32768;
}
cBuffer[si] = mix;
}
}
}
track.write(sBuffer, 0, i);
//It's always full; full buffer of silence, or of composited audio.
totalBytesWritten += Synth.AUDIO_BUFFER_SIZE;
//.. queueNewInputStreams ..
publishProgress(totalBytesWritten);
if (isCancelled()) break;
}
} catch (IOException e) {e.printStackTrace();}
return Long.valueOf(totalBytesWritten);
}
我目前在这条线上收到 BufferUnderflowException
:ByteBuffer.wrap(bBuffer).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer().get(sBuffer);
。
怎么可能出现缓冲区欠载?我只是将 byte[] 转换为 short[].
请帮忙!
我已经发布了我的整个函数,希望这个更完整的代码示例和相当适应性强的用法可以帮助其他人。
(P.S。byte[] 到 short[] 的转换之后是一些脆弱的硬剪辑,我什至还没有调试,但也将不胜感激)
您的解决方案似乎不错,我看到两个问题和一个潜在问题:
- 短数组的长度:必须是字节数组的一半,否则会下溢
- 短裤的总和必须是短裤的平均值而不仅仅是总和,否则你只会得到噪音
- (潜在问题)您通过 InputStream 读取的数组长度不能完全免费,因为您必须为每个 InputStream 求和 2 个字节(那么它必须是一个偶数数组)并且您应该注意单声道与单声道。立体声音频文件(如果立体声,左声道有 2 个字节,右声道有 2 个字节交错)
在这里你可以找到一个片段,我会用它来求和两个 WAV 数组(16 位,单声道)
Random random = new Random();
int bufferLength = 20;
byte[] is1 = new byte[bufferLength];
byte[] is2 = new byte[bufferLength];
byte[] average = new byte[bufferLength];
random.nextBytes(is1);
random.nextBytes(is2);
short[] shorts1 = new short[bufferLength/2];
ByteBuffer.wrap(is1).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer().get(shorts1);
short[] shorts2 = new short[bufferLength/2];
ByteBuffer.wrap(is2).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer().get(shorts2);
short[] result = new short[bufferLength/2];
for (int i=0; i<result.length; i++) {
result[i] = (short) ((shorts1[i] + shorts2[i])/2);
}
ByteBuffer.wrap(average).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer().put(result);
对于 32 位立体声,解决方案可能是
Random random = new Random();
int bufferLength = 8 * 50;
byte[] is1 = new byte[bufferLength];
byte[] is2 = new byte[bufferLength];
byte[] average = new byte[bufferLength];
random.nextBytes(is1);
random.nextBytes(is2);
System.out.println(bytesToHex(is1));
System.out.println(bytesToHex(is2));
int[] ints1 = new int[bufferLength/4];
ByteBuffer.wrap(is1).order(ByteOrder.LITTLE_ENDIAN).asIntBuffer().get(ints1);
int[] ints2 = new int[bufferLength/4];
ByteBuffer.wrap(is2).order(ByteOrder.LITTLE_ENDIAN).asIntBuffer().get(ints2);
int[] result = new int[bufferLength/4];
for (int i=0; i<result.length; i++) {
result[i] = ((ints1[i] + ints2[i])/2);
}
ByteBuffer.wrap(average).order(ByteOrder.LITTLE_ENDIAN).asIntBuffer().put(result);