在 Swift 5 中流式传输麦克风音频

Question

我在 2 个 IOS 设备之间建立了工作连接。从一台摄像机向另一台设备发送实时流是可行的。但现在我想发送麦克风音频，但这不起作用。我没有收到任何错误，但我只收到 "click"-noices.

我也看到了，字节数传输了，但是不知道哪里出错了。

下面是发送和接收函数。我还插入了流媒体功能，它可以很好地传输视频。

发件人：

func recorder() {
        let settings : Dictionary = ["AVSampleRateKey" : 44100.0,
            "AVNumberOfChannelsKey" : 1,
            "AVFormatIDKey" : 1819304813,
            "AVLinearPCMIsNonInterleaved" : 0,
            "AVLinearPCMIsBigEndianKey" : 0,
            "AVLinearPCMBitDepthKey" : 16,
            "AVLinearPCMIsFloatKey" : 0]

        audioFormat = AVAudioFormat.init(settings: settings)

        audioEngine = AVAudioEngine.init()
        audioEngine?.inputNode.installTap(onBus: 0, bufferSize: 4410, format: audioEngine?.inputNode.outputFormat(forBus: 0), block: {buffer, when in         
            let audioBuffer = buffer.audioBufferList.pointee.mBuffers
            let data : Data = Data.init(bytes: audioBuffer.mData!, count: Int(audioBuffer.mDataByteSize))
            let arraySize = Int(buffer.frameLength)
            let samples = Array(UnsafeBufferPointer(start: buffer.floatChannelData![0], count:arraySize))
            self.streamData(data: data, len: 4410)
        })

        // Start audio engine
        self.audioEngine?.prepare()
        do {
            try self.audioEngine?.start()
        }
        catch {
            NSLog("cannot start audio engine")
        }
        if(self.audioEngine?.isRunning == true){
            NSLog("Audioengine is running")
        }
    }

sender streamData（适用于视频等）

func streamData(data : Data, len : Int)
    {
        var baseCaseCondition : Bool = false
        var _len : Int = len
        var _byteIndex : Int = 0
        func recursiveBlock(block: @escaping (()->Void)->Void) -> ()->Void {
            return { block(recursiveBlock(block: block)) }
        }
        let aRecursiveBlock :()->Void = recursiveBlock {recurse in
            baseCaseCondition = (data.count > 0 && _byteIndex < data.count) ? true : false
            if ((baseCaseCondition)) {
                _len = (data.count - _byteIndex) == 0 ? 1 : (data.count - _byteIndex) < len ? (data.count - _byteIndex) : len
                NSLog("START | byteIndex: %lu/%lu  writing len: %lu", _byteIndex, data.count, _len)
                var bytes = [UInt8](repeating:0, count:_len)
                data.copyBytes(to: &bytes, from: _byteIndex ..< _byteIndex+_len )
                _byteIndex += (self.outputStream?.write(&bytes, maxLength: _len))!
                NSLog("END | byteIndex: %lu/%lu wrote len: %lu", _byteIndex, data.count, _len)
                recurse()
            }
        }
        if (self.outputStream!.hasSpaceAvailable){
            aRecursiveBlock();
        }
    }

接收者：

func stream(_ aStream: Stream, handle eventCode: Stream.Event) {
...
    case Stream.Event.hasBytesAvailable:
      var thePCMBuffer : AVAudioPCMBuffer = AVAudioPCMBuffer.init(pcmFormat: (self.audioEngine?.inputNode.outputFormat(forBus: 0))!, frameCapacity: AVAudioFrameCount(mlen))!
      thePCMBuffer.frameLength = thePCMBuffer.frameCapacity
      let channels = UnsafeBufferPointer(start: thePCMBuffer.floatChannelData, count: Int(thePCMBuffer.format.channelCount))
      _ = mdata?.copyBytes(to: UnsafeMutableBufferPointer(start: channels[0], count: Int(thePCMBuffer.frameLength)))

      if((self.player?.isPlaying) != nil){
        DispatchQueue.global(qos: .background).async {
          // Background Thread
          DispatchQueue.main.async {
            // Run UI Updates
            self.player?.scheduleBuffer(thePCMBuffer, completionHandler: {
               NSLog("Scheduled buffer")
               NSLog("\(self.player!.isPlaying)")
               let arraySize = Int(thePCMBuffer.frameLength)
               let samples = Array(UnsafeBufferPointer(start: thePCMBuffer.floatChannelData![0], count:arraySize))
               for sample in samples{
                  NSLog("\(sample)")
               }
            })
          }
        }
      }
     }
   mdata = Data.init()
   mlen = DATA_LENGTH
   }
  break;
  ...
}

Answer 1

找到了我猜的解决方案。我尝试了一个模拟器和一个真实的设备。现在我读到，由于不同的采样率存在问题。运行在 2 个设备（或 2 个模拟器）上运行正常。

在 Swift 5 中流式传输麦克风音频

Stream microphone audio in Swift 5

ios

stream

audio-streaming

swift5